Why most products fit in digital projects like a square peg in a round hole, leading to systematic failure.
The introduction hurdles of Agile methodologies in organizations are now things of the past. Any new customer I deal with has had an Agile practice in place for many years, often decades now, at least for their technology-centric endeavours. Despite this wide success —duly earned and founded in genuine improvements— digital transformations and technology initiatives continue to show embarrassing levels of failure. This lack of success manifests both in failing to achieve time and cost targets as well as failing to provide real —or perceived— business value.
Common sense dictates that if a good idea doesn’t show positive results, it’s not such a good idea. When this happens, one usually goes back to previous ways of working or searches for alternatives. Rest assured, that’s not going to happen with Agile. But then, the question remains and, given the billions of dollars enterprises spend on information technologies, is worth being answered: Why Agile methods haven’t solved this decades-long chronic failure issue? The rock-bottom root cause is that IT investments are left to a single unit —usually called the IT function, or just IT. What’s wrong with this is that most IT units are project-oriented while Agile methods are product-oriented. Applying Agile methods to project-oriented goals is like trying to fit a square peg in a round hole. As such, teams working in Agile mode are straitjacketed by the way technology teams are organized to provide their capabilities to your organization.
Single-Desk IT units are project-oriented while Agile methods are product-oriented, and that’s like trying to fit a square peg in a round hole.
What you will learn below is that, as long as the typical engagement model for IT talent in enterprises is not radically changed, digital teams will struggle to deliver value. But first, let’s recap on the fundamentals of Agile product centricity.
Solving Genuine Issues
Acceptance of Agile methods in organizations is no surprise. It solves issues that were once plaguing the progress of digital projects. The classical approach was to analyze everything, then design everything, then build everything that was designed. This type of development approach is called waterfall. A huge proportion of projects conducted this way ended up canned. Too many projects burned lots of gas before organizations realized that dozens of persons were not working on the right thing. The consequences were devastating: white elephant projects that lasted for years before finally being stopped as well as thousands of smaller projects lasting much longer than expected and acting as business boat anchors, slowing down your ability to change.
Iteration to the Rescue
Agile ways of working take a different approach from waterfall sequencing: iterating. When there is uncertainty about an end product, making iterative development reduces the risk. You design and build a smaller version or a portion of the end product to test the adequacy. Then you build another part, or a bigger version, and test again. That way, you burn less gas before realizing you weren’t building the right solution. You can then adjust the target you’re aiming for or completely stop burning gas until you find the right target.
It is simple, smart and proven to be effective. In other fields, such as the automotive industry, they call it prototyping. You analyze, then design a fake car using a chassis from another car, with a body made of wood or putty. You then show the resulting prototype to groups of people to get their reactions. When you think you know exactly what needs to be done, then you embark on the rest of the costly process.
The Product-Project Clash
Without going into the technical details of Agile methods, it’s important to understand that iterative development revolves around a product. The product is whatever you are developing or improving. There are two categories of such products, and the difference is very important to grasp.
Front-End Products
In the first category, a product is easy to picture because your organization is marketing it or it is one of the core services you provide. For example, a software shop that creates phone apps, or a publisher that provides a platform for viewing educational videos or a governmental tribunal that settles residential lease matters that offers a hearing scheduling platform. In these first-category product cases, especially when they generate revenue, the concept of a product is quite similar to any other products such as mortgage loans, life policies, airline tickets or dishwasher soaps. These products have brand names, target customers, cost structures, selling price structures, profitability, life cycles, etc.
Back-End Products
There’s also a second —and widely spread— category. More often than not, the product in an Agile context is something more obscure or hidden behind your organization’s main reason for being: an order entry application for a phone operator, a monitoring system for railway convoys, or an automated loan risk evaluator. These systems are trickier for a product owner (PO) to own. In the order entry example, that system may be used to sell dozens of ‘real’ phone products, which means the revenue stream becomes difficult to evaluate. Whether of the first or the second category, digital teams working with Agile methods consider them as products.
Product Management
There is a very important role in force across all organizations that use Agile ways of working, the Product Owner (PO) that confirms the centrality of the product. The product owner has very important responsibilities, mainly in providing guidance on priorities, making sure that requirements are well understood and that the product evolves according to the known strategies. A PO is supposed to be a permanent role, lasting the whole lifespan of the product. Products are imagined, designed, developed, launched, monitored, measured, maintained, modified and eventually replaced or retired. As long as there is a product, there should be a busy PO close by, pampering the baby. For the first category of products, that comes naturally.
When it comes to the less visible digital systems of the second category, it’s another story. The PO role doesn’t blend that naturally because the baby isn’t sold to real paying customers and is often not well understood because of its technical nature. The product becomes a tech component that digital teams know and maintain, and for which it’s difficult to find a PO outside of the technical ranks. For the second category of products, I’ve seen inexistent POs, acting POs, named but absent POs, POs with little time left for that role, POs with insufficient product knowledge, managers as POs, business analysts as POs and not enough knowledgeable POs with sufficient time to play such an important role.
“Real” products —the ones that you make revenue from— are easy to see as such. But all the rest are obscure internal or technical creations that are difficult to manage as products.
Measured Performance Drives the Show
The reason behind these role shortcomings is simple: non-IT business people do not see these systems as products, and technical teams are not geared for managing them as products. There’s a void. Most technical teams are part of a bigger, single organizational unit, usually headed by a CIO whose performance is never measured by how well these products are managed. Still, in 2022, delivering digital change endeavours on time and on budget are among the topmost expectations business executives have for their IT arm. Managing Agile’s products is not even on the list.
Projects Are Not Products
The preferred conduit for delivering technology-dependent change remains the project, and all IT departments are well geared to manage them. Since most organizations delegate all IT responsibilities to one functional unit, it becomes a project-oriented department, meaning that it is organized, staffed and prepared for delivering projects. A central aspect of projects is that they are temporary endeavours by definition. Anything that happens before or after a project is deemed out of the scope of that project. Projects are driven by project managers (PMs) who have in mind the temporary nature of the endeavour they are responsible for. POs, on the other hand, manage permanent assets. One thing is certain: project managers are not product managers.
Project-Driven Agile
Going Agile doesn’t stop your IT function from still being project-oriented. Using Agile methods has never replaced the existence of investments and projects. Agile or not, your organization has an investment cycle, supported by known processes, schedules, gates and committees. These processes have been put in place for good reasons, not the least being that any enterprise has limited financial and human resources. It must then choose what to invest in and carefully follow that investment through projects. However, as paradoxical as it may seem, good project management practices do not mean project success. In the specific case of corporate IT, it is a setup for failure.
Organized for Failure
Project orientation —with or without Agile methods— has perverse effects on organizations: many aspects of the good management of digital technologies are left behind because they deal with longer-than-project-term aspects. The consequences are disastrous. Project-oriented digital teams are mediocre at managing long-term performance. They are dunces at managing as assets the systems they created, since they outlive the projects themselves. They are not very good at understanding the tangible value that digital systems bring. Neither are they shining at controlling quality for criteria that could positively affect other projects or some future position. These shortcomings inevitably boomerang back to subsequent projects and products alike. They create extraneous complexity, more costs and more delays that make projects fail.
Radical Change Needed
We’ve just come full circle within a vicious cycle that creates complexities that, in turn, make projects fail. But how can this cycle be broken? The answer lies in the original, rock-bottom cause behind all this: the organizational model used to engage digital teams in your enterprise. The fact that all IT-related chores are pushed under a single umbrella creates conflicts of roles that are impossible to juggle. One has to make a choice between irreconcilable responsibilities such as delivering projects or owning and managing products. And when performance measures put at the top of the list being on time and on budget on project delivery, then project delivery wins. Field observations concur: short-term, project objectives always win.
Hence, the only way out is to completely rethink the way digital teams provide their services and who’s responsible for what. The possible areas of change vary, but one thing is sure: choosing the status quo and waiting to see will not work. If you’re interested on following the topic, subscribe to my newsletter for upcoming articles, books, conferences and educational events.