Why Agile doesn't work.

I hide my agile expertise in order not to be excluded from jobs...

This is not another post about how “Agile is dead”; plenty of those already exist.

This is a post about why Agile often suffers a poor reputation, especially in the enterprise.

Don’t get me wrong, executives love the word “agile” they don’t stop saying it. They all loudly proclaim the use of the latest “agile methodologies” in their software development organizations.

So why is the word forbidden at the fashion retail hereditary oligarchy by which I was once employed?

That’s a longer story I’ll maybe share in another post.

But the gist of it is power structures.

I’m not going to define agile in this post; feel free to check out the Agile manifesto to see what 17 white male programmers had in mind when they gathered at a ski resort to discuss improving the software development business.

The most popular framework in agile, by far, is Scrum. Read The Scrum Guide to learn what the Scrum Alliance thinks Scrum is. You’ll be doing better than 90% of certified Scrum coaches if you read it. Most people have never read the Scrum Guide.

If you want to be even further ahead of the curve, read the paper that initially coined the term and codifies the concept of Scrum as an approach to working by reading The New New Product Development Game written by Nonaka and Takeuchi in 1986.

Of course, thought leaders be thought leaderin’ so good luck getting anyone to attribute anything to these predecessors. It’s just “Ken and Jeff invented Scrum!” and “David Andersen invented Kanban!” (not Toyota, or Ohno…. anyway, more fun consulting culture)

Ultimately, the issue isn’t a framework, certification scheme, or process.

The Agile movement is no different from the attempts at improvement that preceded it. W. Edwards Deming had Total Quality Management; Lean was bastardized by “Six Sigma,” I’m sure Taylor’s “scientific management” suffered from its detractors in his day.

So, without getting into the details of Scrum or Kanban or, god help me, “The Scaled Agile Framework for the Enterprise,” why do these frameworks and approaches fail so frequently?

The answer is our existing power structures maximize control while minimizing coherence. Executives don’t want clarity of purpose because they cannot maneuver politically as easily as they do while they lead through fear and fiat.

Scrum, Lean, and Kanban all have essential elements of team-level autonomy. Actual Lean focuses on maximizing value to customers, but leaders want to use lean to eliminate waste (i.e., jobs). The single biggest source of waste in the enterprise is its executives. Yet, somehow, there’s very little interest by the people signing the checks to make their leaders (i.e., themselves) effective.

Ultimately, when you start with the essentials of these approaches, you end up with the following:

• Transparency

• Autonomy

• Strategic Clarity

None of these things are good for executives.

Transparency, Autonomy, and Strategic Clarity

I’ve done a lot of work trying to help product organizations clarify their corporate strategies and prioritize their limited operational capacity to maximize the flow of value to their customers. The main problem with Don Reinertsen’s approach to quantifying the cost of delay is that many executives do not want those things quantified.

Why wouldn’t an executive want to know the numbers behind the returns they see on their investments?

If the prioritization approach is transparent and coherent, the rank-and-file has enough information to question the strategic direction or suggest alternative approaches to that end goal. This isn't good for the fiat leader because they don’t make decisions based on the highest value to the customer (and return to the business) in the shortest time. They frequently change priorities across the board because of some political reason. Some other more powerful executive said, “Do it this way and get this to me now, or you’re fired.” at least the system of fear in place makes people fear for their jobs if they’re seen to disagree with the ‘little kings’ at the top.

Suddenly, your autonomous delivery team can say, “Wait, but if we switch to working on that, we’ll lose this much money on delaying these more important, higher-value initiatives!”

But “I have to impress my imbecile CTO, so we all have to shift to this approach” doesn’t play well with the rank and file.

Or “The board of directors handed down this edict that we must do this thing first, thus delaying something with considerably higher return” doesn’t play well.

I had one director say the quiet part out loud when critiquing the strategic clarity approach, he said: “Sometimes the VP promises something to another VP, and even if that thing isn’t the optimal thing to work on or delays something very critical, it’s important we do not require a VP to explain himself to anyone.” because autonomy isn’t important, the VPs political gains trump all priorities.

The tech leaders I’ve encountered at many enterprise organizations market themselves as technocrats. They’re not claiming to be primarily business experts; they’re claiming to be super technical programmers who can implement changes to the technology stacks that will introduce efficiencies or technical improvements. This is unfortunate because most actually great engineers don’t pursue roles in leadership, so you often end up with someone who’s just a profoundly arrogant sociopath and who can always fall back to “You’re not as knowledgable as I am; otherwise, you’d understand.” For some reason, we’ve stopped believing that people should be able to convey their ideas.

Ultimately, like TQM and Lean, many of these ideas work. Breaking down risk into smaller batches and prioritizing the highest risks first works. Reducing the amount of overhead, or work in progress, speeds delivery. Creating more autonomous cooperative power structures unlocks the creativity and passion in rank-and-file workers in a meaningful way while allowing for coherent, meaningful governance.

The hereditary oligarchy fashion retailer had all these great performative values.

”Act like an owner.”

“Inverted pyramid structure”

”Use your best judgment”

But in practice, they were the most authoritarian, highly political regime full of abusive psychopaths I’ve encountered.

“Act like an owner,” but we will govern every tiny decision, and we will overrule any dissenting views. We’ll fire anyone we disagree with “without cause” simply for being disagreeable.

The fact of the matter is that organizations will always need consultants because the optics of bringing in consultants has utility to the leaders. They get to show that they’re investing in change. But if any actual change rears its ugly head, they will crush it. Many say “consultants are just ripping the company off because they don’t end up changing anything and just bill.”

Well, as a consultant who was constantly at risk of losing my job, and in the US that means my family loses their health insurance, I can tell you the message I got loud and clear.

”If you change things, you’ll be angrily fired “without cause.” If you gladhand and compliment the executives for their genius, you’ll be quite well compensated.

Until we change the despotic power structure of the American corporate enterprise, we’ll never see any meaningful change happen. Until laborers have the power to protect themselves from rampant abuses from leadership, we’ll continue to see all this dysfunction.

I recently saw a consulting “thought leader” on a podcast saying, “If only we could lock these execs in a room and show them how much overloading their organization was costing them in terms of an inability to get anything done, we’d see change happen” <paraphrases>

I’ve done this long enough to know the two responses to that approach.

1. I don’t get it!
2. You’re fired!

Well, they’re usually phrased in the following way:

1. That’s too academic to work here; we live in the real world.

2. I want to thank you for all your excellent work here and tell you how excited I am to work together again in the future.

I have more to say about this subject but don’t want to write an entire book on Substack.

Feel free to tell me in the comments how wrong I am and all the stories of wonderful egalitarian enterprise technology executives you’ve worked with and enjoyed. :-)