Did the Agile Manifesto make us Lazy?

Summary

All but a few pitch the Agile Manifesto as the greatest thing since sliced bread. Consultants and trainers earn large amounts of money by selling Agile as the solution for all development-related problems. To them, the Agile Manifesto is like the Bible.

In reality, many Agile projects take forever to complete or are abandoned after failing to deliver the desired results. I see companies rehiring the project managers and architects they fired a couple of years ago as part of implementing Agile. These companies went through the Peak of Inflated Expectations and are now crawling out of the Trough of Disillusionment.

Lets start with the good stuff.

Every new day is more unpredictable and dynamic than the day before. The higher the uncertainty of a market, the more the business and IT crave for flexibility and experimentation. In these situations, the business cannot articulate and fixate all requirements upfront, a prerequisite of traditional waterfall programming. At first sight, Agile enters the scene therefore as the knight in shining armor as it:

• puts the customer first,
• delivers rapidly (e.g. develop and deploy most valuable requirements first),
• strives to eliminate waste (e.g. no elaborated requirements analysis for a proof of concept),
• amplifies learning (e.g. customer feedback after every sprint demo),
• makes decisions as late as possible (e.g. adjust priorities backlog along the way),
• empowers the team (e.g. encourage instead of motivate),
• has built-in integrity in (e.g. soft controls, authority based on expertise not hierarchy), and
• sees the whole (e.g. work towards a shared business goal).

All good stuff, which makes Agile such an easy sell. However, it is not a silver bullet.

The bad stuff – no longer a means to an end

Several founding authors of the Agile Manifesto declared Agile dead. Not literally, but they expressed their frustration with the commoditization of their brainchild. Quoting Andrew Hunt:

” What was supposed to be an adaptive, collaborative, and customer-centric approach to development has instead become an over-hyped, buzzword-ridden exercise that delivers almost no value to the organizations that have embraced it.”

Dave Thomas, another founding author is even more pessimistic:

” The word “agile” has been subverted to the point where it is effectively meaningless, and what passes for an agile community seems to be largely an arena for consultants and vendors to hawk services and products.

….

Once the Manifesto became popular, the word agile became a magnet for anyone with points to espouse, hours to bill, or products to sell. It became a marketing term, coopted to improve sales in the same way that words such as eco and natural are.”

One can hire Agile Developers and Agile Testers, buy Agile Tools and become an Agile Certified Practitioner. Apart from the fact that agile is an adjective, not a noun, the Agile Manifesto is “a set of values based on trust and respect for each other and promoting organizational models based on people, collaboration, and building the types of organizational communities in which we [the 13 founding authors] would want to work.” It is not a set of strict rules or methodology. The founding authors expected us practitioners to be pragmatic and use our common sense.

But ‘mushy’ stuff is difficult to sell, let alone certify against. Trainers, consultants and tool vendors need easy to convey bullet lists in their PowerPoint presentations.

The bad stuff – It makes us lazy

At the start of the project, the business executive acting as the Product Owner asks the team: build a green car. Two weeks later, at the end of the sprint demo: “Hmmm, one second thought, paint it red”. The following week the business executive visits a potential client (result: paint it white!). Two months later, after visiting a conference with some colleagues: black is new white! Another twelve months pass and out comes a blue car with yellow dots and white stripes on the roof.

Ask critical questions and the typical answer will be along the lines of “Its Agile. Everybody is doing it this way.”

When the scrum team is funded by IT, considerable waste is almost inevitable as there is no incentive for the business nor the team to think things through. Not only feature-wise, but also regarding architecture, security and so on. Even when the business funds the project, wasteful practices can go on for a while, especially when senior executives don’t feel comfortable enough to call the bluff of the Theorists/Evangelists/Zealots.

Contrary to waterfall and Spiral Model, there is no explicit exit strategy in Agile. Agile loves changing requirements, even late in the development process. It is one of its strengths, but also a potentially expensive weakness. Left unmanaged (read: lack of executive oversight), the Product Owner and the rest of the team will make sure there is a filled backlog. There are always features, defects, performance issues, optimizations of the Ops team and tweaks.

Hence, proper governance, portfolio management, program management, performance management and even project management did not suddenly become obsolete with the introduction of Agile. Don’t let trainers and consultants fool you. When you have to go back to your CEO, COO, CFO for the fifth time to ask for more budget, you either have a really good story, or it is most likely time to continue your career somewhere else.

Unmanaged, Agile hides low business maturity (e.g. sloppy requirements) and low IT maturity (e.g. code quality, velocity).

The bad stuff – Power imbalance

Too much focus on early visibility and desires of the business inevitably results in two large and expensive buckets. The first is filled with technical debt and the another with defects.

Unless the team (and/or CIO/CTO) is stong enough to bring balance to the following two forces:

• Feature-focus of business. The payback period of a secure, modular and scalable architecture is much longer than some cool visual gimmick that catches the eye of potential customers. And what about the choice between solving a defect or adding a functional feature? Guess which one is most likely to make it to the top of the backlog when technology illiterate business representatives are driving the backlog? Too one-sided focus on visibility ensures that the attention span of the team is limited to the optics of the current and maybe next sprint demo. As a result, the expected life cycle of at least five years described in the business case approved by the COO and CIO never makes it to the backlog. When technology illiterate business executes are in charge,  technology debt and the defect backlog become orphans.
• Power-imbalance. This one acts as an multiplier for the first one. Agile pushes IT into the role of Faithful Servant, a positioning described by Van der Zee (I) as “Always helping, wherever possible, no questions asked.” Very different from a Business Partner, which is described as “Top quality, knows what we want, always one leap ahead.” The lower seniority of the developers and scrum master and more technology illiterate the business, the larger the risk (both probability and impact) the project will fail. Giving full authority to the business puts the developers in an underdog position. They are restricted to executing what the Product Owner tells them.

Security (e.g. OWASP 10), architecture, reliability, upgradability, performance and maintainability are neither easy to visualize nor atomize into two-week sprints. They are not part of the happy flow, nor can they be done piecemeal. The same applies to defects. While they can be stored in an ever growing defect backlog, eventually the business has to face the music.

The bad stuff – self organizing utopian  

 Quoting Moscar’s article Why Agile Isn’t Working: Bringing Common Sense to Agile Principles published on CIO.com: “You cannot replace accountable and responsible project management with some immature utopian myth of self-organization. That’s no science for success.”

Self-organization is a source of flexibility, productivity, creativity and even efficiency. It is also a very complex concept to translate from theory to practice. Furthermore, self-organization doesn’t mean that the team:

• doesn’t have managers or that they get to decide what problems to solve or what product to build.
• doesn’t have any tooling constraints, or architectural constraints, or budget constraints, or timing constraints.
• decides who is on the team, how big the team is, or how much money everyone on the team should make.

At least as important in my experience: commitment and the willingness to burn the midnight oil  are not enough to be effective as a team. The widely adopted Situation Leadership Model from Hersey and Blanchard explains that individual performance depends on both the willingness and ability to perform a task. Most of the challenged projects suffered from decisions made by individuals who were ‘unable but confident’.

Everybody has one or more blind spots: the willingness to take responsibility for a task, but unaware he or she lacks one or more of the required competences or skills. Think of:

•  business executives setting unrealistic expectations
• IT promising the business something without overseeing the consequences (e.g. resources, planning budget)
• unable to look beyond theory (e.g.  in reality, the business does not have unlimited budget)
• assuming there is no difference between soft skills required to manage a waterfall project or Agile project
• estimating project velocity after developing two simple features and testing the happy-path
• assuming a skyscraper can be build the same way as a one-story house
• ignoring an almost non-existent velocity in the first part of the project, assuming the team will magically make up the difference during the second half
• developing an application module in Java even though the rest of the application portfolio is based on C# (When asked why: “the team wanted to try something different”)

Hence, self-organization is a great goal to strive for, but it is dangerous to underestimate the impact of operationalizing this utopian concept.

Furthermore, most leaders and managers have work within the boundaries dictated by the skill sets, culture, experience and budget available to them. Coaching, training, 360 degree feedback, evaluation meetings and performance reviews can only achieve so much. Somebody used to twenty years waterfall development (being exactly told what to do) will have a very hard time to transition to an Agile approach. Coaching and training can make an introvert but highly experienced and capable developer a bit less introvert. He or she is however unlikely to ever become the life and soul of the party.

Effective self-organization requires leadership, time and common sense from all stakeholders involved.