Agile Won’t Be Killed

UncleBob wrote this good piece on whether Scrum could Kill Agile.

To me, Agile is a mindset, a state of mind, a way to be. It is neither a (set of) coding practice(s), a concrete process, or a management technique.

Agile is not TDD, XP, FDD, Crystal, Scrum, Lean, Kanban, Six Sigma.

Agile is about being responsive to the current and future needs of the business within the current context, so that you can do the very best possible given your team and budget. Read these again:

  • Individuals and interactions over processes and tools
  • Working software over comprehensive documentation
  • Customer collaboration over contract negotiation
  • Responding to change over following a plan

The “we value the left-hand items over the right-hand items” is precisely loose because it all depends on the context and the business needs. Believe me, we argued mightily over small wording changes in that Snowbird meeting! It’s just that in general, our collective experiences had born out the truths that we were more effective and responsive to delivering valuable software when we emphasized the left-hand items. I was coming from the government background, and I was determined to demand we push for truth and honesty in software. No more stupid 70% done and no working software to show for it!

I submit that the Agile Manifesto tenets are irrefutable ideas and will stand the test of time.

The Non-agile Manifesto

Let’s try a little negative test (if you agree to these, I’ll poke you with a hot stick):

  • We will build software with processes and tools — that way we don’t need to worry about people issues and effective communication.
    • That way we can hire any individual regardless of skill, and forgo all verbal/personal interactions in favor of solely written words. Even better if those written words are automatically transformed into code. Maybe we can get non-coder tools! After all, people are merely fungible assets/resources, and software is factory-work — with processes and tools, and a horde of low-paid serfs, we can crank it out!
  • We would rather spend exorbitant sums of money producing comprehensive documentation than have tangible, working results early on.
    • We start with complete requirements specifications (often 400 pages), that follow our company standard template.
    • Even our Use Cases follow a mandatory 3-level deep path, with proper exception and alternate paths worked out.
    • We link the requirements items into detailed design documents — which include system design diagrams, interface specifications, and detailed object models.
    • If we don’t write it all down now, we’re likely to forget what we wanted. And if we don’t do it to the n-th degree, the developers might screw it up.
    • Writing it all down up front allows us to go on vacation while the process and tools “write” the code from the detailed specs/diagrams. Sweet.
    • In addition, we love to be rewarded by reaching meaningless intermediate deadlines that we place on our 1500-node Gantt chart.
    • When we combine all of the upfront work with a important deadlines, many of the senior managers can get promoted due to their great commitment to generating reams of cool-looking documents. By the time the sh!t hits the fan when folks realize the “ship it” deadline is missed, the senior managers are no longer involved.
    • Besides, if we actually built software instead of writing all sorts of documents thinking about building software, our little ruse would be exposed! (BTW: if the human race had done nothing more than documented the ideas about how best to go about procreating, we wouldn’t be here to waste software budgets. w00t!)
  • We prefer to write a tight Statement of Work & force a strict fixed-price contract rather than actually work towards a common goal with the client.
    • The agreement is based on the 400-page, fully-specified requirements document, and we pad the cost estimate with a 400% profit margin.
    • We then hire dozens of people to argue during the Change Control Review Board monthly meetings about re-writing code to deliver what you wanted versus what you asked for when you thought you knew what you wanted (and wrote it down in that 400-page doc).
    • Contract negotiation pissing matches are such great uses of our collective resources and always result in perfect software! We love our fine print 🙂
    • With a 400% padding, the projects are too big to fail.
    • Once we are in it for 1 or 2 million and 50% done and 2x schedule overrun, who would ever say “No” to a contract extension? Who better to get you to the goal line than the same folks who squandered away your treasure, pissed away the calendar, and delivered no working software yet?
    • We like to appear like we’re just about done… Asymptote? Never heard of one.
  • There’s nothing more appealing than following a 1500-node Gantt chart. To the letter and subletter. Change sucks.
    • Especially a Gantt chart that has been built with tender loving care to include resource allocations, inter-project dependencies, and partial resource allocation assignments for matrix-style organizations.
    • We love hiring a small army to ensure that we drive the entire team to meet every micro-task deadline even when they no longer make any sense.
    • The real fun is collecting the “actuals” data from the developers assigned to each task.
    • And nothing sweeter than seeing 90% of our tasks being started, and 75% of those being 67% completed, and 25% of them being 99% completed — the roll-up summary to management is such a compelling story.
    • Changing such a beautiful plan that took 4 man-years to develop, that incorporates all of the comprehensive non-code documents, and is an appendix in the contract, is no small feat!
    • Better to produce the software according to plan even if nobody wants it that way. That’s our motto, and we’re not going to change!

Waterfall Sucks

Waterfall — as practiced — will die under it’s own weight. Waterfall is a bad idea when applied broadly to all sorts of classes of software projects. Eventually people will wake up and realize that — for many situations — there are better ways to doing software than waterfall. (Waterfall done on one feature at a time is a fine practice 🙂

I remember seeing folks who worked for Big Consulting Firm go into an organization to do all of the Use Cases for 9 months straight. Their documents were then handed off to some other firm to do the analysis and design. And finally, yet another firm would do the implementation based on the sum total of documents. Since the written word left nothing to the imagination, these projects always delivered perfectly. Oh, wait. Standish Group says that really isn’t how things worked out. This might be an okay way to go about hardware or construction projects — since you can be more deterministic up front, and there are more common parameters that are “agreeable” to most folks in terms of real interface documents (like engineering or architectural drawings are pretty specific and not ambiguous). But to anyone that has any experience in software development, waterfall seems ludicrous from Day One! Who would ever think that you could hand off all features as they went through each of the major phases with no loss of understanding? Who would ever think it is good to wait FOREVER to see any tangible results? After all, we’re NOT bending metal or breaking ground. It’s freaking software! Sigh.

Waterfall seemed ludicrous to me my very first time. I was exposed to Mil-STD-2167 while doing DOD contracting in the 80s. I think I followed along like a good little well-behaved junior engineer my first time. Probably even my second. Then my brain kicked in and said — this is stupid for our situation. We’re doing software development for R&D projects, we’re not doing assembly line work for a brick factory. We can’t know everything up front, and it is risky to think you can! So I started to look for better ways, and began to not do wasteful practices, even though they were prescribed. I began to do the right thing for the customer — and they agreed to the <ahem> “waivers.” (I was applying the “Better to beg forgiveness, than ask permission” principle I learned from my good friend and mentor, USMC Colonel Rich “Magic” Dinkel, fighter pilot extraordinaire.)

I sound like I am bashing people that did Waterfall… I don’t want to be that harsh. People did what they thought was best at the time. I think many times we fall into a trap of wishful thinking. And even in the face of overwhelming observations to the contrary, we soldier on in hopes things will work out like we want. (Besides, it’s in the contract!) Rarely is hope ever a good strategy. Maybe never? So folks doing waterfall certainly had good intentions, and it was compartmentalized enough to reward intermediate victories and give an illusion of progress, but there’s no denying it’s just a bad idea.

Agile didn’t kill Waterfall per se. Sure, maybe it helped accelerate the community’s realization that there were better ways to do software than a giant, rigid, 4-stage assembly line.

Waterfall killed waterfall.

Scrum Won’t Kill Agile

I say let the Scrum Fun continue! Make tons of money! Waste as much budget as you can. Brag that you are doing Scrum and have hired 20 CSMs to whip your developers into shape!

The more people and HR departments that think there is some magic bullet, the more companies that might fail and lose business. This makes way for smarter, leaner, more entrepreneurial start-ups to eat their lunch. After all, I would wager that very few small organizations are falling for the Scrum Will Save The Day stupidity that is seemingly gripping some (large?) portion of our community.

Convenient Cover?

Here are a few phrases that come to mind regarding the Scrum dustup, and I’ll add a new one…

  • A friend once quipped that “Off-shoring is just a cheaper way to fail” — implying that if you can’t build software locally, you probably can’t build it 12 time zones away either. But at least the costs are lower.
  • No one was ever fired for hiring IBM
  • We are doing Scrum — all the rage in Management By Magazine circles!

In other words, it is often far easier to go with the flow, follow the herd, live by the plan, and say you did what you were told. It is much harder to buck the system and call out the ineffective and wasteful activities, and continuously try to improve. It’s hard to buck inertia!

Conclusion

Nothing will ever kill agile.

Bold, I know. However, I view it like the USA’s founding principles — immutable laws of freedom. Agile is about freedom to do the right thing.

How can you ever deny the correct way to do software is to do what’s best for the business?

How can you deny the Agile Manifesto tenets?!

Successful software will continue to be developed in an agile manner, regardless of the terms that might become popularized.

13 thoughts on “Agile Won’t Be Killed

  1. Pingback: Tweets that mention Behold: Agile Won't Be killed -- Topsy.com

  2. Naresh Jain

    Great post Jon. You have nailed it. I’m surprised why we (including myself) spend so much valuable time complaining about Scrum or anything else for that matter. Life is too short for that.

    To your question:

    “How can you ever deny the correct way to do software is to do what’s best for the business?”

    Do we know the correct way to build software? Is there one correct way? Is Agile really the answer?

    I don’t have all the answers. I ponder on these questions many times, esp. when I find counter-intuitive things “magically” work.

    For Ex: in many aspects “Apple” is not Agile in the way the manifesto defines. Yet, I love their products and their have changed the way I interact with devices.

  3. jon Post author

    @Naresh, agile is an attitude that helps guide your choices when deciding how to build software. So yes, from that perspective, I submit it is *the* one correct way. Agile is in the eye of the beholder as well. Apple may have found the most agile way for them to build devices and software that are all obviously Apple products and loved by consumers. And I can guarantee it isn’t a Waterfall process.

  4. Pingback: Great Entry – well worth the read > Agile Won’t Be Killed « Technical Debt | My Thoughts Enclosed…

  5. Pawel Brodzinski

    I don’t agree with waterfall part. Waterfall solves a specific set of well-defined problems. Waterfall enforces a lot of planning which, in some projects, is a good thing. Also, no matter how suboptimal linear process can be in terms of building software, there are clients who expect (and pay for) software to be built this way.

    Then we tend to put waterfall label on every chaotic process we can think of, which is wrong. The bad process is just that – the bad process. It doesn’t matter if it is waterfallish or agilish.

    While, as you know, I work with agile I’m far from simplistic view of world: waterfall = bad, agile = good. Not every waterfall project requires 1500-node Gantt chart. Not everyone requires extensive documentation. And to be honest I’ve never heard that tools or documentations are more important than people. This approach is not a feature of process, but a feature of company culture. You can perfectly have Scrum implemented in a company which doesn’t care about people at all. And yes I know you’d call it bad agile.

    Anyway, agile values are healthy so they will survive. At some point in the future the next big thing is going to come and we may start using agile as a bad example then. It may happen not because agile values are bad (they are not), but because most of agile implementations will be bad at some point in the future (which is I guess rather safe prediction).

    By the way: since agile became mainstream a few years ago it hurts from the same sicknesses formal methods do – businesses built over certifications, method-in-the-name-only usage etc. And that’s exactly why it happens, and will happen, from time to time that we hear about the end of agile.

    However, eventually we will learn which methods to use in which situations. There will be place for waterfall, agile, the next big thing and another big thing which comes next. At least that’s how I look at the whole thing at the moment.

  6. Pingback: Recommended Nothing Sweeter Than

  7. jon Post author

    @Pawel, regarding waterfall, I specifically left an “out” by referring to being applied broadly to all sorts of classes of s/w projects. There are some where it might be okay — though I bet you can still do better with a more agile approach. And, yes, I am being very harsh on waterfall, for effect.

    I think it is hard to defend doing “all” requirements at once, all design and analysis at once, all implementation at once, all testing at once, and then *finally* show it to the end user.

    I think waterfall remains in force because it is easier than agile. I have said all along, that AGILE IS HARD. It requires greater courage to accept the reality that software is not always so perfectly predictable. Waterfall is much easier to work with for so many parts of the organization. The customer only has to participate at the beginning and then again at the end. We can use an elite design team to spec things out. Then we can completely wash our hands of the implementation by shipping it offshore.

    Contrast that with requiring the customer or business person to pay continuous partial attention to the silly development team. That we might change our minds about the requirements that we were so sure of just one month ago, once we see the system working.

    Anyway, I challenge folks doing waterfall to find the inefficiencies and waste in their process and go more agile. Even a little bit at a time would be better.

    And, as I have said many times in public, waterfall works great — in one or two week doses 🙂

  8. Pawel Brodzinski

    I’m not going to say that waterfall is great or universal. After all if you had a big, multi-year project with, say, half-year long iterations would you consider it more waterfall or more agile or what?

    And then, you’re completely right that scaling down linear process to small, two-week long doses works good (although I’d be far from stating it’s enough to move to agile).

    The world isn’t black and white – it’s gray. We have a lot of mixes between canonical waterfall (whole project in one iteration) and canonical agile approaches (two-week long iterations and client engaged once every two weeks at worst). As long as these mixes are done consciously I’m fine with that.

    That’s the basic reason why I don’t agree with such radical approach as yours, even though we agree on principles.

  9. Sylvain Hellegouarch

    Scrum won’t kill Agile per se. You shouldn’t have read it literally either. What could happen is that doing Scrum becomes a synonym of being agile. This is what could ultimately kill off folks that have clearly understood better.

    We may end up with the same mess as Open Source opposite Free Software. The former is all about being pragamatic, the latter is about the true philosophy. Both can co-exist nicely but sometimes it’s nice to remind people that they aren’t the same for deep reasons.

  10. Warren

    It seems to me that Agile, the name, will inevitably be freighted with the negative form of all the positive things you have stated. If I read you correctly, and I think I have, the death of Agile will happen when most people realize that it really means, among people far lazier and less brilliant than you will turn it into. I am not saying that you will kill it. You would save it, if you could.

    But the people who will kill agile, will kill it by using Agile as their excuse for:

    (1) never having to produce any documentation at all.

    (2) never having to produce a product at all, but rather working towards it slowly, incrementally, and perhaps, asymptotically.

    (3) having a clever answer for everything, and who think of technical debt more in terms of “jobs for life” than anything else.

    In short, enough ships will sink that claimed to be flying the Agile flag, that the name will tarnish. And the clever people will pick a new name for it, and move on.

    Warren

  11. jon Post author

    @warren,
    Oooh, I like this : “having a clever answer for everything, and who think of technical debt more in terms of “jobs for life” than anything else”

    My hopes is that competition drives these sorts out of a job. But I have also met the one or two fellas along the way that are clever and able to forever fool management. I even tried firing one of them for cause. I ended up leaving that company to start my own consulting firm.

    But, new name or not, I still think the concepts of agile practices will survive and continue to be used in companies that wish to be as effective as they can be.

  12. Pingback: Agile Startups | Coldstreams Technology, Innovation & Social Media

Leave a Reply