Agile Won’t Be Killed

Agile is about the freedom to do the right thing.

UncleBob wrote this good piece on whether Scrum could Kill Agile.

To me, Agile is a mindset, a state of mind, a way to be. It is neither a (set of) coding practice(s), a concrete process, or a management technique.

Agile is not TDD, XP, FDD, Crystal, Scrum, Lean, Kanban, Six Sigma.

Agile is about being responsive to the current and future needs of the business within the current context, so that you can do the very best possible given your team and budget. Read these again:

  • Individuals and interactions over processes and tools
  • Working software over comprehensive documentation
  • Customer collaboration over contract negotiation
  • Responding to change over following a plan

The “we value the left-hand items over the right-hand items” is precisely loose because it all depends on the context and the business needs. Believe me, we argued mightily over small wording changes in that Snowbird meeting! It’s just that in general, our collective experiences had born out the truths that we were more effective and responsive to delivering valuable software when we emphasized the left-hand items. I was coming from the government background, and I was determined to demand we push for truth and honesty in software. No more stupid 70% done and no working software to show for it!

I submit that the Agile Manifesto tenets are irrefutable ideas and will stand the test of time.

The Non-agile Manifesto

Let’s try a little negative test (if you agree to these, I’ll poke you with a hot stick):

  • We will build software with processes and tools — that way we don’t need to worry about people issues and effective communication.
    • That way we can hire any individual regardless of skill, and forgo all verbal/personal interactions in favor of solely written words. Even better if those written words are automatically transformed into code. Maybe we can get non-coder tools! After all, people are merely fungible assets/resources, and software is factory-work — with processes and tools, and a horde of low-paid serfs, we can crank it out!
  • We would rather spend exorbitant sums of money producing comprehensive documentation than have tangible, working results early on.
    • We start with complete requirements specifications (often 400 pages), that follow our company standard template.
    • Even our Use Cases follow a mandatory 3-level deep path, with proper exception and alternate paths worked out.
    • We link the requirements items into detailed design documents — which include system design diagrams, interface specifications, and detailed object models.
    • If we don’t write it all down now, we’re likely to forget what we wanted. And if we don’t do it to the n-th degree, the developers might screw it up.
    • Writing it all down up front allows us to go on vacation while the process and tools “write” the code from the detailed specs/diagrams. Sweet.
    • In addition, we love to be rewarded by reaching meaningless intermediate deadlines that we place on our 1500-node Gantt chart.
    • When we combine all of the upfront work with a important deadlines, many of the senior managers can get promoted due to their great commitment to generating reams of cool-looking documents. By the time the sh!t hits the fan when folks realize the “ship it” deadline is missed, the senior managers are no longer involved.
    • Besides, if we actually built software instead of writing all sorts of documents thinking about building software, our little ruse would be exposed! (BTW: if the human race had done nothing more than documented the ideas about how best to go about procreating, we wouldn’t be here to waste software budgets. w00t!)
  • We prefer to write a tight Statement of Work & force a strict fixed-price contract rather than actually work towards a common goal with the client.
    • The agreement is based on the 400-page, fully-specified requirements document, and we pad the cost estimate with a 400% profit margin.
    • We then hire dozens of people to argue during the Change Control Review Board monthly meetings about re-writing code to deliver what you wanted versus what you asked for when you thought you knew what you wanted (and wrote it down in that 400-page doc).
    • Contract negotiation pissing matches are such great uses of our collective resources and always result in perfect software! We love our fine print ๐Ÿ™‚
    • With a 400% padding, the projects are too big to fail.
    • Once we are in it for 1 or 2 million and 50% done and 2x schedule overrun, who would ever say “No” to a contract extension? Who better to get you to the goal line than the same folks who squandered away your treasure, pissed away the calendar, and delivered no working software yet?
    • We like to appear like we’re just about done… Asymptote? Never heard of one.
  • There’s nothing more appealing than following a 1500-node Gantt chart. To the letter and subletter. Change sucks.
    • Especially a Gantt chart that has been built with tender loving care to include resource allocations, inter-project dependencies, and partial resource allocation assignments for matrix-style organizations.
    • We love hiring a small army to ensure that we drive the entire team to meet every micro-task deadline even when they no longer make any sense.
    • The real fun is collecting the “actuals” data from the developers assigned to each task.
    • And nothing sweeter than seeing 90% of our tasks being started, and 75% of those being 67% completed, and 25% of them being 99% completed — the roll-up summary to management is such a compelling story.
    • Changing such a beautiful plan that took 4 man-years to develop, that incorporates all of the comprehensive non-code documents, and is an appendix in the contract, is no small feat!
    • Better to produce the software according to plan even if nobody wants it that way. That’s our motto, and we’re not going to change!

Waterfall Sucks

Waterfall — as practiced — will die under it’s own weight. Waterfall is a bad idea when applied broadly to all sorts of classes of software projects. Eventually people will wake up and realize that — for many situations — there are better ways to doing software than waterfall. (Waterfall done on one feature at a time is a fine practice ๐Ÿ™‚

I remember seeing folks who worked for Big Consulting Firm go into an organization to do all of the Use Cases for 9 months straight. Their documents were then handed off to some other firm to do the analysis and design. And finally, yet another firm would do the implementation based on the sum total of documents. Since the written word left nothing to the imagination, these projects always delivered perfectly. Oh, wait. Standish Group says that really isn’t how things worked out. This might be an okay way to go about hardware or construction projects — since you can be more deterministic up front, and there are more common parameters that are “agreeable” to most folks in terms of real interface documents (like engineering or architectural drawings are pretty specific and not ambiguous). But to anyone that has any experience in software development, waterfall seems ludicrous from Day One! Who would ever think that you could hand off all features as they went through each of the major phases with no loss of understanding? Who would ever think it is good to wait FOREVER to see any tangible results? After all, we’re NOT bending metal or breaking ground. It’s freaking software! Sigh.

Waterfall seemed ludicrous to me my very first time. I was exposed to Mil-STD-2167 while doing DOD contracting in the 80s. I think I followed along like a good little well-behaved junior engineer my first time. Probably even my second. Then my brain kicked in and said — this is stupid for our situation. We’re doing software development for R&D projects, we’re not doing assembly line work for a brick factory. We can’t know everything up front, and it is risky to think you can! So I started to look for better ways, and began to not do wasteful practices, even though they were prescribed. I began to do the right thing for the customer — and they agreed to the <ahem> “waivers.” (I was applying the “Better to beg forgiveness, than ask permission” principle I learned from my good friend and mentor, USMC Colonel Rich “Magic” Dinkel, fighter pilot extraordinaire.)

I sound like I am bashing people that did Waterfall… I don’t want to be that harsh. People did what they thought was best at the time. I think many times we fall into a trap of wishful thinking. And even in the face of overwhelming observations to the contrary, we soldier on in hopes things will work out like we want. (Besides, it’s in the contract!) Rarely is hope ever a good strategy. Maybe never? So folks doing waterfall certainly had good intentions, and it was compartmentalized enough to reward intermediate victories and give an illusion of progress, but there’s no denying it’s just a bad idea.

Agile didn’t kill Waterfall per se. Sure, maybe it helped accelerate the community’s realization that there were better ways to do software than a giant, rigid, 4-stage assembly line.

Waterfall killed waterfall.

Scrum Won’t Kill Agile

I say let the Scrum Fun continue! Make tons of money! Waste as much budget as you can. Brag that you are doing Scrum and have hired 20 CSMs to whip your developers into shape!

The more people and HR departments that think there is some magic bullet, the more companies that might fail and lose business. This makes way for smarter, leaner, more entrepreneurial start-ups to eat their lunch. After all, I would wager that very few small organizations are falling for the Scrum Will Save The Day stupidity that is seemingly gripping some (large?) portion of our community.

Convenient Cover?

Here are a few phrases that come to mind regarding the Scrum dustup, and I’ll add a new one…

  • A friend once quipped that “Off-shoring is just a cheaper way to fail” — implying that if you can’t build software locally, you probably can’t build it 12 time zones away either. But at least the costs are lower.
  • No one was ever fired for hiring IBM
  • We are doing Scrum — all the rage in Management By Magazine circles!

In other words, it is often far easier to go with the flow, follow the herd, live by the plan, and say you did what you were told. It is much harder to buck the system and call out the ineffective and wasteful activities, and continuously try to improve. It’s hard to buck inertia!


Nothing will ever kill agile.

Bold, I know. However, I view it like the USA’s founding principles — immutable laws of freedom. Agile is about freedom to do the right thing.

How can you ever deny the correct way to do software is to do what’s best for the business?

How can you deny the Agile Manifesto tenets?!

Successful software will continue to be developed in an agile manner, regardless of the terms that might become popularized.

Plan & Estimate, Only if You Must

It must have been the morning to challenge planning efforts, as GeePawHill did here. I was answering a post that had gotten around to discussing budgeting and planning for interruptions to fix legacy apps while teams are building new stuff.

When all of the sudden, I blurted this out:

Kanban is simple, in theory. Simply do one feature or bug fix all the way to being released. AKA focus. But there is a lot of things that have to be working well, for example: the cost of testing has to be low so that it doesn’t cost a fortune in treasure or time to get in that new feature or bug fix. (Expensive testing often leads to bundling a bunch of stuff into a given release and widening the time window, and introducing parallel development techniques.)

And, regarding budgeting in to prepare for the unknown maintenance efforts… That’s but one way to handle things. Another way is simply let the people who care about getting things done participate in setting your priorities. If you were given 10 things to do in a month, and halfway through you get an 11th thing to do. You probably don’t need to budget for that unknown. You simply ask for help deciding which of the original 10 should get axed to make room for the new “emergency” item.

Like it or not, you are all in it together, it is a zero sum game. If you fix the resources, fix the schedule, the amount of work that gets done has to give when new work is injected. Deal with it in an honest manner.

Hard to describe in words… and had to know your exact context… but what you are facing is probably more of an organizational issue than anything else.

All we software folks do for a living is produce features and fix bugs to keep paying customers happy. Just keep trying to improve your team’s efficiencies and squeeze out wasteful practices.

When you are producing software as an internal organization, you can sometimes be brave and forgo lots of planning stuff. If you have the priorities set, and if you have a delivery schedule, and if you understand your costs of releasing, you can make a pretty simple and smooth process. I’ve done it many times in companies formerly steeped in waterfall fantasy land. Why waste all sorts of time making estimates if you don’t have to?

The answers to these questions can help you decide how much ceremony your processes need:

  • Does it matter if the team does 5 things in an iteration or does 10?
  • Does it matter if they estimated 10, but only wound up being able to do 5?
  • Or, if they bid 5 and found they could do more and gobbled up 10, is that a problem?
  • Is it more important to hit a date with at least some features?
  • Or, is it more important to produce specific features as fast as you can, even if you miss the date?
  • Does the team need to be accurate on their estimates? How accurate and why?

Many times, we fall into traps of wanting to manage software process as if it were a brick-making factory. Unless you live or die by doing software as a fixed-bid project, you can often really get simple and make software really fun in your organization by doing less ceremonial planning crap!

Getting Started w/ Best Practices & Design Principles

Praveen had a seemingly innocent question here and there were lots of good answers…

If we are following Agile Methodology (Scrum) & We are in Java/J2ee Domain,

What are the Best Practises & Design Principles need to be followed
while designing the applications.

Big question is an understatement… here’s what I would advise:

  • Understand the business domain
    • because, if you get that wrong, it doesn’t much matter how well-crafted and well-tested your code is
    • Design using Separation of Concerns as the granddaddy of all patterns to sear into your brain.
    • Be sure you use a layered approach to your architecture — if I catch you putting SQL in your javascript, I’ll come over there and stab you in the toe
    • Design in the Large to avoid excessive dependencies
    • Design in the small to adhere to all those good practices like SOLID
    • Consider looking into the Domain Neutral Component — within which a shockingly huge number of domains tend to fit.
  • Architect and build the application in a highly consistent manner — much like Rails enforces.
    • If you have a dozen domain items that need the same sort of treatment from persistence to UI, please don’t get creative and do persistence 5 different ways.
    • Pick one and stick with it for the sake of consistency!
    • I’d rather a consistent implementation that is mediocre, than one with 5 different approaches (one stellar, one horrid, and the rest mediocre). Why you might ask? Because it is easier to refactor when things are consistent ๐Ÿ™‚
  • And, of course, apply an agile process…
    • Involve a customer/proxy to ensure you understand the requirements from the business perspective, do not rely on documents — but if you must, then you need to exaggerate the agile process and make it more granular. That is, to compensate for the likelihood that the written word is ambiguous, you need to show small steps of progress on a feature and get rapid feedback to ensure you are not mis-interpreting the requirements doc.
    • Build a vertical slice of your design to prove it out and to serve as a model for how you will work on your app
    • If you have stringent performance needs, prove them out early
    • Show working code early and often
    • Get yourselves set up on an automated build
    • Try to build out the features using something like BDD or TDD
  • General
    • Be patient with your learning
    • Be impatient to see results, don’t let any team member or task go dark for more than a couple of days
    • Create the lightest-weight process possible
    • Don’t do anything to excess without frequent feedback
    • Always foster a team attitude to question everything with boldness — including my advice

Hope that helps

Ideas for a Distributed Team

In response to a posting on Agile Project Management…

I am starting up a ‘Distributed’ Agile Project and looking at tools to assist the distributed team communicate the status of work undertaken, stories, estimates and burn down/up charts as well as general collaboration (forums, document sharing etc).
There are tools such as Agile Wall, Mingle, Trac (with plugins etc).
I am ideally looking for an externally hosted solution (something that we will not be setting up in-house).
I have a core team of 15 people and with an additional 10 people that would want access. I am looking for any ideas or suggestions on online tools that you may have.
Thanks in Advance.

I’ve been doing this sort of thing since 2000, so it is very easy and very possible to be extremely effective as a distributed team.

Here is a very simple approach using Basecamp (and you can get an add-on for burndowns if you like that sort of thing). I’ve been doing this on a small project since March or so.

Then I set up other simple collaboration tools (looked like this on many projects, though you might have to march around that hodgepodge of a site to find more tool ideas):

  • Wiki: Project stuff that is not issues or code needs a home; a.k.a.,
    (you can use basecamp, though not as extensive as something like confluence)
  • Chat Room for being the virtual cubicle farm and water cooler. Skype or IRC works great. You can establish key words that call out for attention, like “@ALL I’m going to kick the TEST server.” You can even have textual scrums if voice is not always possible. For a while, we had a bot that recorded the chats and logged them to the wiki ๐Ÿ™‚
  • Issues/Bugs: You need a place to store your lists of things to get done. Some call that an issue tracker; others, simply a To-Do list. I have used Jira for a l-o-n-g time for both features and bugs. I have also used Lighthouse — which is just fine too.

Atlassian Studio is worth a look. You can add in continuous integration and code reviews and what not.

My suggestion: start really small and humble with the tiniest process.

That really simple Basecamp process I blogged about? We actually got to PRODUCTION with much less ceremony. There were none of the “To be QAd” or “To Be Released” because we did it more in iteration chunks, so we knew the overall state of a given iteration (in dev, qa, or released).

The slightly greater ceremony of QA and Release buckets allowed me to answer the need from the stakeholders and other non-developers to know the state of the project at a glance. It’s easy to see what is in the backlog, or what is in progress, or waiting QA, etc. Now we’re doing Kanban. I crank issues and (rare ๐Ÿ™‚ bugs through one at a time usually.

Anyway, have fun!

Response to “Three Things I Don’t Like in Agile Community”

Andy wrote some interesting observations in his “Three Things…” post:

1. Dogmatism
Agile is about adaptive, creative approach to complex work yet amazingly average agilists are the most dogmatic people I know.

Part of dogmatism is also treating agile […] as the all-encompassing solution to problems not only in software development, not only in IT, but everywhere. [This] is a far cry from healthy pragmatism that I think is at the core of all good management.

I see this is a common thread among those who have issues with Agile. Naturally, if someone is presenting a methodology at a conference, I can see where it might sound dogmatic. However, if an Agile Consultant is engaged with a team and preaches “Do it My Way or Else” in spite of vehement team opposition, well, that strikes me as non-agile.

For me, agile has always been more about the proper state of mind than dogma. We need to prefer to do those things that are best for the business/customer goals over doing things that are not as helpful at getting the project closer to “done.” Sure, there are practices associated with agile methods, but simply doing those practices don’t make someone agile per se.

No, what makes someone agile is a combination of what they are doing, how they are doing it, and all within the cultural and business context in which things are being done.

For example, you just can’t walk into an elderly long-term care facility and do yoga like you would with 20-30 year-olds. (Well, you could, but that would be stupid.) Similarly, you can’t simply ram your favorite XP tools and practices and Magic Scrumbans into a steeped-in-old-ways, very stodgy organization full of legacy software written in a 30 year-old, no-longer-supported language. Well, you could, but that would be stupid.

So, Andy, if you spot one of us “Agilistas” spouting off a dogma, smack us upside the head and say “Work with me, son!”

2. Sectarianism

Kanban, Scrum, XP โ€“ everyone follows their own method, and basically says others are useless or at least not as good. […] Again, this is in the face of core principles of agile.

Hmmmm. The only good method I ever practiced was the one I was doing at the time it was working right. While I can understand someone who is training in a strict methodology being sectarian, that is not the same as consulting. I attempt to be a generalizing specialist (as Scott Ambler likes to refer to it…) and use a blend of tools, practices, modalities, and therapies.

Andy, if you find yourself being preached to by a consultant (not a trainer) that is one-dimensional, take it for what it is worth. A single, focused, dimension — nothing more.

3. Domination by consultants Most if not all agilists that write, teach and coach do only this, and have not run a software project (or a business) hands on for quite a while.

Well, the lack of real-world experience should be obvious by perusing their background. While trainers can be good at getting across certain practices and methodologies, and while coaching dozens of teams provides valuable exposure, it does not substitute for actual production experience. I learned a great deal traipsing around dozens of companies to help train and consult to kick-start projects — lots of cross-pollination potential as well! Similarly, consultants with great experience at building real world systems and helping teams do that, may not be the best down-and-dirty, nitty-gritty trainer for a given technique.

It’s a trade-off. First off, don’t confuse training on a skill, with consulting to help achieve a business goal. I think you have to understand what the purpose of the consulting engagement is, and who was brought in for that task. If you just want to learn the mechanics and lingo of Scrum, it takes only two days — it’s that simple. But tell any Project Manager with real world software project experience, and they’ll tell you that you don’t know much after just a few days of training.

So, if you ran into an agilista trainer that has not been on a real world project to deliver production code in a long while — or ever, then you should not expect that sort of knowledge. It doesn’t mean they cannot provide you with valuable training.

However, I will also say that not much is new under the sun in software development in the past 30 years. I mean, seriously. The same core issues exist in trying to roll out a piece of software to production as did 30 years ago. Admittedly, I don’t recall having quite as much fun as I do now with delivering production code with Ruby, Rails, Cucumber, RSpec, github, Mongo, gems… The ecosystem makes it so much easier to “do the right thing” with BDD/TDD for example.

Andy, I can appreciate your observations about the “Agile Community” — like so many others I have read on the web. To that end, I suppose we need to re-double our efforts at re-stating that Agile is a State of Mind, not a set of dogmatic practices.

And, in the end, Andy, caveat emptor. The Agile mantra should be one of empowerment and individual freedom to please the customer in the best way possible. You are ultimately responsible for doing the right thing. Agile ideas, processes, tools, trainers and consultants are just that — and nothing more. We are not the answer, we just try to help. In the end, the team is the answer.

Responding to “What’s Next for Agile”

In this post, there are some scary assumptions. I wonder where they come from?

  • “But where does [the Manifesto] say that the customer gets what he wants?”
  • “Mary talked about the next inflection point in agile โ€“ getting back the customer focus.
  • “‘Agile’ until now was about the processes and tools that improve software development.”
  • “[We] now have to align with the business goals of the organization –ย  provide what the customers need.”

Everyone is entitled to their opinion. But this is sooooooooooooo far away from what Agile is all about, I find it shocking.

My primary response:

Well, if you aren’t building software for customers, then you can do what you want.

Even though the customer is mentioned explicitly in the very first principle, I submit it is obvious that you are to satisfy the customer to build valuable software. Always. We ain’t no stinkin’ charity for developers to play around with shiny new software trinkets. (At least not overtly <g>.)

Anyway, “satisfying the customer” is simply a “given” behind each one of us that met to author the Manifesto.

And, who ever suggested moving away from the customer (implied by the need to “get back”)? That is completely anti-agile.

Agile is also not “about the processes and tools” as the post suggests — if anything, Agile is about a state of mind to do the right thing in every circumstance.

How do these ideas get out there about Agile?

I’m confused.

Agile Manifesto Simplified

JB Rainsberger took a stab at simplifying the Agile Manifesto.

Original (24 words):

  • Individuals and interactions over processes and tools
  • Working software over comprehensive documentation
  • Customer collaboration over contract negotiation
  • Responding to change over following a plan

I’ll see your 3 bullets (24 words), JB,

  • People working together over precise rules and pre-packaged solutions
  • Working software over promises
  • Doing whatโ€™s needed over doing what we said was needed

and raise (well, reduce) you:

  • People working together over dogma
  • Working software over promises
  • Doing whatโ€™s needed now

24 words to 13 words. Same context? You decide.

More to follow ๐Ÿ™‚

Refactoring Cucumber Features

I was inspired by this post (You’re Cuking It Wrong) to re-look at some feature specs to see if I could improve them.

I went from here:

Feature: Season Management
Scenario: Create a Season
  Given an organization like "My YMCA"
  When I add a season like "Summer Basketball" for "My YMCA" from "1 May" to "31 August"
  Then I should see the "Summer Basketball" season for "My YMCA" in a list of the organization's seasons

Scenario: Add a Registration Setup
  Given the "Summer Basketball" season for "My YMCA"
  When I add a setup like "Summer Signup" open from "1 April" to "30 April"
  Then I should see "Summer Signup" under the "Summer Basketball" season

to here:

Scenario: Create a Season
  Given an organization like "My YMCA"
  When I add "My Season" season
  Then I should see the "My Season" season

Scenario: Add a Registration Setup
  Given the "My Season" season for "My YMCA"
  When I add a setup like "My Season Signup"
  Then I should be able to signup under "My Season"

This also led to improvements in the steps. For example, by getting rid of the unnecessary dates, I also removed some unintended “brittleness.” As an example, this:

When /^I add a season like "([^"]*)" for "([^"]*)" from "([^"]*)" to "([^"]*)"$/ do |season, org, from, to|
  visit organizations_path
  click_link org
  click_link "New Season"
  fill_in "Name" , :with => season
  fill_in "season_starts_at" , :with => from
  fill_in "season_ends_at" , :with => to
  click_button "Create"


When /^I add "([^"]*)" season$/ do |season|
  visit organizations_path
  click_link # Created in a prior scenario, no need to pass in
  from =      # Create more appropriate from/to values
  to = from + 120*24.hours
  click_link "New Season"
  fill_in "Name" , :with => season
  fill_in "season_starts_at" , :with => from
  fill_in "season_ends_at" , :with => to
  click_button "Create"

Subtle, maybe, but certainly more clear and less likely to need attention if/when the solution changes.

Be an Agile Survey Monkey

Survey Schmurvey!

There’s seemingly quite a bit of a dust-up over assorted agile surveys:

  • Scott Ambler’s 2010 Scrum Certification Survey.
  • VersionOne’s “State of Agile” survey.

I did Scott’s simple survey last week. I did the VersionOne not-so-simple survey the other day.

After someone dissed Scott’s efforts and praised the VersionOne effort, I replied:

I could poke serious holes in the “State of Agile” survey. I mentioned a flaw or two back to VersionOne (i.e., I replied to the email with the link).

But I won’t. It isn’t worth my time.

Surveys are extremely hard to design. I submit that the VersionOne survey has a flawed model of development projects underlying the questionsโ€”not that it stopped me from supplying answers. And when you toss it out to self-selection, it gets even weirder to draw major conclusions. And to get subjective info on being more productive with agile, etc., is anything but accurate.

If people make decisions based solely on someone’s survey (or certification or education), it’s a free country.

Good luck with that.

I would never be so foolish.

Tale of the Demanding Consultant

So, I just heard that a consultant canceled a gig at a company because โ€” get this โ€” the company did not move off of a big VCS system prior to his Scrum (etc.) engagement. (In the words of one friend, it sounds a bit like a temper tantrum or elitism.)

In my opinion, any consultant worth their salt would simply deal with it… the choice of VCS is *not* an issue. Neither is the choice of “Use Cases” or “User Stories.”

If you think these sorts of things are a big deal,

  1. go directly to jail,
  2. do not pass go,
  3. do not collect $200.


Consider this a “Bad Smell” should you face such a pre-engagement demand!