Category Archives: process

Measuring Effectiveness: The Software Industry’s Conundrum

On “Measuring Effectiveness”

I’ve been saying for seemingly decades, this is (one of?) our nascent software industry’s biggest conundrums (maybe even an enigma?).

Just how do you prove a team is “doing better?” Or that a new set of processes are “more effective?”

My usual reaction when asked for such proof:

“Ok, show me what you are tracking today for the team’s performance over the past few years, so we have a baseline.”

Crickets…

But seriously, wouldn’t it be awesome if we could measure something? Anything?

For my money:

Delivering useful features on top of a reasonably quality architecture, with good acceptance and unit test coverage, for a good price, is success.

Every real engineering discipline has metrics (I think, anyway, it sounds good — my kids would say I am “Insta-facting™”).

If we were painting office building interiors, or paving a highway, we could certainly develop a new process or a new tool and quantitatively estimate the expected ROI, and then prove the actual ROI after the fact. All day long.

In engineering a new piece of hardware, we could use costing analysis, and MTBF to get an idea on the relative merits of one design over another.

We would even get a weird side benefit — being relatively capable at providing estimates.

In software, I posit this dilemma (it’s a story of two teams/processes):

Garden A:

  • Produces 15 bushels (on average) per month over the growing season
  • Is full of weeds
  • Does not have good soil management
  • Will experience exponential (or maybe not quite that dramatic) production drop off in ensuing years, requiring greater effort to keep the production going. Predictability will wane.
  • Costs $X per month to tend

Garden B:

  • Produces 15 bushels (on average) per month over the growing season
  • Is weed free and looks like a postcard
  • Uses raised bed techniques, compost, and has good soil management
  • Will experience consistent, predictable, production in ensuing years
  • Costs $Y per month to tend

I could make some assertions that $Y is a bit more costly than $X… Or not. Let’s assume more costly is the case for now.

To make it easier to grok, I am holding the output of the gardens constant. This is reflected by the exorbitant rise in cost in the weedy Garden A to keep producing the same bushels per month… (I could have held the team or expense constant, and allowed production to vary. Or, I could have tried to make it even more convoluted and let everything vary. Meh. Deal with this simple analogy!)

 

Year 1 2 3 4 5 6 7 8 9 10
Garden A 100 102 105 110 130 170 250 410 730 1600
Garden B 120 120 120 120 120 120 120 120 120 120

If we look at $X and $Y in years 1 through 10, we might see some numbers that would make us choose B over A.

But if we looked at just the current burn rate (or even through year 4), we might think Garden A is the one we want. (And we can hold our tongue about the weeds.)

But most of the people asking these questions are at year 5-10 of Garden A, looking over at their neighbor’s Garden B and wanting a magic wand. The developers are in the same boat… Wishing they could be working on the cooler, younger, plot of Garden B.

What’s a business person/gold owner to do? After all, they can’t really even see the quality of the garden, they just see output. And cost. Over time. Unless they get their bonus and move on to the next project before anyone finds out the mess in Garden A. Of course, the new person coming into Garden A knows no different (unless they were fools and used to work in Garden B, and got snookered into changing jobs).

Scenario #2

Maybe we abandon Garden A, and start anew in a different plot of land every few years? Then it is cheaper over the long haul.

Year 1 2 3 4 5 6 7 8 9 10
Garden A 100 102 105 100 102 105 100 102 105 100
Garden B 120 120 120 120 120 120 120 120 120 120

I think the reason it is so challenging to get all scientific about TQM, is that what we do is more along the lines of knowledge work and craftwork, compared to assembly line.

The missing piece is to quantify what we produce in software. Just how many features are in a bushel?

I submit: ask the customer what to measure. And maybe the best you can do is periodic surveys that measure satisfaction (sometimes known as revenue).

Matt Snyder (@msnyder) tweeted me a nice video: Metrics, Metrics, Everywhere – Coda Hale


Easing New Developer Ramp-up Time

On a recent healthcare start-up team, it grew from my buddy (who moved on after 6 months or so) and I, to a handful of developers/sub-contractors.

Here is how we tried to make it fairly efficient. We just started tracking what was needed, getting feedback as each new developer went through the process, and improving the instructions and process along the way. If something was missing, we added it. If something was not clear, the new developer could amend the wiki.

By the 3rd new developer, or so, we had it down to where they could get started and begin a legitimate issue in less than a half day — from getting set up to being able to commit and deploy a new “feature.”

There was a section at the top that shared a good team chat room session with a new remote developer:

Getting Started Chat Room Conversation

That was followed by the FAQ-like list of links:

One of the first reasons I wanted to make it easy for a new team member to get rolling, was so that our friend Max — who would be doing our QA from Russia — could get started. As we added the first couple of devs, we probably decreased the start-up time as follows:

As part of “Getting Started,” I would include a simple Jira issue that helped them ensure that everything was working and that they followed our dev process:

  • Git and Dev and database (MongoDB) environment obviously had to be set up
  • Access to Jira to assign themselves to the issue, and move it — Kanban style — to the In Progress state.
  • Commit the work and the passing tests
  • Drag the Jira issue to “Done”

 

Since Atlassian’s Confluence Wiki does a stellar job at versioning pages, I actually looked back to see how the page grew and morphed over time. It started out rather modestly (and empty):

After it grew a bit bloated:

It was successively refactored into its current state, here is a snippet of the 70 versions that this page underwent from March 2011 through July 2012.

Wikis, like code, need to be tended to, nurtured, and refactored to provide the best value.

Jira vs Post-It Notes

Had some recent discussions on the subject of using tools or just sticky notes…

for me, a digital board representation of the kanban style 3-column view is merely one v-e-r-y small aspect of using something like jira.

though i could probably live with just post-it notes, i bet it would be an interesting challenge for me. i have been:

  1. doing only distributed work for so long (since late 90s)
  2. using jira so long, it is as simple as a pen and paper (or post-its)
  3. dissatisfied trying to do a project with something simpler like Basecamp, without the richness of jira
  4. working on long-term projects that have different people rolling through, and years of life (4000+ issues)

and i ask myself what other benefits do i derive from jira (plus, admittedly, a companion wiki)? why might i be uncomfortable with just post-its?  do i have an unnecessary “crutch” in the form of jira? hmmm… do i do more than is necessary in the way of using jira/wiki to add other pertinent documentation for the project?

well, here are the following things i can think of off the top of my head:

  • it is easy to
    • assign myself to an issue
    • move it to be in progress
    • move it to be resolved
  • our virtual chats hover around the greenhopper view
    • we’ll edit stuff on the fly as needed
    • quickly create a new issue if something pops up during the call
    • everybody refreshes their browser
  • we put a fair amount of supporting docs — details, helpful things — into the issue (or into the wiki and then that link goes in the issue)
    • that way you always know where to look 🙂
    • you can find it 24×7 and regardless of your GeoLoc
    • sometimes we’ll have skype chats and voice calls about the issue, maybe about design ideas. I’ll shove those things into the issue.
  • we can link related issues
  • we can use the search to look up old issues and refresh ourselves on what was asked for/done
  • we can have asynchronous, threaded conversations in the form of comments
  • we can track any myriad of other stuff against the issue
  • i log my hours against the issue
    • useful for billing if you need it…
  • i can use it to help generate pretty charts for management
  • we use the issue status to trigger QA (in addition to our chats)
  • i use jira to help ensure product version notes are up to date
    • jira manages the iterations
  • pretty easy to maintain a backlog, even chunking it up into groups if needed
  • easy to indicate that an issue has been rejected, and why

the point is, i find it incredibly useful to have modern technology at my fingertips…

in my experience, there is so much more to a project tracking “tool” than what post-it notes would seem to represent.

[important]but i have NO (zero, nada, zilch) experience being part of a co-located team using post-it notes. so take it with a grain of salt :-)[/important]

Considering Sprint Length

A friend of mine had an interesting situation:

  • Novel product, many unknowns
  • Multiple teams grouped into 3 product areas
  • Experience doing 3-week sprints

Only a fool would do anything other than the 30-day official sprint cycle that I saw on some website and in a few books.

(Just kidding. Unfortunately, like most of agile development, context has a tremendous impact on what you choose to do, process-wise.)

A lot could go into what the Optimal Sprint Length should be… You could ponder the dependent variables and try and guess an optimal length to optimize the independent variable(s) — which would be, what, maybe cost and rate of feature delivery and quality? You could do the “democratic process” and allow the team to vote, or even do “rock-paper-scissors” to figure out 2 or 3 weeks.

However, what if we built a continuum of sprint lengths for the sake of discussion. On the one end, we start at the idealization of doing one useful feature at a time and deploying it immediately — think simple web app. Anything longer than this is a compromise based on some (hopefully valid) reason. On the other extreme, we could wait until the entire system is done before deploying or integrating, maybe after 6 months or a year.

The cost of “batching up” the “work in process” at the upper end of long sprint lengths, is pretty obvious to everyone. I submit, that if you agree with (or experience first-hand) the premise that batching work has a non-linear impact on overall cost (including the hidden and subtle cost of everything that we know is bad with waterfall), then it stands to reason one might favor shorter cycles and less batching.

Not to digress, but the parallels exist in industry. To allow WIP to be large, and to allow certain parts of the process to run at high levels of batching, is a risk. A risk that the items in the batch, once released into the wild, are discovered to not be as valuable as first thought. Well, it’s water over the dam, time and effort you will never get back. (Think: extra features built because someone thought they would be useful, and it turned out that the marketplace thought otherwise.) Nonetheless, sometimes weighing the risks will lead you to some level of batch that makes the most sense.

There is often much more to the decision on sprint length than purely the development team. For example, what is the cost of QA? If the cost of QA is no different for 1 feature at a time versus a week’s worth of features, than QA cycle time/cost is not an issue. However, if it requires a week of QA time to regression test the system in the case of even a single small feature or bug fix, then you have a serious input into what the optimal sprint length should be.

Naturally, one could do development sprints at one frequency, and QA sprints at another… and even customer ship sprints at a completely other cycle time.

Regarding multiple teams… this is a solution that can be recursively applied, much like you would at a software architectural level. If the teams are horribly coupled, your costs will balloon and no amount of pondering sprint lengths will have a significant impact. If the work dependencies are carefully controlled between the teams, sprint length could vary between teams due to their own local reasons.

Much like the QA process can be a “tax” on each Sprint, what other taxes does your process incur? Running down to a one week sprint will likely reveal expensive parts of the process that could be ripe for improving.

So having said all of that… Here’s a thought. Why not simply agree to try out a few different lengths for enough sprints to get a feel for the differences. Try one week sprints for the next 6 weeks. Try 3 week Sprints three times. See if you can monitor metrics that will tell you what worked better. Consider that different teams might also work at different frequencies to test the “costs” of thinking the teams should be synchronized.

Much like with our USA republic, surely don’t let democratic, mob rule win the day.

Do I Still Do Domain Modeling?

Got a very nice “blast from the past” contact (4 levels deep) on LinkedIn. Scott was a member of a team where we went through object modeling for their business application.

The reason I wanted to contact you was two fold:

First for some reason that training has stuck with me more that many trainings and I still go back to the cobweb section of my brain and bring it out every time I have a object modeling task, so thanks you did a good job helping me.  This is true for many of the people that were there.

Second as I have been given another task of modeling a large system from scratch I was wondering  about how you feel about the Domain-Neutral model that was presented in the training you did for us.  I have found it useful over the years but do you still use this model in anything that you design this many years in the future?

Thanks for the help.
Regards,Scott

And yes, I still do domain modeling the basic way that we did 11 years ago (gulp). I only have my old copy of Together anymore, so I am stuck back in time there. In person, I always use post-it notes, flip-charts, and markers. Once I want to make a computer version, I’ll use different tools for modeling depending on the need. For example, UMLet is a great little simple tool to bang out a quick diagram in no time flat and that can be used by anybody.

Another good approach from what we went through 10 years ago, is to follow the color modeling order to help in the discovery process… that is focusing on the following as you walk through the assorted primary user scenarios:

  1. First trying to discover what time-sensitive aspects does the business care about most (the pinks)
  2. Then looking towards the roles that may be at play there (yellows).
  3. Are there any specific things (like contracts, purchase orders — greens)?
  4. And finally, any descriptive elements (blues) to go alongside the greens?

For the past 18 months, I am in major love with Ruby, Rails, and MongoDB — plus all of the surrounding tools and community (see my recent blog posts). Ruby the object-oriented language I wanted when I was doing C++, and MongoDB/MongoMapper is the OO-like DBMS that I always wanted. Putting it all together makes for real productive, high-quality, consistent development — I can quickly model out domain things and try them in a working application.

I’ve never done an adequate job of explaining how I like to approach doing just-enough design up front, but you can find a few snippets here:

Some tips that I find useful when doing initial up-front Domain Modeling (what you are about to do):

  • Spend time with subject matter experts and the business/product owners
  • Build out as you gather high-level features
  • Do breadth first, then depth
  • Only go into details when it will help reduce an unacceptable level of risk

That last bullet is key. I like to do just enough up-front work to please the stakeholders and myself that we can answer the question about: how much? and when? to the level of desired specificity. When you attempt a high-level estimate at a given feature, and it is too big for your comfort, then you can get more detail and break it down further so that it becomes acceptable.

If you do not do enough up front, and your estimate is off by an order of magnitude or three, you might upset the client!

On the flipside, if you do too much up front (detailed modeling), then you risk not getting frequent, tangible, working results into the hands of the client soon enough.

It’s a big balancing act.

You Can Start at Login!

On one of the lists I hang out at, Marton was wondering about how to start off a project that required authenticated users…

How would you decouple the sign-up feature from the login feature to keep the stories independent and testable from the UI only. The acceptance criteria should not fiddle with any implementation details such as concrete url-s or field types. I’d like to leave that entirely to the developers. Let’s say I have the following story coming to mind in a workshop:

As an Anonymous user I want to sign up to Example.com website to enjoy the benefits of a registered user.

Acceptance Criteria:
1. I navigate to the signup page
2. I enter my full name, desired username and my password
3. I have to verify my password to avoid any typos
4. I submit my information

And here comes the “tricky” bit:
5. I can log in with the credentials I provided.

Some folks suggested he start on real functionality first, others implied login was a lousy first story.

My take: It’s not a hard story to start at and have a meaningful test that can grow over time.

For example, to start, you can simply check that the response has “Login Succeeded” ( or “Login Failed” for testing that a bogus login attempt does indeed fail).

In Cucumber flavor:

Scenario: Registering as a new user
    Given I am a new user
    When I visit the site
    Then I can register

Scenario: Logging in as a registered user
    Given I am a registered user
    Then I can login
    And enjoy the beauty of the website

Or, even more simply:

Scenario: Successful Login
    When I login as admin
    Then I should be logged in

Scenario: Failed Login
    When I login as asdf56ghasdkfh
    Then I should be not logged in

And your steps would hide the logic for filling in the login form and checking for success:

Given /^I login as "([^"]*)"$/ do |login|
  @login_name = login
  visit login_path
  fill_in "login", :with => login
  fill_in "password", :with => "password"
  click_button "login_button"
end

Then /^I should be logged in$/ do
  response.should contain "Login Succeeded"
end

Then /^I should not be logged in$/ do
  response.should contain "Login Failed. Please try again."
end

Your login details can change over time, adding the password confirmation box, handle validation errors, etc.

Your next step might be to show Admin-user-only functionality as I have here (snagged from a current app):

Feature: Quickly exercise the primary UIs just to make sure nothing blows up

  Background: We need to pre-populate the database with the simulator results, and be logged in as admin
    Given I login as "admin"

  Scenario: Brief tour around the Admin UI
    Then I should be able to click on "Dashboard" and see "Admin Management"
    And I should be able to click on "Accounts" and see "Search"
    And I should be able to click on "Sign out" and see "Login to Your Account"

And so it goes, step-by-step.

Hope that helps!

As an update… one of the list contributors went so far as to indicate “Login has Zero business value, always.” No wiggle room there!

well, okay, if you are parsing the act of logging in to achieve a status of an authenticated user to provide a means of secure access as separate from the need to have secure access, well, ok.

yes, i didn’t separate the two. The point was merely to show the OP how you could easily write his desired login and registration story, and it is not that hard.

As an aside, another project I did with two friends required a “marketing” site where you could sign up and see info about the core application. The project also required the site where registered users would do the real work of the application.

Since I knew login would be easy to add, I suggested to my one friend how to do it, and proceeded ahead with the other functionality. This was based on knowing full well that I would be able to feather in the user authentication and access rights into the core app once they were completed in the marketing app.

So, I am not hard over one way or the other on this issue, and in the past 12 months have done it both ways on different projects. Hence my surprise at folks’ having such stark “black and white” edicts banning ever starting at Login as a story.

Now, were I on a project where there was some very risky bit that, should we not solve that problem adequately, nothing else matters, guess where we start? Not login. 🙂

Geesh.

Oh, George posted a reply of sorts here.

The Website Development Anti-Pattern

I don’t want to come down too hard on the author, he looks like a real nice, friendly guy. Writing a post like this took time, effort, thought, and care — which I applaud.

But when I read this post on Website Development — A Practical Methodology, I had flashbacks to Waterfall-esque DoD projects. Seriously? A phased approach with no partial deliveries, no iterations, no “sneaking up on the solution?”

Is this a post from, like, 1986? A phased approach?

  • “Once everything has been appropriately documented, it is time for the Implementation Phase.”
  • “Once implementation is complete, it is time to test.”
  • “When the testing is complete… you are ready to deploy.”

I humbly submit that you might want to intersperse some of those activities into a more iterative approach to reduce the time and cost of getting things wrong and discovering it late in the phases.

I just checked the date of the post again.

It is 2011.

Someone pinch me.

Plan & Estimate, Only if You Must

It must have been the morning to challenge planning efforts, as GeePawHill did here. I was answering a post that had gotten around to discussing budgeting and planning for interruptions to fix legacy apps while teams are building new stuff.

When all of the sudden, I blurted this out:

Kanban is simple, in theory. Simply do one feature or bug fix all the way to being released. AKA focus. But there is a lot of things that have to be working well, for example: the cost of testing has to be low so that it doesn’t cost a fortune in treasure or time to get in that new feature or bug fix. (Expensive testing often leads to bundling a bunch of stuff into a given release and widening the time window, and introducing parallel development techniques.)

And, regarding budgeting in to prepare for the unknown maintenance efforts… That’s but one way to handle things. Another way is simply let the people who care about getting things done participate in setting your priorities. If you were given 10 things to do in a month, and halfway through you get an 11th thing to do. You probably don’t need to budget for that unknown. You simply ask for help deciding which of the original 10 should get axed to make room for the new “emergency” item.

Like it or not, you are all in it together, it is a zero sum game. If you fix the resources, fix the schedule, the amount of work that gets done has to give when new work is injected. Deal with it in an honest manner.

Hard to describe in words… and had to know your exact context… but what you are facing is probably more of an organizational issue than anything else.

All we software folks do for a living is produce features and fix bugs to keep paying customers happy. Just keep trying to improve your team’s efficiencies and squeeze out wasteful practices.

When you are producing software as an internal organization, you can sometimes be brave and forgo lots of planning stuff. If you have the priorities set, and if you have a delivery schedule, and if you understand your costs of releasing, you can make a pretty simple and smooth process. I’ve done it many times in companies formerly steeped in waterfall fantasy land. Why waste all sorts of time making estimates if you don’t have to?

The answers to these questions can help you decide how much ceremony your processes need:

  • Does it matter if the team does 5 things in an iteration or does 10?
  • Does it matter if they estimated 10, but only wound up being able to do 5?
  • Or, if they bid 5 and found they could do more and gobbled up 10, is that a problem?
  • Is it more important to hit a date with at least some features?
  • Or, is it more important to produce specific features as fast as you can, even if you miss the date?
  • Does the team need to be accurate on their estimates? How accurate and why?

Many times, we fall into traps of wanting to manage software process as if it were a brick-making factory. Unless you live or die by doing software as a fixed-bid project, you can often really get simple and make software really fun in your organization by doing less ceremonial planning crap!

Getting Started w/ Best Practices & Design Principles

Praveen had a seemingly innocent question here and there were lots of good answers…

If we are following Agile Methodology (Scrum) & We are in Java/J2ee Domain,

What are the Best Practises & Design Principles need to be followed
while designing the applications.

Big question is an understatement… here’s what I would advise:

  • Understand the business domain
    • because, if you get that wrong, it doesn’t much matter how well-crafted and well-tested your code is
    • Design using Separation of Concerns as the granddaddy of all patterns to sear into your brain.
    • Be sure you use a layered approach to your architecture — if I catch you putting SQL in your javascript, I’ll come over there and stab you in the toe
    • Design in the Large to avoid excessive dependencies
    • Design in the small to adhere to all those good practices like SOLID
    • Consider looking into the Domain Neutral Component — within which a shockingly huge number of domains tend to fit.
  • Architect and build the application in a highly consistent manner — much like Rails enforces.
    • If you have a dozen domain items that need the same sort of treatment from persistence to UI, please don’t get creative and do persistence 5 different ways.
    • Pick one and stick with it for the sake of consistency!
    • I’d rather a consistent implementation that is mediocre, than one with 5 different approaches (one stellar, one horrid, and the rest mediocre). Why you might ask? Because it is easier to refactor when things are consistent 🙂
  • And, of course, apply an agile process…
    • Involve a customer/proxy to ensure you understand the requirements from the business perspective, do not rely on documents — but if you must, then you need to exaggerate the agile process and make it more granular. That is, to compensate for the likelihood that the written word is ambiguous, you need to show small steps of progress on a feature and get rapid feedback to ensure you are not mis-interpreting the requirements doc.
    • Build a vertical slice of your design to prove it out and to serve as a model for how you will work on your app
    • If you have stringent performance needs, prove them out early
    • Show working code early and often
    • Get yourselves set up on an automated build
    • Try to build out the features using something like BDD or TDD
  • General
    • Be patient with your learning
    • Be impatient to see results, don’t let any team member or task go dark for more than a couple of days
    • Create the lightest-weight process possible
    • Don’t do anything to excess without frequent feedback
    • Always foster a team attitude to question everything with boldness — including my advice

Hope that helps

Ideas for a Distributed Team

In response to a posting on Agile Project Management…

I am starting up a ‘Distributed’ Agile Project and looking at tools to assist the distributed team communicate the status of work undertaken, stories, estimates and burn down/up charts as well as general collaboration (forums, document sharing etc).
There are tools such as Agile Wall, Mingle, Trac (with plugins etc).
I am ideally looking for an externally hosted solution (something that we will not be setting up in-house).
I have a core team of 15 people and with an additional 10 people that would want access. I am looking for any ideas or suggestions on online tools that you may have.
Thanks in Advance.
Regards
Neal

I’ve been doing this sort of thing since 2000, so it is very easy and very possible to be extremely effective as a distributed team.

Here is a very simple approach using Basecamp (and you can get an add-on for burndowns if you like that sort of thing). I’ve been doing this on a small project since March or so.

Then I set up other simple collaboration tools (looked like this on many projects, though you might have to march around that hodgepodge of a site to find more tool ideas):

  • Wiki: Project stuff that is not issues or code needs a home; a.k.a.,
    (you can use basecamp, though not as extensive as something like confluence)
  • Chat Room for being the virtual cubicle farm and water cooler. Skype or IRC works great. You can establish key words that call out for attention, like “@ALL I’m going to kick the TEST server.” You can even have textual scrums if voice is not always possible. For a while, we had a bot that recorded the chats and logged them to the wiki 🙂
  • Issues/Bugs: You need a place to store your lists of things to get done. Some call that an issue tracker; others, simply a To-Do list. I have used Jira for a l-o-n-g time for both features and bugs. I have also used Lighthouse — which is just fine too.

Atlassian Studio is worth a look. You can add in continuous integration and code reviews and what not.

My suggestion: start really small and humble with the tiniest process.

That really simple Basecamp process I blogged about? We actually got to PRODUCTION with much less ceremony. There were none of the “To be QAd” or “To Be Released” because we did it more in iteration chunks, so we knew the overall state of a given iteration (in dev, qa, or released).

The slightly greater ceremony of QA and Release buckets allowed me to answer the need from the stakeholders and other non-developers to know the state of the project at a glance. It’s easy to see what is in the backlog, or what is in progress, or waiting QA, etc. Now we’re doing Kanban. I crank issues and (rare 🙂 bugs through one at a time usually.

Anyway, have fun!