So, you’ve got technical debt, eh?
Ward Cunningham coined the term, and Martin Fowler has some good takes on the subject, and Ron Jeffries as well.
It’s an old concept that I remember talking about, maybe circa 2000… That sort of insidious growth of “software barnacles” that slow down development, or worse, bankrupt your company.
Am I Infected With Technical Debt?
If you think you have technical debt, but aren’t sure… Then follow this lengthy process to determine the prognosis with certainty.
Do you suspect you have Technical Debt?
- If YES, then you have Technical Debt
- If NO, then you likely still have Technical Debt
Funny, aren’t I? This stems from a time when I worked with a chef, and asked “Tommy, do you think this smells bad?” “Throw it out,” he said, without giving it a sniff or a second thought. In other words, the risk of being wrong wasn’t worth the cost of the “waste.” If you think you have technical debt, then simply assume you do have some technical debt, and deal with it – maybe not as brusquely say “throw it out.”
Good Technical Debt
This is best described by a true story from when I was at TogetherSoft.
It’s been a while, and my memory might be a bit fuzzy, and the actual dialog is just a guess, but this actually happened. Sales meeting at a Large Financial Firm in San Francisco, ca., 1999. Me and the sales rep.
“If we can get Feature X, then we will buy $1,000,000 worth of your Together ControlCenter [UML Modeling / IDE]”
Jon goes back to the dev shop in Russia:
“Team, here’s a new feature we need, and I have hacked together an approach.”
“Ewww, Jon, you know there is a better way. We must change our architecture to support that.”
“Yes, but that will delay our getting this $1MM deal, and my hack will work.”
“Oh! Then let’s do the hack. Just promise me that we’ll do it right ASAP!”
“Indeed, I have already submitted an issue for the next release to immediately rip out this hack and do it the right way. In the mean time, we’ll get One Million Dollars, and we’ll get feedback on the feature.”
Now that is good technical debt! Done for a very positive reason, and not left in the code base to get worse over time. Just like some debt in business is good, so too in software. But if you leave the bad stuff in place for too long, it will turn into BAD debt!
Bad Technical Debt
This is the “classic” problem easily identified by any or all of the following symptoms:
- Just look at this code, it sucks
- Change in one area causes breakage in (seemingly) unrelated area
- We have a lot of bugs, especially in that area of code over there
- Developers know to avoid changes in certain parts of the code base
- It costs a fortune to add new features
Often, the code base starts off good enough, but somehow erodes over time. Maybe the new feature requires a bit of architectural changes, maybe a new table or two. But you find a way to “jam” it in that will “work” for now. Or maybe you don’t even realize there is a better alternative because you are either not skilled enough, or are not wise enough (grin).
In either case, this sort of “one-off” side-stepping of the proper solution, while seemingly innocent enough, can have additional, unforeseen consequences. Even if you catalogued this debt, can you really understand how it might compound in the future? Maybe the next developer comes along and builds off of your one-off, tacking on yet more debt. And then before both of those “stains” are cleaned up, two more small tweaks are built on top of the existing not-so-good code.
And so it grows, in a typically non-linear fashion. A few years later, a new member joins, points out the cruft, and your team wonders how it got so out of hand… One stain at a time, that’s how.
Don’t be that guy. Stop it before it happens.
Measuring Technical Debt
Manually, you can track technical debt as you run across it. That is, submit a “ticket” to address the technical debt, describing things like:
- where it is found
- what negative impacts does it have now
- what negative impacts will it have in the future
- cost to fix
While applying metrics tools is a good start (I championed adding dozens of audits and metrics to our Together ControlCenter tool), it is lacking in one major fundamental piece: tying the metrics to the real world.
That is, just because I can tell you the horribly complex bits of your code (my favorite metric is cyclomatic complexity), that doesn’t directly translate to your technical debt. Now, if that complexity lies in a heavily-used area or a frequently changing area of your code, now you have a potential handle on “costs” associated with that complexity.
“Hey, boss, if we clean up this core module, we can get features out the door much faster. It really slows us down every time we have to wade through this muck.”
Michael Feathers has a well-liked book out there on tackling Legacy Code.
SonarQube has built an algorithm to express the technical debt lurking inside code using their plug-in.
You can start to track high-level metrics on your projects:
- Feature “throughput” (of course, size matters, in terms of comparisons) to see if you are slowing down or compare against other projects.
- Bug count
- Consecutive days running without a major event that everyone hates
Naturally, some of these could be normalized using number of devs, or usage statistics, or whatever makes sense for your situation and metrics. If you have lots of projects going on, try out a few different metrics to see if any trends emerge.
Of course, if only we had a “tape measure” for technical debt, life would be easy. Alas, at most, we’re just guessing at the current costs, and especially guessing at the future compounding costs.
Another idea is to measure how good or bad the code base is by providing a survey to the development team. Track this subjective measure over time.
You can point Sonar at your source. And then have it find all of your technical debt so you can start fixing it. Just kidding! Though sonar will help you tailor the metrics it gathers, and help you manage “technical debt,” it still requires the intervention of a sentient being to make the right calls.
Planning A Way Out
Much easier said than done — especially if you have spent years digging yourself into this mess.
Catalog your debt. Make it just as visible as your features and bug lists.
Determine the root causes for the different types of debt, and see if you can detect a pattern and fix it!
Hopefully, the business is helping drive the choices that are made regarding what the development team works on next. And hopefully, the development team keeps any critical-to-fix technical debt at the forefront of the conversations regarding iteration/release planning.
Of course, hope is never a good strategy, Best to come to some agreement about allotting time to tackle the priority-ranked list of technical debt items.
Prevention is the Best Remedy
Though you can catalog, prioritize, and start to eliminate your existing Technical Debt, that is all for naught unless you get a handle on preventing it in the first place!
It would be demoralizing for your team to be diligently cleaning up the Technical Debt muck, only to find new projects adding it faster than it is being removed.
Eliminate the causes of the Technical Debt. Eliminate the rewards of allowing Technical Debt. If a manager gets rewarded for getting a project done ahead of schedule regardless of the quality, guess what you just encouraged? The same thing could happen if you give $1000 for every bug fixed — you’ll get more bugs produced.
Avoiding the technical debt requires a holistic set of practices that conspire to help maintain a lean code base:
- Get better at developing software in general. I mean, adopt a truly agile process that involves the business so that trade-offs between quality, schedule, and incurring technical debt can be more easily made.
- Keep the design simple
- Follow the domain
- Use BDD to ensure you have your code protected by acceptance-level tests
- Use TDD to ensure your core classes are well-tested
- Refactor mercilessly
- Employ as many Clean Code techniques as possible
- Constantly strive to improve
- Consider the costs and value to the business of everything you do
- And so much more…
Culture is Key
If your organization rewards folks for developing software that strikes the proper balance between quality and cost and schedule, then it should be easy to not let Technical Debt build up. However, if your company thrives on jumping to the next “brush fire” to excitedly stomp out; or never has the guts to tackle the technical debt identified, then either
- change your own little piece of the world and do the right thing
- change the organization
- buy more rolls of duct tape to keep your head from exploding
- or find a better job where software is treated more like a valuable business asset
What, No Silver Bullets?
At the risk of upsetting those of you who came looking here for a silver bullet to magically detect and destroy Technical Debt, you’re going to be disappointed.
It took work, inattention to detail, possibly poor craftsmanship, to build up the bad debt. It will take work, paying attention, and having professional craftsmen to pay off the debt.
thanks for sharing. This is the best and concise article on technical debt, I’ve ever read.
btw.: is TDD the shortcut for “technical debt developing” ?
Vielen Dank for your kind words. LOL on the suggested meaning of TDD — I think you are right for some folks
Jon – just brillant now if only I could make a $1 millon off adding a hack 🙂
FYI I wrote about the ugly side a few years ago: http://www.infoq.com/articles/technical-debt-levison
I was going to ask you if posts like http://xprogramming.com/blog/2009/02/19/technical-debt-2/ by Ron Jeffries (which exemplifies the view that technical debt is better dealt with by refactoring daily rather than by asking the customer for permission to do a ‘big refactoring’story) would change the content of your ‘Planning A Way Out’ section, but then I noticed you left a comment on that post, so I guess it has not influenced your opinion much.
But doesn’t this http://tech.groups.yahoo.com/group/scrumdevelopment/message/55626 (again, by Ron Jeffries) influence your view?
Ward’s and Ron’s “discussion” seemed to be about avoiding the sort of “insidious” technical debt that creeps in over time during development. I maintain that many people are happily coding away, not recognizing that they are leaving technical debt in the code/design.
Regarding the other article, I can certainly agree that you shouldn’t merely go around and clean up technical debt in the code in an ad hoc fashion — unless you truly have nothing better to do. If you are working in that code area, you should consider cleaning up some or all of the debt — depending on the severity.
My rule of thumb: if it is hard to add a feature (especially one that seems straightforward), then maybe you need to “fix” the design/code to make it easier to add the new feature. So do the clean-up/refactoring, and then add you new feature.
Great message. Admitting it is the first step. I think that identifying it could be a good second. The analysis we provide could be of great help. Have you had any experience with source code analysis and reducing technical debt? We have, though I think it is strange that so much visual/manual inspection is going on. Why do you think this is?
Good blog. I believe you will find mine interesting.
Some (of many) posts about Sonar:
Merci beaucoups for sharing your Sonar work.
My name is Bhavin, and I work for Packt Publishing. We have recently published a new book titled “Instant Cucumber BDD How-to” and was wondering if you would be interested in featuring the book on your blog/website in the form of a book review.
Written by Wayne Ye, this book will cover basics of Cucumber in a Behaviour Driven Development (BDD) style and explain the essence of Cucumber. For more information and to view the table of contents, please visit our book web-page : http://www.packtpub.com/cucumber-behaviour-driven-development/book
I came across your profile through a Google search and see that you have published useful articles on the topic. Considering your expertise in Cucumber BDD, I feel you’d be one of the best persons who could provide an honest, unbiased review of the book. I’m confident that your thoughts about this book will prove to be an interesting read.
In case this subject interests you and you’d be interested in writing a review on your blog or Amazon (if a review on your blog is not possible), do let me know. I have with me limited e-copies to give and I’d be delighted to send you a complimentary e-copy of the book on your confirmation.
Please feel free to ask if you have any doubts about reviewing, timelines or otherwise, I’ll be glad to help.
Have a nice day.