The Trouble with Technical Debt

This article is not about the perils of accumulating technical debt, nor the challenges in paying it down. Instead, it is a call to action to the developer community that we change the way we talk about scheduling technical debt.

It's a constant sore spot and the source of many arguments between developers and product owners: How do you make time for refactoring? When do you pay down your technical debt? I say we developers need to change how we make our proposals.

We ask our product owners to prioritize the backlog: to attach a business value to each user story and put them in order. Then we provide a size estimate for each story, indicating its relative cost in time and resources, and voilà: a prioritized list of requirements with a cost and a benefit for each.

Then we propose investing some time in cleaning up a lurking problem. The work will improve our velocity in the future. By how much? A lot. And how long will it take? A while. So please fit this chunk of work into your carefully prioritized backlog.

You see what we have here, folks: an ArgumentNullException. The product owner has sorted a list of objects based on their cost and their benefit, and then we try to place an object into the list with an undefined cost and an undefined benefit. No amount of arguing is going to make that work.

The first counter-argument I receive from folks is that the business should trust us or, from the more progressive thinkers, that we need to earn the trust of the business. Um, perhaps. But imagine you were the person prioritizing your own list. Even if you trusted your own motives perfectly, how would you prioritize an item with unknown cost and unknown benefit amongst a list of items with known costs and benefits? And that's assuming perfect trust; I'm skeptical of our discipline to admit that substandard, icky code can be good enough. Developers (the good ones, anyway) have a strong desire for elegance and simplicity.

The second counter-approach I hear is timeboxing: allocate a fixed amount of time to refactoring and only spend that much time. That fills in a value in the "cost" column, but it does not address the undefined benefit. Also, can you think of a scenario where you've undertaken a significant refactoring, the clock ticks over, but you're not yet in a stable state? You either roll back the changes (and realize zero benefit), or muscle through (increasing the cost)? For the timeboxing approach to honestly reflect your cost, you must work with the discipline to make many small, beneficial changes, and stop when the clock runs out. Are you up for that?

Speaking of size, be sure to consider the testing cost. Technical debt accumulates in the areas folks are afraid to change. Those are usually business-important, application-spanning areas. Your refactoring will likely touch many features in the application, warranting thorough regression testing. How good is your automated test suite? Be responsible about how much testing you'll need, and be honest about how much time it will take.

Okay, agile developer community. If we're going to get technical debt work and refactoring into a sprint, we need to apply the same rigor that we ask from our product owners. Cut the work into a discrete unit. Write acceptance criteria so you'll know when you've finished. Make some kind of estimate of the benefit; talk with your teammates about how you might quantify such benefit. Using the acceptance criteria, estimate the complexity and size, same as if the work had been requested by your product owner. Observe and reflect on the predictive accuracy of your cost and benefit estimates to improve your estimates in the future. Start conservative to build experience and credibility over time.

Most product owners are happy to invest time in making the product more stable and making the team faster on future features. We need to give them estimated costs and estimated benefits so they can fit that work amongst other business priorities.

No comments: