Technical debt gets real

The cost of fixing poorly written code is apparently going up these days, leading to the question of how much good agile development is actually doing in the modern enterprise.

There are a lot of assumptions packed into that question and few hard answers so I will stipulate up front that this post is filled primarily with crass speculation. Let's look at what we know and what we don't.

What we know is that Cast Software maintains and analyzes a database containing 365 million lines of code from 160 different companies. Last year, automated analysis of that code base for certain structural defects was run and costs to fix each defect was estimated at $2.82 "per line of code in the application." This year, the cost rose to $3.61 per line of code in the application. On that basis, Information Week concludes, "hurry-up development passes along problems to enterprise IT."

This sounds like a job for... DevOps!

But wait... let's talk about what we don't know. We don't know what development methodology was used for each of those apps; since agile is getting more play in and outside the enterprise, it's quite possibly one factor, but we don't have any idea how large of one or what the increase in the surveyed base agile projects might comprise. We don't really know how the cost to fix the code is arrived at; Cast mentions some assumptions for programmer costs and time-to-resolve but it's not clear if those assumptions were identical year-to-year or not. The cost to fix also makes the assumption that the problems are being fixed, which may not hold try in many cases since "The 'violations' that it [the analysis tool] finds are not necessarily going to stop an application from running." In some cases, it's not certain that the identified practices actually result in reproducible bugs in the production environment.

I am a fan of hard data in the service of practice analysis, and so I applaud the efforts here to put some real numbers behind code maintenance. "Technical debt" has become a popular buzzword lately, deployed most frequently in support of arguments that it is worth taking the time to do things right the first time. But those are mostly theoretical arguments, and although this study attempts to put some numbers behind them, they are overly general for use in actual decision support. Abstractions are a necessary part of coding, but if you need to evaluate technical debt in your organization, base it on measurable outcomes and not theoretical studies of somebody else's problems.