Data-Driven Management of Technical Debt

Technical debt communicates the tradeoff between the short-term benefits of rapid delivery and the long-term value of developing a software system that is easy to evolve, modify, repair, and sustain. Like financial debt, technical debt can be a burden or an investment. It can be a burden when it is taken on unintentionally without a solid plan to manage it; it can also be part of an intentional investment strategy that speeds up development, as long as there is a plan to pay back the debt before the interest swamps the principal.

In this blog post, I discuss the SEI's current work in technical debt. The goal of this work is to develop analysis techniques, as well simple practices to help software engineers and decision makers decide how best to manage the effect of technical debt on their software projects.

The Technical-Debt Metaphor

We have been talking about technical debt for a couple of years now, and increasingly we see that this metaphor resonates with developers and decision makers. However, while all software developers understand intuitively what technical debt is, they lack clear guidance and proven techniques on how to identify it, how to concretely describe it, and how to account for it within the software-development life cycle. Consequently, in the past several years we have also seen increasing amount of research addressing all of these aspects.

Technical debt can be thought of as a design strategy. The financial metaphor is apt because, as with finances, getting into debt in software development by optimizing short-term goals creates tangible value. It represents a tradeoff, a choice of one option over another. But selection of this option must be managed through the duration of the project, with a clear understanding of the penalties that result. The danger lies in choosing the option and making the tradeoff, but then forgetting about, ignoring, or underestimating the consequences.

Although many Agile-development teams follow simple practices of labeling their technical debt, talking about it, and trying to minimize it with heuristics, most management of technical debt today is ad hoc. In recent years, static-analysis tool vendors and standards groups have sought to assess whether certain code-policy rules might be more critical for accumulating technical debt. While these developments are promising, more research is clearly needed, in particular analyzing and monitoring design issues that tend to result in technical debt that's harder to fix.

DoD and other government acquisition managers must be able to assess what kind(s) of technical debt their developers and software contractors are creating when they make decisions. Solutions that rely primarily on code analysis fail to differentiate among design issues that lead to accumulating rework costs, leaving the question of which issues are most significant unanswered. Without this information, issues that contribute to accumulating technical debt could be given low priority, allowing the accumulation to continue unabated. The challenge today is to provide development teams and the government with tools and practices to analyze the quality of the software and consequences of specific design choices.

How Research Can Help

In our work at the SEI, we are developing tools that integrate data from multiple, commonly available sources to pinpoint problematic design decisions and quantify their consequences in a repeatable and reliable way for uncovering technical debt. Design problems, which frequently result from optimizing for delivery speed, are a critical part of long-term software costs. Detecting such design issues with tool support is a high priority for technical-debt analysis. The goal of our analysis is to help teams focus on rework-causing issues earlier in the life cycle. Improving identification of such issues and quantifying their effect on accumulating rework provides data to help DoD control life-cycle costs, mitigate technical risk, and reduce cycle times.

Our research approach includes the following components, which represent aspects of the technical debt analysis workflow and tool chain, shown in the figure above.

• Automatically extracting potential technical-debt issues from issue trackers, using data-mining techniques. We use machine-learning techniques to develop a classifier that inspects development discussions, issues, and comments and identifies how particular issues accumulate over the duration of the project. We apply these techniques to actual transcripts of the conversations of developers to understand how they are talking about technical debt. This tool is helping us to understand the actual artifacts that developers are talking about related to architectural decisions around code and software architecture testing. The goal for this part of our work is to track technical debt through system defects and key features providing evidence for why a designated technical-debt label is essential to managing technical debt. This approach is similar to how we manage vulnerabilities, new features, user stories, and bugs.
• Extracting location of design issues in code bases using information from code analyses and software repositories. Static-code-analysis tool vendors have been working in the commercial space trying to help individuals solve their code policies, but we repeatedly hear that code policies are actually not the bottom-line problem for technical debt. More importantly, how do you identify overarching design concepts that have been violated over and over again? For that we analyze and correlate data from multiple sources (issue trackers, code-repository histories, and static-code-analysis results) to identify the design issues as candidate technical debt items.
• Ranking each candidate technical-debt item (e.g., as based on bug churn, change churn, size of affected code units, and developer feedback). Technical debt is about change--being able to anticipate change and understand its impact. The challenge is to recognize those changes from the data that you have based on the commit history and to discover what you can do objectively without having to talk to multiple people. We are investigating whether there is some correlation in the amount of change that occurs in areas where we find more design issues so that we can help organizations prioritize where to focus their attention and resources.

Benefits of a Software Engineering Practice of Technical Debt

The heuristics we are developing through these analysis techniques are helping to identify examples of shortcuts with consequences that projects should address sooner rather than later. If there is a tradeoff that has significant consequences, the project needs to decide when to do a significant re-architecting effort and how it will be tracked, as well as how to collect the data necessary to communicate to management that it needs to be done. Characterizing such issues in terms of technical debt, especially when supported by solid analytical data, helps to enlist the attention and support of stakeholders and to motivate organizational action.

As we continue to work with engineers and project managers on the phenomenon of technical debt, we are increasingly convinced that management of technical debt deserves a place alongside requirements engineering, software architecture, testing, and design as an essential software engineering practice. All large-scale software-intensive systems have technical debt; whether or not technical debt is being actively managed can be a key differentiator between the success and failure of a project or system.

In the next few months we will write posts that explain each of these areas we have introduced here in further detail. In particular, we will provide

• an overview of our issue classification for identifying technical-debt discussions;
• design-rules analysis foreshadowing candidate technical-debt items and their prioritization; and
• a toolbox of practices for software engineers to manage technical debt.