Posted on by Technical Debtin
By Stephany Bellomo
Senior Member of the Technical Staff
Software Solutions Division
What is technical debt? Why identify technical debt? Shouldn't it be captured as defects and bugs? Concretely communicating technical debt and its consequences is of interest to both researchers and software engineers. Without validated tools and techniques to achieve this goal with repeatable results, developers resort to ad hoc practices, most commonly using issue trackers or backlog-management practices to capture and track technical debt. We examined 1,264 issues from four issue trackers used in open-source industry and government projects and identified 109 examples of technical debt. Our study, documented in the paper Got Technical Debt? Surfacing Elusive Technical Debt in Issue Trackers, revealed that technical debt has entered the vernacular of developers as they discuss development tasks through issue trackers. Even when developers did not explicitly label issues as technical debt, it was possible to identify technical debt items in these issue trackers using a classification method we developed. We use our results to motivate an improved definition of technical debt and an approach to explicitly report it in issue trackers. In this blog post, we describe our classification method and some implications of tracking debt for both practice and research.
Identify and Classify Technical Debt
To understand how software developers use issue trackers to communicate technical debt, we conducted an exploratory study of four issue trackers: the Chromium and CONNECT Health IT Exchange open-source projects and two government IT projects. We found that when determining whether an issue represents technical debt, experts apply unspoken and informal rules and practices. We observed this in a manual examination of 1,264 issues from the four issue trackers, as well as from the technical debt literature (for example, Guo 2011, Li 2014, Potdar 2014, and Ernst 2015). Our goal in developing a classification method for technical debt is to make identification of it explicit and to enable repeatable classification of issues. Here we summarize the key decision points in our process:
Our classification method therefore refines the definition of technical debt as "design work related to software units that may carry present or anticipated accumulation of extra work." We use the term "design work" in the general sense to mean that some aspect of the problem is caused by the structure or quality of the software design,(and so we include implemented system design under this definition). Figure 1 illustrates our current understanding of the categories and a flow tree for classifying technical debt. The approach we used to classify technical debt in issue trackers can also help practitioners define and identify technical debt in their development projects.
Figure 1: How we worked through the process of identifying and classifying technical debt.
Issue trackers serve as an entry point for communicating technical debt because developers use them to manage task priorities. Survey feedback from developers indicated that even when they include details about technical debt in an issue tracker, they may not prioritize paying back the debt or they may address its symptoms but not the underlying causes. Our findings offer some practical improvements to make technical debt more visible.
There are standards (such as ISO/IEC 25010:2011) for providing bug reports with enough information so they can be reproduced and fixed. These essential properties are encoded in predefined fields in issue trackers. These fields are necessary but not sufficient for describing technical debt. Recent research (such as Zazvorka 2013 and Li 2014) on technical debt has offered templates for reporting technical debt. This work has similar goals to our work, but the templates use concepts driven by the financial analogy, such as estimating principal and interest, that do not overlap with the daily tasks of developers.
Our analysis demonstrates that technical debt becomes concrete when it relates to software units, as opposed to software process artifacts, such as requirements or documentation. This refined scope helps developers and their stakeholders understand technical debt as the accumulation of technical debt items associated with a software-reliant system.
A technical debt item is a single element of technical debt that connects a set of development artifacts. It has consequences for the quality, value, and cost of the system. Likewise, it is triggered by causes related to process, management, context, and business goals. A technical debt item should be described using the properties in Table 1, which are based on the concepts for classifying technical debt (shown in bold).
Introducing these properties can help developers understand tradeoffs and the longer-term consequences of technical debt. It can also help them make a case for additional resources when communicating to management. We suggest that developers use these properties to write issue descriptions that make technical debt visible and to increase the degree of automation possible in classifying issues as technical debt. Table 2 shows an example of organizing the issue text according to these properties from a CONNECT issue.
The properties can also help developers parse the issues and identify what is ambiguous or missing. For example, without explicit information about debt accumulation, the issue cannot be properly classified nor the tradeoffs understood. Technical debt should foster dialogue between business and technical actors. Classifying issues as technical debt allows developers to justify budgeting project resources for paying back debt similarly to allocating a budget for fixing defects, instead of continuing to pay the ongoing cost of addressing only the symptoms.
After reviewing the results of our study, we hypothesize that developers can use automated text analysis and machine-learning techniques to discover technical debt issues more systematically. To explore this hypothesis, we ran a search on a sample set of the issues we examined for the following words: duplicate, custom, workaround, inconsistent, hack, legacy, rewrite, cleanup, refactor, and refresh. We hypothesized that we would find a statistically significant difference between the percentage of issues that contain one of these key words and are technical debt when compared with the set of issues that contain a key word but were not classified as technical debt. We observe that 67 percent of the issues contained one of the key words and were classified as carrying debt. Only 8 percent fell in the latter category. These findings suggest that automated word searches of key concepts related to technical debt hold promise, but more experimentation is needed with larger data sets.
Assessing accumulation was one of the biggest challenges we observed with systematically classifying technical debt issues in this study. Such an assessment is hard for two reasons:
Our findings indicate several future research opportunities, and our plans include the following:
We welcome your feedback on our classification for technical debt and technical debt item description in the comments section below. If you would like to try out the classification approach, the decision flow description from our study can be found in a classification guidance accessible on the SEI website. This study is part of a wider SEI effort on technical debt. If you would like to collaborate on a study of managing technical debt, please get in touch.
Read about our recent work on managing technical debt in the following publications: