Software Engineering Institute

April 9, 2018—When the DoD wants to build a new weapon system or expand an existing system’s capabilities, software increasingly plays a starring role. Software often demands the longest lead time of all system components, and it’s expected to evolve over the entire life of the system. And though software makes many new capabilities possible, it also expands system costs. In 1997, software accounted for about 45 percent of a system’s costs. In 2020, that number is projected to be 80 percent or more.

Because we are now in an era in which software costs can limit military capability, understanding and controlling these costs is critical. The SEI is attacking the problem in several ways, starting with analyses that provide a clearer picture of the current state of software development. The SEI’s David Zubrow led the development of the DoD Software Factbook, which provides an analysis of the most extensive collection of software engineering data owned and maintained by the DoD. “The Factbook is important because it translates raw data into information that is frequently sought after across the DoD, including how much a software system might cost and how long it might take,” Zubrow said. “It provides practical heuristics to estimate and improve program funding and plans going forward.”

Several other lines of research at the SEI are getting at the root cause of rising software costs. “While important, it’s not enough to know costs are going up or to accurately predict the increase,” said Robert Stoddard, who is leading efforts at the SEI to apply causal modeling to large volumes of software development data. “To contain costs, we need to understand which factors drive costs and which factors we can control.” By applying these new modeling and data mining techniques, Stoddard looks to uncover relationships that will provide a basis for better acquisition policy, practice, and management.

Future anticipated rework, sometimes referred to as technical debt, has already been identified as a big contributor to rising software sustainment costs. The SEI is developing a clustering and ranking algorithm and prototype that can analyze and correlate data from multiple sources—including issue trackers, code repository histories, and static code analysis results—to identify the most significant design issues that contribute to technical debt and the associated amount of rework. “Enabling the identification of design issues and quantifying their impact on sustainment and modernization efforts will provide data the DoD needs to control lifecycle costs, mitigate technical risk, and reduce cycle times, all goals of the Better Buying Power initiative,” said the SEI’s Ipek Ozkaya.

Another way of combating rising software acquisition costs is to improve efficiency by making sure contractors cooperate, which is becoming harder to do as government programs move away from the use of a lead systems integrator. Misaligned incentives between contractors or with the program management office can cause wasted effort, lost time, and poor results. The SEI’s William Novak is leading a research effort that uses game theory to frame these situations and then applies agent-based modeling to quantify and evaluate them. “Different types of incentives—financial, strategic, and social—affect contractors to different degrees,” said Novak, “and the combined impact of multiple incentives can be more effective across a range of contractors in changing their behavior.” This research allows the simulation of candidate incentive mechanisms in the context of an acquisition program model, to see which combinations of contractor incentives can produce the best acquisition outcomes. This approach, which is being piloted, shows promise for solving some of the incentive problems that can plague acquisition performance.

The following publications offer more information about SEI research efforts to control software acquisition costs:

• Department of Defense Software Factbook, http://resources.sei.cmu.edu/library/asset-view.cfm?assetid=502651
• "Why Does Software Cost So Much? Toward a Causal Model," http://resources.sei.cmu.edu/asset_files/presentation/2017_017_001_495821.pdf

This article originally appeared in the 2017 SEI Year in Review.