Software Engineering Institute

Software-intensive acquisition programs continue to experience recurring cost, schedule, and quality issues despite long awareness of these problems, and their persistence indicates they are more difficult to resolve than one might think. As a result, since the 1990s the Carnegie Mellon Software Engineering Institute (SEI) has performed Independent Technical Assessments (ITAs) of mid- to large-sized software-intensive acquisition programs that have experienced problems, conducting interviews and reviewing documents to produce findings and recommendations for corrective action.

To better understand the persistent nature of the problems they encountered, SEI researchers analyzed data collected from 13 unclassified ITAs conducted over five years in a variety of systems. This analysis revealed that while almost all programs face both technical and programmatic issues, the most significant software-related challenges that Department of Defense (DoD) programs face are due to management and oversight concerns. This presentation reviews these “top 10” findings and compares results with prior DoD analyses to examine trends over time.

To explore these problems further, the SEI also looked at those underlying program dynamics that recur across acquisition programs to help identify root causes. Many of the behaviors contributing to the problems could be explained by the presence of “misaligned incentives” (e.g., trading off long-term value for short-term payoff or undermining group objectives to get individual gains) that drive decision making and create poor program outcomes. This presentation explains a set of recurring dynamics that drive the key high-level findings of the ITA analysis and provides qualitative models of each adverse behavior.