Posted on by Software Sustainmentin
Software sustainment involves coordinating the processes, procedures, people, information, and databases required to support, maintain, and operate software-reliant aspects of DoD systems. The 2011 book Examination of the U.S. Air Force's Aircraft Sustainment Needs in the Future and its Strategy to Meet Those Needs states
The Air Force is concerned that the resources needed to sustain its legacy aircraft may increase to the point where they could consume the resources needed to modernize the Air Force.
With millions of lines of code riding on aircraft and automobiles, the cost of software sustainment is increasing rapidly. Several studies show that the cost of sustainment is already as much as 70 percent of the total cost for the life of the software. All the armed services face similar challenges, including deciding how to improve the efficiency and productivity of sustainment organizations and how much should be invested in these improvements. This blog post describes an SEI research initiative aimed at developing an economic model to help anticipate costs and postpone the potential tipping point when sustaining current products is less attractive than replacing legacy systems.
Balancing Stakeholders and Resources
The software sustainment problem is particularly complex for the Department of Defense (DoD) because funding decisions involve an understanding of tensions between three different perspectives:
Our research is motivated by the need to help the DoD make decisions about allocating resources between sustainment work (supporting the warfighter) and improving the performance of sustainment organizations in ways that optimize long-term value to the armed services. Performance improvements are needed in many situations, including
This SEI research initiative is developing an economic model to support decisions about allocating investments in various performance improvement alternatives. The model will analyze various factors, such as demand for sustainment, capacity of an organic workforce to perform sustainment activities, as well as timing of funding in terms of its impact on long-term costs and readiness of aircraft fleets.
I, along with fellow researchers Sarah Sheard, Andrew Moore, William Nichols, and Mike Phillips, have spent the last year exploring several issues related to software sustainment. Part of our work included developing a systems dynamics model to study how funding decisions and the timing of implementation of changes in sustainment organizations affect the performance of both sustainers and warfighters. We theorized that small differences in funds and timing can have large impacts on performance. This type of system dynamics model uses stocks and flows to represent sustainment performance over time.
Foundations of Our Approach
Jay Forrester, a professor at Massachusetts Institute of Technology, founded system dynamics in the 1950s. Systems dynamics modeling studies the changes in many interrelated variables. Since its inception, system dynamics has been used as a modeling approach in the study of economics and organizations.
With many factors changing simultaneously, simpler economic models (such as return-on-investment and net present value) are insufficient. The interaction of the many inputs to sustainment work can cause emergent effects, such as a sudden and dramatic change in ability to meet demand. Through modeling and analysis research, we are looking for the minimum amount of data that can be used to forecast a sudden and dramatic change (which is also known as a "tipping point"). Forecasting the tipping point gives decision makers time to take action before a problem becomes intractable.
We seek to answer the following questions through our research:
Through this approach, we aim to help DoD acquisition programs better plan their financial investments to ensure long-term software sustainment and deliver the best value for taxpayer dollars.
While the construction of a model that exhibits the expected changes in the performance of both mission and sustainment is a necessary step, much of our research will focus on sources of data. Real data collected from real programs will be needed to calibrate the model for making real decisions, including
The Systems Dynamics Model
The basic goal of a simulation model is first to represent the normal behavior of a system and then stimulate it with a new input to see how the responses change.
The model that we have developed represents the behavior of the different players in the sustainment process, including the warfighter, the technical capability of the sustainment organization, and the capacity of the sustainment organization to deliver the work. We are testing the system response to various scenarios, such as
Our current model of sustainment consists of five basic process loops:
Each scenario outlined above entails several decisions and stimulates response curves from the model. The response curves help decision makers forecast how deferring decisions or reallocating resources affect both warfighters and sustainment organizations. We will consider our systems dynamic model "good" if decision makers believe they are able to make faster decisions and if the data from the model makes it easier to get sponsor support for the decisions.
Initial Findings and Future Work
A recent study by the U.S. Army provided us with information confirming the importance of understanding the cycles of demand for sustainment work. From that study, we identified three patterns of release in the software sustainment lifecycle:
The modernization release is often a response to some technology change. The timing (at least four years) has an interesting correlation. Moore's Law suggests the cost-performance of processors doubles every 18 months. A modernization release every four years skips about two processor generations. A modernization release every seven years skips about six generations.
Modernization releases require the sustaining organization to develop new practices and tools and to retrain some existing personnel. Funding for these internal changes is often hard to justify and may be delayed. These delays, in turn, can cause the sustaining organization to fall so far behind the technology curve that a new acquisition contract is often required. If a contractor does win the development contract, the organic sustainment organizational capability may decline while the bulk of funds go to the contractor.
Other organizations have studied various aspects of the sustainment problem. The Army examined costs of software sustainment and the work breakdown structure of sustainment as it is currently performed. Jack McGarry, author of the book Practical Software Measurement: Objective Information for Decision Makers, charted for the U.S. Army the tremendous variability in the demand for sustainment work, with the cost of the largest releases up to 10 times that of the smallest releases. The frequency of the largest releases appears to track to large-scale technology change, occurring about every four to seven years. In that period of time, the speed of processors increases by nearly 10 times, and the number of transistors on a chip also increases by 10 times. This finding suggests that technology change is an important variable in our systems dynamic model.
The research we've conducted thus far has revealed that our system dynamics model exhibits the expected and observed behavior of product sustainment. The model, however, has not yet been calibrated to apply to a real situation. We are actively seeking an opportunity to study a real sustainment program and to collaborate on calibrating the model. We believe that data collection will not be hard to implement and the data potentially can be used for other purposes by sustainers.
If you are interested in collaborating with SEI researchers on this initiative, please send an email to firstname.lastname@example.org, or leave contact information in the comments section below.
We welcome your feedback on our research and look forward to hearing if you'd like to help us achieve our project goals. Please leave feedback in the comments section below.
To read An Examination of the U.S. Air Force's Aircraft Sustainment Needs in the Future and its Strategy to Meet Those Needs by the National Academies Press, please visit
To read the SEI technical report, Sustaining Software-Intensive Systems, please visit
To see a presentation on Sustaining Software Intensive Systems -- A Conundrum please visit http://www.dtic.mil/ndia/2005systems/wednesday/lapham.pdf
To read Business Dynamics: Systems Thinking and Modeling for a Complex World by John Sterman, please visit https://www.amazon.com/Business-Dynamics-Systems-Thinking-Modeling/dp/007238915X
To view the proceedings of the 2012 PSMSC User Group, please visit
To read the Air Force Scientific Advisory Board report, Sustaining Aging Aircraft, please visit http://oai.dtic.mil/oai/oai?verb=getRecord&metadataPrefix=html&identifier=ADA562696