Software products are often used for two decades or more. Several researchers have shown the cost of maintenance and sustainment ranges between 40- and 80 percent of the total lifecycle cost with a median estimate near 70 percent. Sometimes executives have asked, Why does software sustainment cost so much? This blog turns the question around to ask, Can we get better value from our continuing software investment? Of course, the answer is affirmative. We can examine changes in mission objectives, technology and environment to see how to position our product for the best use and value just as we do with other products.

This blog post outlines a 5-step process for release planning. There are suggestions for intake of change requests, measurement of value and developing the plan. It supports DoD sustainment policy regarding 12-month budgets for Operations and Maintenance (O&M) and 24-month budgets for Research and Development (R&D). The basic steps are as follows:

1. Classifying change requests by mission goal
2. Measuring the gaps
3. Budget allocation for O&M and R&D funds
4. Defining selection sets
5. Negotiating the final change set

This is the third post on sustaining software product lines; however, most of this particular post is applicable to any software intensive system. The portions specific to product lines will be appropriately identified.

Classifying change requests by mission goal

Every system will have multiple users or stakeholders. Each stakeholder has a unique perception of system value. In classifying change requests, we must associate the change request to the stakeholder. For example, a logistics person is not likely to be concerned with a real-time performance measure where it is essential to a pilot and aircraft safety. Typically at least three viewpoints are needed for classification: operations, logistics, and command. Value then can usually be captured by one of the following four categories:

• Improving Mission-Capable Availability. Product defects (bugs) cause users to waste time solving and reporting the problem. Usable time improves by eliminating the defect.
• Restoring Product Functionality. A function that has been working properly can be lost or limited by a change in an external system. Effort is require to restore the mission value. The more systems are interconnected, the more frequently this problem occurs.
• Improving Mission Value and Performance. New functions are added to expand mission context. Speed and other performance improvements such as usability also add value to mission.
• Mitigating Future Risk and Improving Future Product Sustainment. This area deals with risks based on cyber attacks and safety concerns. Investing in design changes may be executed as a bet on the future before requirements are fully articulated.

Measuring the Gaps

As detailed in our first post, we next attempt to devise an assessment of value related to the four goals described in the previous section. The value assessment will help with the prioritization process.

Corrective: Fixing Bugs or Software Defects

The primary purpose of fixing bugs is to improve mission capable availability. It is difficult to measure software availability, but we can measure improvement in availability. A useful measure for prioritization is the risk priority number (RPN). A bug effectively compromises the expected performance for some period of time. To calculate the RPN, we observe frequency of occurrence, lost time associated to the occurrence, and the difficulty of detecting the occurrence. The product of these three numbers gives us a priority for the most problematic bugs. Once the selected bugs are fixed, report improved availability based on the sum of reduction in lost time multiplied by the frequency of occurrence for each bug.

Adaptive: Maintain existing functionality in the face of external changes

The purpose of adaptive change is to maintain the product performance in the face of changes to other systems and products. Sources of change may be the operating system, the platform, an external communications link, and many others. The external change may increase operational effort (cost), reduce mission performance (lost value), cause support problems (cost), or other consequences. Thus we can measure loss of value by multiplying frequency of use times cost (or lost value) to set the priority of change.

Perfective: Enhance product to improve performance or add functionality

Functional changes and performance enhancements are intended to improve the value-in-use. Typical goals include improving mission performance, new mission capability, or reduced learning curve. Sometimes operating costs may be reduced. To get a total value proposition we must multiply a frequency of use by a value for the improvement. This measure must be estimated. Check with multiple stakeholders for an estimate. Err on the high side for the estimate.

Preventive: Reduce risk posed by cyber threats; improve maintainability and support

Preventive measures are justified by making a bet on the future. For example, DoD regulation requires an information assurance (IA) activity to reduce cyber threats. Such bets should be understood in the same context as buying an option on a stock or purchasing a commodities futures contract and must always have a limited time frame. Efforts to reduce technical debt or improve maintainability also fall in this category. Since IA must be included in the budget, a modest additional allocation can be added for other preventive work. Reason by proposing a future scenario with duration of 3-to-5 years, and set a value on the realization of the outcome. Rank the proposed changes by value, but reduce the value by dividing by the number of years into the future before value will be realized.

Based on the four separate measures, prioritize changes within each type separately. Prepare for the release plan by estimating the high ranking items. If multiple stakeholders are involved, identify the stakeholder who has the most to gain before ranking.

Budget Allocation for O&M vs. R&D

Delivery of new releases must occur with sufficient frequency to address both the changing mission and the effects of changes to external systems that would compromise product performance. Typically, the resulting cadence is 6-to- 24 months between releases. A program that uses a release cycle longer than 36 months may find requirements changing faster than its ability to deliver products.

In DoD sustainment efforts, the program determines two budget categories for funding the product release:

• O&M budgets are for 12-month periods.
• R&D budgets cover a two- to three-year funding cycle.

The Army has spent significant time analyzing the relative costs of O&M vs. R&D. In 2012, the results of the Army study were presented at the 2012 PSM Conference (Slide 62) in the following chart:

The dark purple bars at the base of the chart represent a typical annual O&M budget. Hence, the light blue portion represents a temporary boost in the O&M budget to fix extra bugs, make adaptive changes, and other modest improvements. The lavender and pink bars represent the longer cycle for the R&D budget used to enhance the product and make larger adaptive changes. The pink, peak values represent significant upgrades (sometimes 7x-to-10x the cost) over the O&M annual budget.

Each bar represents a separate release, but the cycle time is unknown because this chart reflects information collected across multiple programs. It is interesting to note that the major upgrades occurred, on average, every seven-to-eight releases. The period of time to introduce seven releases is likely to be in the range of three and a half- to-ten years. This longer period suggests that the occurrence of a technology change or two may drive the larger update.

The O&M annual budget is weighted heavily to corrective actions and adaptive fixes. Only modest perfective changes can be covered within O&M funding. The DoD requires Information Assurance, a preventive type change, with every release.

The R&D budget is intended to provide the adaptive, perfective, and preventive changes to improve functionality, product performance, and program future performance. The budget pie will reflect the difference by allocating a smaller amount to corrective changes and place more emphasis on the perfective changes.

As illustrated in the pie chart below, the proposed budget should be allocated by change types according to mission strategic needs. If a program office determines that the mission goals must be expanded, then more weight should be applied to perfective changes. Some allocation to preventive changes is warranted if future technology changes are anticipated. This approach brings the operations, training, and logistics communities into the planning process sooner and supports agile practices.

A release plan involves selecting a set of changes that fit within the proposed budget amount. Various operational, logistics, and support stakeholders may rate value and frequency of use differently. The differences may change some rankings of specific change requests. These selection sets will be presented and discussed in the decision process.

Since changes to core components of a product line may affect multiple products, there must be some consensus building and coordination of releases. Lapses in coordination and configuration management cause increasing complexity of versioning for multiple customers. Eventually, this becomes costly and increased delays between releases. Two distinct goals exist for product lines:

• keeping the core components as similar as possible across all products
• incorporating some portion of new functions and technology from individual products into the core components when possible and beneficial

Each of these goals warrants some consideration during budgeting.

Decisions that involve multiple attributes of value and multiple stakeholders require formal decision-making methods. The presentation of multiple decision sets addresses the complexity of the decision. These methods are often abbreviated as MAUT (multi-attribute utility theory) and MCDM (multi-criteria decision methods). The methods are somewhat complex but do converge when traditional multi-voting methods do not. For a description of such decision methods, see "An Analysis of Multi-Criteria Decision Making Methods" from International Journal of Operations Research Vol. 10 No. 2 (Page 56). Guenther Ruhe's book Product Release Planning: Methods, Tools and Applications is also a good source.

The decision process for maintaining the core is the same but must include the representatives of the core as an independent stakeholder. The inherent quality of the core is probably higher than the non-core systems. Therefore the core should see a lower percentage of budget allocated to corrective changes. Adaptive, perfective, and preventive changes should receive an increased portion.

Selecting the change set for core components will also involve discussion of whether to incorporate certain product changes into the core, but the total number of changes is likely to be relatively small.

The five steps described in this post are useful individually. Step one, at intake time, adds classification data to the change request, so it can be evaluated or measured in step two. Step three sets a budget based on the strategic needs of the program bringing the executives into the conversation early. Step four, design sets, establishes the conversation between viewpoints based on value, and step five helps to de-politicize negotiations.

Measurement is key. By including stakeholders in the development of measurement everyone gets a more useful picture of value. People with a better understanding of value are more likely to support a cost increase. Hence we have moved the conversation away from, Why does it cost so much?

Additional Resources

Read the first post in this series, Decisions for Sustaining a Software Product Line.

Read the second post in this series, Governance of a Software Product Line: Complexities and Goals.

Read the series of blog posts by Mike Phillips on Efficient and Effective Software Sustainment.

View the podcast Software Sustainment and Product Lines by Mike Phillips and Harry Levinson.