Software sustainment is growing in importance as the inventory of DoD systems continues to age and greater emphasis is placed on efficiency and productivity in defense spending. In part 1 of this series, I summarized key software sustainment challenges facing the DoD. In this blog posting, I describe some of the R&D activities conducted by the SEI to address these challenges.

Primary Sustainment Activities
The term software sustainment is often used synonymously with software maintenance. Sustaining software for the DoD, however, requires attention to certain issues (such as operations and training) that are less essential in commercial software maintenance. There are four primary categories of software sustainment activities:

• Corrective sustainment diagnoses and corrects software errors after release.
• Perfective sustainment upgrades existing software to support new capabilities and functionality.
• Adaptive sustainment modifies software to interface with changing environments.
• Preventive sustainment modifies software to improve future maintainability or reliability.

SEI Sustainment R&D
The software engineering research community has devised various approaches to improve software sustainment. For example, tools for detecting software modularity violations help identify eroding design structure (referred to whimsically as "bad code smells") so the code can be refactored to enhance its sustainability. Likewise, intelligent automated regression testing frameworks help ensure that changes to legacy software work as required and that unchanged parts have not become less dependable.

SEI sustainment strategies. Over the past several decades, the SEI has created methods and guidelines for sustaining, migrating, and evolving legacy systems. For example, the SEI has devised strategies for modernizing legacy systems and reusing legacy components in service-oriented architecture (SOA)-based systems. These strategies employ risk-managed, incremental approaches that encompass changes in software technologies, engineering processes, and business practices. In addition, the SEI has created techniques for measuring the effectiveness of software sustainment practices. These techniques can be used to help decision-makers choose a course of continued sustainment, replacement, or selecting which redundant legacy systems to keep and which to retire.

Software product lines. Legacy DoD systems comprise a wide range of software variations, such as network, hardware, and software configurations; different algorithms; and different security profiles. This variation is a key driver of total ownership costs because it impacts the time and effort required to assure, optimize, and manage system deployments and configurations throughout the lifecycle. To manage this variation effectively, the SEI helped pioneer software product lines (SPLs), which have been applied in DoD systems to manage software variation while reusing large amounts of code that implement common features needed within a particular domain. Software sustainment costs (particularly SPL testing) for an SPL-based family of systems can be reduced because reusable components in the SPL are maintained and validated in one place, instead of separately within each application.

Team Software Process. The Team Software Process (TSP) is another approach pioneered by the SEI that managers and engineers can use to sustain legacy software projects. TSP is a team-centric, time-boxed approach to developing software. By using TSP, organizations can better plan, measure, and improve software development productivity so they have more confidence in sustainment quality and cost estimates. The U.S. Air Force and other DoD and industry organizations have applied TSP successfully to manage software sustainment in large-scale weapons systems for the U.S. Air Force, as well as other DoD and industry organizations.

Software architecture. The SEI has also focused extensively on software architecture, which comprises the structure of the software elements in a system, the externally visible properties of those elements, and the relationships among them. SEI research has shown that a solid understanding of software architecture--and the associated methods, infrastructure, and tools--is essential to modify and improve software-reliant systems correctly, dependably, rapidly, and cost effectively throughout the lifecycle. Likewise, successful sustainment of software-reliant DoD systems requires techniques and tools for evaluating and improving software engineer and manager competence with respect to software architecture, including the following:

• Understanding, analyzing, and engineering tradeoffs among system properties (such as performance, dependability, and security) that are critical to achieving desired levels of quality in software-reliant systems as they evolve. These properties are quality attributes that determine system viability throughout the sustainment phase.

• Using architecture-centric practices to elicit quality attribute requirements and to design and analyze changes that are needed throughout sustainment of systems at all scales. Architecture-centric practices can be used to plan system releases and address sustainment challenges pertaining to integration and operational problems due to inconsistencies between system and software architectures.

• Applying architecture principles for systems-of-systems and ultra-large-scale systems to develop architecture design and analysis principles that help document and account for socio-technical interactions, decentralized control, and continuous evolution and sustainment environments where failures/changes are the norm. For example, some soldiers or support staff on the battlefield are capable of creating or modifying existing systems in response to needs that were not anticipated by the designers of the original systems.

SEI assessments, workshops, and red teaming. The SEI regularly works with DoD programs to conduct independent technology assessments, reviews, and "red teams" that apply many of the methods and approaches described above to review the planning for--and conducting of--sustainment of DoD systems. For example, architecture practices such as the Architecture Tradeoff Analysis Method (ATAM) can help DoD programs elicit stakeholder input to identify likely long-term sources of change throughout the sustainment phase.

The SEI's experience helping DoD programs transition from the production phase of acquisition to the sustainment phase of acquisition indicates that the DoD often focuses on how its contracts and contractors will change rather than on how its program offices will need to change. The SEI helps acquisition programs plan for these transitions to sustainment and has collected lessons learned from these activities into software acquisition planning guidelines (including Guideline #4: Software Sustainment). An interesting trend is that DoD programs are increasingly interdependent and interoperable, leading to sustainment interdependencies that require new coordination. To address this need, the SEI developed interoperable acquisition workshops to bring program offices together and draft plans that address sustainment.

Information assurance and software security. Increasing requirements for interdependence and interoperability also yield new challenges for information assurance and software security in legacy systems. In particular, many legacy systems were developed as isolated enclaves. With the advent of net-centric systems-of-systems, however, these legacy enclaves are being interconnected in ways that subject them to vulnerabilities not anticipated by their original designers.

For example, legacy systems programmed in languages like C may be susceptible to buffer overflows that will not occur until they are connected to a network. Moreover, maintainers may not resolve these types of vulnerabilities correctly. They might, for instance, simply add input validation to eliminate a particular path to a buffer overview vulnerability rather than remove the out-of-bounds write.

The CERT Secure Coding Team works with developers and maintainers to eliminate these and other types of vulnerabilities by establishing secure coding standards and processes for conformance testing against these standards. Likewise, the CERT Vulnerability Analysis Team can use an analysis of vulnerabilities based on secure coding rule violations to help handle the response. Legacy software systems can also undergo conformance testing against a secure coding standard in the CERT Source Code Analysis Laboratory (SCALe) to detect and eliminate vulnerabilities before the software is deployed. SCALe has also been used by DoD program offices to access the quality of legacy code to inform modernization versus replacement decisions.

Related SEI Blog Posts
SEI researchers have written several blog postings that are relevant to the sustainment of software-reliant DoD systems. For example, Rick Kazman's posting on Measuring the Impact of Explicit Architecture Documentation focused on understanding the value of documenting software architectures for complex, software-reliant systems. Thorough software architecture documentation helps engineers who sustain DoD software understand how they can refactor, maintain, and update the software without introducing new defects or degrading existing capabilities.

Ipek Ozkaya's posting on Enabling Agility by Strategically Managing Architectural Technical Debt examined how metrics extracted from the code and module structures of software can help repay technical debt, which is a conceptual framework for understanding how and when to defer design choices during the planning or execution of a software project. Repaying technical debt via refactoring and re-architecting is an effective strategy to alleviate architectural dependencies that impact system-wide architectural rework and minimize software decay during sustainment.

Steve Rosemergy's posting on A Framework for Evaluating Common Operating Environments described a framework for exploring the interdependencies among common language, business goals, and software architecture when evaluating the sustainability of proposed software solutions.

We Want to Hear Your Thoughts
This post has just scratched the surface of the solutions that meet the challenges of sustaining software-reliant DoD systems. While the SEI has expertise in methods and tools related to software sustainment, the DoD faces deeper and broader challenges than any one organization (or blog post) can address. We welcome your feedback in the comments section below on ways to improve the technologies and ecosystems needed to sustain DoD software effectively.

Additional Resources:

More information about sustaining software-reliant DoD systems is available below.

To read about software sustainment practices for the DoD, please visit www.stsc.hill.af.mil/resources/tech_docs/gsam4.html, especially chapter 16.

To read about the SEI's work in software architecture, please visit
www.sei.cmu.edu/architecture

To read about the SEI's work with the Team Software Process (TSP), please visit
www.sei.cmu.edu/tsp

To read about the SEI's work in Software Product Lines, please visit
www.sei.cmu.edu/productlines

To read about the SEI's work in system of systems and SOA, please visit
www.sei.cmu.edu/sos

To read about the SEI's work on Ultra-Large-Scale Systems, please visit
www.sei.cmu.edu/uls

To read about the SEI CERT's work in secure coding, please visit
www.cert.org/secure-coding/