The Department of Defense (DoD) has become deeply and fundamentally reliant on software. As a federally funded research and development center (FFRDC), the SEI is chartered to work with the DoD to meet the challenges of designing, producing, assuring, and evolving software-reliant systems in an affordable and dependable manner. This blog post--the first in a multi-part series--outlines key elements of the forthcoming SEI Strategic Research Plan that addresses these challenges through research and acquisition support and collaboration with DoD, other federal agencies, industry, and academia.

The DoD's Growing Reliance on Software

In a recent SEI blog post, Doug Schmidt noted that the percentage of avionics specification requirements involving software control rose from about 8 percent for the F-4 in 1960 to 45 percent for the F-16 in 1982. By 2000, the percentage of software-related requirements had grown to 80 percent for the F-22. It is reported that by 2006, fully 90 percent of the F-35's requirements had become software- related. Vehicles are also increasingly software-reliant. In the case of commercial automobiles, for example, it is reported that fully half of the functional features are, in one way or another, software reliant.

This growing reliance on software now affects most aspects of DoD systems, including mission data systems, radar and sensors, flight and engine controls, communications, mission planning and execution, weapons deployment, test infrastructure, program lifecycle management systems, and software integration labs. This reliance on software also affects back office or "enterprise" systems that may, at first blush, seem relatively more routine or precedented, but which can also present significant challenges.

For example, the US Air Force has faced well-documented difficulties in developing the Expeditionary Combat Support System, which was recently cancelled after spending seven years and $1 billion on a logistics management system that had "negligible" capability. The challenge is not just derived from the extent to which software is used to manifest features, but also by the growing role of software as an integrating fabric that links systems together into system-of-systems (SoS) configurations that can span across multiple mission domains, including traditional back-office functions.

The point of these observations is that the DoD has become deeply software reliant. In other words, software is a strategically significant building material for major systems of all kinds. The technology and practices associated with software are also evolving rapidly and at a pace that is increasing rather than diminishing. Indeed, it is foolish and, in fact, dangerous to think that software technology is somehow approaching a plateau in role and capability.

The lack of a plateau is evident in the details of recent developments in technology and practice--examples include big data frameworks, machine learning tools, advanced framework-and-apps architectures, safe programming languages, cyber-physical architectures, resilient architectures, software-assurance analytics, design modeling, cost estimation, agile practices, and the like. As software-related technology evolves, organizations that are deeply reliant on software capability must take an active role in engaging with the technology and the technology ecosystem.

The SEI has a critical role in working with the DoD to adapt to rapid changes in technology in diverse mission contexts, including the development, assurance, and sustainment for cyber-enabled missions. The SEI's role also includes addressing the challenges of managing dimensions of risk, cost, schedule, quality, and complexity for software-reliant systems.

To address the challenges outlined above, the SEI Strategic Research Plan contains four elements:

• architecture-led incremental iterative development (ALIID)
• designed-in security and quality (evidence-based software assurance)
• a set of DoD-critical component capabilities relating to cyber-physical systems (CPS), autonomous systems, and big data analytics
• cybersecurity tradecraft and analytics

Our understanding of these challenges derives from several sources, including our own experience and reports from our many collaborators in the DoD, other government agencies, industry, and academia. Our strategic plan also builds on national studies from the Defense Science Board and, most recently, the National Research Council's study titled Critical Code: Software Producibility for Defense and the White House Comprehensive National Cybersecurity Initiative (CNCI).

These strategic elements are intended primarily to drive the research dimension of the SEI portfolio of activities, but they also inform, in an essential way, the visionary features of SEI collaborative work with DoD and other government partners. The technical strategy is intended to make explicit the more important visionary features in SEI activity. The remainder of this blog posting describes the first strategic element.

Architecture-Led Incremental Iterative Development (ALIID)

The goal of ALIID is to enable iterative and incremental development of highly capable and innovative systems with acceptable levels of programmatic risk. While small-team agile methods are well established in most sectors, the challenges of scaling up to larger efforts, including SoS, remain profound. With ALIID (or, "agile at scale"), SEI researchers are working to enable larger-scale iterative and incremental development by the DoD.

Diverse experience in industry suggests that this type of development is almost always accomplished on the basis of early commitment to effective architectural structures. There are two principal reasons for this:

1. Architectural structures have enormous influence on quality and security outcomes for systems. Without early consideration of quality and security attributes, it may be infeasible to "bolt them on" later in a process. Indeed, for many categories of systems, early architectural decisions can be a greater influence on success than nearly any other factor.
2. Architectural structures strongly influence the work breakdown structure for major software-intensive systems. The ability to achieve an effective granularity of effort--breaking larger tasks into feasible sub-tasks--is an architecture-based outcome at nearly every level of scale.

It is easy to see the benefits of the "framework and apps" model ubiquitously adopted in the commercial sector for mobile devices, web applications, GUI development, and other categories of systems. This model is a compelling example of architecture-enabled scale-up. At the SEI, we aspire to that same approach in defense software; for example, in the development of principles of design for systems-of-systems. Military leaders have described a shift from platform-focused approaches to payload-focused approaches. This shift--which is premised on improving agility while lowering costs and risks--is enabled by appropriate architectural concepts.

While there are many areas of technical development that are needed to aggressively advance this vision, it is nonetheless possible in many domains to advance development approaches based on these concepts. The evidence of this feasibility is the widespread adoption of architecture-led approaches in the commercial sector for products, software as a service (SAAS) services, and the like. Indeed, many commercial and in-house government development organizations already employ practices that are increasingly consistent with the ALIID vision. It is a major challenge, however, to adapt these ideas to the kind of arm's length engagement with development organizations characteristic of DoD major defense acquisition programs.

Here are four areas of particular emphasis in our ALIID program:

• Architectural structures and practices. This area includes identification of concepts of operations for SoS, ultra-large-scale systems, and the like. This area also includes the development of practices for early validation, to assist in getting reliable early assessment of quality and security outcomes associated with particular architectural choices.
• Measurement and process models for cost, progress, and engineering risks. Any approach based on incremental and/or iterative development must incorporate effective practices for estimating costs, progress, risks and then use these practices to reward developers for value earned. Cost estimation techniques, for example, could be enhanced to evaluate both estimated mean values and variances and support ongoing re-estimation as a means to assess progress. Risk-reduction practices such as prototyping or modeling and simulation may lead, for example, to both reduced "cost to complete" and also narrowed variances (i.e., greater confidence that the estimates are accurate).
• Incentives and acquisition practices. How can acquisition practices be developed to build on advances in measurement and process models that feature a structuring of incentives that enables government and contractors to collaborate effectively in development of architectures and in iteration at scale?
• Software sustainment. The reality of many sustainment efforts is that they are really supporting a continual evolution of systems. Planning for continuous evolution, for example in the form of identifying and separating dimensions of variability, is a necessary feature of architectural design.

What's Next

The next blog posting in this series focuses on the three remaining elements in the SEI Strategic Research Plan, which are (1) designed-in security and quality (evidence-based software assurance), (2) a set of DoD critical-component capabilities relating to cyber-physical systems (CPS), autonomous systems, and big data analytics, and (3) cybersecurity tradecraft and analytics.

The SEI Strategic Research Plan is designed to ensure that the SEI conducts high-quality and high-impact work that benefits the DoD by identifying and solving key technical challenges facing current and future DoD software-reliant systems. This strategy itself undergoes continual evolution and improvement; the broad range of SEI engagements enable us to continually refine the strategy as the technology advances, the mission evolves, and our understanding improves. We welcome engagement from our partners and stakeholders in the improvement and refinement of this strategy.

Additional Resources

To download the report Critical Code: Software Producibility for Defense, please see www.nap.edu/catalog.php?record_id=12979

To download the Report of the Defense Science Board Task Force on Defense Software (2000), please visit https://apps.dtic.mil/docs/citations/ADA385923

To view on-demand presentations from the SEI Agile Research Forum, please visit https://www.sei.cmu.edu/publications/webinars/index.cfm and search for "Agile Research Forum"