Software-enabled capabilities are essential for our nation's defense systems. I recently served on a Defense Science Board (DSB) Task Force whose purpose was to determine whether iterative development practices such as Agile are applicable to the development and sustainment of software for the Department of Defense (DoD).

The resulting report, Design and Acquisition of Software for Defense Systems, made seven recommendations on how to improve software acquisition in defense systems:

1. A key evaluation criterion in the source selection process should be the efficacy of the offeror's software factory.
2. The DoD and its defense-industrial-base partners should adopt continuous iterative development best practices for software, including through sustainment.
3. Implement risk reduction and metrics best practices in new program and formal program acquisition strategies
4. For ongoing development programs, task program managers and program executive officers with creating plans to transition to a software factory and continuous iterative development.
5. Over the next two years, acquisition commands in the various branches of the military need to develop workforce competency and a deep familiarity of current software development techniques.
6. Starting immediately, the USD(R&E) should direct that requests for proposals (RFPs) for acquisition programs entering risk reduction and full development should specify the basic elements of the software framework supporting the software factory.
7. Develop best practices and methodologies for the independent verification and validation (V&V) for machine learning (ML).

All seven recommendations are important for the DoD. The U.S. Congress therefore mandated that the DoD implement them, in the 2019 National Defense Authorization Act (NDAA), section 868. Specifically, the 2019 NDAA states, "Not later than 18 months after the date of the enactment of this Act, the Secretary of Defense shall . . . commence implementation of each recommendation submitted as part of the final report of the Defense Science Board Task Force on the Design and Acquisition of Software for Defense Systems."

As this post details, while some of the recommendations are particular to DoD's practices, two of them— the modern software factory and V&V for ML— stand out in their importance for software engineering research.

The Modern Software Factory

In a recent SEI Podcast, I stated that when we talk about a software factory, we are not referring to the concept from decades ago: developers working in an environment that was more like a factory. The software factory of today focuses on building an environment of tools and practices around programmers to help them work creatively and effectively. Creating this type of environment includes support systems to help manage configuration control and automated testing to detect software flaws that adversaries may exploit as vulnerabilities.

Software-factory-focused environments include nearly continuous feedback loops. Feedback can improve the productivity of software programmers, teams, processes, and progress. Through feedback loops, programmers and teams can evaluate and re-evaluate how well they are moving toward the goal of timely deployment of affordable, trustworthy, and robust software-enabled capabilities.

These environments also feature the testing of software upon its check-in to source code repositories. In this way, developer teams test software at least daily, finding issues that they can remediate more quickly and with less cost.

Industry's success at creating these environments provides a model for the DoD, which has traditionally followed requirements-based approaches. The challenge of a requirements-based approach is that developers often do not understand the full scope of those requirements at the start of a software development project. What's more, these requirements can change over the course of the software development lifecycle. However, the software factory's environment of continuous integration allows users to interact with the system as it is being developed and play a role in determining whether developers are moving the in the correct direction.

Independent V&V of ML

One promising technology to address our control over ML systems is runtime assurance enforcers that can recognize when the software's behavior is outside established boundaries and trigger change to bring it within those boundaries. This would allow the DoD to use ML techniques in increasingly complex non-deterministic systems that interact with the physical world (e.g., aircraft).

Active research at the SEI in runtime assurance (RA) techniques (see here, here, and here) starts with the insight that current RA approaches have limitations, for which research is needed. RA technology today is applied to single domains (i.e., types of analyses), relies on unverified and potentially inconsistent enforcers that can be circumvented easily, and requires system-wide re-verification when an enforcer is changed, added, or removed.

The SEI research seeks to address those challenges with tools and techniques to formally verify correctness of an enforcer implementation against its policy, combine multiple enforcers and resolve any inconsistencies between their behavior, deploy enforcers so that they cannot be circumvented by an attacker, and verify that the enforcers react on time to prevent physical consequences (e.g., aircraft crashes).

Proposed ML research at the SEI involves the development of new algorithms that will make ML systems behave more predictability and the development of a tool we call FAITH, the Fast AI Test Harness. In this way, developers with limited ML experience would be able to verify the quality attributes of ML components within a modern software development process, enabling faster and cheaper development of AI capability.

Looking Ahead

The organizations that embrace the concepts of the software factory and independent V&V for ML rely heavily on the individual work team that is actually building the software. We need to adjust our systems to trust those teams of experts and listen to them, understand their problems, and give them the creative space to do what they do best: develop and deploy software systems.

Additional Resources

Read the Defense Science Board report Design and Acquisition of Software for Defense Systems.

View the SEI Podcast The Role of the Software Factory in Acquisition and Sustainment.