Software is the principal, enabling means for delivering system and warfighter performance across a spectrum of Department of Defense (DoD) capabilities. These capabilities span the spectrum of mission-essential business systems to mission-critical command, control, communications, computers, intelligence, surveillance, and reconnaissance (C4ISR) systems to complex weapon systems. Many of these systems now operate interdependently in a complex net-centric and cyber environment. The pace of technological change continues to evolve along with the almost total system reliance on software. This blog posting examines the various challenges that the DoD faces in implementing software assurance and suggests strategies for an enterprise-wide approach.

Over the past decade, the DoD has been increasingly challenged to develop, implement, and continually evolve comprehensive enterprise software assurance policies, guidance, and infrastructure capabilities to anticipate the impact of technological change and the dominance of software. As a result, there is significant uncertainty about DoD's institutional software assurance capability to achieve the level of confidence that software functions as intended and remains free of vulnerabilities across legacy systems and systems being acquired. Although the DoD has taken various initiatives to examine software assurance issues and craft comprehensive assurance strategies, this uncertainty and its impact on warfighter performance persist as a major problem.

Congressional Concerns about Software Assurance

Congress has become increasingly concerned about the DoD's progress in implementing its software assurance strategies and capabilities. For example, the National Defense Authorization Act (NDAA) for FY 2011 (Section 932) mandated that the Secretary of Defense develop and implement a strategy for assuring the security of software and software-based applications for all covered systems by no later than October 1, 2011. The FY 2012 NDAA was the first law to provide strong policy guidance to secure both new and legacy software from attack throughout the software development lifecycle.

More recently, Section 933 of the FY 2013 NDAA directs the DoD to implement a baseline lifecycle software assurance policy. This policy requires the use of appropriate, automated vulnerability analysis tools in computer software code. These tools are intended for using during development, operational testing, and operations and sustainment phases, through retirement for specific types of systems.

The Scope of the DoD Software Assurance Challenge

The DoD faces several challenges in creating and implementing an institutional software assurance capability for systems in acquisition, as well as legacy systems. Such a capability must be multidimensional (including policy, guidance, process, practice, tool, and workforce concerns) to encompass reliability, security, robustness, safety, and other quality-related attributes critical to achieving the level of confidence that software functions as intended and is free of vulnerabilities. This software assurance capability must be broad in scope and rigorous in its discipline, but with enough flexibility and adaptability to address the challenges discussed below.

Software assurance policies and capabilities should span a spectrum of system and use-case challenges. These policies and capabilities need to account for the fact that the DoD maintains a diverse and complex systems portfolio including business and enterprise network information systems; modeling and simulation; automated tools for design, test, and manufacturing; complex C4ISR; and autonomous, semi-autonomous, and manned systems.

This diverse and complex systems portfolio must operate in a net-centric cyber environment where each system acts as an information node in one or more networks. Although DoD systems operate in this environment, they are overwhelmingly acquired as individual systems, rather than being managed in a systems-of-systems or portfolio context for acquisition and sustainment. More than 100 systems alone are classified as major defense acquisition programs. The software assurance use case challenge also spans legacy systems that may be in operation and sustainment for multiple decades. During their lifetime, these systems undergo continuous evolution to meet warfighter performance needs and to continue to operate in the net-centric environment.

Another issue that must be considered in formulating software assurance capabilities and policies is the limited visibility at the DoD level of the size and demographics of the total software inventory. As a result, there is no ongoing analysis of this ever changing inventory to inform software acquisition and sustainment enterprise decisions as well as assurance policy, program, and investment decisions.

Further complicating matters is the fact that the software supply chain is not well integrated vertically in terms of prime-to-subcontractor flow-down of software assurance requirements, independent verification and validation (IV&V), and test and evaluation to enable consistent visibility, traceability, and integrated testing.

The myriad of assurance type acquisition requirements creates a confusion of stove-piped policies that sustainment and acquisition program managers must navigate, understand, and make actionable. For example, government and industry acquisition program managers face a variety of policies and requirements for information assurance, software assurance, trusted systems, program protection planning, anti-tamper, and the like. The acquisition community is therefore often confused about what these and other various terms mean, why they are important, what needs to be done and how, and, finally, how to evaluate achievement of the desired outcome.

Another challenge is that the DoD's current assurance policies are the result of incremental changes over many years, rather than originating from a coherent and integrated strategy that recognizes the rapid pace of technology and software change. As a result, there is a need for an overarching, integrated assurance framework (ideally evidence-based) that can be continually adapted to address emerging needs and that communicates more effectively to officials who plan and execute acquisition and sustainment programs.

Finally, the DoD's sustainment and acquisition communities are decentralized geographically and organizationally with limited, on-going visibility at the DoD enterprise level of the state of the software assurance infrastructure (including tools, practices, and workforce) to inform policy and resource decisions.

Towards a Strategic DoD Software Assurance Approach

The DoD must deal with a critical strategy issue in addressing the software assurance challenge. DoD's approach relies on a decentralized approach to software assurance. In this approach, each program (acquisition and legacy) addresses software assurance (and overall system assurance) within the acquisition and contract strategy for that specific program. This approach can enable the proliferation of an ever-increasing variety of approaches to software assurance for individual systems that must operate in a systems-of-systems environment. In 2010, the DoD initiated the Program Protection Plan (PPP) requirement for acquisition programs to consolidate in one document a number of existing assurance-type policy requirements. This policy is certainly a step in the right direction to elevate the criticality of assurance in all phases of the system lifecycle. However, the policy allows each program to satisfy assurance requirements in different ways and relies on idiosyncratic (and potentially conflicting) software assurance tools and practices of multiple contractors.

The persistence of concerns and their consequences drives the need to consider the merits of creating and organizing the DoD software assurance policy and infrastructure around a standard infrastructure. This infrastructure should include policies, contract requirements, practices, tools, and a workforce that is continually refreshed. This infrastructure should also be continually evaluated for its value to achieve consistency across programs, within the DoD's software sustainment organizations, and the SoS network environment.

This shift to an enterprise software assurance approach should include creating a meta-assurance framework that synthesizes myriad legacy assurance models into a more technologically current model. This model should effectively communicate to acquisition and sustainment managers what must be done; how to implement it; and then how to evaluate outcomes. An effective enterprise software assurance approach should also include the means to continually and comprehensively assess and improve the state of the art of the DoD's software assurance infrastructure to develop a baseline of past, current, the gaps, and then identify the impacts of emerging technology. Moreover, the DoD needs enterprise strategy, plan, and performance measures for the inventory and continual analysis of the software portfolio to inform corporate decisions about software assurance policies, programs, and investments.

In addition to these policy frameworks and assessments measures, the DoD also needs to enhance workforce competencies for software assurance within the context of a broad- based enterprise strategy to improve software acquisition management policy, practices, and competencies. Since these workforce issues cannot entirely be addressed by applying existing best practices, a deliberate and executable DoD research and development investment strategy is also needed to advance capabilities in software assurance and vulnerability detection to address infrastructure gaps and future needs. This investment strategy should consider the following technical areas:

• Architecture and composition principles that enable separate evaluation of individual components with the possibility of combining results to achieve aggregate assurance judgments. These principles are motivated by the reality of modern software supply chains, which are rich and diverse in sourcing and geography.
• Modeling and analytics to support the diversity of quality attributes significant to DoD and infrastructural systems including modeling, simulation, and design techniques that support critical security attributes.
• Exploration of development approaches that incorporate creation of evidence in support of assurance claims into the process of development. This evidence-based assurance can harmonize incentives to create designs and implementations that can more readily support evaluation.
• Evaluation and other techniques to support the use of more opaque components in systems, including binary components and potentially dangerous components from unknown sources.

Summary

Software assurance is a complex domain that is one element of a larger mission assurance framework needed by the DoD to encompass reliability, security, robustness, safety, and other quality-related attributes. Although software assurance nests with the DoD's overall software strategy, its related policies, plans, and infrastructure have not kept pace with the impacts of advancing technology and reliance on software to achieve warfighter performance. To address critical software assurance challenges, the DoD must shift to an enterprise led and managed infrastructure.

Additional Resources

To download a PDF of the report Critical Code: Software Producibility for Defense, please go to
https://www.foxebook.net/critical-code-software-producibility-for-defense.

To read the SEI report Making the Business Case for Software Assurance, please go to
https://resources.sei.cmu.edu/library/asset-view.cfm?assetID=8831.

Acquisition and sustainment workforce competencies for software assurance are a critical element in creating a robust infrastructure. For an overview of the software assurance competency domain, please go to https://resources.sei.cmu.edu/library/asset-view.cfm?assetID=47953.