Posted on by Architecturein
The Department of Defense (DoD) has become deeply reliant on software. As a federally funded research and development center (FFRDC), the SEI is chartered to work with the DoD to meet the challenges of designing, producing, assuring, and evolving software-reliant systems in an affordable and dependable manner. This blog post is the second in a multi-part series that describes key elements of our forthcoming Strategic Research Plan that address these challenges through research, acquisition support, and collaboration with the DoD, other federal agencies, industry, and academia.
The first postin this series focused on Architecture-Led Incremental Iterative Development. This part focuses on the remaining three elements of our strategic plan: (1) designed-in security and quality (evidence-based software assurance), (2) a set of DoD critical component capabilities relating to cyber-physical systems (CPS), autonomous systems, and big data analytics, and (3) cybersecurity tradecraft and analytics.
Evidence-based Software Assurance and Certification
The goal of the second element in the SEI Strategy Research Plan is a dramatic reduction in the cost and difficulty of making assurance judgments related to quality and security attributes. Achieving this goal is particularly important as systems become more complex and evolve more rapidly. Current approaches for certification and accreditation are largely based on an after-the-fact evaluation of a snapshot of a system.
While after-the-fact approaches are effective for certain well-defined categories of components and systems, they tend to break down as systems increase in complexity, scale, and dynamism. They also tend to hinder ongoing evolution, rapid reconfiguration, dynamic loading of components, autonomy, and composition and interlinking of systems-of-systems. Put simply, these established techniques do not scale up, and they do not work well for the emerging software framework-based systems now prevalent in commercial and infrastructural applications.
The industry folklore has long asserted that quality-related activities, including security-related assurance, can consume half of total development costs for larger systems. For example, the IBM Systems Journal states that, in a typical commercial development organization, "the cost of providing [the assurance that the program will perform satisfactorily in terms of its functional and nonfunctional specifications within the expected deployment environments] via appropriate debugging, testing, and verification activities can easily range from 50 to 75 percent of the total development cost." Additionally, after-the-fact evaluation practices can add a year or more to the elapsed time required to develop and deploy software-reliant systems.
Commercial systems, including products and software as a service (SaaS), cloud-based systems, tend to undergo a relatively rapid and continual evolution. For many of our DoD and infrastructural systems, we similarly need to support a continuous evolution.
Some areas of particular technical emphasis include
Part of the SEI Strategy Research Plan addresses incentives for developers to adapt their architectural structures, development process, and tooling to better accommodate the idea of amassing evidence--during the development process--that can support an eventual assurance claim with respect to quality and security attributes critical to the operation of the particular system. Our work benefits from the fact that the development process can be naturally incremental through the composition of components and the incremental validation of assurance claims. This development process is supported by the partial accumulation of engineered artifacts and evidence.
These ideas of "designed-in security" build on the fourth theme of the Networking Information Technology Research & Development (NITRD) Program plan, "Trustworthy Cyberspace: Strategic Plan for the Federal Cybersecurity Research and Development Program."
Critical Component Capabilities
The goal of the third element in the SEI Strategic Research Plan is to enhance DoD software capability in several areas that have critical and pervasive roles in DoD software-reliant systems. These areas include composable, cyber-physical systems (CPS), autonomous and distributed systems, and high-performance, data-intensive computing.
Each of the areas presents challenges:
Cybersecurity Tradecraft and Analytics
The goal of the fourth strategic element is to advance analytic capability in support of diverse aspects of the cybersecurity mission. These aspects include analytics and situational awareness for malware, vulnerability categorization and assessment, vulnerability information management, network activity analysis, threat characterization and assessment, organizational security, and many other dimensions of operational response, remediation, and recovery.
This capability builds on a range of data assets related to adversarial tradecraft, malware, vulnerabilities, insider threats, and other results of experience with large numbers of cybersecurity-related incidents. There are diverse purposes of this strategic element, including:
The next blog posting in this series focuses on our approach for evaluating and validating SEI research projects. The SEI Strategic Research Plan is designed to ensure that the SEI conducts high-quality and high-impact work that benefits the DoD by identifying and solving key technical challenges facing current and future DoD software-reliant systems. This strategy itself undergoes continual evolution and improvement; the broad range of SEI engagements enable us to continually refine the strategy as the technology advances, the mission evolves, and our understanding improves. We welcome engagement from our partners and stakeholders in the improvement and refinement of this strategy. Please leave comments below or contact us directly at firstname.lastname@example.org.
To download the report Critical Code: Software Producibility for Defense, please see www.nap.edu/catalog.php?record_id=12979
To download the Report of the Defense Science Board Task Force on Defense Software (2000), please visit http://oai.dtic.mil/oai/oai?verb=getRecord&metadataPrefix=html&identifier=ADA385923
To view on-demand presentations from the SEI Agile Research Forum, please visit
For more information about the Networking Information & Technology Research & Development Program, please visit
Visit the SEI Digital Library for other publications by Bill.