SEI Insights

SEI Blog

The Latest Research in Software Engineering and Cybersecurity

In my preceding blog posts, I promised to provide more examples highlighting the importance of software sustainment in the U.S. Department of Defense (DoD). My focus is on sustaining legacy weapons systems that are no longer in production, but are expected to remain a key component of our defense capability for decades to come. Despite the fact that these legacy systems are no longer in the acquisition phase, software upgrade cycles are needed to refresh their capabilities every 18 to 24 months. In addition, significant modernization can often be made by more extensive, focused software upgrades with relatively modest hardware changes. This third blog post describes effective sustainment engineering efforts in the Army, using examples from across its software engineering centers. These examples are tied to SEI research on capability maturity models, agility, and the Architecture Analysis and Design Language (AADL) modeling notation.

This is the first post in a three-part series.

Software and acquisition professionals often have questions about recommended practices related to modern software development methods, techniques, and tools, such as how to apply agile methods in government acquisition frameworks, systematic verification and validation of safety-critical systems, and operational risk management. In the Department of Defense (DoD), these techniques are just a few of the options available to face the myriad challenges in producing large, secure software-reliant systems on schedule and within budget.

In an effort to offer our assessment of recommended techniques in these areas, SEI built researchers built upon an existing collaborative online environment known as SPRUCE (Systems and Software Producibility Collaboration Environment), hosted on the Cyber Security & Information Systems Information Analysis Center (CSIAC) website. From June 2013 to June 2014, the SEI assembled guidance on a variety of topics based on relevance, maturity of the practices described, and the timeliness with respect to current events. For example, shortly after the Target security breach of late 2013, we selected Managing Operational Resilience as a topic.

Ultimately, SEI curated recommended practices on five software topics: Agile at Scale, Safety-Critical Systems, Monitoring Software-Intensive System Acquisition Programs, Managing Intellectual Property in the Acquisition of Software-Intensive Systems, and Managing Operational Resilience. In addition to a recently published paper on SEI efforts and individual posts on the SPRUCE site, these recommended practices will be published in a series of posts on the SEI blog. This post, the first in a three-part series by Robert Ferguson, first explores the challenges to Monitoring Software-Intensive System Acquisition (SISA) programs and presents the first two recommended best practices as detailed in the SPRUCE post. The second post in this series will present the next three best practices. The final post will present the final two recommendations as well as conditions that will allow organizations to derive the most benefit from these practices.

By Donald Firesmith
Principal Engineer
Software Solutions Division

Due to advances in hardware and software technologies, Department of Defense (DoD) systems today are highly capable and complex. However, they also face increasing scale, computation, and security challenges. Compounding these challenges, DoD systems were historically designed using stove-piped architectures that lock the Government into a small number of system integrators, each devising proprietary point solutions that are expensive to develop and sustain over the lifecycle. Although these stove-piped solutions have been problematic (and unsustainable) for years, the budget cuts occurring under sequestration are motivating the DoD to reinvigorate its focus on identifying alternative means to drive down costs, create more affordable acquisition choices, and improve acquisition program performance. A promising approach to meet these goals is Open Systems Architecture (OSA), which combines

This blog posting expands on earlier coverage of how acquisition professionals and system integrators can apply OSA practices to effectively decompose large monolithic business and technical architectures into manageable and modular solutions that can integrate innovation more rapidly and lower total ownership costs.

By Douglas Gray
Information Security Engineer
CERT Division

In leveraging threat intelligence, the operational resilience practitioner need not create a competing process independent of other frameworks the organization is leveraging. In fact, the use of intelligence products in managing operational resilience is not only compatible with many existing frameworks but is, in many cases, inherent. While it is beyond the scope of this blog to provide an in-depth discussion of some of the more widely used operational resilience measurement and decision-making best practices--including the CERT® Resilience Management Model (CERT-RMM), Operationally Critical Threat, Asset, and Vulnerability Evaluation (OCTAVE) Allegro methodology, the NIST Risk Management Framework (RMF), Agile, and the Project Management Body of Knowledge (PMBOK)-- this blog post, the second in a series, provides a discussion of how to operationalize intelligence products to build operational resilience of organizational assets and services.

By David Svoboda
Senior Member of the Technical Staff
CERT Division

Whether Java is more secure than C is a simple question to ask, but a hard question to answer well. When we began writing the SEI CERT Oracle Coding Standard for Java, we thought that Java would require fewer secure coding rules than the SEI CERT C Coding Standard because Java was designed with security in mind. We naively assumed that a more secure language would need fewer rules than a less secure one. However, Java has 168 coding rules compared to just 116 for C. Why? Was our (admittedly simplistic) assumption completely spurious? Or, are there problems with our C or Java rules? Or, are Java programs, on average, just as susceptible to vulnerabilities as C programs? In this post, I attempt to analyze our CERT rules for both C and Java to determine if they indeed refute the conventional wisdom that Java is more secure than C.

By Douglas Gray
Information Security Engineer
CERT Division

What differentiates cybersecurity from other domains in information technology (IT)? Cybersecurity must account for an adversary. It is the intentions, capabilities, prevailing attack patterns of these adversaries that form the basis of risk management and the development of requirements for cybersecurity programs. In this blog post, the first in a series, I present strategies for enabling resilience practitioners to organize and articulate their intelligence needs, as well as relevant organizational information, establish a collaborative relationship with their intelligence providers, organize and assess intelligence, and act upon intelligence via frameworks such as the CERT® Resilience Management Model (CERT-RMM), Operationally Critical Threat, Asset, and Vulnerability Evaluation (OCTAVE) Allegro methodology, the NIST Risk Management Framework, Agile, and the Project Management Body of Knowledge (PMBOK). Subsequent postings in this blog series, we discuss how these common resilience, risk, and project-management frameworks can be leveraged to integrate threat intelligence into improving the operational resilience of organizations.

By Donald Firesmith
Principal Engineer
Software Solutions Division

There are more than 200 different types of testing, and many stakeholders in testing--including the testers themselves and test managers--are often largely unaware of them or do not know how to perform them. Similarly, test planning frequently overlooks important types of testing. The primary goal of this series of blog posts is to raise awareness of the large number of test types, to verify adequate completeness of test planning, and to better guide the testing process. In the previous blog entry in this series, I introduced a taxonomy of testing in which 15 subtypes of testing were organized around how they addressed the classic 5W+2H questions: what, when, why, who, where, how, and how well. This and future postings in this series will cover each of these seven categories of testing, thereby providing structure to roughly 200 types of testing currently used to test software-reliant systems and software applications.

This blog entry covers the four, top-level subtypes of testing that answer the following questions:

  • What-based testing: What is being tested?
    • Object-Under-Test-based (OUT) testing
    • Domain-based testing
    • When-based testing: When is the testing being performed?
      • Order-based testing
      • Lifecycle-based testing
      • Phase-based testing
      • Built-in-Test (BIT) testing

After exploring the what-based and when-based categories of testing, this post presents a section on using the taxonomy, as well as opportunities for accessing it.

By Julien Delange
Member of the Technical Staff
Software Solutions Division

For decades, safety-critical systems have become more software intensive in every domain--in avionics, aerospace, automobiles, and medicine. Software acquisition is now one of the biggest production costs for safety-critical systems. These systems are made up of several software and hardware components, executed on different components, and interconnected using various buses and protocols. For instance, cars are now equipped with more than 70 electronic control units (ECUs) interconnected with different buses and require about 100 million source lines of code (SLOC) to provide driver assistance, entertainment systems, and all necessary safety features, etc. This blog post discusses the impact of complexity in software models and presents our tool that produces complexity metrics from software models.