search menu icon-carat-right cmu-wordmark

A Summary of Key SEI R&D Accomplishments in 2011

A key mission of the SEI is to advance the practice of software engineering and cyber security through research and technology transition to ensure the development and operation of software-reliant Department of Defense (DoD) systems with predictable and improved quality, schedule, and cost. To achieve this mission, the SEI conducts research and development (R&D) activities involving the DoD, federal agencies, industry, and academia. One of my initial blog postings summarized the new and upcoming R&D activitieswe had planned for 2011. Now that the year is nearly over, this blog posting presents some of the many R&D accomplishments we completed in 2011.

Our R&D benefits the DoD and other sponsors by identifying and solving key technical challenges facing developers and managers of current and future software-reliant systems. Our R&D work focuses on the following four major areas of software engineering and cyber security:

  • Innovating software for competitive advantage. This area focuses on producing innovations that revolutionize development of assured software-reliant systems to maintain the U.S. competitive edge in software technologies vital to national security.
  • Securing the cyber infrastructure. This area focuses on enabling informed trust and confidence in using information and communication technology to ensure a securely connected world to protect and sustain vital U.S. cyber assets and services in the face of full-spectrum attacks from sophisticated adversaries.
  • Advancing disciplined methods for engineering software. This area focuses on improving the availability, affordability, and sustainability of software-reliant systems through data-driven models, measurement, and management methods to reduce the cost, acquisition time, and risk of our major defense acquisition programs.
  • Accelerating assured software delivery and sustainment for the mission. This area focuses on ensuring predictable mission performance in the acquisition, operation, and sustainment of software-reliant systems to expedite delivery of technical capabilities to win the current fight.

Following is a sampling of the SEI's R&D accomplishments in each of these areas during 2011 with links to additional information about these projects.

Innovating Software for Competitive Advantage

Although the SEI advocates software architecture documentation as a software engineering best practice, the specific value of software architecture documentation has not been established empirically. The blog posting Measuring the Impact of Explicit Architecture Documentation describes a research project we conducted to measure and understand the value of software architecture documentation on complex software-reliant systems, focusing on creating architectural documentation for a major subsystem of Apache Hadoop, the Hadoop Distributed File System (HDFS).

The SEI has developed algorithms and tools for optimize the performance of cyber-physical systems without compromising their safety. The blog posting Ensuring Safety in Cyber-Physical Systems describes a safe double-booking algorithm that reduces the over-allocation of processing resources needed to ensure the timing behavior of safety-critical tasks in cyber-physical systems. A subsequent posting describes an algorithm for supporting mixed-criticality operations by giving more central processing unit (CPU) time to functions with higher value while ensuring critical timing guarantees.

Together with researchers at CMU, the SEI has worked to develop cloudlets, which are localized, lightweight servers running one or more virtual machines on which soldiers can offload expensive computations from their handheld mobile devices, thereby providing greater processing capacity and helping conserve battery power. The blog posting Cloud Computing for the Battlefield describes a cloudlet prototype the SEI developed to recognize faces on an Android smartphone. A subsequent posting describes how the SEI is using cloudlets to help soldiers perform other mission capabilities more effectively, including speech and imaging recognition, as well as decision making and mission planning.

SEI-developed methods and tools allow soldier end-users to program their smartphones to provide an interface tailored to the information they need for a specific mission. The blog posting A New Approach for Handheld Devices in the Military motivates the need for soldiers to access information on a handheld device and described software we are developing to enable soldiers to tailor the information for a given mission or situation. A subsequent blog posting describes the challenges the SEI encountered when equipping soldiers with end-user programming tools.

Other SEI-developed methods and tools help reduce the time and effort needed to re-certify mission- and safety-critical real-time embedded software systems (RTESs) after significant changes have be made, such as migrating a single-core RTES to a multi-core platform, significant code refactoring, or performance optimizations. The blog posting on Regression Verification of Real-time Embedded Software focuses on research in applying regression verification (which involves deciding the behavioral equivalence of two closely related programs) to help the migration of RTESs from single-core to multi-core platforms. A subsequent posting describes regression verification tools and techniques that the SEI is building to conduct static analysis of RTESs.

Securing the Cyber Infrastructure

A large percentage of cybersecurity attacks against DoD and other government organizations are caused by disgruntled, greedy, or subversive insiders, employees, or contractors with access to that organization's network systems or data. The blog posting Protecting Against Insider Threads with Enterprise Architecture Patterns describes work that researchers at the CERT® Insider Threat Center have been conducting to help protect next-generation DoD enterprise systems against insider threats by capturing, validating, and applying enterprise architectural patterns. These patterns can be used to ensure that the necessary agreements are in place (IP ownership and consent to monitoring), critical IP is identified, key departing insiders are monitored, and the necessary communication among departments takes place to mitigate the impact of insider threats.

The SEI has been conducting research to help organizational leaders manage critical services in the presence of disruption by presenting objectives and strategic measures for operational resilience, as well as tools to help them select and define those measures. The blog posting Measures for Managing Operational Resilience describes how the SEI has been exploring the topic of managing operational resilience at the organizational level for the past seven years through development and use of the CERT Resilience Management Model (CERT-RMM). The CERT-RMM is a capability model designed to establish the convergence of operational risk and resilience management activities and apply a capability level scale that expresses increasing levels of process performance.

New malicious code analysis techniques and tools being developed at the SEI will better counter and exploit adversarial use of information and communication technologies. The blog posting Fuzzy Hashing Techniques in Applied Malware Analysis describes a technique the SEI has developed to help analysts determine whether two pieces of suspected malware are similar. A subsequent posting discusses types of malware against which similarity measures of any kind (including fuzzy hashing) may be applied. Other blog postings on Learning a Portfolio-Based Checker for Provenance-Similarity of Binaries and Using Machine Learning to Detect Malware Similarity describe our research on using classification (a form of machine learning) to detect "provenance similarities" in binaries, which means that they have been compiled from similar source code (e.g., differing by only minor revisions) and with similar compilers (e.g., different versions of Microsoft Visual C++ or different levels of optimization). Yet another blog posting A New Approach to Modeling Malware using Sparse Representation describes our use of suffix trees, zero-suppressed binary decision diagrams, and sparse representation modeling to create a rapid search capability that allows analysts to quickly analyze a new piece of malware.

Advancing Disciplined Methods for Engineering Software

Recent SEI research aims to improve the accuracy of early estimates (whether for a DoD acquisition program or commercial product development) and ease the burden of additional re-estimations during a program's lifecycle. The blog posting Improving the Accuracy of Early Cost Estimates for Software-Reliant Systems describes challenges we have observed trying to accurately estimate software effort and cost in DoD acquisition programs, as well as other product development organizations. A subsequent post explores a method and tools the SEI is developing to help cost estimation experts get the right information into a familiar and usable form for producing high quality cost estimates early in the lifecycle.

A notable new approach at the SEI combines elements of the SEI's Architecture Centric Engineering (ACE) method, which requires effective use of software architecture to guide system development, with its Team Software Process (TSP), which is a team-centric approach to developing software that enables organizations to better plan and measure their work and improve software development productivity to gain greater confidence in quality and cost estimates. The blog postings Combining Architecture-Centric Engineering Within TSP and Using TSP to Architect a New Trading System describe how ACE was applied within the context of TSP to develop system architecture to create a reliable and fast new trading system for Groupo Bolsa Mexicana de Valores (BMV, the Mexican Stock Exchange).

Over the last several years, the SEI hosted a series of workshops that brought together leaders in the application of measurement and analytical methods in many areas of software and systems engineering. The workshops helped identify the technical barriers organizations face when they use advanced measurement and analytical techniques, such as computer modeling and simulation. The blog posting on Using Predictive Modeling in Software Development: Results from the Field describes the technical characteristics and quantified results of models used by organizations at the workshops.

Accelerating Assured Software Delivery and Sustainment for the Mission

The SEI has been assisting large-scale DoD acquisition programs in developing systematically reusable software platforms that provide applications and end-users with many net-centric capabilities, such as cloud computing or Web 2.0 applications. The blog posting A Framework for Evaluating Common Operating Environments explains how the SEI developed a Software Evaluation Framework and applied it to help assess the suitability of common operating environments for the U.S. Army.

Methods and processes that enable large-scale software-reliant DoD systems to innovate rapidly and adapt products and systems to emerging needs within compressed time frames were another area of exploration for the SEI. A series of blog postings details our research on improving the overall value delivered to users by strategically managing technical debt, which involves decisions made to defer necessary work during the planning or execution of a software project, as well as describing the level of skill needed to develop software using Agile for DoD acquisition programs and the importance of maintaining strong competency in a core set of software engineering processes.

Teams at the SEI also have been researching common problems faced by acquisition programs related to the development of IT systems, including communications, command, and control; avionics; and electronic warfare systems. A series of blog postings covers acquisition problems, such as

  • misaligned incentives, which occur when different individuals, groups, or divisions are rewarded for behaviors that conflict with a common organizational goal
  • the need to sell the program, which describes a situation in which people involved with acquisition programs have strong incentives to "sell" those programs to their management, sponsors, and other stakeholders so that they can obtain funding, get them off the ground, and keep them sold
  • the evolution of "science projects," which describes how prototype projects that unexpectedly grow in size and scope during development often have difficulty transitioning into a formal acquisition program, and
  • the tragedy of common infrastructure and joint programs, which arises when multiple organizations attempt to cooperate in the development of a single system, infrastructure, or capability that will be used and shared by all parties.

The SEI also developed a collaborative method for engineering systems with critical safety and security ramifications. A series of blog postings on this topic explores problems with safety and security requirements, examines key obstacles that acquisition and development organizations encounter concerning safety- and security-related requirements, and explains how the Engineering Safety- and Security-related Requirements (ESSR) method overcomes these obstacles.

Concluding Remarks

As you can see from the summary of accomplishments above, 2011 has been a highly productive and exciting year for the SEI R&D staff. Naturally, this blog posting just scratches the surface of SEI R&D activities. Please come back regularly to the SEI blog for coverage of these and many other topics we'll be doing in 2012. As always, we're interested in new insights and new opportunities to partner on emerging technologies and interests. We welcome your feedback and look forward to engaging with you on the blog; as always we invite your comments below.

About the Author