As part of our mission to advance the practice of software engineering and cybersecurity through research and technology transition, our work focuses on ensuring the development and operation of software-reliant Department of Defense (DoD) systems with predictable and improved quality, schedule, and cost. To achieve this mission, the SEI conducts research and development (R&D) activities involving the DoD, federal agencies, industry, and academia. As we look back on 2012, this blog posting highlights our many R&D accomplishments.

Our R&D benefits the DoD and other sponsors by identifying and solving key technical challenges facing developers and managers of current and future software-reliant systems. R&D work at the SEI focuses on the following four general areas of software engineering and cybersecurity:

• Securing the cyber infrastructure. This area focuses on enabling informed trust and confidence in using information and communication technology to ensure a securely connected world to protect and sustain vital U.S. cyber assets and services in the face of full-spectrum attacks from sophisticated adversaries.
• Advancing disciplined methods for engineering software. This area focuses on improving the availability, affordability, and sustainability of software-reliant systems through data-driven models, measurement, and management methods to reduce the cost, acquisition time, and risk of our major defense acquisition programs.
• Accelerating assured software delivery and sustainment for the mission. This area focuses on ensuring predictable mission performance in the acquisition, operation, and sustainment of software-reliant systems to expedite delivery of technical capabilities to win the current fight.
• Innovating software for competitive advantage. This area focuses on producing innovations that revolutionize development of assured software-reliant systems to maintain the U.S. competitive edge in software technologies vital to national security.

Following is a sampling of the SEI's R&D accomplishments in each of these areas during 2012, with links to additional information about these projects.

Securing the Cyber Infrastructure

A large percentage of cybersecurity attacks against DoD and other government organizations are caused by disgruntled, greedy, or subversive insiders, employees, or contractors with access to that organization's network systems or data. Over the past 12 years, researchers at the CERT Insider Threat Center have collected incidents related to malicious activity by insiders from a number of sources, including media reports, the courts, the United States Secret Service, victim organizations, and interviews with convicted felons.

The blog post Developing Controls to Prevent Theft of Intellectual Property described controls that researchers have developed to prevent, identify, or detect theft of intellectual property. A subsequent posting, A New SIEM Signature Developed to Address Insider Threats, explored controls developed to prevent, identify, or detect IT sabotage.

Another aspect of insider threat research focused on identifying enterprise architecture patterns that protect an organization's systems from malicious insiders. Enterprise architecture patterns are organizational patterns that involve the full scope of enterprise architecture concerns, including people, processes, technology, and facilities. Our goal with this pattern work is to equip organizations with the tools necessary to institute controls that will reduce the incidence of insider compromise. A blog post described research to create and validate an insider threat mitigation pattern language that focuses on helping organizations balance the cost of security controls with the risk of insider compromise. A final post described exploratory research to determine the indicators that insiders might demonstrate prior to attacks.

New malicious code analysis techniques and tools being developed at the SEI will better counter and exploit adversarial use of information and communication technologies. Through our work in cybersecurity, we have amassed millions of pieces of malicious software in a large malware database. Analyzing this code manually for potential similarities and identifying malware provenance is a painstaking process. The blog post Modeling Malware with Suffix Trees outlined a method to create effective and efficient tools that analysts can use to identify malware more effectively.

Another approach for identifying similarity in malicious code involves leveraging analyst insights by identifying files that possess some property in common with a particular file of interest. One way to do this is by using YARA, an open-source project that helps researchers identify and classify malware. YARA has gained enormous popularity in recent years as a way for malware researchers and network defenders to communicate their knowledge about malicious files, from identifiers for specific families to signatures capturing common tools, techniques, and procedures. The post Writing Effective YARA Signatures to Identify Malware provided guidelines for using YARA effectively, focusing on selection of objective criteria derived from malware, the type of criteria most useful in identifying related malware (including strings, resources, and functions), and guidelines for creating YARA signatures using these criteria.

Our security experts in the CERT Program are often called upon to audit software and provide expertise on secure coding practices. The blog posting Improving Security in the Latest C Programming Language Standard detailed security enhancements-- bounds-checking interfaces and analyzability--from the December 2011 revision of the C programming language standard, which is known informally as C11. Another post described our work on the CERT Perl Secure Coding Standard, which provides a core of well-documented and enforceable coding rules and recommendations for Perl, which is a popular scripting language.

Advancing Disciplined Methods for Engineering Software

According to a February 2011 presentation by Gary Bliss, director of Program Assessment and Root Cause Analysis, to the DoD Cost Analysis Symposium, unrealistic cost or schedule estimates frequently cause a program to breach a performance criterion. To help the DoD address this need, the SEI has continued its research into improving the accuracy of early cost estimates (whether for a DoD acquisition program or commercial product development) and ease the burden of additional re-estimations during a program's lifecycle. The blog posting Quantifying Uncertainty in Early Lifecycle Cost Estimation (QUELCE) outlines a multi-year project conducted by the SEI Software Engineering Measurement and Analysis (SEMA) team. QUELCE is a method for improving pre-Milestone A software cost estimates through research designed to improve judgment regarding uncertainty in key assumptions (which are termed program change drivers), the relationships among the program change drivers, and their impact on cost.

QUELCE asks domain experts to provide judgment not only on uncertain cost factors for a nominal program execution scenario, but also for the drivers of cost factors across a set of anticipated scenarios. A second installment in the series on QUELCE described efforts to improve the accuracy and reliability of expert judgment within this expanded role of early lifecycle cost estimation.

On a separate front, a series of blog posts detailed a pilot of the Team Software Process (TSP) approach at Nedbank, one of the four largest banks in South Africa. The first post described how TSP principles allowed developers at the bank to address challenges, improve productivity, and thrive in an agile environment. The second post detailed how the SEI worked with Nedbank to address challenges with expanding and scaling the use of TSP at an organizational level. The third post explored challenges common to many organizations seeking to improve performance and become more agile and concluded by demonstrating how SEI researchers addressed these challenges in the TSP rollout at Nedbank.

The TSP pilot teams at Nedbank made significant behavioral changes that not only improved the quality of the software but also team members' work lives by decreasing the need for evening and weekend overtime. The teams were able to make these improvements because they had project-specific measurements to guide their decisions, and they had the authority to implement those decisions. Based on the results of the pilots, Nedbank decided to implement TSP throughout the organization.

Accelerating Assured Software Delivery and Sustainment for the Mission

Another area of exploration for the SEI was methods and processes that enable large-scale software-reliant DoD systems to innovate rapidly and adapt products and systems to emerging needs within compressed time frames. The SEI has focused research efforts on improving the overall value delivered to users by strategically managing technical debt, which involves decisions made to defer necessary work during the planning or execution of a software project, as well as describing the level of skill needed to develop software using Agile for DoD acquisition programs and the importance of maintaining strong competency in a core set of software engineering processes. A blog post on this topic discussed how an architecture-focused analysis approach helps manage technical debt by enabling software engineers to decide the best time to rearchitect--in other words, to pay down technical debt.

SEI researchers also focused their efforts on common problems faced by acquisition programs related to the development of IT systems, including communications, command, and control; avionics; and electronic warfare systems. Long development cycles don't fit with user expectations, and in the DoD, it can take up to 81 months to acquire or develop a new technology.

To help bridge this gap, the SEI in 2012 hosted the Agile Research Forum, which brought together researchers and practitioners from around the world to discuss when and how to best apply agile methods in the mission-critical environments found in government and many industries. A multi-part series of blog posts highlighted key ideas and issues addressed at the forum, including

• Applying Agile at-Scale for Mission-Critical Software-Reliant Systems. This post, which recapped presentations by Anita Carleton, director of the SEI's Software Engineering Process Management Program and Teresa M. Takai, chief information officer for the DoD, highlighted key ideas and issues associated with applying agile methods to address the challenges of complexity, exacting regulations, and schedule pressures.
• Agile Methods: Tools, Techniques, and Practices for the DoD Community. This post, which recapped a presentation by Mary Ann Lapham, highlighted the importance of collaboration with end users, as well as among cross-functional teams, to facilitate the adoption of agile approaches into DoD acquisition programs.
• Strategic Management of Architectural Technical Debt. This post, which recapped a presentation by Ipek Ozkaya, discussed the use of agile architecture practices to manage strategic, intentional technical debt.
• Balancing Agility and Discipline at Scale. This post, which recapped a presentation by James Over, manager of TSP, advocated the building of self-managed teams, planning and measuring project process, designing before building, and making quality the top priority, among other principles associated with applying agile methods at-scale.
• Applying Agility to Common Operating Platform Environment Initiatives. This post, which recapped a presentation I gave at the forum, highlighted the importance of applying agile methods to common operating platform environments (COPEs) that have become increasingly important for the DoD to deliver enhanced integrated warfighting capability at lower cost, reduce acquisition and new technology insertion cycle time, and establish sustainable business and workforce strategies to support these goals.

One problem that occurs when organizations are trying to adopt new practices is a disconnect in business strategy, values, and management style. These aren't the only disconnects, but they are representative of what we often see when working with DoD or other regulated organizations trying to adopt agile methods. In the first installment in an upcoming series of blog posts, we introduced an analysis method called Readiness and Fit Analysis (RFA) that has been used for multiple technologies and sets of practices, most notably for adoption of CMMI practices, to identify and suggest strategies for mitigating common adoption risks.

Engineering the architecture for a large and complex system is a hard, lengthy, and complex undertaking. System architects must perform many tasks and use many techniques if they are to create a sufficient set of architectural models and related documents that are complete, consistent, correct, unambiguous, verifiable, usable, and useful to the architecture's many stakeholders. The SEI has developed the Method Framework for Engineering System Architectures (MFESA), which is a situational process engineering framework for developing system-specific methods to engineer system architectures. The first post in a two-part series on MFESA provided a brief historical description of situational method engineering, explained why no single system architectural engineering method is adequate, and introduced MFESA via a top-level overview of its components, describing its applicability, and explaining how it simultaneously provides the benefits of standardization and flexibility. The second post in the series took a deeper dive into the four components that comprise MFESA:

• the MFESA ontology, which defines the foundational concepts underlying system architecture engineering
• the MFESA metamodel, which defines the base superclasses of method components
• the MFESA repository, which defines reusable method components
• the MFESA metamethod, which defines a process for creating project-specific methods using method components from the MFESA repository

Innovating Software for Competitive Advantage

For more than 10 years, scientists, researchers, and engineers used the TeraGrid supercomputer network funded by the National Science Foundation (NSF) to conduct advanced computational science. The SEI recently joined a partnership of 17 organizations and helped develop the successor to the TeraGrid called the Extreme Science and Engineering Discovery Environment (XSEDE). The first post in a multi-part series focused on the nature of software engineering practice in the context of the TeraGrid socio-technical ecosystem.

From war zones in Afghanistan to disaster relief in countries like Haiti and Japan, today's warfighter faces many challenges. The Advanced Mobile Systems Initiative at the SEI continues to focus on meeting the needs of the warfighter at the tactical edge, which is a term used to describe hostile environments with limited resources. Warfighters who use handheld devices face problems ranging from information obscurity (i.e., a lack of awareness of the available information) to information overload (i.e., too much information, coupled with an inability to locate truly vital information.). Through the development of context-aware mobile applications, which was described in a recent blog post, SEI researchers are exploring alternative sources of data that would not only push the limit of what could be done with user context, but also focus on the extremely challenging environment at the tactical edge.

Another research effort is aimed at addressing a common phenomenon that takes place when the software development and maintenance effort involves several programmers and spans months or years: the source code exhibits an actual architecture that gradually diverges from the intended architecture. A post on the SATURN Network Blog describes one researcher's experience using Checkstyle and pre-commit hooks on Subversion to verify the conformance between code and architecture.

Concluding Remarks

As you can see from this summary of accomplishments, 2012 has been a highly productive and exciting year for the SEI technical staff. Moreover, this blog posting just scratches the surface of SEI R&D activities. Please come back regularly to the SEI blog for coverage of these and many other topics we'll be doing in the coming year. As always, we're interested in new insights and new opportunities to partner on emerging technologies and interests. We welcome your feedback and look forward to engaging with you on the blog, so please feel free to add your comments below.

Additional Resources

For the latest SEI technical reports and papers, please visit
www.sei.cmu.edu/library/reportspapers.cfm

For more information about R&D at the SEI as well as opportunities for collaboration, please visit
www.sei.cmu.edu/research/