Software Engineering Institute

Many organizations measure just for the sake of measuring, with little or no thought given to what purpose and business objectives are being satisfied or what questions each measure is intended to answer. However, meaningful measurement is about transforming strategic direction, policy, and other forms of management decision into action and measuring the performance of that action. 

Effective measures express the extent to which objectives are being met, how well requirements are being satisfied, how well processes and controls are functioning, and the extent to which performance outcomes are being achieved. The basic goal of measurement and analysis is to provide decision makers with the information they need, when they need it, and in the right form. In recent years, researchers have begun to turn their attention to the topic of software security assurance and how to measure it.

Software security assurance is justified confidence that software-reliant systems are adequately planned, acquired, built, and fielded with sufficient security to meet operational needs, even in the presence of attacks, failures, accidents, and unexpected events. For several years, various groups within the software engineering community have been working diligently to identify practices aimed at developing more secure software. However, efforts to measure software security assurance have yet to materialize in any substantive fashion, although some foundational work has been performed.

As a result of the software engineering community's interest, the CERT® Program at Carnegie Mellon University's Software Engineering Institute (SEI) has chartered the Security Measurement and Analysis (SMA) Project to advance the state-of-the-practice in security measurement and analysis. The SMA Project builds on the CERT Program's core competence in software and information security as well as the SEI's work in software engineering measurement and analysis. The purpose of this new research project is to address the following three questions:

• How do we establish, specify, and measure justified confidence that a software-reliant product is sufficiently secure to meet operational needs?
• How do we measure at each phase of the development or acquisition life cycle that the required/desired level of security has been achieved?
• How do we scale measurement and analysis approaches to complex environments, such as large-scale, networked, software-reliant systems (e.g., systems of systems)?

In essence, the three research questions examine how decision makers (e.g., development program and project managers as well as acquisition program officers) can measure and monitor the security posture of large-scale, networked, software-reliant systems across the life cycle and supply chain.