Software Engineering Institute | Carnegie Mellon University

SEI Insights

Insider Threat Blog

Real-World Work Combating Insider Threats

Why Is Measurement So Hard?

Posted on by in

Developing security metrics within an organization is an ongoing challenge. Organizations want to know "Am I secure enough?" While this is the common question, it lacks context. Organizations vary in size, mission, risk appetites, and budget for security. There is no "one size fits all" for security metrics.

Security metrics should demonstrate return on investment for the security program. Every dollar spent on security is a dollar less somewhere else. Security metrics need to validate that the security program adds business value. To do this, metrics must be directly tied to the business objectives of the organization.

Almost anything can be measured. However, that doesn't mean it should be measured. An organization needs to determine the security metrics that will drive its internal action by deriving metrics that tie to business objectives. Instead of asking, "Am I secure?" an organization should ask these key questions:

  • What should I be measuring to determine if I am meeting my performance objectives for security?
  • What is the business value of being more secure?
  • What is the value of a specific security investment?

Tying the development of metrics to its business objectives helps an organization identify the most valuable security measures to measure and report. If what you are measuring doesn't drive action, then perhaps it should not be measured at all.

Measurement should be used to manage risk. And to manage well, you have to make technically competent, well-informed decisions. Decision makers value clear, concise data to help them make decisions. Security metrics can inform decisions made strategically, tactically, and operationally within the organization. When developing a metrics program, consider the audience. Senior management cares about security metrics for governance and oversight and alignment with the strategic direction of the organization. Middle management cares about security measures to oversee security management and to make decisions around improvement activities. Operators care about security metrics to ensure controls are configured and managed appropriately.

Metrics must provide actionable information for decision makers, but the metric must be balanced with the cost of measuring. Measurement can be very expensive. Consider whether the metric will enable better understanding of progress toward achieving goals and objectives. If not, it's probably not worth the investment in measuring.

For more information about developing metrics, come see us at InfoSec World Conference & Expo 2018, March 19-21, in Orlando, FL, where Jason Fricke and I will be discussing, "Metrics that Matter - Incident Management."

More from Katie C. Stewart

Posts


View other blog posts by Katie C. Stewart.

Other Publications

Visit the SEI Digital Library for other publications by Katie.