search menu icon-carat-right cmu-wordmark

A New Approach for Critical Information Systems Protection

Headshot of Anne Connell.

The source of a recent Target security breach that allowed intruders to gain access to more than 40 million credit and debit cards of customers between Nov. 27 and Dec. 14, 2013, has been traced to a heating, ventilation, and air conditioning (HVAC) service sub-contractor in Sharpsburg, Pa., just outside of Pittsburgh, according to a Feb. 5 post on a Wall Street Journal blog.

The post stated that the intruders were able to gain access to Target's system after stealing login credentials from one of Target's HVAC subcontractors, who had been given remote access. This breach demonstrates how any vulnerability in a critical information system can be exploited to disrupt or harm the normal operation of any commercial or industrial sector. In this blog post, we will present a tool we have developed that increases a security incident responder's ability to assess risk and identify the appropriate incident response plan for critical information systems.

We define a critical information system as a computer-controlled information system that manages the operation and essential assets of any commercial or industrial sector, including

  • energy delivery
  • backup generators
  • water
  • sewer systems
  • airports
  • railway
  • public transportations
  • oil and natural gas
  • emergency medical services
  • information technology (IT) systems
  • business management systems

A compromise of any of these critical systems and its assets in terms of confidentiality, integrity or availability (CIA Triad of Information Assurance) typically leads to the loss of:

  • financial stability
  • revenue
  • stakeholder trust
  • customer confidence
  • competitive advantage
  • key technologies
  • property
  • life

As the Target breach illustrates, organizations face large-scale distributed attacks on a regular basis. Another example of a similar breach is the compromise of networks of the three top medical device makers, according to an article published Feb. 10 in the San Francisco Chronicle.

A report prepared by the Minority Staff of the Homeland Security and Governmental Affairs Committee presented in February 2014 stated "in the past few years, we have seen significant breaches in cybersecurity which could affect critical U.S. infrastructure... Nuclear plants' confidential cybersecurity plans have been left unprotected. Blueprints for the technology undergirding the New York Stock Exchange were exposed to hackers."

As a result, organizations and law enforcement officials are increasingly interested in investing in tools that will allow them to assess the security of their information systems, and identify and address any vulnerabilities. Often times, the approach to investigating and assessing critical information systems involves interviews with individuals who own and maintain such systems and recording the details about such systems in handwritten notes that are later transcribed into a word processing program.

Due to the lack of a systematic process and standardized terminology, however, this approach is not effective, nor sustainable. It prevents people from accessing historical data from past assessments and leveraging the experience of previous assessors. Moreover, it does not afford real-time assessment and prevents people's ability to compare and identify patterns of data. Finally, effective collaboration on assessments is nearly impossible, as it is cumbersome to share and expand upon the collected information.

As we outlined in a paper presented at IEEE's Technologies for Homeland Security Conference in November 2013, The CERT Assessment Tool: Increasing a Security Incident Responder's Ability to Assess Risk, organizations and law enforcement officials tasked with assessing and/or responding to large-scale attacks on critical information systems typically face three challenges:

  • It is hard for organizations and security incident responders to establish adequate levels of trust. Organizations are reluctant to share information and communicate when effective response to such attacks requires them to share information including networks, diagrams, and logs.
  • Even after establishing adequate levels of trust, it is extremely hard to manage the sheer number of tasks and processes in the assessment and response plan. This plan includes identifying the vulnerability or a threat, determining if adequate controls are in place, mitigating the vulnerability if necessary, sharing the response plan, participating in collaborative decision making, and sharing information and analysis across the team of incident responders.
  • Site administrators who own the critical information systems are at the core of the assessment and response plan and thus must be involved during the entire duration of the assessment and/or the event. This proves challenging because site administrators may need to make modifications to a system that they depend on, so the solution involves them in the process.

Foundations of Our Approach

In 2010, at the request of a federal government agency, we began work on development of a customer-specific version of a tool and accompanying training component that would enable that agency to conduct critical information systems assessments.

After we deployed the tool to the agency, we received positive feedback that it was an operational solution with a built-in training path for critical information systems assessment. We received additional feature requests for the tool, including scaling the methodology to other domains within the organization. This feedback led to the development of the CERT Assessment Tool prototype, which provides a framework for assessing and addressing risk in critical information systems in various domains. The prototype, which is described in our paper, proves the concept of facilitating multiple people to actively collaborate, investigate, plan, and respond to ensure the confidentiality, integrity and availability of their critical information systems and the assets they manage.

Our approach, which can be tailored to any commercial or industrial domain, is based on established methodologies and principles for risk management and information assurance, including NIST's Recommended Security Controls for Federal Information Systems and Organizations (800-53) and Risk Management Guide for Information Technology Systems (800-30). The CERT Assessment Tool's hierarchical data model addresses the identified need to standardize the terminology and taxonomy used during assessments. Having this data hierarchy and structure enables the application to create a systemic approach to assessment.

Data Model

Below we have included an image of a data model, which shows the hierarchy and relational qualities that each assessment has to present and future objects in the system. Each assessment can have multiple sites. For each site there are multiple systems, and for each system there are at least six system attributes collected to describe the platform, configuration, vulnerabilities, controls, mitigations, and responsible contact.


The CERT Assessment Tool incorporates machine learning and decision support systems to provide actionable information and guidance that is repeatable. The tool and its systems reduce the reliance on experts, making knowledge more accessible and available when it is needed. The application utilizes Drools, an open-source business rule management system, to enable users to create domain-specific rules.

Our approach defines a concrete process of information collection and assessment, guiding the user through various stages of assessment, through the implementation of the security plan. The approach also incorporates a role-based model that defines the entities involved in the assessment and their responsibilities.

How the CERT Assessment Tools Works

The prototype of the CERT Assessment Tool currently runs on any Windows computer; however, the prototype can be architected for mobile and tablet deployment as well, which will provide flexibility to users and afford a nearly real-time assessment on site.

The application enables users to record the interview with a site/system administrator and tag the information about the system to create object meta data, and establish a standardized assessment taxonomy.


The image above illustrates areas where the tool segments the functionality for the assessment. The left-hand side of the image lists the systems and all of the tagged information. The center section is the free-form text area to support unrestricted information collection and tagging. The right panel provides real-time guidance questions based on the user's inputs and expert system push notifications should a system be tagged with a known vulnerability. For a proper critical information system (e.g., HVAC, energy delivery, elevators) assessment, the user is required to enter and tag the following information:

  • System Name
  • Characteristics - e.g. type of connection, updating/patching process, model number, etc.
  • Contacts - e.g. any person associated with the system
  • Vulnerabilities - any flaws or weaknesses in the system security
  • Controls - any safeguards or measures implemented to minimize or eliminate the identified vulnerabilities
  • Mitigations - any risk-reducing controls recommended from the risk assessment

Once the information is tagged, the system creates push notifications that help the user ask additional interview questions, identify vulnerabilities or recommend possible mitigations. One of the sources the application taps for these recommendations is the National Vulnerability Database (NVD). For example, if one of the tagged system characteristics is Windows 7, the application will search the NVS and display the common vulnerabilities for Windows 7 to provide the user with additional context for the assessment.

Once enough information is tagged and the system reaches a certain threshold of available information from past assessments, it is possible to create automatic rules using the Drools engine to provide specific expert advice about system vulnerabilities, existing controls, and mitigations.

If we apply this approach and methodology to the HVAC system that was compromised during the Target breach, the initial (very simplified) assessment taxonomy would look like this:

System Name: HVAC


  • Vendor-Updated
  • Windows 7
  • Internet-Connected

Contacts: Fazio Mechanical Services - Vendor


  • Remote Access
  • Active Directory Credentials

Controls: Password Policy

Based on this information, the CERT Assessment Tool with its guidance questions and decision support system would be able to recommend or alert the user that a remote access capability with active directory credentials is a vulnerability that needs to be properly mitigated to prevent a security breach.

The CERT Assessment Tool can be used for assessments of a wide variety of information systems, which may or may not be deemed critical by the organization:

  • Industrial Control Systems (ICS) that manage any kind of industrial production, including the following:

  • SCADA (Supervisory Control and Data Acquisition) computer information systems monitor and control industrial, infrastructure or facility-based processes in commercial buildings, airports, hospitals such as HVAC and energy distribution.

  • DCS (Distributed Control System) information systems control a manufacturing system, process, or any kind of dynamic system in which the controller elements are not centralized in one location but are distributed throughout the system with each component sub-system controlled by one or more controllers. Examples of such systems are electrical power grids, water management systems, and traffic signals.

  • Distributed File Systems (DFS) that allow access to files from multiple hosts sharing via a computer network, such as the NFS from Sun Microsystems and DFS from Microsoft
  • Network Information Systems (NIS) that manage networks such as gas supply or telecommunications

Looking Ahead

Given the increasing complexity and frequency of attacks on critical information systems, organizations and security incident responders need a tool that will allow them to effectively collaborate on assessments and security planning.

The CERT Assessment Tool allows users to take a more pro-active role in risk management and gain an enhanced situational awareness of all systems on site. The application is the first line of defense in making users aware of the security of their information systems.

Additional Resources

Read the paper, The CERT Assessment Tool: Increasing a Security Incident Responder's Ability to Assess Risk, by Anne Connell and Todd Waits.


Get updates on our latest work.

Each week, our researchers write about the latest in software engineering, cybersecurity and artificial intelligence. Sign up to get the latest post sent to your inbox the day it's published.

Subscribe Get our RSS feed