In June, representatives of organizations in the government, military, and industry sectors--including American Express and PNC--traveled to Pittsburgh to participate in a crisis simulation the SEI conducted. The crisis simulation--a collaborative effort involving experts from the SEI's Emerging Technology Center (ETC) and CERT Division--involved a scenario that asked members to sift through and identify Internet Protocol (IP) locations of different servers, as well as netflow data. Participants also sorted through social media accounts from simulated intelligence agencies, as well as fabricated phone logs and human intelligence. Our aim with this exercise was to help cyber intelligence analysts from various agencies learn to think critically about the information they were digesting and make decisions that will protect their organizations in the event of a cyber attack or incident and increase resilience against future incidents. This blog post, the second in a series highlighting cyber intelligence work from the ETC, highlights the importance of critical thinking in cyber intelligence, as well as a three-step approach to taking a more holistic view of cyber threats.

The importance of applying critical thinking to cyber intelligence cannot be overstated. In our work with organizations, we have noticed that when a new threat arises, instead of holistically assessing it, organizations often simply request the latest, greatest analytic tool or contract out the work to third-party intelligence providers. As a former intelligence analyst--prior to joining the SEI, I served as a counterintelligence and counterterrorism analyst for the Naval Criminal Investigative Service (NCIS)--I know from experience that the operational tempo required for intelligence analysts to keep pace with the ever-changing cyber environment is overwhelming at best. While technology and external resources offer value, analysts also need to critically assess the information they receive.

In 2013, the Defense Science Board echoed a similar sentiment. In their report, Resilient Military Systems and the Advanced Cyber Threat they included the following among their recommendations to improve DoD systems' resilience: "Refocus intelligence collection and analysis to understand adversarial cyber capabilities, plans and intentions, and to enable counterstrategies."

Foundations of Our Work

Our work in cyber intelligence started in 2012 with a request from the government to assess the state of the practice of cyber intelligence. Our work on that initial project involved an examination of the cyber intelligence practices of 30 organizations (6 from government and 24 from industry), specifically their strategic approaches to cyber intelligence. Our work focused on identifying the methodologies, processes, tools and training that shaped how these organizations assessed and analyzed cyber threats. As detailed in an earlier blog post, our work on this project resulted in an implementation framework that captured best practices.

When this work concluded, several participant organizations approached the ETC about leading an effort that would research and develop technical solutions and analytical practices to help people make better judgments and quicker decisions with cyber intelligence. As a result, ETC launched the Cyber Intelligence Research Consortium.

The first year of this consortium focused primarily on continuing our research in cyber intelligence, as well as identifying best practices and challenges. Nearly four years after our initial research began, we have noted clear examples of a strategic shift among participant organizations with respect to cyber intelligence. They are investing resources in hiring intelligence analysts from a pool of vetted and qualified experts, and they are investing significant resources in acquiring tools and tradecraft. However, they are not yet making effective use of the intelligence provided by these resources.

In both government and industry, organizational resilience in the wake of an attack relies on an analyst's ability to holistically assess a threat. The remainder of this post proposes a three-step approach for holistically approaching a cyber threat.

Three Steps to Holistically Assess Cyber Threats

First and foremost, applying critical thinking--which brings together all the skills shown in the "conceptual framework" above--to cyber threats improves an analyst's ability to accurately evaluate and estimate a threat's potential to impact and expose its target. My ETC colleagues and I propose a three-step approach to holistically assess cyber threats:

1. Establish a baseline of how the threat will be analyzed. This step involves outlining the approach so that the analyst uses all the skills represented in the conceptual framework. Since the framework is non-linear, the components can be approached in whatever order makes sense.
2. Leverage creative brainstorming. When facing a potential cyber threat, analysts don't have the luxury of time to stare into space and wait for an "ah ha" moment. Creative brainstorming techniques such as those found in human-centered design accelerate the time it takes to get to an "ah ha" moment. To enhance an analyst's creative brainstorming skills, I recommend looking at recent brainstorming research including that by Luma Institute, specifically their 36 techniques for creative brainstorming and practice them daily.
3. Perform the holistic threat assessment. The assessment evaluates the threat from the three perspectives shown in the figures below:
   
   
   

The three steps outlined in this approach enable analysts to avoid intelligence tunnel vision and seek to understand all causes and effects of relevant threats, which can significantly improve the efficiency and effectiveness of cyber intelligence efforts.

Two Case Studies

This section presents two examples of how a holistic approach to cyber intelligence improved an organization's ability to counteract cyber threat.

• A civil service federal government agency was monitoring open source publications from an entity known to sponsor cyber threat actors who frequently targeted the agency. By examining threat actor potential, organizational impact, and target exposure, the agency came to understand the motivations of the sponsored cyber threat actors, what effects a successful attack might have on the organization, and what vulnerabilities existed that exposed the agency to the threat. This approach enabled the agency's analysts to narrow down the types of data likely to be targeted, to work with network security experts to create diversions and honey pots, and to proactively defend against the threat.
• An organization we worked with from the retail sector focused on the extracurricular activities of its CEO, who was active with companies, non-profits, and policy institutes in a capacity that had nothing to do with his responsibilities at the retail company. The company's cyber intelligence analysts knew this and maintained an awareness of his activities, so when hacktivists publicly threatened attacks against one of the institutes, the analysts knew this could have implications for the retail organization. This attention to target exposure (their CEO's connection with a targeted entity), along with an examination of threat actor potential and organizational impact, enabled analysts to prepare for and successfully stop the attack that eventually happened.

Wrapping Up and Looking Ahead

While the ETC will continue in its efforts to combat the dizzying operational tempo of cyber intelligence with technology, it is equally important to focus on enhancing analytical brainpower. As intelligence analysis pioneer Dr. Richards J. Heuer once observed,

Analysts at all levels devote little attention to improving how they think. To penetrate the heart and soul of improving analysis, it is necessary to better understand, influence, and guide the mental processes of analysts themselves.

There is an elegance involved in designing, developing, and delivering ways for analysts to enhance critical thinking skills. We are working on several fronts to help intelligence analysts acquire these skills:

• I teach a graduate-level strategic cyber intelligence course through Carnegie Mellon's Information Networking Institute (INI); the second offering of the course wrapped up at the end of the fall semester, and I plan to teach it again in Fall 2016. In that class, I emphasize the importance of taking a holistic approach to analyzing cyber threats. My colleague, Jared Ettinger, and I also are working with INI to develop a second cyber intelligence course to be taught for the first time in spring 2017.
• Our Cyber Intelligence Research Consortium is continuing to meet and work on real cyber intelligence problems facing real organizations. We will hold a tradecraft lab in April featuring cyber intelligence experts, discussion amongst members, and output of student projects being run through CMU's Heinz College about cyber intelligence strategy. We are also in the early stages of planning for another crisis simulation for August 2016.

We welcome your feedback on this research and suggestions for future posts in the comments section below.

Additional Resources

View the slides for my presentation, Be Like Water Applying Analytical Adaptability to Cyber Intelligence, which I presented at the 2015 RSA Conference.

Learn more about the ETC's Cyber Intelligence Research Consortium.