Well-known asymmetries pit cyber criminals with access to cheap, easy-to-use tools against government and industry organizations that must spend more and more to keep information and assets safe. To help reverse this imbalance, the SEI is conducting a study sponsored by the U.S. Office of the Director of National Intelligence to understand cyber intelligence best practices, common challenges, and future technologies that we will publish at the conclusion of the project. Through interviews with U.S.-based organizations from a variety of sectors, we are identifying tools, practices, and resources that help those organizations make informed decisions that protect their information and assets. This blog post describes preliminary findings from the interviews we have conducted so far. Our final report, which will include an anonymized look at the cyber intelligence practices of all the organizations we interviewed, will be released after the conclusion of the study in 2019.

We are currently accepting additional U.S.-based organizations for the study. If your organization would like to participate, please contact cyber-intel@sei.cmu.edu. There is no cost to participate other than your cyber intelligence team's time (2-3 hours for an interview with our SEI team). As a participant, your organization will receive a private comparative analysis of your own cyber intelligence practices based on benchmarks we develop as part of the study. This analysis can help you make improvements, understand your strengths, and justify investments.

Cyber Intelligence Best Practices

In late 2017 we began our cyber intelligence study. Over the past few months, we have noted a few practices high-performing organizations share. Although this list will certainly grow over the course of the study, we have identified the following cyber intelligence best practices based on our initial research:

• Know your environment.
  In our interviews for this study, high-performing organizations demonstrate in-depth knowledge of their environment. This means that they know their assets, people, intellectual property, patents, business units, and data. They also know what they don't know, i.e., they know where their gaps are.
  
  High-performing organizations also focus on the who and why of a threat. They consider threat actors, motivations, and capabilities, and they map that information to known vulnerabilities and assets threat actors may target. These organizations know what critical assets they have, and what their competitors don't have. Their cyber intelligence analysts and team leaders get out and talk to employees working on sensitive projects to understand how to protect those projects.
• Make the most of new technologies.
  Emerging technologies, such as advanced computing, applied artificial intelligence and machine learning, and human-machine interaction, are already revolutionizing the activities associated with cyber intelligence. By learning about and adopting these technologies, your cyber intelligence team can process more information and work smarter when it comes to analyzing that information.
  
  High-performing organizations have used machine learning to process articles and other intelligence information to isolate relevant information, freeing analysts to focus their efforts on specific threats. Adding machine learning experts, software developers, and those with data science experience to your team can be a huge win. With these experts, you can develop tools to fit your organization's unique needs.
• Make the most of your informal sources and networks.
  Cyber intelligence analysts at high-performing organizations rely on peers they know well and trust, even--and sometimes especially--when those peers are outside their own organizations. The development of these peer-to-peer relationships from networking or past jobs or degree programs is valuable, particularly for identifying trends and in determining the credibility of cyber threat data.
• Hire curious critical thinkers--not just technical experts--for your cyber intel team.
  When it comes to building your cyber intelligence team, curiosity, critical thinking, an analytical mind, strong interpersonal skills, communication skills, and a good team fit is just as important as hiring someone with strong technical skills. In our interviews, we are hearing that training new team members on technical tools if often easier than teaching critical thinking. New hires don't necessarily need to know the tools from the start, but with a passion to learn, critical thinkers can become extremely skilled at using them to get the right information quickly.

Cyber Intelligence Biggest Challenges

Many cyber intelligence teams struggle with the same difficulties. In our initial interviews, we have identified the following shared challenges:

• Establishing a clear definition of cyber terminology.
  Our preliminary interviews indicate that agreement on cyber terminology is still a challenge across industries and even within organizations. Different and often overlapping definitions exist for terms like cyber intelligence, cyber threat intelligence, and even cybersecurity. Some organizations see cyber intelligence and cyber threat intelligence as the same thing and use the terms interchangeably; some see them differently. Lack of clear definitions leads to confusion when establishing roles, assessing threats, making a strategy for defense, and understanding what information to report.
• Incorporating analytical tradecraft into cyber intelligence.
  We are finding in our current research that analytical tradecraft is still not fully instantiated into cyber intelligence performance. The 2015 Intelligence Community Directive 203 (ICD 203) and CIA's structured analytical techniques are good sources for analytical tradecraft. For example, are cyber intelligence reports objective, timely, and sourced correctly, and do they use estimative language and clear lines of argumentation? Do teams apply red teaming, the analysis of competing hypotheses, and what-if analysis? Without analytical tradecraft, cyber intelligence teams run the risk of providing unreliable and inaccurate intelligence in poorly written reports to decision makers.
• Communicating cyber to leadership.
  For some organizations, communicating cyber to leadership is easy because the CISO or direct manager has a technical background, or the organization is small enough for people to simply walk down the hall and talk with managers. Yet communicating cyber to leadership is still a challenge for many organizations. We have recently met with organizations whose cyber intelligence teams have not briefed their boards about cyber threats in years. We have also met with cyber intelligence teams that are buried in layers upon layers of bureaucracy, making it hard for them to get the right data to the right of level of leadership in a timely manner. Some cyber intelligence teams report to leaders who lack any technical background, or leaders from law enforcement backgrounds who are overly focused on threat attribution.
• Demonstrating return on investment in cyber intelligence for decision makers.
  Related to communicating cyber to leadership is demonstrating ROI. Making sure decision makers understand the value your cyber intelligence program provides is essential to ensuring your cyber intelligence efforts are funded and supported. Organizations have shared with us their difficulties in framing cyber intelligence as producing a return on investment when decision makers see cyber intelligence as cost avoidance.
  
  Merely being in compliance with increasingly strict regulations or being faster than the slowest gazelle in the herd when it comes to patching and other threat defenses is not enough to demonstrate ROI. Consider the costs of mitigating a threat from a threat actor who is specifically interested in a particular technology your organization might be working on. If that technology is compromised before you go to market, you might never be able to reap the profits that could be generated once that technology goes public. In this scenario, the investment in a cyber intel team that can understand threat actors, what technology they are interested in, and why they want your particular technology can enable your organization to make prudent and proactive decisions on how to protect that technology.

Study Background and Structure

This project is an update to the ODNI-sponsored Cyber Intelligence Tradecraft Project we completed in 2013 with around 30 participating organizations. The report from that project has been widely cited in academic and commercial publications and is being used as a study text in university courses.

The current study will culminate in a public report based on anonymized information from our interviews with organizations. Our overall analysis will consider how organizations are performing in the following categories:

• Environment-- Analysts assess the environment to establish the scope of the cyber intelligence effort and the data needed to accomplish it.

• Data gathering--Through automated and labor-intensive means, analysts explore data sources, collect information, and aggregate it to perform analysis.
• Functional analysis--Analysts use gathered data to perform technical and tailored analysis, typically in support of a cyber security mission. Functional analysis answers the where, when, and how aspects of a cyber threat.
• Strategic analysis--Analysts apply a strategic lens to functional data and report this intelligence to a decision maker or use it to influence the environment. Strategic analysis aims to answer the "who" and the "why" and weaves in the functional data.
• Reporting and feedback--After intelligence is disseminated to decision makers, they provide feedback and/or use the intelligence to influence the environment.

By providing a public report on the state of cyber intelligence of U.S.-based organizations, we aim to enable organizations to improve their practices and understand the most pressing problems plaguing cyber intelligence efforts across the country.

Additional Resources

Read a summary of key findings of the 2013 Cyber Intelligence Tradecraft Project.