Modern software systems are constantly exposed to attacks from adversaries that, if successful, could prevent a system from functioning as intended or could result in exposure of confidential information. Accounts of credit card theft and other types of security breaches concerning a broad range of cyber-physical systems, transportation systems, self-driving cars, and so on, appear almost daily in the news. Building any public-facing system clearly demands a systematic approach for analyzing security needs and documenting mitigating requirements. In this blog post, which was excerpted from a recently published technical report, we present the Hybrid Threat Modeling Method that our team of researchers developed after examining popular methods.

While security can be analyzed at the networking and code levels to prevent buffer overflows, SQL injection attacks, and other issues, there is value in creating a mindset of defensive thinking early in the requirements engineering process. Thinking defensively means that for every new requirement or feature, we need to think about how it could be abused or defeated by adversaries--this mindset underlies the approach to threat modeling.

Based on our discussions with practicing threat modelers, we adopted the following definition:

A threat modeling method (TMM) is an approach for creating an abstraction of a software system, aimed at identifying attackers' abilities and goals, and using that abstraction to generate and catalog possible threats that the system must mitigate.

Threat modeling is performed by a variety of organizations, commercial, government, and everything in between. It is a required part of Program Protection Plans within the DoD.

Foundations of Our Work

In our initial research into threat modeling, we experimented with the following threat modeling methods to determine which of these methods was clearly the most efficient and effective in identifying valid threats, while minimizing the number of false positive errors:

• PnG. A persona non grata (PnG) represents an archetypal user who behaves in unwanted, possibly nefarious ways. However, like ordinary personas, PnGs have specific goals that they wish to achieve and specific actions that they make take to achieve their goals. Modeling PnG can therefore help us think about the ways in which a system might be vulnerable to abuse, and use this information to specify appropriate mitigating requirements. The PnG approach makes threat modeling more tractable by asking users to focus on attackers, their motivations, and abilities.
• Security Cards. The Security Cards approach to threat modeling emphasizes creativity and brainstorming over more structured approaches, such as checklists, to help users identify unusual or more sophisticated attacks. The method is suitable for purposes ranging from fundamental learning about security threats to aiding professionals in system design.
• STRIDE. Perhaps the most well-known and widely used TMM, STRIDE represents the state of the practice. STRIDE relies on a checklist evaluation approach based on the six threat categories to identify specific threats to a system: Spoofing identity, Tampering with data, Repudiation, Information disclosure, Denial of service, and Evaluation of privilege. Invented in 1999 by Kohnfelder and Garg and implemented at Microsoft, STRIDE uses system model and the related data flows.

In our initial study, we compared the PnG approach, Security Cards, and STRIDE. Two scenarios were used: an aircraft maintenance scenario and a drone swarm scenario. We found the following:

• Subjects who used the STRIDE method did not report a lot of false positives, but the teams generally obtained inconsistent results. The threats reported seemed to have more to do with the makeup of specific teams and their background or experience.
• With Security Cards, the teams of participants exhibited higher effectiveness. Almost all types of threats were found by teams using Security Cards. This approach, however, produced many false positives.
• With Persona non Grata, our research participants reported fewer false positives, and consistency among the teams increased (that is, we could have more confidence that threats detected by PnG would be reported consistently by many different teams). However, PnG threat modeling also tended to detect only a subset of threat types, which could indicate that certain threat classes were not easily detected with this method.

No individual threat model included all identified threats. We therefore explored the idea of crowdsourcing the task of threat identification. Our approach used information retrieval techniques to analyze the identified threats and collate them and provided the results to a human analyst to assist with construction of a unified threat model. The results of this prior research led to our development of the Hybrid Threat Modeling Method (hTMM) as detailed below.

Developing a Hybrid Threat Modeling Method

Our earlier research showed that no one TMM excelled in all of the aspects of interest. We therefore conducted additional work aimed at developing a "best of breed" approach that combines the strengths of each existing approach. Since the three TMMs that we studied were all intended to be stand-alone approaches--with different expectations about inputs and outputs--it was not feasible just to apply one after the other. Our research goal for this subsequent phase of the work was to create such a hybrid approach and explore whether it represented an improvement over existing approaches.

In developing a hybrid TMM (hTMM), we sought the following characteristics:

• yields minimal false positives
• does not overlook threats - encourages thinking outside the box so as not to miss subtle or unusual threats
• produces consistent results regardless of who is doing the threat modeling - provides a level of confidence in the results when applied by any appropriate team
• is cost-effective (doesn't waste time)
• produces empirical evidence to support its efficacy
• suggests a prioritization scheme
• is easy to learn, intuitive
• is clearly superior for specific types of systems

Steps for the Hybrid Threat Modeling Method

As described in our technical note, the steps in the hTMM are as follows:

1. Identify the system you will be modeling. Execute Steps 1-3 of SQUARE or a similar security requirements method, e.g.

a. Agree on definitions.
b. Identify a business goal for the system, assets, and security goals.
c. Gather as many artifacts as feasible.

2. Create a large initial set of possible threats by applying Security Cards in the following way:

a. Distribute the Security Cards to participants either in advance or at the start of the activity. Include representatives of at least the three following groups of stakeholders: system users/purchasers, system engineers/developers, and cybersecurity experts. You may find that within each of those categories, there are multiple distinct perspectives that must be represented. Other relevant stakeholders can be included, as well:

i. System users/purchasers would include those purchasing or acquiring the system, end users, and other groups with a vested interest in the system. For example in a scientific research organization, stakeholders could include the scientists conducting research, the executive directors of the organization, human resources, and information technologists managing the system. Each role would have its own ideas about assets that must be protected and potential attackers.
ii. Cybersecurity experts could be part of a separate specialized team or integrated into the project team. They could include roles such as system administrators, penetration testers or ethical hackers, threat modelers, security analysts, and so on.
iii. The engineer/development team members could range from systems engineers, requirements analysts, architects, developers, testers, and so on.

b. Have the participants look over the cards along all four dimensions: human impact, adversary's motivations, adversary's resources, and adversary's methods. To familiarize themselves with the type of information on the cards, have participants read at least one card from each dimension, front and back.

c. Use the cards to support a brainstorming session. Consider each dimension independently, and sort the cards within that dimension in order of how relevant and risky it is for the system overall. Discuss as a team what orderings are identified. It's important to be inclusive, so do not exclude ideas that seem unlikely or illogical at this point in time. As you conduct your brainstorming exercise, record the following:

i. If your system were compromised, what assets, both human and system, could be impacted?
ii. Who are the personae non gratae who might reasonably attack your system and why? What are their names/job titles/roles? Describe them in some detail:

1. What are their goals?
2. What resources and skills might the PnG have?

iii. In what ways could the system be attacked?

1. For each attack vector, have you identified a PnG (or could you add a PnG) capable of utilizing that vector?

3. After the data in Step 2 has been collected, you will have enough information to prune the listed attacks, based on which have PnGs that are unlikely, and which have no realistic attack vectors could be identified. Once this is done, for the remaining attacks:

a. Itemize their misuse cases. This expands on HOW the adversary attacks the system. The misuse cases provide the supporting detailed information on how the attack takes place.

4. Summarize the results from the above steps, utilizing tool support, as follows:

a. Actor (PnG): Who or what instigates the attack?
b. Purpose: What is the actor's goal or intent?
c. Target: What asset is the target?
d. Action: What action does the actor perform or attempt to perform? Here you should consider both the resources and the skills of the actor. You will also be describing HOW the actor might attack your system and its expansion into misuse cases.
e. Result of the action: What happens as a result of the action? What assets are compromised? What goal has the actor achieved?
f. Impact: What is the severity of the result (high, medium, or low)
g. Threat type: (e.g., denial of service, spoofing)

5. After the preceding steps are done, you can continue with a formal risk assessment method, using these results, and the additional steps of a security requirements method (such as SQUARE), perhaps tailoring the method to eliminate steps you have already accounted for in the threat modeling exercise.

Steps 1 and 5 are activities that precede and follow the bulk of the threat modeling work. We felt it was necessary to include these, to understand where hTMM fits into lifecycle activities, specifically security requirements engineering.

Our focus today in our work on hTMM is scaling up to larger systems and measuring the effort requirements as well as the results, in terms of the effectiveness of the method for identifying appropriate threats. We are also examining the feasibility of the method outcomes for generating improved security requirements, so that threats can be identified early in the lifecycle when proper mitigations can be designed and implemented most easily. We are currently applying hTMM as part of research work funded by the DoD with application on realistic programs.

Wrapping Up and Looking Ahead

In this blog post, we described three threat modeling methods used in our earlier research project: Security Cards, Persona non Grata, and STRIDE. The purpose of the earlier project was to evaluate these approaches in terms of measurable outcomes, including how many relevant threats they identified, how many false positives were generated, and the consistency with which they are applied across different teams.

There was no clear winner in the earlier study. We therefore developed a hybrid method that attempted to meld the desirable features of all three methods. As part of that activity, we developed some threat profile examples using our standard scenarios, which is detailed in our technical note. We then applied a hybrid thread modeling method, or hTMM, to one of our standard scenarios. We are currently engaging in larger pilot studies to validate and/or revise the method at scale. Ultimately, we hope to collect data to support the efficacy of the hybrid method.

We welcome your feedback on this research in the comments section below.

Additional Resources

Read the SEI technical note from which this blog post has been excerpted.