SEI Insights


Vulnerability Insights

E Pluribus, Que? Identifying Vulnerability Disclosure Stakeholders

Posted on by in

On September 29, Art Manion and I attended the first meeting of the Multistakeholder Process for Cybersecurity Vulnerabilities initiated by the National Telecommunications and Information Administration (NTIA), part of the United States Department of Commerce. There has been ample coverage of the meeting in blogs (e.g., by Dr. Neal Krawetz and by Cris Thomas), mailing lists, and media reports, so I won't attempt to duplicate that information.

During the course of the meeting, I became increasingly aware that no one had actually specified who the "stakeholders" are. The name of the process implies there are more than one, but nowhere in the accompanying materials was there a comprehensive list of who is perceived to hold a stake in this process. In this post, you'll find my take on who those stakeholders are and how the set of players has changed over the past decade.

Vox Populi

The concern I have with the multistakeholder process is that vulnerability disclosure is too important a topic to let discussions and decisions fall only to those who are sufficiently motivated and funded to represent their views. First, there are stakeholders that may be unaware of their role in the matter. Second, there are other stakeholders that, while aware, lack the resources to contribute their time, efforts, or travel budgets to participate in meetings, conference calls, and the like.

It's in the interest of these two groups that I want to acknowledge their existence and ensure that they are given due consideration in whatever conversations ensue.

Who are these stakeholders?

In a talk I gave at RVASec (video, slides) this past summer, I included a survey of vulnerability disclosure process models. One of these models is particularly relevant here, as it covers much of the same territory as the NTIA Multistakeholder Process for Cybersecurity Vulnerabilities.

I'm referring to the National Infrastructure Advisory Council (NIAC) Vulnerability Disclosure Framework (NIAC VDF), released in 2004. In fact, the NIAC VDF has an entire section dedicated to defining stakeholders. Here are some relevant excerpts:

Stakeholders may be grouped into four major categories: discoverers, vendors, users, and coordinators.

Discoverers include individuals or organizations that find vulnerabilities. Subgroups include researchers, security companies, users, governments, and coordinators.

Vendors develop or maintain information system products or services that may be vulnerable. Subgroups include information security teams, product security teams, incident response teams, researchers, communications coordinators, legal officers, and operators.

End Users and Organizations ... includes everyone using a vendor's product that could be affected by a vulnerability. Subgroups include governments, critical infrastructure owners and operators, and service providers. Each of these groups may also have information security teams, incident response teams, researchers, communications coordinators, legal officers, and operators.

Coordinators can manage a single vendor's response or multiple vendors' responses to a vulnerability. Coordinators may also serve as unbiased, independent evaluators of severity, and may act as a medium for communicating with the public and multiple users and vendors. Additionally, coordinators may be able to enhance international reporting, especially in support of organizations that are prohibited from reporting issues to non-citizens. A coordinator may be in a good position to study relationships among vulnerabilities and recognize trends. The most important attribute of a coordinator is to be trusted.

Having reviewed the NIAC VDF's survey of stakeholders, I noticed that a number of things have changed in the decade since that work was completed. Discoverers, vendors, users, and coordinators are still present and relevant, but there are a few new ones that the NIAC VDF didn't address. Additionally, some of the stakeholder subgroups have grown to the point where their concerns may in fact be distinct enough to warrant their treatment as top-level stakeholder groups of their own.

End Users

In the past, one might have argued that only computer users were affected by vulnerabilities and their disclosure: this is no longer the case. The scope of the citizenry affected by cybersecurity vulnerabilities has widened considerably. Affected users now include those who have smart phones, watch smart TVs, use credit cards or ATMs for banking and/or shopping, drive cars, fly in airplanes, go to the hospital for diagnostic imaging or intravenous medicine, live in houses with smart meters, etc. The list goes on to include nearly everyone, and "opting out" is not a viable position for most people to take.

Around the time of the NIAC VDF report, it was controversial when the CERT/CC advised users to "use a different browser" in response to a vulnerability in the most popular browser of the day (VU#713878). (See also followup coverage from IBM Systems Magazine.) However consider the implications today if we were to issue similar advice: "use a different phone," "drive a different car," or "use a different bank". If those phrases give you pause (as they do me), you have recognized how the importance of this issue has grown.

Cloud Providers

Traditional software vendors were covered by the NIAC VDF, and in it service providers were included as a subgroup of end users. Insofar as cloud-based services are built on top of traditional computing platforms, that much is still true. However, as cloud-based services (e.g., software, platform, and infrastructure as a service) have risen to prominence, they have also distinguished themselves from traditional software vendors since their development, deployment, and delivery processes for security fixes tend to be much more direct.

The ascent of DevOps tactics, techniques, and practices among such providers means that the time from code commit to last vulnerable system patched can sometimes be measured in minutes. To be sure, development and delivery processes in traditional software environments have accelerated considerably since 2004 as well, but the fact that the service provider has direct control over the vulnerable systems makes a significant difference in its ability to mitigate vulnerabilities across all its users in short order.

Internet of Things

Another class of software vendors are the purveyors of Internet of Things (IoT) products. These kinds of products and services are on the opposite end of the deployment spectrum from cloud-based services. Whether we're talking about cars, televisions, medical devices, airplanes, sensors, home automation, or industrial control systems, too often today the patch deployment process involves going out and physically touching the thing that needs to be updated.

Patch deployment will likely improve as more connected things get Over-the-Air (OTA) update capabilities, but there is already a large installed base of systems lacking such features. Furthermore, systems at the lower end of the price range might have "fire and forget" assumptions built into their pricing model, meaning that there is neither the technical means to deliver updates nor the support capability in place to even develop them in the first place.

A second issue with these devices is their supply chain, whereby the vendor of the final product actually has very little to do with the hardware, firmware, or software development of the system they sell. I've previously described additional differences in my blog post, Vulnerability Analysis and Discovery for Emerging Networked Systems.

Mobile Platforms and Applications

Mobile devices present yet another class of stakeholders that has grown distinct since 2004. The device vendors themselves are most akin to IoT vendors, but app developers can be quite a diverse bunch, ranging from very large traditional software companies, to cloud services, to a student with a good idea and a few hours of coding.

Some mobile devices have multi-stage, serial supply chains, each step of which can stand in the way of security updates reaching their intended beneficiaries (i.e., the users). Will Dormann discussed this issue in his recent Supporting the Android Ecosystem post. In both the mobile and IoT spaces, high-viscosity supply chains are bad for end-user security.

Bug Bounties and Commercial Vulnerability Response Programs

In recent years, a new class of coordinator has emerged in the form of commercial bug bounty programs. Many individual vendors have established programs to compensate security researchers for their efforts in discovering vulnerabilities in the vendor's products. Creation of a bug bounty program has been noted as an indicator of maturity in vendors' vulnerability response efforts.

In some cases, vendor bug bounty programs are enabled by other companies who provide tools and services to facilitate vulnerability coordination. Companies such as BugCrowd, HackerOne, Synack, and Cobalt offer turnkey solutions for vendors who want to bootstrap their own vulnerability response program.

While bug bounty programs help address the vulnerability coordination needs of individual vendors, there still are vulnerabilities that require larger scale coordination. In particular, multivendor coordination remains a challenge for many organizations. As individual vendors have become more mature in their handling of vulnerabilities in their products, the role of multivendor coordination has increased in importance for more traditional vulnerability coordinators like the CERT/CC, NCSC-NL, NCSC-FI, and JPCERT/CC.


Information Sharing and Analysis Organizations (ISAOs) and Centers (ISACs) are non-government entities that serve various roles in gathering, analyzing, and disseminating critical infrastructure cybersecurity information across private sector organizations of various sizes and capabilities.

These organizations have only begun to emerge in earnest within the past few years, but they are already actively involved in the coordination and deployment of vulnerability mitigations. Furthermore, it seems likely that at least a few critical infrastructure sectors will need to become involved further upstream in the coordination of the vulnerability discovery, disclosure, and remediation process.

The United States and Other Governments

Governments are multifaceted stakeholders in regards to cybersecurity vulnerabilities and their disclosure. Whereas in 2004 the NIAC VDF noted the government's stake as the owners and operators of vulnerable networks and systems, the list below shows just how much more broad the issues of vulnerability discovery, coordination, disclosure, and mitigation have become in the government's role of, well, governing.

In the past few years we have observed the following:

  • Special Assistant to the President and Cybersecurity Coordinator, Michael Daniel, described the need to strike a balance between network defense and intelligence collection.
  • The Federal Trade Commission reached settlements with companies, in part due to their lack of "a clearly publicized and effective channel for receiving security vulnerability reports" (pdf).
  • The Department of Commerce Bureau of Industry and Security proposed rules for intrusion software that leverage zero day vunerabilities. (Our detailed comments on these rules are available.)
  • The Office of Personnel Management was hit with two separate but related cybersecurity incidents in which millions of individuals' sensitive and personally identifiable information was compromised.
  • The Food and Drug Administration (FDA) hosted a public meeting intended to "catalyze collaboration in the health care and public health sector to more fully address medical device cybersecurity."
  • The Government Accountability Office reported findings of vulnerabilities in the aviation security system.
  • The National Highway Traffic Safety Adminstration (NHTSA) declared its goal to "be ahead of potential vehicle cybersecurity challenges, and seek ways to address or avoid them altogether."
  • The Department of Commerce NTIA kicked off the multistakeholder process that prompted this post in the first place.

Increasingly, governments are being asked to take on new roles in cyberspace that are a logical extension of their roles in the physical world. For example, many regulated industries (medical, transportation, aviation, etc.) have long had mandatory reporting requirements for safety issues that are governed by specific agencies.

The FDA Medical Device Reporting process enables oversight and detection of potential device-related safety issues. The NHTSA collects reports of vehicle safety issues, which helps to drive its investigation and recall processes. The FAA offers a number of safety reporting capabilities as well.

As these industries move toward increasing connectivity, agencies with regulatory oversight responsibilities will likely see an increased demand to extend their safety monitoring to include security issues (especially for security issues that directly impact safety). As just one example, the House Energy and Commerce Committee recently held hearings on proposed legislation that would penalize car hacking.

Beyond just documenting observed issues, some government agencies take an active learning approach when failures occur. The aforementioned FDA and NHTSA reporting programs serve this purpose, but other programs exist as well. For example, the National Transportation Safety Board is explicitly tasked with investigating transportation accidents, and NASA has elevated learning from experience into a science. This kind of continuous improvement process has demonstrated its effectiveness in a variety of environments and seems to provide a good model for cybersecurity vulnerabilities in both the private and public sectors.

The United States is not alone in realizing that vulnerability discovery, disclosure, and remediation is important to national interests. These cybersecurity issues have been global for quite some time. The EU Parliament recently held hearings on modernizing export controls and the trade in zero-day vulnerabilities. Meanwhile, a quick glance at the vulnerability database catalog being developed by the Forum of Incident Response and Security Teams (FIRST) gives a good indication of the international interest in this problem space.

Value-Focused Thinking

In preparation for the NTIA meeting, I looked for documentation of the multistakeholder process, hoping to find a well-documented process with a relatively structured format that would move from initiation to completion of whatever the process goals were. And while I did find evidence that the phrase has been used repeatedly by NTIA, I was unable to find any process documentation to enlighten me.

What I did come across though, were references to Ralph L. Keeney's 1992 book Value-Focused Thinking: A Path to Creative Decisionmaking. In it, Keeney outlines a process that "consists of two activities: first deciding what you want and then figuring out how to get it." He contrasts this with "alternative-focused thinking," where "you first figure out what alternatives are available and then choose the best of the lot." There is a good summary of the book over at Less Wrong if you want to know more: however, I want to briefly highlight two points Keeney makes in Chapter 3, Section 8:

When multiple stakeholders are involved, it is essential to have a facilitator guide the elicitation of objectives from each stakeholder, structure each set of objectives into an objectives hierarchy, and aggregate the hierarchies into a single fundamental objectives hierarchy. The facilitator is typically an independent analyst hired by the decisionmaker, typically a government organization, because of his or her skills and impartiality to work on the specific decision being considered.

He goes on to recommend that constructing a broad and diverse set of objectives implies input from a broad and diverse set of stakeholders. Selecting the stakeholder groups and those individuals who represent them is important to both the effectiveness of the process and its perceived legitimacy by the various stakeholders who might be unable to participate directly.

In selecting participants, it is important to take into account both fairness and perceived fairness with which different groups are treated ... In practice, the careful selection of from three to five groups representing different perspectives in a decision problem is likely to provide a comprehensive list of objectives. The number of participating groups necessary to provide fair treatment is more problem-dependent.


I remain hopeful that the NTIA Multistakeholder Process for Cybersecurity Vulnerabilities will bear fruit by increasing mutual understanding and respect among the various stakeholders. A values-focused approach seems promising as a way to integrate a wide range of concerns and interests among such a diverse group. But, for that process to be perceived as both legitimate and fair, care must be taken to ensure that it is inclusive and gives a voice to those who will be affected by its outcome.

The NIAC Vulnerability Disclosure Framework offers a good starting point for identifying key stakeholders: discoverers, vendors, users, and coordinators. However, it would be remiss if the multistakeholder process failed to recognize the increased importance of users, cloud providers, IoT and mobile vendors, ISAOs and ISACs, and governments to the vulnerability disclosure landscape.

Epilogue: Regarding the Title

It's always a risk to construct a phrase in a language you haven't studied, moreso when you intend for that to be the first three words of the title of a piece. You are likely familiar with the phrase "E Pluribus Unum" from the Great Seal of the United States. This Latin phrase translates to "Out of Many, One."

In composing this post, I attempted to answer the question: If this is a multistakeholder process, who are these stakeholders of which you speak? Condensing this question by allusion to the Great Seal, I have attempted to ask: "Out of Many, Whom?" as "E Pluribus, Que?"

To come up with this translation, I consulted with a few folks who actually studied Latin some time ago (i.e., high school). Among those who were able to overcome their PTSD from that experience long enough to consult on this question, the conclusion was that "que" represents the neuter plural form of both the nominative (who) and accusative (whom) case. "Unum" in the original is the neuter singular form of the nominative case, so this seemed to be the most appropriate substitution. I could of course be wrong, and if so: mea culpa. :-)

About the Author

Allen Householder

Contact Allen Householder
Visit the SEI Digital Library for other publications by Allen
View other blog posts by Allen Householder