The CERT Coordination Center (CERT/CC) has been receiving an increasing number of vulnerability reports regarding Internet of Things devices and other embedded systems. We've also been focusing more of our own vulnerability discovery work in that space. We've discovered that while many of the vulnerabilities are technically the same as in traditional IT software, the coordination process has some substantial differences that will need to be addressed as the Internet of Things grows.

As an example, my coworker Joel Land recently completed a project to survey vulnerabilities in common home WiFi routers. He found quite a few vulnerabilities and we've gotten a number from independent researchers since then as well. You can find all the Vulnerability Notes we've published about home routers in our Vulnerability Notes Database. The project caused some discussion within our group about how to coordinate vulnerabilities in consumer products. There are certainly technical differences in discovering vulnerabilities in IoT and other embedded devices, many of which were enumerated in Allen Householder's CERT/CC blog posts here and here. We've discovered that there are quite a few logistical differences as well, and we're still batting around ideas for how best to handle them.

Home routers are only one case of a more general trend. If you take a stroll through your nearest big-box electronics store, you'll see dozens of items that connect to the Internet. DVRs, cameras, printers, lighting, appliances, and now even Barbie dolls! Our experience is that these all share a lot of the same security controls, or lack thereof. We haven't actually examined the Internet-connected Barbie (Matthew Jakubowski has), but we've seen a lot of similar vulnerabilities in the other devices. We've also found similarities in business and support models, supply chains, and underlying components. Since the challenges we've encountered in home routers are also likely to be challenges with these other devices, we can use our home router project to extrapolate a series of general challenges.

Challenge #1. Vendors are new to vulnerability disclosure - and security.

The CERT Coordination Center has been handling vulnerabilities in software for 27 years, through much of the debate over disclosure of vulnerabilities. In the past ten to fifteen years, most mature software companies have come to the conclusion that coordinated disclosure is a benefit to them and to their customers. Companies like Microsoft and Google have public ways to report vulnerabilities, standard methods of patching, bug bounty programs, and other practices. However, many of the companies that make consumer products do not yet have these practices in place. These companies are experts in making cameras, or lights, or appliances, rather than software. Of course, home router vendors obviously make devices that connect to the Internet, and can reasonably be expected to be familiar with best practices in security and vulnerability disclosure. They have a different problem....

Challenge #2. Business models.

Software companies have some inherent advantages. Once created, their product can be duplicated and sold as many times as needed, for virtually no additional unit cost. It's easy and cheap to change the product or add new features. Profit margins are large compared to many products. Consumer products are fundamentally different. Profit margins are much lower, and since each unit costs money to be produced, those costs need to be minimized. Changing the hardware is difficult and even changing the software is a challenge to users who are accustomed to auto-updating software on desktops and mobile devices. All of these factors mean the cost to fix vulnerabilities is more noticeable. Having to reengineer a product invalidates the original profit models. It also costs money to get fixes to all of the units - in warehouses, on store shelves, in homes, etc. A vendor may have to provide technical support to users who are uncomfortable updating their own hardware. All of these change the economics of the product.

Challenge #3. Customer expectations.

Consumers today are mostly aware that if they buy a computer or smartphone, that device will need to periodically have software updates. However, when consumers buy a home router, camera, lightbulb, or refrigerator, they expect to just unbox it and use it as-is, potentially for years. They don't think about the fact that their new device runs software and, increasingly, connects to the Internet. So even if CERT, researchers, and vendors all work together to make a patch available concurrently with publication of a vulnerability, users may not get the patch in a timely manner, if ever. More and more vendors are designing systems that will be securely and automatically updated over the air, but that number is still far from 100%.

Challenge #4. Contacting vendors.

There are vendors who are still new to vulnerability reporting and don't make it easy for anyone to contact them about security issues. They may not have a widely publicized "security@" email address or other way to contact them. Their social media, customer support or public relations departments may not know how to handle vulnerability reports. They may not have a dedicated PSIRT (Product Security Incident Response Team) or other product security team. Unfortunately, some of them simply do not respond, or respond but do not understand what we (or independent researchers) are telling them.

Challenge #5. Supply chain.

To make contacting the vendor even more complicated, it's not always clear who to contact for certain vulnerabilities. The same consumer product may be sold under different brand and model names, resold, or privately labeled. Many consumer product companies are little more than marketing firms or integration shops that assemble finished products from components. The vulnerable software may be in any one of those components. These components may even change in non-obvious ways as different suppliers compete for the business. Many electronic components are sold as functionally equivalent - but not technically equivalent - variations of a market leader's technology. Furthermore, once you get down to the component level, the suppliers are often located in foreign countries, so there are language problems, a lack of contacts, and foreign business processes to deal with.

In summary, there are several challenges in IoT industries that are either unique or more prominent than they are in traditional software. The number of IoT and embedded devices is likely to increase dramatically in the coming years, making this all the more challenging. On the plus side, the information security industry has gone through the growing pains of vulnerability disclosure with the software industry already. Hopefully this gives us a head start on addressing vulnerabilities in new industries and new types of products.