Posted on by CERTin
Knowing what assets are on a network, particularly which assets are visible to outsiders, is an important step in achieving network situational awareness. This awareness is particularly important for large, enterprise-class networks, such as those of telephone, mobile, and internet providers. These providers find it hard to track hosts, servers, data sets, and other vulnerable assets in the network.
Exposed vulnerable assets make a network a target of opportunity, or "low-hanging fruit" for attackers. According to the 2012 Data Breach Investigations Report, of the 855 incidents of corporate data theft reported in 2012, 174 million records were compromised. Of that figure, 79 percent of victims were targets of opportunity because they had an easily exploitable weakness, according to the report. This blog post highlights recent research in how a network administrator can use network flow data to create a profile of externally-facing assets on mid- to large-sized networks.
Network flow data, which is an aggregation of the header information contained in datagrams (packets), can be used to create profiles of network traffic, detect malicious activity, and determine appropriate traffic prioritization settings. Network flow data includes information about communicating pairs of IP addresses, and the ports and protocols on which they communicate, as well as aggregated byte counts and flags used.
Network administrators can use network profiling to consider how decisions about configuration changes will affect the rest of the assets on their network. Security administrators can evaluate the profiles to identify assets that violate policy and suspicious activity, while business administrators can use the profiles to help guide long-term decisions regarding network security.
The intent of this research by the CERT Network Situational Awareness Team was to create a step-by-step guide for using network flow to inventory or profile a network that includes thorough explanations of why certain steps were chosen so that administrators could understand the process and tailor the steps for their environments. We on the research team focused our analysis on creating a profile of externally facing assets on mid- to large-sized networks that serve thousands to hundreds of thousands of users. We used data from a medium-sized enterprise network that allowed us to access its typical data usage. By focusing on network flow data, we had much less data to deal with than if we collected all traffic on a network (full packet capture). Focusing on headers also allowed us to avoid issues with confidentiality and privacy because we were not actually collecting payload information.
We then parsed the data we collected using the System for Internet-Level Knowledge (SiLK). SiLK is an open-source tool developed by the CERT Network Situational Awareness Team that is an efficient network flow collection and storage infrastructure that will accept flow data from a variety of sensors.
To produce relevant results, the process we developed for network profiling must complete within a fixed amount of time. For networks with relatively stable assets, this process could take place over one or two months. For fast-changing networks, the process could take place in as little as one to two weeks. By following these steps, a network administrator profiling a network will obtain a list of public-facing assets, information about the ports through which each asset is communicating, and other pertinent information, such as the external IP addresses to which the asset is connecting.
Our approach can be broken down into the following steps:
Challenges in this Approach
Relying only on network flow data to create a network profile is inherently more inaccurate than using full packet capture. As long as administrators are aware of the limitations of using flow data (as explained in the report documenting the guide), useful results can be produced by following the steps in the guide.
Our team is currently examining how administrators customize and use the results of network profiles. We are investigating how to automate these steps and implement a continual update process. We are also considering development of a new step-by-step guide using this same approach with a different tool. For example, ARGUS is another network flow analysis tool with a slightly different approach than SiLK. Please feel free to suggest other tools for investigation in the comments section below.
The technical report describing this research, Network Profiling Using Flow, may be downloaded at
Our approach is based on System for Internet-Level Knowledge (SiLK), an open-source tool developed by the CERT Network Situational Awareness Team. You can download SiLK at
The Network Situational Awareness (NetSA) group in the CERT Program at the SEI developed and maintains a suite of open-source tools for monitoring large-scale networks using flow data. These tools have grown out of our work on the AirCERT project, the SiLK project, and the effort to integrate this work into a unified, standards-compliant flow collection and analysis platform. You can view that suite of tools at
Visit the SEI Digital Library for other publications by Austin