Posted on by Best Practices in Network Securityin
By Angela Horneman Network Intelligence Analyst CERT Division
This post is also authored by Tim Shimeall and Timur Snoke.
In July of this year, a major overseas shipping company had its U.S. operations disrupted by a ransomware attack, one of the latest attacks to disrupt the daily operation of a major, multi-national organization. Computer networks are complex, often tightly coupled systems; operators of such systems need to maintain awareness of the system status or disruptions will occur. In today's operational climate, threats and attacks against network infrastructures have become far too common. At the SEI's CERT Division Situational Awareness team, we work with organizations and large enterprises, many of whom analyze their network traffic data for ongoing status, attacks, or potential attacks. Through this work we have observed both challenges and best practices as these network traffic analysts analyze incoming contacts to the network including packets traces or flows. In this post, the latest in our series highlighting best practices in network security, we present common questions and answers that we have encountered about the challenges and best practices in analysis of network traffic, including packets, traces, or flows.
How critical is the role of the network traffic analyst in an organization's security operations center (SOC)?
Angela: A network traffic analyst looks at communications between devices. In a security context, they do it to detect threats, such as undetected malware infections, data exfiltration,attempts, unauthorized device access, etc. Network traffic analysis is one part of security analysis that provides insight into communications between technological assets into how they are being used and how they can go wrong. The more assets talk to each other, the more important network analysis becomes. Some types of misuse are much easier to find looking at communications (e.g., DoS, malware signaling) than events that are captured on a host (e.g., login attempts, virus detections).
Not all organizations need a full-time network analyst. Smaller organizations may have a security team where everyone handles all aspects of security. The larger or more complex an organization's network becomes, however, the more important it is to have one or more people whose primary responsibility is to protect from, detect, and respond to network-involving events.
There are two views applied to network analysis that are not mutually exclusive: analysis as a set of rules, playbook, or scripted (automated) workflow versus analysis as a hunting, awareness, or exploratory process. Both views are needed. The first view is appropriate for handling common threats--spam carrying malicious attachments, virus detections, etc. The second view is needed to handle activity that is harder to detect, such as advanced persistent threats, data exfiltration, etc. The first view is a rote activity. The second view is very creative.
Tim: The network traffic analyst is the one who watches what is happening on the network as opposed to on its hosts. The network traffic analysts tend to look at the wide scope of the activity, as opposed to specific changes on hosts.
Despite that wide scope, network analysts work in partnership with people that do host forensics and examine what is happening on the computers on the network, as opposed to what is happening on the network itself. The analysts provide an unbiased look at the information moving across the network, whether malicious or not. This unbiased view lets analysts also operate in partnership with network traffic engineers who examine whether things happening that are supposed to be happening. Traffic analysts either work on one side or the other: Are things being blocked that are supposed to be blocked or are things happening that are supposed to be happening?
Timur: The analyst is the one who understands how things work on the network, and when they aren't working, why they aren't working. This analysis includes understanding at a deep level how things on the network work together. Analysts also look at the utilization of the network between different devices, to determine if there is enough capacity to let the applications run with optimal performance. Analysts monitor what applications run on the network, and how the applications are communicating with each other. The environment that allows attackers to impact networks is often unknown. Network traffic analysis supportsin understanding the baseline of the environment being defended.
What are some of the challenges that network traffic analysts face?
Timur: Although networking is about communications, defending the network is not about just keeping the lights blinking, it is about understanding the mission of the components on the network. Network traffic analysts must work with application owners to make sure that the dependencies are understood and not impacting other parts of the organization. Only by understanding the needs of the enterprise can analysts effectively support efforts in its defense.
Tim: The first challenge that I see is dealing with the myriad of data that is available. Network traffic analysts must review log entries, packet capture, firewall or, logs on affected systems, plus routing information or resolution records (pDNS). To gain a better understanding of network status or malicious activity on the network, a network traffic analyst must understand the role that each of these would play towards completing a picture of the activity on the network.
The malicious activity could be a security event. The network status could be understanding
In each of these cases, the analyst would be integrating a variety of network information to build a consistent picture from the network traffic. The analyst would also be looking to defend the conclusions inherent in that picture.
Angela: I think that one of the biggest challenges faced by network analysts is lack of clear missions and priorities. This problem can arise from a lack of organizational technology and cyber usage policies or organizational technology and cyber usage policies that lack specific detail. For example, a policy may state that devices must be kept up-to-date but doesn't include any timeline, such as patches must be applied within one week of availability. When it is unclear what an organization allows, it is hard to figure out what constitutes a security event. When an organization's policies are permissive (e.g., organizations not willing to block access to some websites or allow peer-to-peer file sharing), it is hard to find threats in all the noise that users generate.
Unclear missions and priorities can also arise from poorly defined analysis processes, analyst roles and authority, and tools and data available for analysts. In other words,management and resources can make it hard for analysts to focus on threats that would have high impact. For example, if the analysis tools, data, and/or processes only exist for virus detection or suspicious URLs sent in emails, but the organization has important intellectual property, protecting that intellectual property will take a backseat to dealing with filtering email and cleaning up potential virus infections.
In addition, unclear priorities can arise from an analyst's lack of understanding of existing assets, how they are used, and work products of various organizational departments (e.g., the work product of a finance department may be payroll).
What best practices in network traffic analysis have you observed?
Timur: Avoid getting too vendor-driven. While tools and building skills with tools are important, analysts need to keep the perspective that the function is important rather than buying into the mindset that their job is to use specific tools.
Tim: Here are some effective best practices that I have observed:
How do you see the role of network traffic analysts evolving? What challenges do you see network traffic analysts facing in the next five years
Tim: Network traffic analysis has historically been an ad hoc activity, requiring high expertise and intense effort. We are going to see more regularization of analysis, based on formalisms that are being developed now. These improvements will allow more clarity and traceability in the analysis process, which are often lacking in common practice. It will also simplify management of analysis activity and make a stronger association between data and conclusions suitable for particular network issues.
Timur: Networks are constantly evolving and the demands on resources are increasing at a steady pace. What network traffic analysts used to manage is no longer as simple. The increased usage of encryption decreases visibility into network traffic and the volume of enterprise applications that are now outsourced to vendors and platforms that are now "in the cloud" make defending the enterprise more challenging. What security controls that exist within the enterprise are not replicated in those externally sourced solutions? How does an analyst defend an application on a cloud service provider that uses a multi-tenant architecture and has an oversubscription model and is encountering resource contention because of issues with a different tenant?
Angela: An upcoming challenge I see is as organizations acquire new products, many of which are beginning to incorporate machine learning (ML) and artificial intelligence (AI). Analysts will therefore need to understand how to validate the results the products produce and make adjustments to their workflows.
What resources are out there for network traffic analysts?
As outlined in, there are a number of resources available to network analysts and security defenders as they contend with rapid-fire increases in global internet protocol traffic:
We welcome your feedback about this work in the comments section below.
Learn more about.
Read Tim Shimeall's blog post,
Read other blog posts in the ongoing series from CERT researchers,.