Software Engineering Institute

Human-driven network forensics activities (such as threat hunting and incident response) focus on identifying the source of potential network threats or other problem incidents. Analysts must sift through large amounts of network data to find forensically relevant events. Full packet payloads (called packet capture, or PCAP) have long been considered as the gold standard of forensic evidence. While full packet capture does contain all relevant forensic information, capturing and storing every packet for an extended time period is often prohibitively expensive and inefficient to analyze in bulk.

Because of these shortcomings, network analysts often turn away from full packet capture to alternative forms of forensic data. Popular alternatives include NetFlow, extended (augmented) flow, and application metadata (DPI). These alternatives provide forensic value and use significantly less disk space than full packet capture, but lack the complete packet payloads needed to fully confirm the presence of malicious activity on the network. This trade-off between the forensic value of data and the size and cost of storing it has caused analysts to seek an optimized balance between full packet payloads and other forms of forensic data.