search menu icon-carat-right cmu-wordmark

Smart Collection and Storage Method for Network Traffic Data

Technical Report
This report discusses considerations and decisions to be made when designing a tiered network data storage solution.
Publisher

Software Engineering Institute

CMU/SEI Report Number
CMU/SEI-2014-TR-011
DOI (Digital Object Identifier)
10.1184/R1/6583826.v1

Abstract

Captured network data enables an organization to perform routine tasks such as network situational awareness and incident response to security alerts. The process of capturing, storing, and evaluating network traffic as part of monitoring is an increasingly complex and critical problem. With high-speed networks and ever-increasing network traffic volumes, full-packet traffic capture solutions can require petabytes of storage for a single day. The capacity needed to store full-packet captures for a time frame that permits the needed analysis is unattainable for many organizations. A tiered network storage solution, which stores only the most critical or effective types of traffic in full-packet captures and the rest as summary data, can help organizations mitigate the storage issues while providing the detailed information they need. This report discusses considerations and decisions to be made when designing a tiered network data storage solution. It includes a method, based on a cost-effectiveness model, that can help organizations decide what types of network traffic to store at each storage tier. The report also uses real-world network measurements to show how storage requirements change based on what traffic is stored in which storage tier.

Cite This Technical Report

Horneman, A., & Dell, N. (2014, September 15). Smart Collection and Storage Method for Network Traffic Data. (Technical Report CMU/SEI-2014-TR-011). Retrieved April 24, 2024, from https://doi.org/10.1184/R1/6583826.v1.

@techreport{horneman_2014,
author={Horneman, Angela and Dell, Nathan},
title={Smart Collection and Storage Method for Network Traffic Data},
month={Sep},
year={2014},
number={CMU/SEI-2014-TR-011},
howpublished={Carnegie Mellon University, Software Engineering Institute's Digital Library},
url={https://doi.org/10.1184/R1/6583826.v1},
note={Accessed: 2024-Apr-24}
}

Horneman, Angela, and Nathan Dell. "Smart Collection and Storage Method for Network Traffic Data." (CMU/SEI-2014-TR-011). Carnegie Mellon University, Software Engineering Institute's Digital Library. Software Engineering Institute, September 15, 2014. https://doi.org/10.1184/R1/6583826.v1.

A. Horneman, and N. Dell, "Smart Collection and Storage Method for Network Traffic Data," Carnegie Mellon University, Software Engineering Institute's Digital Library. Software Engineering Institute, Technical Report CMU/SEI-2014-TR-011, 15-Sep-2014 [Online]. Available: https://doi.org/10.1184/R1/6583826.v1. [Accessed: 24-Apr-2024].

Horneman, Angela, and Nathan Dell. "Smart Collection and Storage Method for Network Traffic Data." (Technical Report CMU/SEI-2014-TR-011). Carnegie Mellon University, Software Engineering Institute's Digital Library, Software Engineering Institute, 15 Sep. 2014. https://doi.org/10.1184/R1/6583826.v1. Accessed 24 Apr. 2024.

Horneman, Angela; & Dell, Nathan. Smart Collection and Storage Method for Network Traffic Data. CMU/SEI-2014-TR-011. Software Engineering Institute. 2014. https://doi.org/10.1184/R1/6583826.v1