In my last post, I presented how to create a YAF application label signature rule that corresponds to a text-based Snort-type rule. In this post, I discuss methods for using Analysis Pipeline to provide context to those signatures.

The context for signatures can take many forms. Some context can be derived from the individual flows that match the signatures. This information is easy to obtain from either SiLK or another traffic analysis tool--just look at the traffic that matched the signature. Analysis Pipeline lets you easily do more. I will discuss three simple options, but Analysis Pipeline can be used for more complex analyses.

Option One

Provide the number or internal addresses communicating with each external address in communications matching the signature. For example,

FILTER sigAppLabel
TYPENAME == out
APPLICATION == 34566
END FILTER

STATISTIC countByExternal
FILTER sigAppLabel
FOREACH DIP
DISTINCT SIP
...
END STATISTIC

Basically, you

1. filter the traffic leaving the network that has the application label of interest
2. create a statistic that counts distinct source IP addresses for each destination IP address

This option tells you how many different internal addresses are actively communicating with each of the external IP addresses that are of interest based on the application label.

Option Two

Find other non-signature-type communication between IP addresses that have one or more flows that match the signature. For example,

FILTER sigAppLabel
TYPENAME == out
APPLICATION == appLabel
END FILTER

INTERNAL FILTER pairs
FILTER sigAppLabel
SIP DIP sigPairList n timespan
END INTERNAL FILTER

FILTER nonSigAppLabel
ANY IP PAIR IN_LIST sigPairList
APPLICATION != appLabel
END FILTER

STATISTIC sumDurations
FILTER nonSigAppLabel
FOREACH ANY IP PAIR
SUM DURATION
...
END STATISTIC

Basically, you

1. filter the traffic leaving the network that has the application label of interest
2. create a list of the source and destination IP address pairs from those communications
3. filter the traffic that has a source and destination IP pair in the list, but does not have the same application label
4. create one or more statistics that gives information for each pair
   Possibilities include
   1. counting the number of different source and/or destination ports used
   2. summing the duration of all flows between the addresses
   3. averaging the size of each packet between the addresses

Option Three

Determine if any IP addresses that correspond to the application label have communications that meet a threshold. For example,

FILTER sigAppLabel
APPLICATION == appLabel
END FILTER

EVALUATION manyBytes
FILTER sigAppLabel
FOREACH ANY IP
CHECK THRESHOLD
SUM BYTES > bytes
...
END CHECK
...
END EVALUATION

Basically, you

1. filter on all (both incoming and outgoing) traffic that has the application label of interest
2. create an evaluation that looks for any IP address that meets or exceeds a desired threshold
   Ideas for thresholds include
   1. The average of the duration exceeds some number of seconds.
   2. The sum of the byte count is at least some number of bytes.

For more ideas, check out the Analysis Pipeline documentation, check out this presentation by Dan Ruef from FloCon 2015 (hint, the application label could act as the blacklist) or contact us.