Traffic Analytics as it applies to the network and security is meant to help IT professionals who need to forensically investigate massive amounts of mostly internally generated logs and flows. Because threat prevention has largely failed in the industry with the gap continuing to widen, network and security analysts are forced to react to events on the network. This means wait for something to occur and then investigate:

• when it happened
• what it did once it was inside
• who else was involved
• what data was compromised

When answering the above, Network Traffic Analytics (NTA) or sometimes called Security Analytics are the umbrella terms that apply. Ultimately, the goal of NTA to improve security posture, reduce risk and gain deeper insight into each and every event. But how? Many organizations are drowning in data with a warehouse full of terabytes of logs and flows yet, still they lack visibility into what’s happening on the network.

What is Network Traffic Analytics
Since the systems claiming threat prevention can’t be relied upon, security professionals are often left pondering for the best alternative courses of action. Unlike Network Traffic Analysis, Network Traffic Analytics (NTA) includes the collection of packets, NetFlow, IPFIX, sFlow and other meta data. This information in turn is used primarily for two routines:

1. Behavior Monitoring: Developing behavior profiles and then automating the monitoring for events outside the profile
2. Incident Response: The forensic investigation of suspicious traffic patterns or events served up by monitoring and detection efforts

Behavior Monitoring
The one – two punch of constantly introduced new applications and increasing amounts of network traffic is leaving security professionals feeling defeated and overwhelmed. One thing that isn’t changing is the way protocols communicate that are used by malware. The miscreants writing the code that is under the hood of most infections, want their devious software to behave like ordinary approved applications.

Consider the analogy of the common cold. Symptoms could include a sore throat, sinus pressure, congestion, coughing, sneezing, etc. By monitoring for these symptoms and triggering events for individual behaviors, we can accumulate a score or threat index that leads to the positive identification of a virus. The higher the score, the more likely we can conclude that the person has a common cold or the flu. End users that aren’t infected who are always sneezing or coughing are given an exclusion from a specific symptom however, they can still accumulate a score. Security analysts can follow this same routine with network traffic but, it requires some work.

Network Traffic Analytics or Security Analytics can be used to monitor for odd DNS communications, excessively long connections, large Internet bound uploads, excessive Tor traffic, etc. Over an extended period of time, infected end systems end up with the highest infection scores. Applications or end systems which trigger false positives for specific monitoring methods can be excluded but, it requires ongoing work on the part of the security team. This is especially true if the organization does not establish ground rules on what is allowed on the network. Machine Learning systems require the same interaction.

The more free and open an organization is regarding what is allowed on the network, the more exclusion work it creates for the security team. The end users alternative to a blocked connection is often easy: use the 4G or LTE connection. Sounds logical however, some organizations argue that they have to keep Internet access completely open and that the risk in doing so is worth it.

One area of NTA that is getting attention is Machine Learning (ML). In theory, ML promises to create profiles of ‘normal’ behaviors without being told what to look for. It then uses its own logic to uncover anomalies. Most of what is in the market that claims ML support is hype or calling itself machine learning when really algorithms are being utilized which have tweakable thresholds. Any threat detection system that claims ML support but, still requires user input is beginning to deviate from a pure ML system. Regardless, despite ML promises, the technology is unproven and false positives are likely to ensue.

We have learned that an ML system for massive NetFlow and IPFIX volumes requires clusters of expensive computers and a network environment where communication changes don’t happen. In reality, new applications are constantly being introduced and their communication patterns frequently change. Even sometimes thought of as evil communication methods such as Tor and peer to peer are leveraged by legitimate applications. As a result, they will trigger false positives in many threat detection systems.

Whether the organization implements communication pattern restrictions or not, all networks are constantly changing environments. This means that Network Traffic Analytics and Security Analytics has to be frequently revisited by updating how the solution is monitoring for unwanted behaviors.

Incident Response
When threats are uncovered, the incident response plan gets thrown into action. After getting the answers to initial questions that allow security analysts to understand context, logs and flow data surrounding the event will be sought out. Forensic investigations almost always require the details these technologies provide and the more history the better.

Logs are great if the malware didn’t delete them. Packets are awesome but, the probes providing them are not always where they are needed and they almost never maintain a history beyond a week. NetFlow and IPFIX however provide a complete enterprise view into traffic patterns because all routers and most switches can export flows. The aggregated nature of flows allows IT to keep months of data all while maintaining 100% historical visibility.

What makes logs and flows even more valuable for incident response is the collection of meta data. This type of information includes details such as:

• The username associated with an IP address
• VLANs and MAC addresses
• Requested Fully Qualified Domain Names
• URLs and URI information 
• IPAM details

When logs and flows are correlated with meta data, the Mean Time To Know (MTTK) surrounding an incident often shrinks. Network Traffic Analytics that can take advantage of meta data by using it to provide more context surrounding an event. Integration with the SIEM is one of the more popular ways to gain access to context. Cisco ISE, Forescout CounterACT and Microsoft Active Directory can also provide unique pieces of information. Whether it’s the flow collection system or the SIEM that collects the context, integration between the two will improve the vitality of any Network Traffic Analytics effort that needs to take place.

UDP Replicators
In order to thwart an infections efforts to delete logs or flows, UDP Replicators or UDP forwarders can be deployed. These system replicate and forward messages to multiple collection points making it harder for miscreants to hide their tracks. These systems can ensure that 99% of the data surrounding an attack is maintained for the Network Traffic Analytics system.

A Business Case for NTA
As stated earlier, when connected to the Internet, threat prevention is impossible. All companies must deal with periodic compromised systems. Incident response is best performed with well-tuned Network Traffic Analytic practices which can include:

• Constantly monitoring for abnormal behaviors over time and scoring end systems
• Correlating flows, logs and meta data to provide better overall context surrounding an event
• Shorten investigation times by providing richer forensic details

Although systems supporting Network Traffic Analytics can pay for themselves in a short amount of time by reducing the Mean Time To Know, they still require a motivated analyst team. Knowing what to look for and an instinct for identifying problem areas requires skilled human interpretation. Check out our Collection, Visualization, and Reporting solution.