As learned from the part 1 blog above, the phrase “Flow Direction” can mean at least two things in the world of NetFlow and IPFIX. To the security professional, flow direction means “who started it”. To the engineer who is developing software to export IPFIX, flow direction tells us where the flow was metered and was it done ingress or egress. This post is about the later.
Here’s IANA’s IPFIX definition of flow direction: The direction of the Flow observed at the Observation Point. There are only two values defined – ingress and egress.
Above, we can see that another term "observation point" has been introduced. As a result, in order to explain flow direction, we need to understand what an observation point (OP) is. The OP tells us where the flow was metered and by metered I mean collected. In an effort to put a visual to these terms, lets consider the diagram below:
NOTICE ABOVE:
Given the above, traffic leaving the host is considered ingress to the router's eth0 and egresses from the router's eth1. The traffic coming back from the cloud is ingress to the router's eth1 and egresses from the router's eth0. As a result, x and y can potentially report both forward and return traffic. Therefore, in IPFIX we need the flowDirection information element because if we are talking about a single interface on the router, we can determine if the metering was done as the traffic was coming ingress which is indicated by 0x00 or if it was coming egress 0x01.
There are some "post-" information elements which report egress traffic counts. The idea is that a single observation point "x" follows traffic through the routing process which can report both the original and post- counters:
Keep in mind that it's unnecessary to have "post-" versions of all the information elements, or to duplicate every element according to whether it observes incoming or outgoing traffic. Developers can simply create an Observation Point on the outgoing interface and report the ingress Information Elements.
NOTE: Above, the flow collector will not know the location of the OP or whether the flow was collected ingress or egress. Knowing this could become important for traffic analysis reasons but, there is no metadata currently available on this. If there was, certainly the sequence of events would become important as well.
Suppose that the observation points are associated with a traffic filter which discards packets. It's crucial to know whether the reported counts represent the original traffic ingressing the filter, or the filtered traffic egressing the filter. Therefore we define the observation as the "incoming" count for consistency and to avoid ambiguity.
MAC addresses may be useful for simple observations based on router interfaces. However, if you're following traffic through a sequence of processes and you want to report the count between each of them, you'll need some other way of identifying the specific observation points.
For example, if you put filtering and sampling processes ahead of your switching infra and you report the layer2FrameTotalCount between each process:
Each of these observation points would report the same MAC addresses, which means they could only be distinguished by their observation point IDs. Working out the details on how to export this series of observations could end up being a fun project!
Questions? Reach out to me or my friend Paul Aitken who was a HUGE help on this post. I’ve written on this topic a few times and with each blog I’d like to think that I’m getting closer to helping those struggling with this area of IPFIX. I wrote a post titled ingress or egress back in June of 2009 then again here in Feb of 2012 and again here in May of 2013. I guess I’ll keep trying.
]]>With VoIP, BitTorrent, Skype, iCloud and the like now on the network, administrators are dealing with even more flows. On the NetFlow and IPFIX reporting side of things, vendors often find that 2-3 issues come into play when scaling NetFlow tools:
High speed NetFlow collection can lead to very large database tables. Large tables, if not indexed or queried correctly can lead to poor performance in traffic analysis reporting. As a consumer, how a vendor deals with enormous amounts of flow data can and should be part of the vendor selection process.
High NetFlow volumes does not necessarily mean you have to use multiple distributed NetFlow collectors. Many NetFlow and IPFIX collectors can handle tens of thousands or even over one hundred thousand flows per second with a single appliance (e.g. Scrutinizer). Distributed NetFlow collection should be configured when sending all of the flows over a wide area link doesn’t make sense. Enterprise NetFlow analysis requires a careful understanding of the IT managers goal, the budget constraints and the potential bottle neck areas on the network.
Work with your vendor to determine if a single flow collector or if distributed NetFlow collection is in your companies best interest. Beware of the necessary add-on modules and remember to ask about the yearly maintenance cost.
Join NetFlow Developments on Linkedin.
]]>