Dropped NetFlow detection should be a major part of the decision making process in your next enterprise NetFlow collector. High volume Netflow collection and reporting without regard to the NetFlow Sequence Numbers should send up red flags to an educated consumer in the market for a reliable NetFlow and IPFIX collector. Here’s why: It is a safe bet that companies serious about network traffic analysis or network traffic monitoring want to know if they are looking at all the data. In many cases they may not be. How would they know?



Are You Missing NetFlow
Routers and switches export flows with something called a flow sequence number.  These flow sequence numbers increment and tell the NetFlow and IPFIX collector that data is missing if a datagram or flow is not received.  If your NetFlow collector is receiving over 100,000 flows per second from hundreds or even thousands of routers, it is nice to know if you can rely on the trends when reports are run.   For example, the Catalyst 6500 NetFlow  exports are not always reliable.   Counting the Flow sequence numbers on a busy Catalyst 6500 reveals a NetFlow overflow with TCAM tables issue that results in reports that display a utilization level on interfaces that are actually dealing with much higher utilization.

Every NetFlow Collector has a limit on what it can handle.  How much it can handle can depend on several components:

• Architecture of the collector
• The amount of preprocessing of NetFlow data (e.g. looking for security threats)
• The version of NetFlow/IPFIX
• The volume of devices sending flows
• The volume of flows from any one device

In the screen shot below, we can see that the Scrutinizer Netflow Analyzer  is receiving nearly 6,000 flows per second from 5 different exporters. NOTE: our Linux collector can handle over 100K flows per second!



After further investigation, we discovered that most if not all of the Missed Flow Sequence Numbers (MFSN) are caused by one device.  See below:



Above you can see the MFSN trend for port 2055.  Notice directly below this trend outlined in red is a similar trend from a single device (i.e. router).  This tells us that the majority of missed flows across all 5 exporters is happening on one device.

What does an increase in MFSN tell us?
The loss of flow exports is usually caused by one of three things:

1. The network dropped some packets
2. The router can’t keep up (e.g. Catalyst 6500)
3. The High Volume NetFlow collector can’t keep up

The above is why NetFlow sequence numbers are becoming increasingly important.  Companies need to know if they can rely on the data: 

• Billing requires accurate data
• Threat Detection requires accurate data

If they can’t rely on the data, what is the source of the problem?

NetFlow v9 vs v5
NetFlow v9 flow sequence numbers are incremented per datagram.  NetFlow v5 flow sequence numbers are incremented per flow inside each datagram.  A NetFlow reporting solution that properly deals with this difference requires fairly sophisticated engineering.  Make sure you ask for it.

NetFlow Collection without Flow Sequence Number
The bottom line: NetFlow and IPFIX collection without Flow Sequence Number counting could be unreliable.  This is especially true when dealing with high NetFlow volumes. 

Join NetFlow Developments on Linkedin.