Most engineers implementing NetFlow or IPFIX know how to get started.  Where they sometimes stumble is in the area of a properly structured export with well thought out relationships between the templates. Today I want to provide an good example.

This  post on building a NetFlow Cache and exporting IPFIX is pretty deep. For this reason, my prior post on Exporting NetFlow or IPFIX   really should be reviewed first.  A flow cache entry in a router or switch is built using the first packet between two hosts and the cache table is maintained for all active connections (i.e. flows).   When a packet comes into the device, its tuple is compared to existing entries in the cache table.  A match of the key fields triggers a flow entry update where packet, byte counts and perhaps other fields are incremented and updated. Packets that don’t match a flow entry are compared to policy (e.g. firewall or ACL rules) and are ultimately dropped or used to create new cache entries.  Flow entries are exported to a flow collector periodically based on timers (I.e. Active Timeout) or flow behaviors.
When talking with a vendor about building out a flow cache, it is very important that we first establish what we want to export.  For most vendor implementations, this decision is easy and they copy NetFlow v5. Some software developers however, want to take their flow export to another level and provide the very best in Network Traffic Monitoring metrics. To do this, a list of all the details that need to be exported should be created.

• All of the data available in NetFlow v5
• VoIP details on caller ID, Codec, Jitter and packet loss
• Cloud service details on round trip time, URLs and deep packet inspection details that identify tricky applications like Skype, Webex and BitTorrent.
• Interface names and speeds to avoid the reporting tools dependency on SNMP
• Syslog and trap message details

The next step is to layout the IPFIX elements needed for each template.  We have to use IPFIX because in all likelihood your company name isn’t Cisco and even if it is, the list includes elements that will end up being variable length fields such as URL and syslog details.  These are, for all practical reasons, difficult to export using NetFlow v9.  Let’s get started on IPFIX template creation and take the list above one item at a time.  Each element name is preceded by an IANA or a dot separated vendor specific IE.

Be sure to take notice that IANA defined IEs are always the preference over vendor specific IEs.  The vendor specific IEs are followed by the vendor Private Enterprise Number (PEN) (e.g. .32473).

• A) Template: All of the data available in NetFlow v5/v9. This template is easy to define because all of the information necessary has been defined by IANA:

• 1 octetDeltaCount
• 2 packetDeltaCount
• 4 protocolIdentifier
• 5 ipClassOfService
• 6 tcpControlBits
• Sent as 0 for non TCP traffic
• 7 sourceTransportPort
• 8 sourceIPv4Address
• 9 sourceIPv4PrefixLength
• 10 ingressInterface
• 11 destinationTransportPort
• 12 destinationIPv4Address
• 13 destinationIPv4PrefixLength
• 14 egressInterface
• 15 ipNextHopIPv4Address
• 16 bgpSourceAsNumber
• 17 bgpDestinationAsNumber
• 21 flowEndSysUpTime
• 22 flowStartSysUpTime
• 95 Application ID
• This element causes another template below
• 148 Flow ID
• This element is used to link to other templates

The above template is used for all protocols not sent by a more specific template.  See below.

• B) Template: VoIP details on caller ID, Codec, Jitter and packet loss.  These elements are exported using a new template that contains meta data related to the flows exported above for the UDP real-time protocol (RTP).  It contains the following IDs:

• 148 Flow ID
• Links to the flow in A)
• 400.32473 Caller ID
• 401.32473 Codec
• 402.32473 Jitter
• 403.32473 Packet Loss

• C) Template: Cloud service details on round trip time, URLs and deep packet inspection details that identify tricky applications like Skype and BitTorrent.  These elements are exported using a meta data template similar to the one created in b).  In this case however, the template is only used for TCP traffic and includes the following IDs:

• 148 Flow ID
• Links to the flow in a)
• 501.32473 Round Trip Time
• 503.32473 URL

• D) Template: Interface names and speeds to avoid the reporting tools dependency on SNMP:
  
  • 10 ingressInterface
  • 82 interfaceName
  • 83 interfaceDescription
  • 82.32473 ifSpeed

IMPORTANT: Always check with IANA for the IE before creating one with the company PEN! This helps guarantee wider acceptance from collectors built by different vendors and improves cross vendor reporting.

• E) Template: Syslog and trap message details:
  
  • 322 observationTimeSeconds
  • 700.32473 Facility
  • 701.32473 Severity
  • 8 sourceIPv4Address
  • 10 ingressInterface
  • 12 destinationIPv4Address
  • 704.32473 Message

• F) Template: Application Name - This template provides the correlation between the application ID and the actual name of the application.  There is an effort underway to standardize this export which would allow consistency across vendors.

• 95 Application ID
• Links to the flow in a)
• 96 Application Name

With the templates above defined, it is time to find an IPFIX software solution that will export the data from your appliance. There are plenty of open source projects on the Internet that can do this.  One favorite is IPFIXify which also takes care of the template refresh (e.g. every 1-5 minutes) that has to be reported periodically.  IPFIXify also exports the RFC 5610 details.  To learn more about the above process, consider purchasing my book on NetFlow and IPFIX titled Unleashing the Power of NetFlow and IPFIX.