Advanced NetFlow Traffic Analysis

Cloud Service Monitoring

Sun, 20 May 2012 12:21:11 -0500

Interest in cloud service monitoring utilities will grow as businesses become more dependent on these types of virtual services. In this post, I’ll outline some great ways to monitor cloud services by leveraging in house hardware namely, Cisco Routers and the use of Flexible NetFlow Performance Monitoring or Medianet exports.

Performance Monitoring Flexible NetFlow (FnF) is being touted by Cisco as the technology to be used for monitoring the quality of media rich applications. These new FnF exports provide details on TCP round trip time and for VoIP it provides jitter and packet loss metrics.

In the first example, lets use an instance where an employee is leveraging the cloud service like Vonage. Below, we are looking at the top 10 of 23 pages of calls.

Above you can see that most of the calls have low jitter values and notice that you can sort on packet loss by clicking on the column header. These FnF exports can be used just as easily for monitoring video performance. Performance Monitoring NetFlow elements allow us to confirm that a cloud service like Vonage (or even Skype) at least most of the time is working for this remote sales person and we can see that they are actively making calls and even the call duration.

In this second example, I filtered for the cloud service salesforce.com and you can see below that scrutinizer is reporting some unhappy connection times.

Notice above that all of the different Cisco Performance Reporting options. There are a bunch of report options that allow you to report on ranges of IP / subnets which help you get a big picture during your regular network traffic management routines. Thresholds can be set on any combination of the above and it is all done with existing routers running IOS 15.X or more recent. What’s great about using Flexible NetFlow is that they aren’t limited to monitoring only popular apps like Microsoft Azure, Google App Engine, etc. This cloud monitoring solution can operate at the transport layer (e.g. TCP, UDP, etc.) or even higher layer by leveraging technologies such as Cisco NBAR which can also deliver details on RTT, availability and packet loss.

Not only can you monitor the quality of individual connections but, you can setup synthetic transactions using IP SLA and monitor / trend the values.

What metrics can you monitor? How about availability, response time or even bytes transferred. Imagine being able to setup notifications if for example an entire subnet witnesses a quality of service above 200 millisecconds for over 10 minutes. This is the kind of monitoring and reporting that allows you to paint a clear picture of the status of your cloud services.

Solutions capable of monitoring cloud services are still evolving and without a doubt, the metrics mentioned in this post will be part of the evolution.

]]> Tags: Cisco NBAR, Cisco Performance Reporting, Cloud monitoring solution, cloud service monitoring, monitoring cloud services Related tags: cloud service, cloud services, flexible netflow, performance monitoring, service monitoring, monitoring
, cloud service monitoring, monitoring cloud services, Cisco NBAR, Cloud monitoring solution, Cisco Performance Reporting

Follow me:

Related Entries TrackBacks | Comments | Tag with del.icio.us | Advanced NetFlow Traffic Analysis Home | Permalink: Cloud Service Monitoring

Temperature Monitoring Solutions with Remote Humidity Sensors

Wed, 09 May 2012 12:33:25 -0500

Generally I write about NetFlow solutions but, today I’ve got temperature monitoring solutions on my mind for a data center we use that requires remote humidity sensors or electronic hygrometer equipment. Server room monitoring isn’t something I consider my forte but, it seemed I couldn’t avoid getting involved with this project.

Water Leak Alarm

Normally electronic professionals are concerned about humidity in that, they don’t want to see the room to dry. Locally, we monitor our server room humidity with a simple appliance that can take inputs from multiple environmental sensors. Since the initial purchase we have added smoke detection sensors, temperature monitors and a few others. A water sensor was the most recent investment. Apparently one of our technicians felt that we needed some type of water leakage monitor as well. Are we really worried about a flood on the 2^nd story of a building? No, we aren’t but, we are concerned about air conditioner or ceiling leaks.

A New Career

Who is responsible for this new web based temperature monitoring, humidity sensoring, smoke detecting, etc. etc. system? Should we give it to the server guy? How about the network guy or maybe the phone system guy? Everyone seems to agree that we need to monitor these things however, no one wants to take on the additional work!

I think it is interesting that our network monitoring solution now encompasses items such as temperature and remote humidity sensors.

]]> Tags: Environmental sensors, Remote humidity sensor, Remote Humidity Sensors, Server room humidity, Server room monitoring, Temperature Monitoring Solutions, Water Leakage Monitor, web based temperature monitoring Related tags: remote humidity, temperature monitoring, humidity sensors, monitoring solutions, sensors temperature, temperature
, Temperature Monitoring Solutions, Remote Humidity Sensors, Server room monitoring, Environmental sensors, Server room humidity, Water Leakage Monitor, Remote humidity sensor, web based temperature monitoring

Follow me:

Sponsored by the Call Center Outsourcing Community & the Virtual Contact Center Outsourcing Community

Cisco ASA Vs. Juniper SRX: NetFlow Reporting

Sun, 06 May 2012 06:32:21 -0500

The Cisco ASA Vs. Juniper SRX was being hotly debated on the Cisco forum. Being a flow analysis company we always ask about NetFlow or IPFIX support before we purchase a network appliance, especially a firewall. Reporting on data in our organization is paramount as “he who stays in the know, stays ahead”. When it comes to firewall reporting, we are looking for:

Traditional flow reporting
Log reporting
Other cool flow exports (e.g. usernames)

I think most people reading our blog are familiar with traditional NetFlow but, what about log reporting? Just about all firewalls today export syslogs and a few export logs in NetFlow datagrams which of course falls into our realm of competency. The Cisco ASA and SonicWALL firewalls support both.

Considering all of the Cisco ASA NetFlow problems (i.e. several) when exporting NetFlow, we can still get some great NetFlow reports on:

Top Talkers, Applications, Protocols, etc.
Usernames which is very helpful in BYOD security monitoring
Top Violated Access Control Lists
Network Address Translations

Example Cisco ASA NSEL Report Below:

The Juniper SRX does not export logs and if you look into the J-Flow configuration (AKA NetFlow) it is basically sampled NetFlow. Who wants to be limited to sampling? It’s sort of like sFlow reporting. From what I read, there is significant overhead associated with J-Flow whereas on the Cisco ASA we haven’t seen any issues.

Exporting NetFlow is also important if you are looking to start detecting advanced persistent threats. Many NetFlow security solutions feed the flows into a host reputation database to try and detect communications with known C&C hosts and the like. NetFlow Sampling would lead to many missed threats.

NetFlow and Firewalls : The Bottom Line

All major players in the firewall market support a Flow technology. The Cisco ASA, Juniper SRX (sampled), SonicWALL, Barracuda, Palo Alto Networks, Checkpoint and Fortinet (sFlow) all understand this. Contact me if you would like your firewall added to this list and if you are a firewall company, we’ll give you a NFR copy of our NetFlow and IPFIX analyzer for development purposes.

]]> Tags: BYOD security, Cisco ASA NetFlow, Cisco ASA Vs. Juniper SRX, J-Flow configuration, NetFlow or IPFIX, netflow security solutions, Sampled netflow Related tags: firewall reporting, exporting netflow, threats netflow, netflow ipfix, netflow, cisco
, Cisco ASA Vs. Juniper SRX, J-Flow configuration, NetFlow or IPFIX, BYOD security, Cisco ASA NetFlow, netflow security solutions, Sampled netflow

Follow me:

TrackBacks | Comments | Tag with del.icio.us | Advanced NetFlow Traffic Analysis Home | Permalink: Cisco ASA Vs. Juniper SRX: NetFlow Reporting

Sponsored by the Call Center Outsourcing Community & the Virtual Contact Center Outsourcing Community

How to roll out BYOD Security: Best Network Management

Tue, 24 Apr 2012 08:29:56 -0500

You’ve been allowing it for months or even years and now you’ve decided to get serious about how to roll out a secure BYOD environment. Supporting BYOD is easy. Making BYOD secure is the tricky part and then there is always network traffic monitoring to make sure your BYOD security strategy is working. Today I’m going to outline several steps that Patrick Sweeney (VP of Product Marketing) at SonicWALL shared with me. These steps should be considered when setting up a ‘reasonably’ safe BYOD environment.

Patrick Sweeney – VP, SonicWALL

1) Establish a Reverse Proxy. A reverse proxy presents your prized servers on the internet and provides a single point of access to all servers and they can force authentication on just about any type of device or operating system. There are other benefits as well such as making it easier to replace backend servers without worrying about host name changes.

2) Provide VPN access. I’m not going to get into a IPSec vs. SSL discussion but, I will say that a single solution for both smartphones and laptops is ideal

3) All remote connections should pass through a firewall before accessing the corporate network. Next Generation firewalls do not allow clear text and constantly scan traffic for malware.

4) Force strong authentication. A one time password can be reused if it is capture by a machine infected with a key logger. They are also simple to implement.

If you are ready to take BYOD security to another level, read on:

5) The machine using a VPN to access the corporate network should be interrogated immediately after logging in to ensure that it is running antivirus software and not using tools such as BitTorrent.

6) Leverage secure virtual desktop environments which are erased and recreated on exit. This is very important if your company has employees accessing the corporate network from random hardware such as an airport kiosk.

7) Enforce a cache cleaner once the user logs off and closes his / her local browser

Want more ideas to keep the business applications optimized in a BYOD and non-BYOD environments:

8) Make sure business applications such as VoIP, Salesforce.com, etc. are prioritized. This is done with deep packet inspection available on most routers, switches and firewalls.

9) Incorporate hardware that can take automatic action on unacceptable behaviors by throttling a users traffic or by stuffing the host into a limited access VLAN

10) Manage and monitor. With technologies such as next generation NetFlow and IPFIX we can now monitor BYOD traffic by incorporating filters for :

User names and viewing all of the devices a user has authenticated onto the network (e.g. laptop, smart phone and tablet)
The vendor ID of MAC addresses can be filtered on. This is time consuming but, effective.
Verify that business applications are getting the bandwidth they need during peak times
Set thresholds that trigger alarms which watch for traffic to certain domains within URLs

The more steps you take to secure BYOD access your network the better and you don’t have to spend a lot of money. BYOD has been here for a while and it is growing rapidly and should be part of most network management solutions.

]]> Tags: best network management, BitTorrent, BYOD Security, BYOD strategy, IP Flow Information Export, IPad, MAC address, making byod secure, NetFlow, network traffic monitoring, secure BYOD access, Security, SonicWALL, supporting BYOD, Virtual private network Related tags: corporate network, business applications, reverse proxy, accessing corporate, patrick sweeney, network
, supporting BYOD, network traffic monitoring, making byod secure, BYOD strategy, BYOD Security, secure BYOD access, best network management

Follow me:

Migrating to Flexible NetFlow : BEST PRACTICES - Mar 08, 2012

Dropped NetFlow : Flow Sequence Numbers - Jan 29, 2012

High Volume NetFlow Collector : Enterprise Traffic Analysis - Jan 03, 2012

TrackBacks | Comments | Tag with del.icio.us | Advanced NetFlow Traffic Analysis Home | Permalink: How to roll out BYOD Security: Best Network Management

Sponsored by the Call Center Outsourcing Community & the Virtual Contact Center Outsourcing Community

Detecting Advanced Persistent Threats with NetFlow and IPFIX

Wed, 11 Apr 2012 21:26:41 -0500

Detecting Advanced Persistent Threats and other Network Security Threats with NetFlow and IPFIX requires advanced flow analytics. For example Top hosts, top applications, top DSCP values, etc., it’s great information and these reports can be used both reactively and proactively but, it’s just the tip of the iceberg as NetFlow can tell us about all kinds of anomalous traffic that may be flying under the radar.

For example, a host scanning the network causes lots of flows but uses little bandwidth. What if you only want TCP, UDP and ICMP protocols on your network. Would you want an alarm if DDP, DDX, IPv6 or IGMP showed up? Because what we want to see about the network can change per business, threat detection systems need to allow for custom algorithms (e.g. monitoring facebook traffic).

Below you can see a more detailed view of the outstanding alarms with additional columns. The first and last violation time stamps are displayed as well as a way to expand and see all the hosts that have violated each algorithm. Lets drill in on the Internet Threats Monitor.

Internet threats is an IP host reputation database that is downloaded by all our NetFlow collectors every hour. This algorithm monitors the source and destination address of each flow to see if it is on the poor reputation list. Positive matches trigger alarms. Here you can click for a menu to gain further insight into the traffic which can help admins be more effective at detecting some Advanced Persistent Threats (IPTs).

NetFlow and IPFIX Threat Messages

NetFlow and IPFIX are being used by Cisco's Smart Logging Telementry and SonicWALL firewalls to send threat detected messages. Expect vendors to get on board in the coming months as although flow technologies can be used to detect some threats, the lack of the entire packet allows the IPS to maintain its primary position in threat detection and prevention.

The Cisco ASA NSEL also exports messages in NetFlow related to flows violating ACLs. Make sure your NetFlow Analyzer supports these types of exports.
]]> Tags: cisco asa nsel, detecting advanced persistent threats, flow analytics, ip host reputation, ipfix reporting, network security threats, threat detection systems Related tags: netflow ipfix, advanced persistent, persistent threats, detecting advanced, threat detection, netflow
, network security threats, ipfix reporting, flow analytics, threat detection systems, ip host reputation, detecting advanced persistent threats, cisco asa nsel

Follow me:

TrackBacks | Comments | Tag with del.icio.us | Advanced NetFlow Traffic Analysis Home | Permalink: Detecting Advanced Persistent Threats with NetFlow and IPFIX

Sponsored by the Call Center Outsourcing Community & the Virtual Contact Center Outsourcing Community

Monitoring BYOD traffic with NetFlow

Sat, 24 Mar 2012 05:48:50 -0500

Monitoring BYOD traffic is a growing concern amonst network administrators. Why? Gartner predicts that 645 million smartphones will be sold in 2012 – a 40% increase from this year. Cell phone reception is often weak on the interior of office buildings and smartphone owners will have their WiFi on. What's more is that many companies are allowing employees onto the corporate net with their personal smart phones in hopes of increased productivity.

Employees using Corporate Bandwidth with Personal Phones
This big increase brings with it big concerns when it comes to network monitoring:

How much bandwidth are all these additional devices collectively using and is it impacting business critical applications?
What applications and web sites are users hitting and what impact are these distractions having on productivity and how often?
What are the security implications introduced by allowing these devices onto the net? Many of these hand held devices do not have antivirus software.

Given that the traffic from a cell phone browsing a web site looks nearly identical to that of a PC hitting the same site, how can a company determine the amount of Internet bandwidth utilized by the combined smart phone devices? To answer this, we need a new flow element.

All hardware accessing the LAN utilizes a six byte hexadecimal MAC address. The first three bytes of this address is reserved to identify the vendor. For example, an iPhone may have an address of E4:CE:8F:C2:9D:AA. The first three bytes E4:CE:8F identifies the vendor ‘Apple’ and it is likely that thousands of other iPhones start with the same 3 bytes. The remaining three bytes C2:9D:AA are unique to the individual iPhone.

Nearly a dozen vendors (e.g. Cisco, Enterasys, Exinda, Juniper, nBox, Sonicwall) are now exporting MAC information in their flow exports. Learn how to export [MAC address with Flexible NetFlow] http://www.plixer.com/blog/netflow/getting-mac-addresses-from-netflow-v9/ . Setting up a simple network monitor will help you proactively keep track of this traffic.

Below is an example of our partnership reporting with Enterasys NetFlow and mIAM exports:

If your NetFlow or IPFIX hardware can export Username, you could click on a username and see the number of devices authenticated by the same user.

Monitoring BYOD traffic is a growing concern and the above report can be run against flow exports from the Cisco ASA, Palo Alto Networks and the SonicWALL (example above). Vendors are always looking for new and innovative ways to filter on this data.

[Gartner predicts that 645 million smartphones] http://www.channelinsider.com/c/a/Mobile-Devices/Mobile-Management-Styles-Driven-by-Consumerization-Gartner-634465/ will be sold in 2012 – a 40% increase from this year.   Cell phone reception is often weak on the interior of office buildings and smartphone owners will have their WiFi on. Many companies are allowing employees onto the corporate net with their personal smart phones in hopes of increased productivity.

Employees using Corporate Bandwidth with Personal Phones
This big increase brings with it big concerns when it comes to network monitoring:
1.   How much bandwidth are all these additional devices collectively using and is it impacting business critical applications?
2.   What applications and web sites are users hitting and what impact are these distractions having on productivity and how often?
3.   What are the security implications introduced by allowing these devices onto the net? Many of these hand held devices do not have antivirus software.
Given that the traffic from a cell phone browsing a web site looks nearly identical to that of a PC hitting the same site, how can a company determine the amount of Internet bandwidth utilized by the combined smart phone devices? To answer this, we need a new flow element.

All hardware accessing the LAN utilizes a six byte hexadecimal MAC address. The first three bytes of this address is reserved to identify the vendor. For example, an iPhone may have an address of E4:CE:8F:C2:9D:AA. The first three bytes E4:CE:8F identifies the vendor ‘Apple’ and it is likely that thousands of other iPhones start with the same 3 bytes. The remaining three bytes C2:9D:AA are unique to the individual iPhone.

<<< monitoringMobilePhoneTraffic.png >>>

Nearly a dozen vendors (e.g. Cisco, Enterasys, Exinda, Juniper, nBox, Sonicwall) are now exporting MAC information in their flow exports. Learn how to export [MAC address with Flexible NetFlow] http://www.plixer.com/blog/netflow/getting-mac-addresses-from-netflow-v9/ . Setting up a simple network monitor will help you proactively keep track of this traffic.

If your NetFlow or IPFIX hardware can export Username, you could click on a username and see the number of devices authenticated by the same user.

<<< sonicwall-Ipfix-username.png >>>

Monitoring BYOD traffic is a growing concern and the above report can be run against flow exports from the Cisco ASA, Palo Alto Networks and the SonicWALL (example above). Vendors are always looking for new and innovative ways to filter on this data.

KEY WORDS:
network monitoring
flexible netflow
network monitor
monitoring mobile phone traffic
monitoring byod traffic

big concerns when it comes to network monitoring:

1. How much bandwidth are all these additional devices collectively using and is it impacting business critical applications?

2. What applications and web sites are users hitting and what impact are these distractions having on productivity and how often?

3. What are the security implications introduced by allowing these devices onto the net? Many of these hand held devices do not have antivirus software.

All hardware accessing the LAN utilizes a six byte hexadecimal MAC address. The first three bytes of this address is reserved to identify the vendor. For example, an iPhone may have an address of E4:CE:8F:C2:9D:AA. The first three bytes E4:CE:8F identifies the vendor ‘Apple’ and it is likely that thousands of other iPhones start with the same 3 bytes. The remaining three bytes C2:9D:AA are unique to the individual iPhone.

<<< monitoringMobilePhoneTraffic.png >>>

Nearly a dozen vendors (e.g. Cisco, Enterasys, Exinda, Juniper, nBox, Sonicwall) are now exporting MAC information in their flow exports. Learn how to export [MAC address with Flexible NetFlow] http://www.plixer.com/blog/netflow/getting-mac-addresses-from-netflow-v9/ . Setting up a simple network monitor will help you proactively keep track of this traffic.

If your NetFlow or IPFIX hardware can export Username, you could click on a username and see the number of devices authenticated by the same user.

<<< sonicwall-Ipfix-username.png >>>

Monitoring BYOD traffic is a growing concern and the above report can be run against flow exports from the Cisco ASA, Palo Alto Networks and the SonicWALL (example above). Vendors are always looking for new and innovative ways to filter on this data.

KEY WORDS:

network monitoring

flexible netflow

network monitor

monitoring mobile phone traffic

monitoring byod traffic

]]> Tags: flexible netflow, monitoring byod traffic, monitoring mobile phone traffic, network monitor, network monitoring Related tags: flexible netflow, network monitoring, network monitor, growing concern, hardware accessing, devices
, monitoring byod traffic, monitoring mobile phone traffic, network monitor, flexible netflow, network monitoring

Follow me:

Related Entries

Monitoring Video Performance with NetFlow - Feb 19, 2012

End to End Visibility : Network Flow Path - Feb 06, 2012

TrackBacks | Comments | Tag with del.icio.us | Advanced NetFlow Traffic Analysis Home | Permalink: Monitoring BYOD traffic with NetFlow

Sponsored by the Call Center Outsourcing Community & the Virtual Contact Center Outsourcing Community

Migrating to Flexible NetFlow : BEST PRACTICES

Thu, 08 Mar 2012 07:12:24 -0500

Migrating to Flexible NetFlow (FnF) is a simple and for some of us, an exciting process. It is exciting because of the deeper and improved insight into network traffic monitoring. If your company is interested in migrating to FnF from traditional NetFlow, there are a few things to consider.
The 5 steps to make this change are easy.

1)    List what business applications you want to monitor? Ask yourself: Do I want to measure performance to Cloud services such as salesforce.com, Skype or in house applications such as SAP?
2)    Decide what metrics you want to export. FnF can export MAC addresses, VLAN details and depending on the hardware, you can export entire packets, Cisco TrustSec, CoS, Jitter, packet loss, TCP latency, NAT details, NBAR, IP SLA, etc. Make sure your NetFlow Analyzer can filter and report on these details else, by our NetFlow collector!

3)    Make sure you are running the proper IOS version. Cisco IOS 15.2(2)T or more recent for the most capable Flexible NetFlow exports which combine Cisco NBAR and Cisco Medianet Performance Monitoring.
4)    Telnet to the router and disable traditional NetFlow and remove all of the commands from the router. Now try to forget about how to setup NetFlow. Commands like ip route-cache flow aren’t going to work anymore.
5)    Then, just setup Flexible NetFlow

Keep in mind that performing step 5 and configuring flexible netflow prior to step 4 will cause some duplicate flow exports. This is not something you want as it will over state utilization in the NetFlow reporting and it can also potentially cause problems for network security monitoring. Either way, it’s really a simple process and a huge value add to your overall network monitoring efforts. Don't forget to setup the new Cisco Performance Routing Flexible NetFlow exports. Call us if you need help.

]]> Tags: configuring flexible netflow, ip route-cache flow, Migrating to flexible netflow, NetFlow reporting, network security monitoring, network traffic monitoring, setup Flexible NetFlow Related tags: flexible netflow, forget setup, netflow exports, migrating flexible, traditional netflow, netflow
, ip route-cache flow, network security monitoring, NetFlow reporting, configuring flexible netflow, setup Flexible NetFlow, Migrating to flexible netflow, network traffic monitoring

Follow me:

Dropped NetFlow : Flow Sequence Numbers - Jan 29, 2012

TrackBacks | Comments | Tag with del.icio.us | Advanced NetFlow Traffic Analysis Home | Permalink: Migrating to Flexible NetFlow : BEST PRACTICES

Sponsored by the Call Center Outsourcing Community & the Virtual Contact Center Outsourcing Community

Monitoring Video Performance with NetFlow

Sun, 19 Feb 2012 02:41:55 -0500

Three years ago I was listening to John Chambers - CEO of Cisco Systems, proclaim that video was going to be the rage. I snickered and though it would be long time before anyone will be monitoring video performance with NetFlow. Well, here we are and John Chambers was right. My daughter is the one that made me realize why video vs. only voice will continue to grow.

Reporting on Skype with NetFlow
During a Skype connection with my daughter who was in my wife's car, my daughter wanted to show me her sneakers and then her book:

I noticed a big difference from when we just talked over the telephone. Seeing my face made her realize that I was fully engaged in what she had to say. She then put her jacket and shoes on and took the mobile phone outside to show me the fort she had built using scraps of wood. I couldn't believe it. She moved the camera in close for me to see things. She then brought be inside and put the phone in front of the dog so that I could say hello to 'Charlie'.

I have to admit, I liked the video especially since I was in London, England and my daughter was in Maine. What I didn't like was the jitter. I'm glad there are tools in our NetFlow traffic analyzer called Scrutinizer to monitor this.

Three years later at Cisco Live 2012 in London I was listening to Chief Cisco Futurist David Evans about the future of networking. I learned that video and data in general over the internet will continue to explode. This time I BELIEVE!

Cisco Performance Monitoring
Plixer was the first Cisco NetFlow Partner to become certified for Cisco Medianet Performance monitoring reports. Check out the VoIP jitter or lost packets in the network monitoring report below.

The above is VoIP with our Asterisk server. Skype traffic uses both TCP and UDP. We can measure the TCP latency during the connection setup with NetFlow Performance monitoring to look at Skype traffic as well. Today, customers can monitor cloud services with NetFlow. The example report below is filtering for the Cisco NBAR detected application: Skype.

Next Generation NetFlow
Keep in mind that these reports require the use of Flexible NetFlow which doesn't use the command ip route-cache flow. Make sure you are running IOS 15.2(2)T or more recent for the most capable Flexible NetFlow exports. The latest version provides even more network latency details than what is displayed above. I'm talking about Cisco IP SLA.

Performance Routing NetFlow
Cisco Performance Routing (PfR) can export IP SLA details using Flexible NetFlow. When a router determines that a connection is a bit congested, it will evaluate existing flows and reroute traffic over different connections ensuring priority to time sensitive traffic. By using PfR and Cisco Performance Monitoring together with Flow Hopper, administrators gain end to end network visibility on a link by link, hop by hop basis all with NetFlow.

The bottom line: Network traffic monitoring with NetFlow is at a whole new level from just two years ago. Join NetFlow Developments on Linkedin and stay on top of the future of NetFlow.

]]> Tags: Flexible NetFlow, Skype Related tags: flexible netflow, performance monitoring, skype traffic, performance netflow, cisco performance, netflow
, medianet performance monitoring, monitor cloud services, monitoring video performance, netflow performance monitoring, network monitoring, network traffic monitoring

Follow me:

Related Entries

Monitoring BYOD traffic with NetFlow - Mar 24, 2012

End to End Visibility : Network Flow Path - Feb 06, 2012

TrackBacks | Comments | Tag with del.icio.us | Advanced NetFlow Traffic Analysis Home | Permalink: Monitoring Video Performance with NetFlow

Sponsored by the Call Center Outsourcing Community & the Virtual Contact Center Outsourcing Community

End to End Visibility : Network Flow Path

Mon, 06 Feb 2012 04:01:15 -0500

Gaining end to end visibility into the path a flow took through the network can be an easy thing to do if you have the right network traffic monitoring tool. Because multiple paths exist between devices, leveraging traceroute or routed topology information may not provide the exact path taken by an end to end flow. To help address this mystery, a couple of utilities exist on the market today.

Mediatrace
Cisco Mediatrace which attempts to provide hop by hop path visibility of a flow (e.g. phone call) though the layer 2 and 3 topology. This application allows admins to view DSCP values, dropped packets, jitter and more on each device from point A to B. Although it is currently limited to Cisco hardware, the concept is solid and proves useful in VoIP and video environments.

Many networks are a hybrid of hardware vendors which may only contain a few Cisco Mediatrace capable routers and switches. Most if not all major routing platforms today from nearly all vendors support some form of NetFlow. What can be done to show flow path in a multi vendor environment.

Flow Hopper™
An Advanced NetFlow Analyzer should include a feature like Flow Hopper ™, a patent pending application which attempts to provide an end to end path of the flow through the layer 3 topology. Flow Hopper is different from Cisco Mediatrace in that it leverages any version of NetFlow or IPFIX from any vendor which includes NextHop in the flow export.

If Flow Hopper determines that an asymmetric flow path exists (i.e. a different route is taken on the return path), the GUI will draw out the connection accordingly. Admins can click on each router or layer 3 switch in the path and view all details exported in the flow template. Changes in element values (e.g. DSCP, TTL, octets, etc.) between ingress and egress metered flows are highlighted.

Medianet Performance Monitoring
Imagine how much easier root cause analysis is when you know the path and you can click on each router in the path to see the changes in the flow along the way. If you are using something like Performance Monitoring for Cisco Medianets, knowing where the jitter or packet loss was introduced can be very helpful. Make sure you are running IOS 15.2.2T or more recent for the most capable Flexible NetFlow exports.

End to end visibility which displays the complete network flow path is a must when trying to perform root cause analysis. If you want to learn more about Advanced NetFlow, join NetFlow Developments on Linkedin.

]]> Tags: Asymmetric flow path, cisco media trace, end to end visibility, Flexible NetFlow, network flow path, root cause analysis Related tags: cause analysis, cisco mediatrace, advanced netflow, attempts provide, click router, netflow
, end to end visibility, network flow path, cisco media trace, root cause analysis, Asymmetric flow path, end to end path

Follow me:

Related Entries

Monitoring BYOD traffic with NetFlow - Mar 24, 2012

Monitoring Video Performance with NetFlow - Feb 19, 2012

TrackBacks | Comments | Tag with del.icio.us | Advanced NetFlow Traffic Analysis Home | Permalink: End to End Visibility : Network Flow Path

Sponsored by the Call Center Outsourcing Community & the Virtual Contact Center Outsourcing Community

Dropped NetFlow : Flow Sequence Numbers

Sun, 29 Jan 2012 03:19:41 -0500

Dropped NetFlow detection should be a major part of the decision making process in your next enterprise NetFlow collector. High volume Netflow collection and reporting without regard to the NetFlow Sequence Numbers should send up red flags to an educated consumer in the market for a reliable NetFlow and IPFIX collector. Here’s why: It is a safe bet that companies serious about network traffic analysis or network traffic monitoring want to know if they are looking at all the data. In many cases they may not be. How would they know?

Are You Missing NetFlow
Routers and switches export flows with something called a flow sequence number. These flow sequence numbers increment and tell the NetFlow and IPFIX collector that data is missing if a datagram or flow is not received. If your NetFlow collector is receiving over 100,000 flows per second from hundreds or even thousands of routers, it is nice to know if you can rely on the trends when reports are run. For example, the Catalyst 6500 NetFlow exports are not always reliable. Counting the Flow sequence numbers on a busy Catalyst 6500 reveals a NetFlow overflow with TCAM tables issue that results in reports that display a utilization level on interfaces that are actually dealing with much higher utilization.

Every NetFlow Collector has a limit on what it can handle. How much it can handle can depend on several components:

Architecture of the collector
The amount of preprocessing of NetFlow data (e.g. looking for security threats)
The version of NetFlow/IPFIX
The volume of devices sending flows
The volume of flows from any one device

In the screen shot below, we can see that the Scrutinizer Netflow Analyzer is receiving nearly 6,000 flows per second from 5 different exporters. NOTE: our Linux collector can handle over 100K flows per second!

After further investigation, we discovered that most if not all of the Missed Flow Sequence Numbers (MFSN) are caused by one device. See below:

Above you can see the MFSN trend for port 2055. Notice directly below this trend outlined in red is a similar trend from a single device (i.e. router). This tells us that the majority of missed flows across all 5 exporters is happening on one device.

What does an increase in MFSN tell us?
The loss of flow exports is usually caused by one of three things:

The network dropped some packets
The router can’t keep up (e.g. Catalyst 6500)
The High Volume NetFlow collector can’t keep up

The above is why NetFlow sequence numbers are becoming increasingly important. Companies need to know if they can rely on the data:

Billing requires accurate data
Threat Detection requires accurate data

If they can’t rely on the data, what is the source of the problem?

NetFlow v9 vs v5
NetFlow v9 flow sequence numbers are incremented per datagram. NetFlow v5 flow sequence numbers are incremented per flow inside each datagram. A NetFlow reporting solution that properly deals with this difference requires fairly sophisticated engineering. Make sure you ask for it.

NetFlow Collection without Flow Sequence Number
The bottom line: NetFlow and IPFIX collection without Flow Sequence Number counting could be unreliable. This is especially true when dealing with high NetFlow volumes.

Join NetFlow Developments on Linkedin.

]]> Tags: Dropped netflow, High netflow volumes, High Volume NetFlow, Netflow flow sequence numbers, network traffic analysis, network traffic monitoring Related tags: sequence numbers, netflow collector, netflow ipfix, dropped netflow, numbers incremented, netflow
, High netflow volumes, Dropped netflow, Netflow flow sequence numbers, network traffic analysis, network traffic monitoring, High Volume NetFlow

Follow me:

Migrating to Flexible NetFlow : BEST PRACTICES - Mar 08, 2012

High Volume NetFlow Collector : Enterprise Traffic Analysis - Jan 03, 2012

TrackBacks | Comments | Tag with del.icio.us | Advanced NetFlow Traffic Analysis Home | Permalink: Dropped NetFlow : Flow Sequence Numbers

Sponsored by the Call Center Outsourcing Community & the Virtual Contact Center Outsourcing Community

Email Monitoring Software: Email Reporting

Fri, 20 Jan 2012 08:31:06 -0500

Email monitoring and email reporting is an important proactive responsibility that is not addressed in some companies. Who are the top email senders and receivers. What are the top domains sending or receiving emails? What are the top email subjects and who is involved with these emails. These types of reports allow you to investigate email issues such as repeated bounce backs and even virus trails.

The screen shots above are from Mailinizer Email Analyzer which is a free email reporting tool.

Why is It Important to monitor email traffic? There are several reasons listed below:

Email Troubleshooting

Was the mail server sending and receiving emails at a specified time and who was receiving them?
Who is sending non-deliverable or bounce back emails and who is the intended recipient?
What expired email addresses are still receiving messages and from whom?
Who is receiving "delayed email" messages from the mail server and how often does this occur?
What mail accounts are used the most or the least?

Email Marketing

When marketing campaigns are emailed out, how many messages have been sent so far?
Who received the email and who did they forward it to?
Report on what social networking sites send the most mail and to whom.
Are select employees/departments constantly sending/receiving emails to certain addresses or domains?

Mail and Network Traffic Details

NetFlow traffic analysis may indicate that the mail server is causing a lot of traffic, but who is sending or receiving all the messages? Mailinizer provides the details.
What sales and support people communicate the most via email and is it work related?
Overall, how many messages are sent and received for a specified time frame?

Routine email monitoring helps ensure better security, business continuity and at the same time improve Exchange organization performance. Routine reports to check include, but are not limited to:

Senders by message and order by count or size
Receivers by message and order by count or size
Domains (sending and receiving) and order by count or size
Conversations (Sender to Receiver) and order by count or size
Conversations with Subject (Sender to Receiver) and order by count or size
Popular Email Subjects and order by count or size
Email Volume and order by count or size

One of the keys to reporting is filtering. Good filtering allows you to narrow in on the data you need to investigate specific issues. Good filtering is crucial when mining email data and reporting on Microsoft Exchange email traffic. A good system empowers email tracking to identify specific account activity or delivery issues.

Mailinizer competitors include Promodag and MailMeter and others. Although I’m not familiar with all of the MailMeter problems or Promodag problems, I can tell you what the short comings are of Mailinizer. It doesn’t have direct access to the actual contents of the email. It is narrowly focused on reporting on where the email came from and where is it going and is limited to the details that Microsoft exports to the event log. The reporting however, is the best I’ve seen and should be part of every server room monitoring solution.

What is interesting about Mailinizer is it’s unique ability to handle large scale Microsoft Exchange email environments through the use of IPFIX technology. What is IPFIX? IPFIX is is used on each mail server to export details about each email received by the mail server. Similar to how NetFlow exports communication details on a router, Mailinizer uses IPFIX to treat email conversations like flows. Check out this Mailinizer video. Whatever email reporting solution you decide on, make sure you are routinely looking at proactive reports. Most solutions will automatically email them to you on a regular basis.

Join NetFlow Developments on Linkedin.
]]> Tags: email monitoring, email reporting, free email reporting, mailmeter competitors, mailmeter problems, promodag competitors, promodag problems Related tags: order count, sending receiving, email reporting, email monitoring, receiving emails, email
, email reporting, email monitoring, mailmeter competitors, promodag competitors, mailmeter problems, promodag problems, free email reporting

Follow me:

Sponsored by the Call Center Outsourcing Community & the Virtual Contact Center Outsourcing Community

NetFlow Behavior Analysis Systems : Limited Impact

Fri, 13 Jan 2012 05:24:48 -0500

NetFlow Network Behavior Analysis (NBA) systems have limited impact on detecting threats. They are only suitable as a second or perhaps third layer of threat detection. Some companies offering NetFlow Threat Detection tools would have you believe otherwise. Don't fall for it. "Gartner says NBA is suitable as a complementary technology to intrusion detection and prevention software, which is effective for addressing network attacks that can be positively identified." As a HUGE NetFlow and IPFIX supporter, I tend to agree that flow technologies can augment security practices, but can't replace them.

What's Missing in NetFlow Threat Detection?
Most NetFlow exports do not include the entire data portion of the packet. Sophisticated threat detection systems require this data in order to compare the contents to a database of signatures.

Can NetFlow Export Packets?
A NetFlow competitor called sFlow exports the entire packet, however its sampling architecture results in frequently missed infected frames. Cisco Smart Logging Telemetry NetFlow technology which is available on the Catalyst 3000 series can export entire packets however, it only exports datagrams that it detected via ACLs. NetFlow-Lite on the Catalyst 4948E can also export entire packets however, in most cases it is configured to sample. The bottom line is that NetFlow and IPFIX collection is not intended for large scale full packet exports. NetFlow does however try to give you 100% of the connection information which can be useful for behavior analysis, but it is still limited.

NetFlow Behavior Analysis systems
Some NetFlow Behavior Analysis systems attempt to detect threats by base lining system behaviors overtime. When a host communicates outside of its normal behavior baseline, its index goes up. If the index goes too high an alarm can be triggered. Because an end systems behavior is constantly changing, the alarm is frequently a false positive.

Certainly NetFlow and IPFIX, but generally not sFlow can be used to accurately detect threats. The point of this post is to educate on why it should only be part of a company's Unified Threat Management solution. NetFlow can be used to accurately detect SYN scans, ICMP redirect issues, DDoS attacks, XMAS scans, etc. In practice, these same algorithms will often also get triggered by legitimate traffic. Experience tells us that IPFIX and NetFlow are ideally suited for accounting and utilization reporting.

Grow Your Internal Threat Detection
Internal threat detection is a growing area of concern in many networks today. Some companies are placing firewalls on backbone links as yet another layer of protection from internal infected hosts. Forrester Research calls for a Zero Trust model where networks are designed from the inside out. "The redesign starts with a black box or network segmentation gateway that can handle high speeds – up to 10G interfaces. The gateway acts like a UTM appliance, but it does much more than provide firewall, antispam and content filtering features. It can add data leakage prevention capabilities, intrusion prevention and encryption to the network" said John Kindervaq, a senior analyst with Forrester Research, Inc. NetFlow is largely about monitoring internal traffic. The Cisco ASA, Palo Alto Networks firewall and the SonicWALL firewall all export NetFlow or IPFIX.

NetFlow's threat detection value belongs as part of a internal UTM effort where potential threats detected are sent to a SEIM which will then look for other messages from appliances witnessing the same behavior of a host. If other threat detection efforts are not detecting the same suspicious behavior perhaps a false positive can be avoided. Here's a thought: maybe the security appliance reporting the bad behavior should have an index whereby its accuracy for detecting legitimate threats could be graded over time.

Although often used as a differentiator by NetFlow reporting companies, the demand for dedicated NetFlow and IPFIX threat detection tools is limited. Detecting security threats with NetFlow Behavior Analysis systems is not often the best primary threat protection due to unpredictable host behaviors and lack of the entire packet. The largest opportunity for threat detection with NetFlow lies in IP Host Reputation lookups.

Join NetFlow Developments on Linkedin.

]]> Tags: ip host reputation, NetFlow Behavior Analysis, Netflow threat detection, NetFlow-Lite, Network threat detection, Smart logging telemetry, threat detection with NetFlow Related tags: threat detection, behavior analysis, netflow ipfix, analysis systems, netflow behavior, netflow

Follow me:

TrackBacks | Comments | Tag with del.icio.us | Advanced NetFlow Traffic Analysis Home | Permalink: NetFlow Behavior Analysis Systems : Limited Impact

Sponsored by the Call Center Outsourcing Community & the Virtual Contact Center Outsourcing Community

High Volume NetFlow Collector : Enterprise Traffic Analysis

Tue, 03 Jan 2012 11:44:32 -0500

A high volume NetFlow collector is a must for many service providers and universities. Because of the nature of the type of Internet traffic created by these types of organizations, enormous amounts of flows are created. People visiting internet search sites such as Google or those of us clicking on all the different links in facebook or youtube often end up creating a new flow with each click.

With VoIP, BitTorrent, Skype, iCloud and the like now on the network, administrators are dealing with even more flows. On the NetFlow and IPFIX reporting side of things, vendors often find that 2-3 issues come into play when scaling NetFlow tools:

The number of flow exporting devices
The number of interfaces across all flow exporting devices
The total volume of flows per second

High speed NetFlow collection can lead to very large database tables. Large tables, if not indexed or queried correctly can lead to poor performance in traffic analysis reporting. As a consumer, how a vendor deals with enormous amounts of flow data can and should be part of the vendor selection process.

High NetFlow volumes does not necessarily mean you have to use multiple distributed NetFlow collectors. Many NetFlow and IPFIX collectors can handle tens of thousands or even over one hundred thousand flows per second with a single appliance (e.g. Scrutinizer). Distributed NetFlow collection should be configured when sending all of the flows over a wide area link doesn’t make sense. Enterprise NetFlow analysis requires a careful understanding of the IT managers goal, the budget constraints and the potential bottle neck areas on the network.

Goals: Does the IT team need NetFlow insight into all areas of the network? What problems are they trying to resolve?
Budget: What is the budget for the new traffic analysis solution? Can they invest in stages? What is the yearly maintenance contract?
Bottle necks: Where are the potential bottle neck areas on the network? Due to budget constraints, it may make sense to purchase a lower license. Focusing on the bottle necks followed up with good proactive reporting may allow the IT team to push off further licensing investments.

Work with your vendor to determine if a single flow collector or if distributed NetFlow collection is in your companies best interest. Beware of the necessary add-on modules and remember to ask about the yearly maintenance cost.

Join NetFlow Developments on Linkedin.