- last edited on by
ciscomoderator
This is a list of NetFlow best practices that I have compiled that I feel should be considered when exporting flow data. It is meant for software developers and consumers who are considering the export of flows for network traffic monitoring. The goal behind these NetFlow best practices is to ensure that you can accurately report on what is on the network.
When I started drafting this blog I kept returning to a few terms that I feel are very important but, shouldn’t be considered part of the 11 NetFlow best practices. I decided to write about these terms first because I think they are important issues when considering best at NetFlow practices. First make sure you understand some basic NetFlow terms:
Observation Point
The location where a router, switch or probe meters flow information (i.e. network traffic) is called the Observation Point (OP). The OP is a location in the appliance where IP packets can be observed. The most common location is the ingress interface although the egress interface is sometimes used as well. In IPFIX the location of the OP can be exported in the template.
Flow Direction
Which either is the case - ingress or egress, the topic of ‘flowDirection’ Information Element (#61) should be understood. The flowDirection as defined in RFC 5102 is: The direction of the flow observed at the Observation Point.
5102bis: This document defines the datatypes and management policy for the information model for the IP Flow Information eXport (IPFIX) protocol. This information model is maintained as the IANA IPFIX Information Element Registry, the initial contents of which were defined by RFC 5102.
If the packets are metered as they enter an interface, the value of the ‘flowDirection’ must be 0 (i.e. ingress). If on the other hand the packets are metered as they exit the interface, the value of the ‘flowDirection’ must be 1 (i.e. egress).
Again, the Observation Point is the interface the flows were metered on. The flowDirection defines whether the flows were metered as they came in ‘ingress’ or went out ‘egress’ the interface. A detailed outline on when to meter flows ingressly or egressly is covered in another post.
Every Observation Point is associated with an Observation Domain (defined below) and that one Observation domain may be a superset of several other Observation Points. For example, one Observation domain can be an entire line card that would be the superset of the individual Observation Points at the line card's interfaces.
Observation Domain
In its simplest form, an Observation Domain OD is a router or switch with multiple interfaces or OPs. However, a single line card with multiple interfaces could be an OD.
The Observation Domain is unique per exporting process and every OP is associated with an OD. Per RFC 5101, it is recommended that Observation Domain IDs also be unique per IPFIX device.
Best NetFlow Practices
In the process of helping over a dozen different vendors implement a flow export, a pattern of consistent problems has emerged.
- SNMP Indexes: Earlier in this book, the source interface was identified as one of the elements in the traditional seven field tuple. In at least four different exports from four different vendors, the source interface provided in the flow did not match the SNMP index. This is a problem that should be avoided by making sure that the ingress and egress interfaces exported in flows always (I.e. MUST) match those instances available via SNMP. Also, in most cases the egress interface should not be 0. If you read the IANA description (or even RFC 5102) you will see. "The value matches the value of managed object 'ifIndex' as defined in [RFC2863]."
- Active Timeout: The active timeout covered earlier should be set to 60 seconds. This is because most reporting tools provide a granularity level at one minute intervals. On a few occasions, vendors have forgotten to implement an active time out resulting in long lived flows and abnormal spikes in the graphs when the data is used to build trends. An active timeout routine should be built into the flow export.
- Inactive Timeout: Flows that not active for a period of time (E.g. 15 seconds) should be exported from the flow cache.
- octetDeltaCount: Long-lived flows as outlined above should be summarized and exported every 60 seconds as specified by the active timeout. The bytes for the flow should provide the delta values every 60 seconds and not the octetTotalCount since the inception of the flow. This is an easy mistake to make due to the similarities between SNMP and flow technologies. SNMP requires the reporting tool to calculate the delta. With flows, the exporter is expected to provide the delta.
- Flow Direction: Flow direction should be implemented as it tells the collector how the connection was metered for ingress and egress flows. In the case of bidirectional flows, it is best not to send the element in the template as there is no reason for it.
- Multiple Templates: Exporting multiple templates is a great way to gain efficiencies in minimizing database growth. This is generally done through the use of a foreign key in the flow data template which relates to the same key found in the meta data or option template. Cisco’s NBAR NetFlow export is a great example of this. Exporting an application name as a string would cost 40-50 bytes per row in the database. Using only an integer in the flow will consume four bytes.
- Template Timeout: The template timeout is discussed in another chapter. In short, it specifies how frequently a template in NetFlow v9 and IPFIX is exported. Until a template is received, data is sometimes cached or dropped by the collector. In most cases, the template is exported every 1-5 minutes.
- NetFlow Volume: A scalable NetFlow and IPFIX collector can handle over 100K flows per second with a single appliance.
- Missed Flow Sequence Numbers: A NetFlow reporting solution that trends missed flow sequence numbers (MFSN) can be invaluable when trying to diagnose whether or not the collector is receiving all the flows from every device sending them. A NetFlow or IPFIX solution that doesn’t trend MFSN can’t be trusted to be accurate.
- Filtering: A NetFlow reporting solution should allow the user to filter on any field exported in the template. AND, OR, GREATER THAN, LESS THAN, EQUAL TO, ETC. should all be supported. It should also allow for thresholds to be set which trigger notifications.
- Reporting: Every vendor assumes they deliver the reports you will want. The ideal network traffic monitoring solution ships with dozens of reports and if you don’t see the report you need, you can create the report yourself. Don’t settle for less as this is becoming an increasingly important feature with technologies such as Cisco Application Visibility and Control NetFlow exports.
Now that the concepts involved with proper flow exports are clear, it is time to make sure that you have invested in a proper NetFlow Collector as flow collection is an equally important part of a proper flow implementation.
