Well it isn’t exactly a death blow to Wireshark or to network security appliances that perform deep packet inspection to detect threats, however, the rising percentage of secure network connections is certainly strengthening the “look to the flows first” position in the NetFlow Vs. Packet Analysis debate. I think there are clear advantages between the two network traffic monitoring technologies and in this post, I’d like to point out where both technologies run into similar limitations.
One of my favorite points we make in our Advanced NetFlow Seminars is that a single flow can represent thousands of packets. We have a slide in the presentation that shows 1364 packets in both directions for a telephone conversation. We show all the packets in Wireshark then, we show a single flow in each direction and each flow shows a counter of 1364 packets. In the class we discuss how flows don’t contain all of the details available in the packets and then we show Cisco’s Performance Monitoring export which includes SSRC and metrics on jitter and packet loss for each call. Although flows don’t include the actual voice payload of each call, I have experience with NetFlow-Lite exports from the 4948E where we pulled the packets out of NetFlow and opened them up in Wireshark. At this point, we could have replayed the actual conversation. Also, by configuring Flexible NetFlow, ISRs are also capable of sending entire packets.
Gartner last year stated that flow analysis should be done 80% of the time and that packet capture with probes should be done 20% of the time.
The point I want to make today regarding flows and packets is around secure connections. Specifically, SSL connections (e.g. https, chat) where the contents of the packets are encrypted resulting in the packet analyzer being unable to see the contents inside the individual datagrams. In this case, the value of having the individual packets for some troubleshooting is diminished greatly simply because we are unable to see the payload. The “more detail” with packets argument is nearly moot when comparing to NetFlow or the proposed standard for NetFlow called IPFIX. Respectively, a single flow would have exported the entire series of packets in the same SSL connection as port 443 with the corresponding address pairs. You might have all the individual encrypted packets with Wireshark but, depending on the issue being trouble shot, it may not do us any good if they are encrypted.
Now lets consider network threat detection. A host infected with malware (e.g. Advanced Persistent Threat) which makes connections to a C&C host are often established outbound. Generally, the IPS, proxy server or firewall won’t even question an outbound connection and let it pass right on through. Even if these security appliances did scrutinize the connection, they would find an SSL encrypted connection which cannot be snooped. Sophisticated signature matching often won’t help detect this type of threat. What can we do?
Some forms of malware can be detected by looking at the communication behaviors. Simply by looking at the TCP flags, the ratio of flows to unique hosts or by comparing flows to a host reputation database we can uncover many potential internal network threats. I wonder if Gartner knew all this when they came up with the 80% number? Perhaps it is more like 90% of network traffic analysis should be done with NetFlow and IPFIX.