Infohub‎ > ‎Articles‎ > ‎

Gathering NetFlow data with a FreeBSD server

Sometimes you need to know what's going on on a segment of your network. And here comes handy the good old network sniffer (someone said Wireshark?).

There are some problems with this: if you know that yesterday night, or last sunday, your network was "slow", how do you find out who were generating traffic? Cisco provides you with Netflow enabling router to export data about flows that are seen on the network (among other things). This data is going to get collected in some sort of database, allowing analysis in any moment.

Until here we have good news. The bad news is that not all Cisco devices support NetFlow, but only high end ones, and sometimes it is requires some special add-on card or software release. Also Cisco analysis software comes at big price.

The last is not a big problem: there are lots of open source and free NetFlow data collectors available. My choice was the nfdump framework that runs smoothly on FreeBSD and has a primitive but somewhat functional web interface nfsen. I will not cover here the setup and installation of these, but it's a simple pkg_add job.

The main remaining problem is how to get NetFlow data if you do not have a mighty Cisco device.

My first solution was using Softflowd. It basically sniffs the traffic from one interface, analyses it and builds up NetFlow status and data to send them to your collector.
This worked no bad, but processor load can be very heavy if you have several Gigabit interfaces, leading to packet loss and general server slowdown. Also the exported data lead me to think that something was missing or not so coherent.

So I switched to using netgraph, a powerful framework part of FreeBSD since lots of time and that I was unaware of.

Honestly I can't say I got the whole picture behind netgraph, but probably I know enough to have it do the job. The idea is to have the NetFlow data generated by a specialized kernel module, ng_netflow, far more efficient than a userspace program.

So let's see an example based on my setup:

I need to sniff some specific VLANs (my DMZ) running on my Catalyst, so I configured a monitor session spitting out packets on a gigabit interface. That interface is connected to my FreeBSD host, on an interface (bge0) renamed dmz. The same machine hosts also nfsen, but it is not relevant here: assume the nfsen machine is called nfsen_host and is listening on port 9996.

Start ngctl, and get an interactive prompt (you can later put all the commands in a file and pass it via -f switch), and type these lines:

mkpeer dmz: netflow lower iface0
name dmz:lower nfdmz
connect dmz: nfdmz: upper out0
mkpeer nfdmz: ksocket export inet/dgram/udp
msg nfdmz:export connect inet/nfsen_host:9996

Here nfdmz is the name I choose form my node, you can change it at your will.

Well, having this done you'll see, nothing... In the sense that no flows will be exported now, because they do not enter FreeBSD IP stack, since it is a span port, not a router port. The trick is to put the dmz interface in promiscuous mode:

ifconfig dmz promisc

That's all. Now from the sensor machine (in my case the same as the Netflow collector one), you can just type tcpdump port 9996 and see NetFlow data going to its destination, ready to be analyzed.

And yes, start top to see cpu usage: you'll notice little load on your server, allowing to have a single machine to monitor high traffic segments without too many hassle.