High-precision NFDUMP data visualization
I was given a similar mandate for as with this task but this time around I did have a lot of historical data already by means of netflow data collected by NFDUMP a high-performance open source netflow collector. The only problem the data as such could not be used for my purpose without a bit of massaging. And here is how I did it.
Create a graph from collected netflow data in the NFDUMP format with a precision of one second.
Note: For lower precision graphs the NfSen visualization package should suffice.
Extract the netflow data
In order to process the NFDUMP binary format with R the data has to be converted into something parsable. The best option is the -o pipe option of the nfdump command. Additionally to suppress statistical data at the end of the output the quiet -q option is to be set. One drawback of the pipe format output is the missing of a descriptive header. To include one the nfdump command is perpended by a simple echo for readability split into two lines.
( echo -n "N1|tstart|mstart|tend|mend|proto|N2|N3|N4|srcip|srcp|"; \ echo "N5|N6|N7|dstip|dstp|N8|N9|N10|N11|flags|tos|packets|bytes"; \ nfdump -R /path/to/nfdump/data -q -o pipe \ ) > netflow.data
The fields starting with a capital N are later discarded of by the R script.
Produce graph with R