Install & configure nfdump with nfsen on Ubuntu server 10.04

NFSEN

This was done using Ubuntu server 10.04 although everything is compiled from source so the commands should be very similar on any linux box. There are also example configs for Cisco ASA 8.2 near the bottom of the post.

I was looking for a netflow collector/analyser that would accept v9 flows from Cisco ASA devices. These devices create v9 “NetFlow Security Event Logging” (NSEL) flows which can include information about security events in addition to the traditional v5 flow info. Luckly there is a specific NSEL version of nfdump that still works with the web based gui nfsen. The original (non NSEL) version of nfdump doesnt support v9 flows at the time of writting. If you dont need v9 support you arent restricted to the NSEL version.

Download, extract, compile and install rrdtool from source. This will install rrdtool to /usr/local/rrdtool obviously change this to your preference.

cd /tmp
wget http://oss.oetiker.ch/rrdtool/pub/rrdtool.tar.gz
tar xzfv rrdtool.tar.gz
cd rrdtool-1.4.5/
./configure -prefix=/usr/local/rrdtool -disable-tcl
make
sudo make install

Now we grab the nfdump source, compile it with nfprofile support thats required by nfsen and install it.

cd /tmp
wget http://sourceforge.net/projects/nfdump/files/nsel/nfdump-1.5.8-NSEL/nfdump-1.5.8-NSEL.tar.gz/download
cd nfdump-1.5.8-NSEL/
./configure –with-rrdpath=/usr/local/rrdtool –with-ftpath=source –enable-nfprofile
make
sudo make install

Now to get the nfsen source and extract it

cd /tmp
wget http://sourceforge.net/projects/nfsen/files/stable/nfsen-1.3.5/nfsen-1.3.5.tar.gz/download
tar xzf nfsen-1.3.5.tar.gz
cd nfsen-1.3.5

Before we can install it we need to add a user for nfsen to run as and configure some parameters in the nfsen-dist.conf file. Or you can use an existing user e.g. www-data and skip this step using the appropriate user substituted in the following steps. (thanks to Rafael Fonseca for this suggestion via his comment below)

sudo useradd nfsen
vi etc/nfsen-dist.conf

Now we need to change the config to reflect the changes below.

$BASEDIR = “/usr/local/nfsen”;
$HTMLDIR    = “/var/www/nfsen/”;
$USER = “nfsen”;
$WWWUSER = “nfsen”;
$WWWGROUP = “nfsen”;

%sources = (
‘m00nies-ASA’ => { ‘port’ => ‘2055’, ‘col’ => ‘#000fff’, ‘type’ => ‘netflow’ },
‘Another-ASA’ => { ‘port’ => ‘2056’, ‘col’ => ‘#0000ff’, ‘type’ => ‘netflow’ },
‘An-IOS-netflow-source’ => { ‘port’ => ‘2057’, ‘col’ => ‘#00000f’, ‘type’ => ‘netflow’ }

Change the sources to your own name instead of MY_ASA and the port you configured the device to send to. Notice each source has its own unique port to send to and non NSEL sources are configured in the same way. Now save the file. Then install and start nfsen with the following command.

sudo ./install.pl etc/nfsen-dist.conf
sudo chown -R nfsen /usr/local/nfsen
sudo /usr/local/nfsen/bin/nfsen start

Tail the syslog file to check for errors.

sudo tail -f /var/log/messages

By default the template is only sent from a Cisco ASA every 30 mins so grab a coffee until nfsen can make sense of the flows.

nfsen should now be accesable via http://MYSERVER/nfsen/nfsen.php if you get a nfsen permission error check the nfsen user has permission to use the socket at /usr/local/nfsen/var/run/nfsen.comm

Cisco ASA netflow config

access-list NETFLOW extended permit ip any any
!
class-map NetFlow-traffic
match access-list NETFLOW
!
!
policy-map global_policy
class NetFlow-traffic
flow-export event-type all destination 10.1.1.3
!
!
flow-export destination Outside 10.1.1.3 2055
!
! Configure how often templates are sent in minutes 30 is default
!
flow-export delay flow-create 30

Check the ASA is exporting flows

m00nies-ASA# show flow-export counters

destination: Outside 10.1.1.3 2055
Statistics:
packets sent 1077495
Errors:
block allocation failure 0
invalid interface 0
template send failure 0

m00nie 🙂

Comments

  • Excellent write-up! There’s only one small typo, in the 4th code block from the bottom up:

    sudo chown -R /usr/local/nfsen

    should be

    sudo chown -R nfsen /usr/local/nfsen

    (But in my case I just avoided the creation of the netflow and nfsen users and used www-data)

  • Hey im facing a problem after installing NFSEN with Nfdump, when i try to access http:///nfsen/nfsen.php it gives me error “ERROR: nfsend connect() error: Permission denied!
    ERROR: nfsend – connection failed!!
    ERROR: Can not initialize globals!”

    Need help what i have done wrong..

    and how to check nfdump is working or not..???

  • Hi Aslam

    Sounds like you need to check the permissions of the socket at /usr/local/nfsen/var/run/nfsen.comm.
    Just to test you could try chmod 777 /usr/local/nfsen/var/run/nfsen.comm if that fixes it you can restrict the permissions further.

    m00nie 🙂

  • Hi ZW

    Sorry I dont quite understand what the question is?
    sudo /usr/local/nfsen/bin/nfsen start is the command to start the collector.

    m00nie

  • HI m00nie
    My question is, in view path / usr / local / nfsen / bin /, which did not run the file.
    And I enter the command, sudo. / Install.pl etc / nfsen-dist.conf, when prompted
    perl to use: [/ usr / bin / perl], I do not know what the input path.
    Ask you, thank you!

  • Hi ZW

    /usr/bin/perl is probably right. You can find the location of perl with the whereis command e.g:

    m00n@Ubuntu_6:~$ whereis perl
    perl: /usr/bin/perl

    Hope this helps

    m00nie

  • Hi!

    – succesfull make install of nfdump
    – successfull install nfsen (i can see plain graphs)

    but my syslog is full with this. Do you have any idea?

    thx

    Jan 11 03:28:32 netmon /usr/local/bin/nfcapd[21237]: Process v9: [0] No table for id 261 -> Skip record
    Jan 11 03:28:32 netmon /usr/local/bin/nfcapd[21237]: Process v9: [0] No table for id 265 -> Skip record
    Jan 11 03:28:32 netmon /usr/local/bin/nfcapd[21237]: Process v9: [0] No table for id 260 -> Skip record
    Jan 11 03:28:32 netmon /usr/local/bin/nfcapd[21237]: Process v9: [0] No table for id 265 -> Skip record

  • Sorry Dude, I just should have to wait a while, ASA sent automatically the table, and everything started working.

    great and usefull post, thank you so much!

    Jan 11 03:39:08 netmon /usr/local/bin/nfcapd[21237]: Process_v9: [0] Add template 256
    Jan 11 03:39:08 netmon /usr/local/bin/nfcapd[21237]: Process_v9: [0] Add template 257
    Jan 11 03:39:08 netmon /usr/local/bin/nfcapd[21237]: Process_v9: [0] Add template 258
    Jan 11 03:39:08 netmon /usr/local/bin/nfcapd[21237]: Process_v9: [0] Add template 259
    Jan 11 03:39:08 netmon /usr/local/bin/nfcapd[21237]: Process_v9: [0] Add template 260
    Jan 11 03:39:08 netmon /usr/local/bin/nfcapd[21237]: Process_v9: [0] Add template 261
    Jan 11 03:39:08 netmon /usr/local/bin/nfcapd[21237]: Process_v9: [0] Add template 262
    Jan 11 03:39:08 netmon /usr/local/bin/nfcapd[21237]: Process_v9: [0] Add template 263
    Jan 11 03:39:08 netmon /usr/local/bin/nfcapd[21237]: Process_v9: [0] Add template 264
    Jan 11 03:39:08 netmon /usr/local/bin/nfcapd[21237]: #012 Suhas: ValidateAndUpdateNselFields :Invalid field type :[152] revcieved
    Jan 11 03:39:08 netmon /usr/local/bin/nfcapd[21237]: Process_v9: [0] Add template 265
    Jan 11 03:39:08 netmon /usr/local/bin/nfcapd[21237]: #012 Suhas: ValidateAndUpdateNselFields :Invalid field type :[152] revcieved
    Jan 11 03:39:08 netmon /usr/local/bin/nfcapd[21237]: Process_v9: [0] Add template 266
    Jan 11 03:39:08 netmon /usr/local/bin/nfcapd[21237]: #012 Suhas: ValidateAndUpdateNselFields :Invalid field type :[152] revcieved
    Jan 11 03:39:08 netmon /usr/local/bin/nfcapd[21237]: Process_v9: [0] Add template 267
    Jan 11 03:39:08 netmon /usr/local/bin/nfcapd[21237]: #012 Suhas: ValidateAndUpdateNselFields :Invalid field type :[152] revcieved
    Jan 11 03:39:08 netmon /usr/local/bin/nfcapd[21237]: Process_v9: [0] Add template 268
    Jan 11 03:39:23 netmon nfsen[21245]: connection on UNIX socket
    Jan 11 03:39:23 netmon nfsen[21245]: comm server started: 2093
    Jan 11 03:39:23 netmon nfsen[2093]: Cmd Decode: get-globals
    Jan 11 03:39:23 netmon nfsen[2093]: Cmd Decode: get-du
    Jan 11 03:39:23 netmon nfsen[2093]: comm child[2094] terminated with no exit value
    Jan 11 03:39:23 netmon nfsen[2093]: Cmd Decode: get-profile
    Jan 11 03:39:23 netmon nfsen[2093]: Cmd Decode: quit
    Jan 11 03:39:23 netmon nfsen[21245]: comm child[2093] terminated with no exit value
    Jan 11 03:39:23 netmon nfsen[21245]: connection on UNIX socket
    Jan 11 03:39:23 netmon nfsen[21245]: comm server started: 2095
    Jan 11 03:39:23 netmon nfsen[2095]: Cmd Decode: get-picture
    Jan 11 03:39:23 netmon nfsen[21245]: comm child[2095] terminated with no exit value

  • Thanks mOOnie, when i use www-data as the $USER $WWWUSER $WWWGROUP, My problem is solved. Firefox can show correctly.
    But after i edit the nfsen.conf file, and add
    ‘S93’ => { ‘port’ => ’23456′, ‘col’ => ‘#000fff’, ‘type’ => ‘netflow’ },
    then i input ./install.pl etc/nfsen.conf, another problem occured:
    Add source ‘S93’Error while setting up channel ‘S93’: Can’t create channel directory:
    ‘/usr/src/nfsen/profiles-data/live/S93’ File exists
    No collector started!

    why and how to solve?
    3Q…

  • I solved this problem.
    You have to delete the directories in /usr/local/nfsen/profile-data/live/
    then run ./install.pl etc/nfsen.conf, It will be ok. 😛

  • why i always have this error 🙁 :

    “Can’t create channel directory: ‘/usr/local/nfsen/profiles-data/live/…File exists
    No collector started!

    while i try to start nfsen, and i try many ways to config source in nfsen-dist.conf or delete the directories in /usr/local/nfsen/profile-data/live/ as this guy above…but the same things always happen. 🙁
    Can you advise me ?
    Thanks in advanced!

  • Hi Anhlee

    Maybe a ls -al /usr/local/nfsen/profiles-data/live/ would show the file that possibly exists?
    Running ./nfsen reconfig in the /usr/local/nfsen/bin directory might help sort things out once you’ve removed all files and directories in profile-data?

    m00nie

  • This is all working fine for me, apart from one thing: in nfsen, I only see flows/sec graphs, not bytes or packets – those just sit at zero. My sources are a couple of ASAs running 8.2.x. Did you have the same problem? I was hoping to get data on the top talkers by protocol, host etc…

  • Hi Howie

    Never come across that particular problem before. Most of my firewalls are running 8.2 and I can use nfsen to display the top 10 by bytes/protocol etc.
    No issues or errors reported in the logs?

    m00nie

  • My error on install:
    Rebuilding profile stats for ‘./live’
    Unable to create graph: No such file or directory
    Error GenGraph: Profile: live, traffic-day: Legend set but no color: peer2 at li bexec/NfSenRRD.pm line 337.

    Any help?

  • I get the same problem as Howie: nfdump shows zero bytes for both packets and bytes. Typical log entries:

    Jan 14 12:25:01 lch-mon1 nfcapd[23506]: Ident: ‘lch-asa1’ Flows: 5242, Packets: 0, Bytes: 0, Sequence Errors: 0, Bad Packets: 0
    Jan 14 12:25:01 lch-mon1 nfcapd[23506]: Total ignored packets: 0

    I am running ASA firmware 8.4(3). I will try upgrading to 8.4(5) since it looks like there has been a change for byte counters:

    http://www.cisco.com/en/US/docs/security/asa/roadmap/asa_new_features.html#wp95483

    “NSEL: Flow-update events have been introduced to provide periodic byte counters for flow traffic”

  • Also some minor corrections to the flow-export parameters:


    ! time in MINUTES for sending templates (default 30)
    flow-export template timeout-rate 1
    ! time in SECONDS for aggregating identical flows
    flow-export delay flow-create 10

  • After upgrading to 8.4(5), and decoding the packets with tshark, I now see two new fields in the flow records:

    Initiator Octets: 8343
    Responder Octets: 2119

    but still no packet counters. And nfdump still shows 0 bytes and 0 packets for each flow, so I guess nfcapd is expecting this information in different Netflow v9 fields.

  • Brian, I’m experiencing the same using ASA 8.3. I can however see the byte counts with nfdump:

    Date flow start Duration Proto Src IP Addr:Port Dst IP Addr:Port Packets Bytes Flows
    2013-01-25 21:03:36.296 0.000 TCP 10.0.0.3:43911 -> 10.0.0.1:443 4.3 G 2.6 M 1
    2013-01-25 21:03:36.296 0.000 UDP 10.99.255.198:55496 -> 10.0.0.239:53 1.0 M 2.6 M 1
    2013-01-25 21:03:36.296 0.000 UDP 10.99.255.198:49541 -> 10.0.0.239:53 1.0 M 2.6 M 1

    But in daemon.log I see no byte count:
    Feb 21 12:15:00 nfsen nfcapd[15643]: Ident: ‘asa-active’ Flows: 27886, Packets: 0, Bytes: 0, Sequence Errors: 0, Bad Packets: 0

    Did you manage to solve yours?

  • There is a new version of nfdump which works with the ASA 8.4 flows: nfdump-1.5.8-4-NSEL

    And the next major release of nfdump will integrate this so you don’t need a separate version. From nfdump-discuss mailing list:

    “New users are encouraged to use upcoming nfdump-1.6.9, which contains full support for CISCO NSEL ASA and NEL Nat devices”

  • “New users are encouraged to use upcoming nfdump-1.6.9, which contains full support for CISCO NSEL ASA and NEL Nat devices”

    Thanks for the info Brain 😛

  • Anyone can guide on the below error after all above steps :

    RRDs object version 1.4004 does not match bootstrap parameter 1.4008 at /usr/lib/perl5/5.8.7/x86_64-linux/DynaLoader.pm line 253.
    Compilation failed in require at libexec/NfSenRRD.pm line 38.
    BEGIN failed–compilation aborted at libexec/NfSenRRD.pm line 38.
    Compilation failed in require at libexec/NfSen.pm line 43.
    BEGIN failed–compilation aborted at libexec/NfSen.pm line 43.
    Compilation failed in require at ./install.pl line 44.
    BEGIN failed–compilation aborted at ./install.pl line 44.

  • Unable to view graphs

    All things are done sharing log meassages below :

    Feb 12 20:10:06 cprakash nfcapd[20921]: Ident: ‘NIXI-DELHI’ Flows: 754, Packets: 19844, Bytes: 2472905, Sequence Errors: 1, Bad Packets: 0
    Feb 12 20:10:06 cprakash nfcapd[20921]: Total ignored packets: 0
    Feb 12 20:10:10 cprakash nfcapd[20924]: lseek() error in nffile.c line 562: Bad file descriptor
    Feb 12 20:10:10 cprakash nfcapd[20924]: Ident: peer1, Failed to create sub hier directories: mkdir() error for ‘/usr/local/nfsen/profiles-data/live/peer1/2014’: No such file or directory
    Feb 12 20:10:10 cprakash nfcapd[20924]: Ident: peer1, Can’t rename dump file: No such file or directory
    Feb 12 20:10:10 cprakash nfcapd[20924]: Ident: peer1, Serious Problem! Fix manually
    Feb 12 20:10:10 cprakash nfcapd[20924]: Ident: ‘peer1’ Flows: 0, Packets: 0, Bytes: 0, Sequence Errors: 0, Bad Packets: 0
    Feb 12 20:10:10 cprakash nfcapd[20924]: Failed to open file /usr/local/nfsen/profiles-data/live/peer1/nfcapd.current.20922: ‘No such file or directory’
    Feb 12 20:10:10 cprakash nfcapd[20924]: killed due to fatal error: ident: peer1
    Feb 12 20:10:10 cprakash nfcapd[20924]: Total ignored packets: 0
    Feb 12 20:10:15 cprakash nfsen[24778]: 0 channels/alerts to profile
    Feb 12 20:10:15 cprakash nfsen[24778]: Update profile live in group .
    Feb 12 20:10:15 cprakash nfsen[24778]: Run expire at Wed Feb 12 20:10:00 2014
    Feb 12 20:10:15 cprakash nfsen[24778]: End expire at Wed Feb 12 20:10:00 2014
    Feb 12 20:10:16 cprakash logger: server.test.com:80 202.141.127.72 – – [12/Feb/2014:20:10:16 +0530] “GET /nfsen/nfsen.php HTTP/1.1” 404 293 “-” “Mozilla/5.0 (Windows NT 6.1; WOW64; rv:27.0) Gecko/20100101 Firefox/27.0”

  • Hi Moonie

    Getting below error:

    Not Found

    The requested URL /nfsen was not found on this server.
    Apache/2.2.3 (Red Hat) Server at 202.141.127.74 Port 80

  • Hi!
    anybody getting this error:
    “Can’t create channel directory: ‘/usr/local/nfsen/profiles-data/live/…File exists
    No collector started!

    Solution:
    delete the folder referring to your device inside profiles-data & profiles-stat.
    Then run ./nfsen reconfig
    ./nfsen start

    (./install.pl etc/nfsen.conf should be run only when installing nfsen 1st time)

Leave a Reply