Exporter to expose iftop styled metrics

[nesa...@gmail.com]

We have a Linux server which acts as an edge server,

Is it possible to export metrics like top IP addresses (not all of them) and how much traffic they are sending to the server as well as total total amount of traffic passing by the network interface?

[Brian Candler]

node_exporter gives you the total amount of traffic passed in and out by each network interface.

There's no out-of-the-box "top IP address" exporter that I know of, and it's probably not a good idea anyway as it could generate extremely high cardinality of timeseries. Even if it's only the top 10 peer addresses, that could change on every scrape.

However, I did once make an experimental netflow exporter using pmacct.  I've just posted the bits here:

nfacctd.conf  (update this to include your own private and public IP address ranges)

pmacct.py

I ran pmacct.py under exporter_exporter:

modules:

  pmacct:

    method: exec

    timeout: 5s

    exec:

      command: /usr/local/bin/pmacct.py

although you could just run it from cron, and write out a file for node_exporter textfile collector.

This takes netflow data from your border router (say), and aggregates it by local IP address, so you can see which of your own devices is sending or receiving the most traffic.  As the number of local IP addresses in your network is bounded, the cardinality is bounded.  Hence with this configuration it will only tell you which devices are the top-talkers on your network, not what they're actually talking to, but you can draw graphs of the traffic sent and received per device.

However, if you have node_exporter running on every server, then you'll already have information about how much traffic is being sent or received by each server.  The intention of this pmacct configuration was for monitoring traffic generated by *client* devices like laptops, where there's no exporter running.

If you wanted to hack this, you could use pmacctd instead of nfacctd to monitor the traffic going through an interface on a single server; and you could change the aggregation to show remote addresses (although as I said before, I think it's a bad idea).

IMO, logging systems provide a better long-term solution.  Packetbeat, logstash (elastiflow), telegraf/influxdb, and others all have ways to analyse flows either going through an interface or received as Netflow packets.