Blackbox Exporter and ICMP

5,744 views
Skip to first unread message

dya...@gmail.com

unread,
Jan 9, 2017, 10:30:49 AM1/9/17
to Prometheus Users
Hi,
I'm having trouble configuring the blackbox exporter for a simple task, ping response times. I have replaced a Cacti installation monitoring 25 server nodes with minimal difficulty. I just need a base example to ping a single node, so I can figure the rest out. I don't know what I'm missing, as I have been able to use the blackbox exporter to monitor my https API endpoints, and my websites.

The server nodes are all in the 192.168.1.0/24 subnet.

Thanks,
David

Julius Volz

unread,
Jan 9, 2017, 1:39:03 PM1/9/17
to dya...@gmail.com, Prometheus Users
In the blackbox_exporter.yml you'll just need:

-------------------------
modules:
  icmp:
    prober: icmp
    timeout: 5s
-------------------------

Then in your prometheus.yml, something like this in your scrape config:

-------------------------
  - job_name: 'blackbox'
    metrics_path: /probe
    params:
      module: [icmp]
    static_configs:
      - targets:
        - google.com
        - twitter.com
    relabel_configs:
      - source_labels: [__address__]
        target_label: __param_target
      - source_labels: [__address__]
        target_label: instance
      - source_labels: []
        target_label: __address__
        replacement: 127.0.0.1:9115  # Blackbox exporter.
-------------------------

That will give you a metric "probe_duration_seconds", which tells you the latency of the last probe (from the view of the blackbox exporter).

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscribe@googlegroups.com.
To post to this group, send email to prometheus-users@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/8da2917a-6e31-4bba-80c3-503737faab48%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

dya...@gmail.com

unread,
Jan 9, 2017, 1:55:15 PM1/9/17
to Prometheus Users, dya...@gmail.com
That's almost what I have now. After following the notes found on github (https://github.com/prometheus/blackbox_exporter/issues/70) I was able to get the Blackbox_exporter working.

One stupid question, when I query probe_ip_protocol for the ping probes, I comes back with protocol 4 (TCP), not protocol 2 (ICMP).

This is my blackbox.yml:
modules:
  ping:
    prober: icmp
    timeout: 5s
    icmp:
      protocol: "icmp"
      preferred_ip_protocol: "ip4"

And the relevant snippit from prometheus.yml
scrape_configs:
  - job_name: 'ping'
    metrics_path: /probe
    params:
      module: [ping]
    static_configs:
      - targets: ['gizmo','donald','daisy']
    relabel_configs:
      # Set the target's address as the target-GET-parameter for the blackbox exporter
      - source_labels: [__address__]
        regex: (.*)
        target_label: __param_target
        replacement: ${1}
      # Set the target's address as the instance name
      - source_labels: [__address__]
        regex: (.*)
        target_label: instance
        replacement: ${1}
      # Set the scrape address to the blackbox exporter
      - source_labels: []
        regex: .*
        target_label: __address__
        replacement: gizmo:9115

Is the returned protocol '4' because of the transfer from the blackbox exporter back to prometheus or does it have to do with my configuration?

Thanks!
David
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To post to this group, send email to promethe...@googlegroups.com.

Julius Volz

unread,
Jan 9, 2017, 2:04:56 PM1/9/17
to dya...@gmail.com, Prometheus Users
As the option 'preferred_ip_protocol: "ip4"' in your config files hints at, this is not the protocol of the payload of the IP packet, but the version of the IP protocol itself (IPv4 / IPv6).

To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscribe@googlegroups.com.
To post to this group, send email to prometheus-users@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/854dbfa9-792d-405c-b86a-22fc1647d83c%40googlegroups.com.

manfred...@googlemail.com

unread,
Nov 25, 2018, 4:54:21 AM11/25/18
to Prometheus Users
Hello

I am wondering. In my case I get this log in the Blackbox-Exporter:

Logs for the probe:
ts=2018-11-25T09:47:31.825547649Z caller=main.go:116 module=icmp target=twitter.com level=info msg="Beginning probe" probe=icmp timeout_seconds=4.5
ts=2018-11-25T09:47:31.825703591Z caller=utils.go:42 module=icmp target=twitter.com level=info msg="Resolving target address" preferred_ip_protocol=ip4
ts=2018-11-25T09:47:31.82689422Z caller=utils.go:65 module=icmp target=twitter.com level=info msg="Resolved target address" ip=104.244.42.129
ts=2018-11-25T09:47:31.826964068Z caller=icmp.go:61 module=icmp target=twitter.com level=info msg="Creating socket"
ts=2018-11-25T09:47:31.827022821Z caller=icmp.go:78 module=icmp target=twitter.com level=error msg="Error listening to socket" err="listen ip4:icmp 0.0.0.0: socket: operation not permitted"
ts=2018-11-25T09:47:31.827067894Z caller=main.go:129 module=icmp target=twitter.com level=error msg="Probe failed" duration_seconds=0.001429711

On the terminal I get the Ping:
# ping twitter.com
PING twitter.com (104.244.42.65) 56(84) bytes of data.
64 bytes from twitter.com (104.244.42.65): icmp_seq=1 ttl=56 time=33.6 ms
64 bytes from twitter.com (104.244.42.65): icmp_seq=2 ttl=56 time=43.6 ms


What is going wrong?

Thanks in advance
Manfred
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To post to this group, send email to promethe...@googlegroups.com.

Ben Kochie

unread,
Nov 25, 2018, 4:58:27 AM11/25/18
to manfred...@googlemail.com, Prometheus Users
Normal ICMP requires extra privileges to be used.  Your ping binary probably has these.

sudo setcap cap_net_raw=+ep blackbox_exporter


Manfred Gnaedig

unread,
Nov 25, 2018, 5:09:17 AM11/25/18
to sup...@gmail.com, promethe...@googlegroups.com
Thanks for the tip!
Now it works like expected ;-)
Reply all
Reply to author
Forward
0 new messages