Blackbox icmp probe relabel_configs

[Kolja Krückmann]

Hi there, hope you're doing well.

I'm fairly new to prom and having a bit of trouble with all the configs.

I installed the Blackbox exporter for testing on a Windows Client. What I just want to do is a simple ping to the blackbox node and see in my dashboard if the client is up or down.

my prometheus.yml:

[...]

  - job_name: 'blackbox-icmp'
    metrics_path: /probe
    params:
      module: [icmp]
    static_configs:
      - targets:
        - 1stclient_ip:9115
        - 2ndclient_ip:9115

    relabel_configs:
      - source_labels: [__address__]
        target_label: __param_target
      - source_labels: [__param_target]
        target_label: instance

When doing this i at least get more or less the correct url, like:

http://1stclient_ip:9115/probe?module=icmp&target= 1stclient_ip %3A9115

but the problem with that is the modulo at the end, which is causing the hole probe to give only 0's... so there has to be something wrong with my relabel_config. Completly without i get the error "400 server not reachable" and when clicking the node the html sais "target parameter is missing"

Can someone tell me what my fault is?

Kind regards - Kolja

[Brian Candler]

I think you're trying to do two things:

1. Get Prometheus to scrape blackbox_exporter on hosts 1stclient_ip and 2ndclient_ip
2. Get blackbox_exporter on those hosts to make an ICMP probe *to itself*

So you're right, you want a scrape address like

http://1stclient_ip:9115/probe?module=icmp&target=1stclient_ip

or you could even do

http://1stclient_ip:9115/probe?module=icmp&target=127.0.0.1

Let's say you want to do the first one of those. You'll need a bit more rewriting to achieve this. Here's one way (which also keeps the 9115 out of the instance label):

  - job_name: 'blackbox-selftest'

    metrics_path: /probe
    params:
      module: [icmp]
    static_configs:
      - targets:
        - 1stclient_ip

        - 2ndclient_ip

    relabel_configs:
      - source_labels: [__address__]
        target_label: instance

      - source_labels: [__address__]
        target_label: __param_target

      - source_labels: [__address__]
        replacement: '${1}:9115'
        target_label: __address__

Personally though, I don't think this check is needed, if you are planning to use those blackbox_exporter instances to probe *other* systems. At that point, the metric 'up' will tell you whether blackbox_exporter is working or not, and you can simply alert on that, if you want to know when blackbox_exporter goes down.

That is, when probing some other target:

up=1, probe_success=1  => blackbox_exporter is working, target is working
up=1, probe_success=0  => blackbox_exporter is working, target is down

up=0   => blackbox_exporter itself is down or unreachable

And if all you want to do is ping 1stclient_ip and 2ndclient_ip, and not do any probe tests from those hosts to other hosts, you could run a single instance of blackbox_exporter on the prometheus server instead.