Multiple blackbox exporter targets per service discovery item

[jkroepke]

Hi,

I currently trying to implement a tags based discovery to probes ports based on Virtual Machine Tags.

Example: I have a tag like prometheus-tcp-probes with values "80,443"

I would like to pass 2 targets to prometheus blackbox exporter. I have a relabel_config like this:

      relabel_configs:
      - source_labels: [__meta_azure_machine_tag_prometheus_probe_tcp]
        regex: ^$
        action: drop
      - source_labels: [__meta_azure_machine_private_ip, __meta_azure_machine_tag_prometheus_probe_tcp]
        action: replace
        regex: ([^:]+)(?::\d+)?;(\d+).*
        replacement: $1:$2
        target_label: __param_target
      - source_labels: [__param_target]
        target_label: instance
      - target_label: __address__
        replacement: "{{ .Release.Name }}-blackbox-exporter.{{ .Release.Namespace }}.svc.cluster.local:9115"
      - action: labeldrop
        regex: __meta_azure_machine_tag_(.+)
      - action: labelmap
        regex: __meta_(.+)

Is it possible to split the values from __meta_azure_machine_tag_prometheus_probe_tcp which contains a comma separated port list, prepend the ip and pass them to target label as array?

[Brian Candler]

No, because:

1. blackbox_exporter only supports a single target per scrape

2. In prometheus relabelling, a single target can only result in a single scrape (or no scrapes, if you drop the target).  You cannot expand a single target to multiple targets.

If the set of ports is fixed, then you can have separate prometheus scrape jobs for each port:

- one for port 80

- one for port 443

- etc as required

and in those scrape jobs, you can use 'drop' actions to drop the target if the given port is not contained with the port list.

Otherwise, you need to move all your target collection logic outside of prometheus, and get prometheus to read it (e.g. using file_sd_configs or http_sd_configs)