how to manage Blackbox config file

[Devi Ojha]

Hello There, 

I have a blackbox config file which keeps on growing, i have almost 30 URLS and loads of ICMP checks plus DNS checks configured to be monitored.

i was thinking to modularize  the config file, like if i could have separate config files for each URLS and just import/include these files in blackbox config.

something to extent of how Prometheus config file handle rules files. (rule_files:)

I didn't see any mention about same in documentation, is it something doable? or how do you handle these config files ?

I understand, you can create templates to generate the config file but it will ultimately a big config file end, so i think include files options would be neat!

Thanks 

Devi

[Stuart Clark]

As with all jobs you can use any of the SD mechanisms, so it might be
useful to use the file_sd mechanism with multiple YAML or JSON files.

--
Stuart Clark

[Devi Ojha]

Thanks Stuart, 

but file_sd_configs works only on Prometheus config file not in balckbox.yml, or does it 

i tried in blackbox.yml and getting the below error :

level=error ts=2020-02-07T13:33:23.681Z caller=main.go:216 msg="Error loading config" err="error parsing config file: yaml: unmarshal errors:\n  line 3: field file_sd_configs not found in type config.plain"

in my current  blackbox.yml file i have my modules defined as below and have 30-40 of them.

#config for skynet

  http_2xx_skyapp:

    prober: http

    timeout: 30s

    http:

      method: GET

      valid_status_codes: [200, 302]

      fail_if_ssl: false

      fail_if_not_ssl: false

      tls_config:

        insecure_skip_verify: true

      fail_if_body_not_matches_regexp:

        - "The Sky"

      proxy_url: <<url>>

      basic_auth:

        username:<<creds>>

        password: <<creds>>

[Stuart Clark]

Sorry. You mentioned having lots of URLs to check so I assumed you were
talking about prometheus.yaml.

Are you needing to define lots of modules in the blackbox exporter
itself?

--
Stuart Clark

[Devi Ojha]

 yes , i have loads of modules defined in blackbox.yml.

 I tried to club as much as possible but since we are checking for "fail_if_body_not_matches_regexp:"  and its different for all applications plus basic auth also changes per URL.

So i was thinking of if i could create a config file per module/url , per icmp checks,dns checks and ultimately refer them in blackbox.yml 

url1-config.yml

url2-config.yml

--

--

--

urln-config.yml

icmp-set1.yml

icmp-set2.yml

-

-

icmp-setn.yml

similarly for DNS configs 

and ultimately include them in backbox.yml

something like

modules:

   module_files:

     - *.yml

Thanks,

Devi

[Brian Candler]

On Friday, 7 February 2020 14:57:09 UTC, Devi Ojha wrote:

and ultimately include them in backbox.yml

something like

modules:

   module_files:

     - *.yml

Unfortunately, blackbox_exporter has only --config.file, not --config.dirs.  You could cat together some pieces though, if they have the right indentation.

As for the prometheus config: you can add labels to groups of targets in file_sd, so you can specify the module to use for each target this way.  I also like to separate the instance label and the target IP address, like this:

==> /etc/prometheus/targets.d/whatever.yml <==

- labels:

    module: url1

  targets:

    - aaaa/1.2.3.4
    - bbbb/5.6.7.8
- labels:

    module: url2
  targets:

    - cccc/9.10.11.12
    - dddd/13.14.15.16

and in prometheus.yml:

  - job_name: blackbox

    file_sd_configs:

      - files:

        - /etc/prometheus/targets.d/*.yml

    metrics_path: /probe

    relabel_configs:

      # When __address__ consists of just a name or IP address,

      # copy it to both the "instance" label (visible to user)

      # and "__param_target" (where blackbox_exporter probes)

      - source_labels: [__address__]

        regex: '([^/]+)'

        target_label: instance

      - source_labels: [__address__]

        regex: '([^/]+)'

        target_label: __param_target

      # When __address__ is of the form "name/address", extract

      # name to "instance" label and address to "__param_target"

      - source_labels: [__address__]

        regex: '(.+)/(.+)'

        target_label: instance

        replacement: '${1}'

      - source_labels: [__address__]

        regex: '(.+)/(.+)'

        target_label: __param_target

        replacement: '${2}'

      # Copy "module" label to "__param_module" so that blackbox_exporter

      # receives it as part of the scrape URL

      - source_labels: [module]

        target_label: __param_module

      # Send the actual scrape to SNMP exporter

      - target_label: __address__

        replacement: 127.0.0.1:9115

[Devi Ojha]

Thanks a lot Brian for the response. Yeah this helps in Prometheus blackbox target configuration but my concern is the 

long list of modules in blackbox.yml config file. dont see a option for that though. 

 

[Brian Candler]

Nope.  Like I say, you can only give a single blackbox.yml, and there's no way to insert URL query parameters into it.

Best I can suggest is you can have a separate config file for each URL as you intended, and just cat them together.

(

echo "modules:"

cat blackbox.d/*.yml

) >blackbox.yml