How to use blackbox for ssl cert monitoring?

[Dmitry]

Hello!

I have blackbox for checking service availability and everything is ok.

I've tried configuring blackbox exporter to monitor ssl certificates for https endpoints. But unfortunately not able to achieve proper results.

Blackbox exporter:

modules:

  http_2xx_example:

    prober: http

    timeout: 15s

    http:

      valid_http_versions: ["HTTP/1.1", "HTTP/2"]

      valid_status_codes: []  

      method: GET

      headers:

        Host: my.host.team

        Accept-Language: en-US

  http_2xx_whaleapp:

    prober: http

    timeout: 15s

    http:

      valid_http_versions: ["HTTP/1.1", "HTTP/2"]

      valid_status_codes: []

      method: GET

      headers:

      no_follow_redirects: false

      fail_if_ssl: false

      fail_if_not_ssl: false

      fail_if_body_not_matches_regexp:

        - "HTTP OK*"

      preferred_ip_protocol: "ip4" 

      tls_config:

        insecure_skip_verify: true

  http_post_2xx:

    prober: http

    timeout: 10s

    http:

      method: POST

Prometheus:

  - job_name: 'blackbox-ssl'

    metrics_path: /probe

    params:

      module: [http_2xx_whaleapp]

    static_configs:

      - targets:

        - "mysite1.com"
        - "mysite2.com"

    relabel_configs:

      - source_labels: [__address__]

        target_label: __param_target

      - source_labels: [__param_target]

        target_label: instance

      - target_label: __address__

        replacement: blackbox-exporter:9115

[Dmitry]

Prometheus returns HTTP status 400 Bad Request

[Brian Candler]

I believe that the http prober needs the targets to be complete URLs like

  - targets:

    - https://mysite1.com

Test it using curl:

curl -v 'blackbox-exporter:9115/probe?module=http_2xx_whaleapp&target=https:%2f%2fwww.google.com'

For checking certificates I use the "tcp" prober instead (since it works with non-https endpoints), in which case the target needs to include the port number, e.g. mysite1.com:443

Working example:

(blackbox)

modules:

  certificate:

    prober: tcp

    timeout: 5s

    tcp:

      tls: true

      tls_config: {}

(targets)

- labels:

    module: certificate

  targets:

    - www.google.com:443

I think you should be wary about using "insecure_skip_verify: true" if you are checking certificates, because it explicitly allows bad certificates to be accepted.  However you should still be able to see the cert expiry timestamps.

[Dmitry]

I have changed both configs with http and tcp prober, with

  - targets:

   - https://mysite1.com

and

- labels:

   module: certificate

 targets:

   - www.google.com:443

and still prometheus returned 400 bad request.

I go into blackbox-exporter logs, and what i see

Module	Target	Result	Debug
http_2xx	my.host.team	Success	Logs
http_2xx	google.com	Failure	Logs

it seems like blackbox use http-prober for both modules.

On Saturday, July 25, 2020 at 5:32:03 PM UTC+3, Dmitry wrote:

[Brian Candler]

When you say "prometheus returns 400 Bad Request", what exactly do you mean?  What request are you sending to prometheus that gets that response?

What matters is the response you get from blackbox exporter, which you can test using "curl" as per the example I showed before  Can you show the curl -v command line you are using and the response you get?