Blaxkbox Panic Error.

[yagyans...@gmail.com]

Hi. I am running Blackbox Exporter v 0.18.0 and quite a few time I am observing error like below.

Nov  7 05:03:11 dh4-k1-infra-prometheus-n2 blackbox_exporter: 2020/11/07 05:03:11 http: panic serving 172.20.10.98:44106: runtime error: invalid memory address or nil pointer dereference

This is happening quite frequently on intervals. As of now, I haven't seen any kind of major impact due to this. But wanted to understand why is this happening and is there something that can be done to mitigate this?

Thanks in advance!

[Brian Candler]

I've not seen this.

What server platform are you running under? (e.g. is it linux-amd64?)  Are you using the release binaries, or did you build it yourself from source? (show output of blackbox_exporter --version)

Is it only http modules that give this error? Can you show the blackbox module config? Are they using TLS?

[yagyans...@gmail.com]

Hi Brain,

I am running under linux-amd64 and using pre-built binary.

[myuser@infra-prometheus ~]# /usr/local/bin/blackbox_exporter --version
blackbox_exporter, version 0.18.0 (branch: HEAD, revision: 60c86e6ce5a1111f7958b06ae7a08222bb6ec839)
  build user:       root@53d72328d93f
  build date:       20201012-09:46:31
  go version:       go1.15.2

So far I have seen this error with http modules only.

Below is the configuration for 2 of my most used modules.

  http_healthcheck:
    prober: http
    timeout: 10s
    http:
      valid_http_versions: ["HTTP/1.1", "HTTP/2.0", "HTTP/1.0"]
      method: GET
      no_follow_redirects: false
      fail_if_ssl: false
      fail_if_not_ssl: false
      tls_config:
        insecure_skip_verify: true
      preferred_ip_protocol: "ip4"

  http_healthcheck_es:
    prober: http
    timeout: 10s
    http:
      valid_http_versions: ["HTTP/1.1", "HTTP/2.0", "HTTP/1.0"]
      fail_if_body_not_matches_regexp: ['green']
      method: GET
      no_follow_redirects: false
      fail_if_ssl: false
      fail_if_not_ssl: false
      tls_config:
        insecure_skip_verify: true
      preferred_ip_protocol: "ip4"

[Brian Brazil]

If you've any Prometheus binary panicking, please file a bug in the relevant repo including the full stack trace.

Brian

 

Thanks in advance!

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/dd15f490-a645-4444-b736-a352a235bf1an%40googlegroups.com.

--

Brian Brazil

www.robustperception.io

[yagyans...@gmail.com]

Okay, thanks a lot, Brian.

Can you please have a look at https://groups.google.com/u/1/g/prometheus-users/c/Cb7lUaqWnbc too?