Ignoring cert verification for a consul-sd target

[Sandip Bhattacharya]

I am trying to bypass tls cert verification for a target fetched by querying consul.

- job_name: "grafana"

consul_sd_configs:

- server: "consul.service.consul:8500"

services:

- grafana

scheme: https

tls_config:

insecure_skip_verify: true

relabel_configs:

- source_labels: [__meta_consul_node]

target_label: hostname

- target_label: __scheme__

replacement: https

The 'insecure_skip_verify' part only seems to be used for accessing consul. But since grafana is running using a self-signed cert as well, I was hoping that its certificate would also not be verified.

But I keep getting this error suggesting that the final target is still getting its cert verified.

May 02 00:00:43 isengard prometheus[53817]: ts=2023-05-02T04:00:43.400Z caller=scrape.go:1353 level=debug component="scrape manager" scrape_pool=grafana target=https://192.168.1.31:3001/metrics msg="Scrape failed" err="Get \"https://192.168.1.xx:3001/metrics\": x509: certificate is valid for 127.0.0.1, not 192.168.1.xx"

Is there a way to bypass tls cert verification for the final target? Any suitable annotation that I can relabel?

I am using Prometheus 2.43.0.

Thanks,

  Sandip

[Brian Candler]

You can add a separate tls_config key at the top level of the scrape job,. This controls TLS used for https scrapes (as opposed to service discovery). See:

https://prometheus.io/docs/prometheus/latest/configuration/configuration/#scrape_config

https://prometheus.io/docs/prometheus/latest/configuration/configuration/#tls_config

Hence:

- job_name: "grafana"

consul_sd_configs:

- server: "consul.service.consul:8500"

services:

- grafana

scheme: https

tls_config:

insecure_skip_verify: true

relabel_configs:

- source_labels: [__meta_consul_node]

target_label: hostname

- target_label: __scheme__

replacement: https

tls_config:

insecure_skip_verify: true

[Sandip Bhattacharya]

Thank you! I totally missed that tls_config section at the job level!