prometheus scrape_config failback to http if https is not available

[mohan garden]

Hi All, 
I am trying to secure the node exporter end points using self signed TLS certificates, here are the configurations:
a) node exporter -  tls_server_config:
cert_file: node_exporter.crt
key_file: node_exporter.key

b)  Prometheus - scrape_configs: - job_name: 'prometheus' scheme: https tls_config: ca_file: prometheus.crt file_sd_configs: - files: - /data/inventory.yml
is it possible to configure the scrape_configs in a way where in if the https collection is not possible then prometheus falls back to http for same job/target?

[Brian Candler]

If your issue is simply that you are in the process of migrating node_exporter configs from http to https, I'd suggest you configure the required scheme for each target using labels in the service discovery:

- labels:

    __scheme__: http

  targets:

    - foo:9100

    - bar:9100

- labels:

    __scheme__: https

  targets:

    - baz:9100

    - quz:9100

Then move the targets from one section to the other as you migrate them. Or at worst, simply scrape all targets twice, and let one fail and the other succeed. When the migration is complete, remove the http targets.  (Obviously you'll want to tweak your alerting rules in this case)

If you really want to try https first and fall back to http for all targets (on the same port? on different ports?) then I don't think Prometheus can do this. But you could write your own proxy, which makes connection type A and then falls back to connection type B, and scrape the proxy. Pass the connection parameters (e.g. target address) as part of the URL.