Using consul sd configs - Unable to scrape while configuring same source with two independent/multiple ports

[Shaam Dinesh]

Problem statement:

I am trying to achieve below things using consul_sd_configs

There was a consul where the instance Neo4j been registered as a service which exposes two metrics end points runs on different ports

1. http://:2004 - neo4j specific metrics
2. http://:9001 - node-exporter metrics

Here I couldn't scrape metrics running on both/multiple ports with same source IP address and tried with 'N' number of combinations but none of the configs have helped.

What did you expect to see?

I would like to discover scrapping across multiple ports on the same source IP by which i can able to get metrics for service specific (In my case Neo4j specific metrics on 2004) and also the node-exporter metrics running on 9100

What did you see instead? Under which circumstances?

This is a very common use-case where every service discovery helps scrapping metrics only on single port but while it comes to node-exporter it runs on different port for obvious reasons also needs a mechanism/configs to scrape

Environment

Ubuntu Linux VM

• System information:

  insert output of uname -srm here

• Prometheus version:

  v2.13.0

• Alertmanager version:

  iNA

• Prometheus configuration file:

consul_sd_configs:
    - server: 'abc.com:8500'
      services: ['neo4j']
  relabel_configs:
    - source_labels: [__meta_consul_service_address]
      regex: '(.*)'
      replacement: '${1}:2004'
      target_label: '__address__'
      action: 'replace'
    - source_labels: [__meta_consul_tagged_address_wan]
      regex: '(.*)'
      replacement: '${1}:9100'
      target_label: 'wan'
      action: 'replace'

[Brian Candler]

On Tuesday, 17 March 2020 08:45:25 UTC, Shaam Dinesh wrote:

I would like to discover scrapping across multiple ports on the same source IP by which i can able to get metrics for service specific (In my case Neo4j specific metrics on 2004) and also the node-exporter metrics running on 9100

You need two separate scrape jobs, with slightly different relabel configs to set __address__ to ${1}:9100 and ${1}:2004.

These two scrape jobs will have include identical settings under 'consul_sd_configs'.  Indeed, I understand that this particular case is optimised, so the SD mechanism is not run twice.