multiple snmp_exporters and single prometheus server config

609 views
Skip to first unread message

Vishnu B

unread,
Aug 13, 2019, 1:37:42 PM8/13/19
to Prometheus Users
I have around 30 DC, and each DC is installed with an SNMP_exporter. i have a central Prometheus server and it has arounf 10 different jobs.

For each DC i am creating different Jobs. only the target and snmp_exporter ip getting changed,  the config file is growing bigger and bigger. is is the correct way of using this?
Please help

```
  - job_name: 'cisco_ipsla_del2'
    scrape_interval: 60s
    scrape_timeout: 60s
    static_configs:
     - targets: ['core1.del2','core2.del2']
    metrics_path: /snmp
    params:
      module: [ipsla]
    relabel_configs:
      - source_labels: [__address__]
        target_label: __param_target
      - source_labels: [__param_target]
        target_label: instance
      - source_labels: []
        target_label: __address__
        replacement: 10.0.24.50:9116


  - job_name: 'cisco_ipsla_sjc2'
    scrape_interval: 60s
    scrape_timeout: 60s
    static_configs:
     - targets: ['core1.sjc2','core2.sjc']
    metrics_path: /snmp
    params:
      module: [ipsla]
    relabel_configs:
      - source_labels: [__address__]
        target_label: __param_target
      - source_labels: [__param_target]
        target_label: instance
      - source_labels: []
        target_label: __address__
        replacement: 10.0.1.50:9116
```


Stuart Clark

unread,
Aug 13, 2019, 1:41:04 PM8/13/19
to Vishnu B, Prometheus Users
Ideally you would have a Prometheus server in each DC and then a central server for aggregated metrics via federation, or using something like Thanos
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Ben Kochie

unread,
Aug 14, 2019, 3:52:27 AM8/14/19
to Vishnu B, Prometheus Users
Typically people use a configuration management template engine to generate their prometheus.yml. Something like Ansible, Chef, etc. This allows  you to expand a large matrix of configurations.

Another option is to use a service discovery method to generate this data.

One other trick is to leverage relabel configs. This configuration example assumes you have a DNS entry for your exporters, like `snmp-exporter.sjc2`.

- job_name: 'cisco_ipsla'

  scrape_interval: 60s
  scrape_timeout: 60s
  static_configs:
   - targets:
     - 'core1.del2'
     - 'core2.del2'
     labels:
     - dc: del2
   - targets:
     - 'core1.sjc2'
     - 'core2.sjc2'
     labels:
     - dc: sjc2

  metrics_path: /snmp
  params:
    module: [ipsla]
  relabel_configs:
    - source_labels: [__address__]
      target_label: __param_target
    - source_labels: [__param_target]
      target_label: instance
    - source_labels: [dc]
      target_label: __address__
      replacement: 'snmp-exporter.$1:9116'


--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/612469fe-6d50-40f9-b460-300ec29e099a%40googlegroups.com.

Vishnu

unread,
Aug 14, 2019, 4:55:26 AM8/14/19
to Ben Kochie, Prometheus Users
Thanks a lot Ben..

the relabel config will better work for me.. 

Regards,
Vishnu
Reply all
Reply to author
Forward
0 new messages