snmp proxy

40 views
Skip to first unread message

Michael Thomas

unread,
Nov 4, 2020, 6:04:00 PM11/4/20
to Prometheus Users
Our SNMP-enabled devices are on a secure, non-routed subnet in our datacenter.  To enable snmp monitoring of them, we use a snmpd proxy[1]. This allows us to grab snmp data from each device using the same IP address, but a different community name.  For example:

# snmpget -v2c -On -c pdu1 snmpd.proxy.host .1.3.6.1.2.1.1.5.0.1.3.6.1.2.1.1.5.0 = STRING: DCS_PDU_10_1
# snmpget -v2c -On -c pdu2 snmpd.proxy.host .1.3.6.1.2.1.1.5.0 .1.3.6.1.2.1.1.5.0 = STRING: DCS_PDU_11_1

Our snmp_exporter(s) run on the same host as prometheus, which is not the same host that the snmp proxy is running on.  I've generated a snmp_exporter snmp.yml config that specified a different module for each device:

servertech_sentry3: &servertech_sentry3
  walk:
  - 1.3.6.1.4.1.1718.3.2.2
  - 1.3.6.1.4.1.1718.3.2.3
[...]
pdu1:
  <<: *servertech_sentry3
  auth:
    community: pdu1

pdu2:
  <<: *servertech_sentry3
  auth:
    community: pdu2

This lets us use a single exporter for all identical devices by specifying a different module, eg


However, I'm uncertain now to use this in my prometheus.yml file so that they show up as different targets under the same job.  I wanted to do something like this:

  - job_name: 'pdu'
    static_configs:
      - targets:
        - 'prometheus.host:9116/snmp?target=snmpd.proxy.host&module=pdu1'
        - 'prometheus.host:9116/snmp?target=snmpd.proxy.host&module=pdu2'

...but prometheus doesn't like having the url path in the target; it only wants a hostname:port.  But in my case, I want to reuse the same hostname:port for multiple targets.

Is this something that is possible?  If not, the only other option I see is to run a separate snmp_exporter instance (listening on different ports) for each snmp device (we have ~50 such devices).  Ultimately I want to have a single job_name, but be able to select data for each individual proxied snmp device in grafana.

--Mike

Brian Brazil

unread,
Nov 4, 2020, 7:00:19 PM11/4/20
to Michael Thomas, Prometheus Users
You will need to use relabelling to set __param_module.

However a simpler idea might be to run the snmp_exporter on the machine the snmpd_proxy is currently running on. Generally you want the snmp exporter near the network devices, not Prometheus, given how chatty SNMP is.

Brian
 

Is this something that is possible?  If not, the only other option I see is to run a separate snmp_exporter instance (listening on different ports) for each snmp device (we have ~50 such devices).  Ultimately I want to have a single job_name, but be able to select data for each individual proxied snmp device in grafana.

--Mike

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/2038b634-8058-4eb1-836f-31acd4b95a49n%40googlegroups.com.


--

Michael Thomas

unread,
Nov 4, 2020, 9:39:46 PM11/4/20
to Prometheus Users
That did the trick!  I didn't understand that relabeling could be used to modify the url path.  Here's the final config that I ended up with:

  - job_name: 'pdu'
    static_configs:
    - targets: [ 'localhost:9117', ]
      labels:
        pduname: 'pdu1'
    - targets: [ 'localhost:9117', ]
      labels:
        pduname: 'pdu2'
    metrics_path: '/snmp'
    relabel_configs:
      - source_labels: [__address__]
        target_label: __param_target
        replacement: '10.13.5.16'
      - source_labels: ['pduname']
        target_label: __param_module

I hadn't tried running the snmp_exporter directly on the snmpd_proxy host because I couldn't get the generator to build on RHEL7.  But it looks like I can run the generator on a compatible host, then copy over the snmp.yml that it produces.  This should allow me to move the snmp_exporter to the machine running the snmpd proxy, at which point I should be able to use the relabel_config example from the documentation.

Thanks for the advice,

--Mike

Brian Brazil

unread,
Nov 5, 2020, 3:33:14 AM11/5/20
to Michael Thomas, Prometheus Users
On Thu, 5 Nov 2020 at 02:39, Michael Thomas <war...@gmail.com> wrote:
That did the trick!  I didn't understand that relabeling could be used to modify the url path.  Here's the final config that I ended up with:

  - job_name: 'pdu'
    static_configs:
    - targets: [ 'localhost:9117', ]
      labels:
        pduname: 'pdu1'
    - targets: [ 'localhost:9117', ]
      labels:
        pduname: 'pdu2'
    metrics_path: '/snmp'
    relabel_configs:
      - source_labels: [__address__]
        target_label: __param_target
        replacement: '10.13.5.16'
      - source_labels: ['pduname']
        target_label: __param_module

I hadn't tried running the snmp_exporter directly on the snmpd_proxy host because I couldn't get the generator to build on RHEL7.  But it looks like I can run the generator on a compatible host, then copy over the snmp.yml that it produces.  This should allow me to move the snmp_exporter to the machine running the snmpd proxy, at which point I should be able to use the relabel_config example from the documentation.

Yes, by design you can run the generator anywhere (but not a Mac, there looks to be some odd netsnmp bug there with fixed size string indexes).

Brian

 
Reply all
Reply to author
Forward
0 new messages