snmp-exporter returning non-printable characters

[BenB]

Hi everyone,

I'm trying to setup the snmp-exporter to poll a Checkpoint Maestro appliance for system stats.  I'm starting small, hitting a single OID, asgCoreIndex OID which returns a string index with the name of each cpu core.  I can do a walk on this OID with snmpwalk and it returns the following:

snmpwalk -On -v3 -u <redacted> -l authPriv -a SHA -A <redacted> -x AES -X <redacted> 1.2.3.4 asgCoreIndex
.1.3.6.1.4.1.2620.1.48.29.1.1.1.0 = STRING: Blade ID
.1.3.6.1.4.1.2620.1.48.29.1.1.2.0 = STRING: cpu0
.1.3.6.1.4.1.2620.1.48.29.1.1.3.0 = STRING: cpu1
.1.3.6.1.4.1.2620.1.48.29.1.1.4.0 = STRING: cpu2
.1.3.6.1.4.1.2620.1.48.29.1.1.5.0 = STRING: cpu3
.1.3.6.1.4.1.2620.1.48.29.1.1.6.0 = STRING: cpu4
.1.3.6.1.4.1.2620.1.48.29.1.1.7.0 = STRING: cpu5
.1.3.6.1.4.1.2620.1.48.29.1.1.8.0 = STRING: cpu6
.1.3.6.1.4.1.2620.1.48.29.1.1.9.0 = STRING: cpu7
.1.3.6.1.4.1.2620.1.48.29.1.1.10.0 = STRING: cpu8
.1.3.6.1.4.1.2620.1.48.29.1.1.11.0 = STRING: cpu9
.1.3.6.1.4.1.2620.1.48.29.1.1.12.0 = STRING: cpu10
.1.3.6.1.4.1.2620.1.48.29.1.1.13.0 = STRING: cpu11
.1.3.6.1.4.1.2620.1.48.29.1.1.14.0 = STRING: cpu12
.1.3.6.1.4.1.2620.1.48.29.1.1.15.0 = STRING: cpu13
.1.3.6.1.4.1.2620.1.48.29.1.1.16.0 = STRING: cpu14
.1.3.6.1.4.1.2620.1.48.29.1.1.17.0 = STRING: cpu15
.1.3.6.1.4.1.2620.1.48.29.1.1.18.0 = STRING: cpu16
.1.3.6.1.4.1.2620.1.48.29.1.1.19.0 = STRING: cpu17
.1.3.6.1.4.1.2620.1.48.29.1.1.20.0 = STRING: cpu18
.1.3.6.1.4.1.2620.1.48.29.1.1.21.0 = STRING: cpu19

Hitting the same OID with snmp-exporter shows what I assume are non-printable characters in the label for asgCoreIndex:

# HELP asgCoreIndex  - 1.3.6.1.4.1.2620.1.48.29.1.1
# TYPE asgCoreIndex gauge
asgCoreIndex{asgCoreIndex="�"} 1
asgCoreIndex{asgCoreIndex="��"} 1
asgCoreIndex{asgCoreIndex="���"} 1
asgCoreIndex{asgCoreIndex="����"} 1
asgCoreIndex{asgCoreIndex="�����"} 1
asgCoreIndex{asgCoreIndex="������"} 1
asgCoreIndex{asgCoreIndex="�������"} 1
asgCoreIndex{asgCoreIndex="��������"} 1
asgCoreIndex{asgCoreIndex="���������"} 1
asgCoreIndex{asgCoreIndex="����������"} 1
asgCoreIndex{asgCoreIndex="�����������"} 1
asgCoreIndex{asgCoreIndex="������������"} 1
asgCoreIndex{asgCoreIndex="�������������"} 1
asgCoreIndex{asgCoreIndex="��������������"} 1
asgCoreIndex{asgCoreIndex="���������������"} 1
asgCoreIndex{asgCoreIndex="����������������"} 1
asgCoreIndex{asgCoreIndex="�����������������"} 1
asgCoreIndex{asgCoreIndex="������������������"} 1
asgCoreIndex{asgCoreIndex="�������������������"} 1
asgCoreIndex{asgCoreIndex="��������������������"} 1
asgCoreIndex{asgCoreIndex="���������������������"} 1
# HELP snmp_scrape_duration_seconds Total SNMP time scrape took (walk and processing).
# TYPE snmp_scrape_duration_seconds gauge
snmp_scrape_duration_seconds 0.097737986
# HELP snmp_scrape_pdus_returned PDUs returned from walk.
# TYPE snmp_scrape_pdus_returned gauge
snmp_scrape_pdus_returned 21
# HELP snmp_scrape_walk_duration_seconds Time SNMP walk/bulkwalk took.
# TYPE snmp_scrape_walk_duration_seconds gauge
snmp_scrape_walk_duration_seconds 0.097562665

my generator.yml file :

modules:
# Maestro CPU
  maestro_cpu:
    auth:
      username: <redacted>
      security_level: authPriv
      password : <redacted>
      auth_protocol: SHA
      priv_protocol: AES
      priv_password: <redacted>
    version: 3
    walk:
      - asgCoreIndex

produces the following snmp.yml file:

# WARNING: This file was auto-generated using snmp_exporter generator, manual changes will be lost.
maestro_cpu:
  walk:
  - 1.3.6.1.4.1.2620.1.48.29.1.1
  metrics:
  - name: asgCoreIndex
    oid: 1.3.6.1.4.1.2620.1.48.29.1.1
    type: DisplayString
    help: ' - 1.3.6.1.4.1.2620.1.48.29.1.1'
    indexes:
    - labelname: asgCoreIndex
      type: DisplayString
  version: 3
  auth:
    security_level: authPriv
    username: <redacted>
    password: <redacted>
    auth_protocol: SHA
    priv_protocol: AES
    priv_password: <redacted>

the debug logs don't appear to indicate any issues:

level=debug ts=2020-11-10T00:19:37.688Z caller=main.go:99 module=maestro_cpu target=1.2.3.4 msg="Starting scrape"
level=debug ts=2020-11-10T00:19:37.688Z caller=collector.go:164 module=maestro_cpu target=1.2.3.4 msg="Walking subtree" oid=1.3.6.1.4.1.2620.1.48.29.1.1
level=debug ts=2020-11-10T00:19:37.783Z caller=collector.go:177 module=maestro_cpu target=1.2.3.4 msg="Walk of subtree completed" oid=1.3.6.1.4.1.2620.1.48.29.1.1 duration_seconds=94.396128ms
level=debug ts=2020-11-10T00:19:37.788Z caller=main.go:110 module=maestro_cpu target=1.2.3.4 msg="Finished scrape" duration_seconds=0.099797031

I'm a newbie when it comes to prometheus - any ideas why I'm not getting the expected response?

Thanks,

Ben.

[Brian Brazil]

Can you share the MIB? Based on the output I'd guess the index is wrong.

--

Brian Brazil

www.robustperception.io

[BenB]

Hi Brian,

thanks for responding.  I've tried attached the mib file to this post but I get an error when submitting.  MIB file can be downloaded from https://supportcenter.checkpoint.com/supportcenter/portal?action=portlets.DCFileAction&eventSubmit_doGetdcdetails=&fileid=83804   OID name is asgCoreIndex.

In order to get some additional debug information, I added an override to my configuration to create a new metric by extracting all alphanumeric and space characters from the original value - as expected this generated parsing errors as the value isn't a float, but I was able to see the value of the original metric in this output and it appears to be correct (unless I'm missing something). My understanding is that under my original configuration, the value of the metric would be added as a label if it isn't a float and the metric value then set to 1 - please correct my if this is wrong.

Debug output is below.

level=info ts=2020-11-10T06:21:24.348Z caller=main.go:149 msg="Starting snmp_exporter" version="(version=0.19.0, branch=HEAD, revision=9dcbc02f59648b21fcf632de1b62a30df70f4649)"

level=info ts=2020-11-10T06:21:24.349Z caller=main.go:243 msg="Listening on address" address=:9116

level=debug ts=2020-11-10T06:21:26.918Z caller=main.go:99 module=maestro_cpu target=10.112.3.99 msg="Starting scrape"

level=debug ts=2020-11-10T06:21:26.918Z caller=collector.go:164 module=maestro_cpu target=10.112.3.99 msg="Walking subtree" oid=1.3.6.1.4.1.2620.1.48.29.1.1

level=debug ts=2020-11-10T06:21:27.028Z caller=collector.go:177 module=maestro_cpu target=10.112.3.99 msg="Walk of subtree completed" oid=1.3.6.1.4.1.2620.1.48.29.1.1 duration_seconds=10

9.883144ms

level=debug ts=2020-11-10T06:21:27.028Z caller=collector.go:428 module=maestro_cpu target=10.112.3.99 msg="Error parsing float64 from value" metric=asgCoreIndex value=cpu9 regex="^(?:^(?

:([A-Za-z0-9 ]+))$)$" extracted_value=cpu9

level=debug ts=2020-11-10T06:21:27.028Z caller=collector.go:428 module=maestro_cpu target=10.112.3.99 msg="Error parsing float64 from value" metric=asgCoreIndex value=cpu15 regex="^(?:^(

?:([A-Za-z0-9 ]+))$)$" extracted_value=cpu15

level=debug ts=2020-11-10T06:21:27.028Z caller=collector.go:428 module=maestro_cpu target=10.112.3.99 msg="Error parsing float64 from value" metric=asgCoreIndex value=cpu17 regex="^(?:^(

?:([A-Za-z0-9 ]+))$)$" extracted_value=cpu17

level=debug ts=2020-11-10T06:21:27.028Z caller=collector.go:428 module=maestro_cpu target=10.112.3.99 msg="Error parsing float64 from value" metric=asgCoreIndex value=cpu0 regex="^(?:^(?

:([A-Za-z0-9 ]+))$)$" extracted_value=cpu0

level=debug ts=2020-11-10T06:21:27.028Z caller=collector.go:428 module=maestro_cpu target=10.112.3.99 msg="Error parsing float64 from value" metric=asgCoreIndex value=cpu3 regex="^(?:^(?

:([A-Za-z0-9 ]+))$)$" extracted_value=cpu3

level=debug ts=2020-11-10T06:21:27.028Z caller=collector.go:428 module=maestro_cpu target=10.112.3.99 msg="Error parsing float64 from value" metric=asgCoreIndex value=cpu6 regex="^(?:^(?

:([A-Za-z0-9 ]+))$)$" extracted_value=cpu6

level=debug ts=2020-11-10T06:21:27.029Z caller=collector.go:428 module=maestro_cpu target=10.112.3.99 msg="Error parsing float64 from value" metric=asgCoreIndex value=cpu7 regex="^(?:^(?

:([A-Za-z0-9 ]+))$)$" extracted_value=cpu7

level=debug ts=2020-11-10T06:21:27.029Z caller=collector.go:428 module=maestro_cpu target=10.112.3.99 msg="Error parsing float64 from value" metric=asgCoreIndex value=cpu11 regex="^(?:^(

?:([A-Za-z0-9 ]+))$)$" extracted_value=cpu11

level=debug ts=2020-11-10T06:21:27.029Z caller=collector.go:428 module=maestro_cpu target=10.112.3.99 msg="Error parsing float64 from value" metric=asgCoreIndex value=cpu12 regex="^(?:^(

?:([A-Za-z0-9 ]+))$)$" extracted_value=cpu12

level=debug ts=2020-11-10T06:21:27.029Z caller=collector.go:428 module=maestro_cpu target=10.112.3.99 msg="Error parsing float64 from value" metric=asgCoreIndex value=cpu14 regex="^(?:^(

?:([A-Za-z0-9 ]+))$)$" extracted_value=cpu14

level=debug ts=2020-11-10T06:21:27.029Z caller=collector.go:428 module=maestro_cpu target=10.112.3.99 msg="Error parsing float64 from value" metric=asgCoreIndex value=cpu18 regex="^(?:^(

?:([A-Za-z0-9 ]+))$)$" extracted_value=cpu18

level=debug ts=2020-11-10T06:21:27.029Z caller=collector.go:428 module=maestro_cpu target=10.112.3.99 msg="Error parsing float64 from value" metric=asgCoreIndex value="Blade ID" regex="^

(?:^(?:([A-Za-z0-9 ]+))$)$" extracted_value="Blade ID"

level=debug ts=2020-11-10T06:21:27.029Z caller=collector.go:428 module=maestro_cpu target=10.112.3.99 msg="Error parsing float64 from value" metric=asgCoreIndex value=cpu2 regex="^(?:^(?

:([A-Za-z0-9 ]+))$)$" extracted_value=cpu2

level=debug ts=2020-11-10T06:21:27.029Z caller=collector.go:428 module=maestro_cpu target=10.112.3.99 msg="Error parsing float64 from value" metric=asgCoreIndex value=cpu10 regex="^(?:^(

?:([A-Za-z0-9 ]+))$)$" extracted_value=cpu10

level=debug ts=2020-11-10T06:21:27.029Z caller=collector.go:428 module=maestro_cpu target=10.112.3.99 msg="Error parsing float64 from value" metric=asgCoreIndex value=cpu1 regex="^(?:^(?

:([A-Za-z0-9 ]+))$)$" extracted_value=cpu1

level=debug ts=2020-11-10T06:21:27.029Z caller=collector.go:428 module=maestro_cpu target=10.112.3.99 msg="Error parsing float64 from value" metric=asgCoreIndex value=cpu4 regex="^(?:^(?

:([A-Za-z0-9 ]+))$)$" extracted_value=cpu4

level=debug ts=2020-11-10T06:21:27.029Z caller=collector.go:428 module=maestro_cpu target=10.112.3.99 msg="Error parsing float64 from value" metric=asgCoreIndex value=cpu5 regex="^(?:^(?

:([A-Za-z0-9 ]+))$)$" extracted_value=cpu5

level=debug ts=2020-11-10T06:21:27.029Z caller=collector.go:428 module=maestro_cpu target=10.112.3.99 msg="Error parsing float64 from value" metric=asgCoreIndex value=cpu8 regex="^(?:^(?

:([A-Za-z0-9 ]+))$)$" extracted_value=cpu8

level=debug ts=2020-11-10T06:21:27.029Z caller=collector.go:428 module=maestro_cpu target=10.112.3.99 msg="Error parsing float64 from value" metric=asgCoreIndex value=cpu13 regex="^(?:^(

?:([A-Za-z0-9 ]+))$)$" extracted_value=cpu13

level=debug ts=2020-11-10T06:21:27.029Z caller=collector.go:428 module=maestro_cpu target=10.112.3.99 msg="Error parsing float64 from value" metric=asgCoreIndex value=cpu16 regex="^(?:^(

?:([A-Za-z0-9 ]+))$)$" extracted_value=cpu16

level=debug ts=2020-11-10T06:21:27.029Z caller=collector.go:428 module=maestro_cpu target=10.112.3.99 msg="Error parsing float64 from value" metric=asgCoreIndex value=cpu19 regex="^(?:^(

?:([A-Za-z0-9 ]+))$)$" extracted_value=cpu19

level=debug ts=2020-11-10T06:21:27.031Z caller=main.go:110 module=maestro_cpu target=10.112.3.99 msg="Finished scrape" duration_seconds=0.112523537

[Brian Brazil]

On Tue, 10 Nov 2020 at 12:27, BenB <benb...@gmail.com> wrote:

Hi Brian,

thanks for responding.  I've tried attached the mib file to this post but I get an error when submitting.  MIB file can be downloaded from https://supportcenter.checkpoint.com/supportcenter/portal?action=portlets.DCFileAction&eventSubmit_doGetdcdetails=&fileid=83804   OID name is asgCoreIndex.

The MIB doesn't match the snmpwalk output, the MIB says the index is a string whereas the output looks like an integer. You'll need to take this up with your vendor, as this looks like a device bug.

Brian

 

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/d4f3516a-8261-4cb3-a60b-687f2ca40817n%40googlegroups.com.

--

Brian Brazil

www.robustperception.io

[benb...@gmail.com]

Hi Brian,

When I manually run an snmpwalk I appear to be getting a string as the response:

snmpwalk -On -v3 -u <redacted> -l authPriv -a SHA -A <redacted> -x AES -X <redacted> 1.2.3.4 asgCoreIndex
.1.3.6.1.4.1.2620.1.48.29.1.1.1.0 = STRING: Blade ID
.1.3.6.1.4.1.2620.1.48.29.1.1.2.0 = STRING: cpu0
.1.3.6.1.4.1.2620.1.48.29.1.1.3.0 = STRING: cpu1
.1.3.6.1.4.1.2620.1.48.29.1.1.4.0 = STRING: cpu2
.1.3.6.1.4.1.2620.1.48.29.1.1.5.0 = STRING: cpu3
.1.3.6.1.4.1.2620.1.48.29.1.1.6.0 = STRING: cpu4
.1.3.6.1.4.1.2620.1.48.29.1.1.7.0 = STRING: cpu5
.1.3.6.1.4.1.2620.1.48.29.1.1.8.0 = STRING: cpu6
.1.3.6.1.4.1.2620.1.48.29.1.1.9.0 = STRING: cpu7
.1.3.6.1.4.1.2620.1.48.29.1.1.10.0 = STRING: cpu8
.1.3.6.1.4.1.2620.1.48.29.1.1.11.0 = STRING: cpu9
.1.3.6.1.4.1.2620.1.48.29.1.1.12.0 = STRING: cpu10
.1.3.6.1.4.1.2620.1.48.29.1.1.13.0 = STRING: cpu11
.1.3.6.1.4.1.2620.1.48.29.1.1.14.0 = STRING: cpu12
.1.3.6.1.4.1.2620.1.48.29.1.1.15.0 = STRING: cpu13
.1.3.6.1.4.1.2620.1.48.29.1.1.16.0 = STRING: cpu14
.1.3.6.1.4.1.2620.1.48.29.1.1.17.0 = STRING: cpu15
.1.3.6.1.4.1.2620.1.48.29.1.1.18.0 = STRING: cpu16
.1.3.6.1.4.1.2620.1.48.29.1.1.19.0 = STRING: cpu17
.1.3.6.1.4.1.2620.1.48.29.1.1.20.0 = STRING: cpu18
.1.3.6.1.4.1.2620.1.48.29.1.1.21.0 = STRING: cpu19

[Brian Brazil]

On Tue, 10 Nov 2020 at 12:58, benb...@gmail.com <benb...@gmail.com> wrote:

Hi Brian,

When I manually run an snmpwalk I appear to be getting a string as the response:

That's the value, the index however in the oid is a single integer followed by a 0 (which is also incorrect).

Brian

--

Brian Brazil

www.robustperception.io

[benb...@gmail.com]

I see, thanks for explaining.  I'll raise a support ticket with the vendor.

Kind regards,

Ben.