Exporter returning HTTP/1.1 500 Internal Server Error

[Renato Tasinafo]

Hello All, 

I wonder if someone one could help me with a weird issue. 

I'm  using the new module paloalto_fw
my generator.yml has: 

```  palo_alto:

    walk:

      - sysUpTime

      - interfaces

      - hrDevice

      - hrSystem

      - hrStorage

      - 1.3.6.1.4.1.25461.2.1.2.1 # panSys

      - 1.3.6.1.4.1.25461.2.1.2.3 # panSession

      - 1.3.6.1.4.1.25461.2.1.2.5 # panGlobalProtect

    timeout: 30s

    auth:

      community: xxxxx```

The config file installed in the exporter looks good OK
```palo_alto:             

  walk:                          

  - 1.3.6.1.2.1.2                

  - 1.3.6.1.2.1.25.1     

  - 1.3.6.1.2.1.25.2    

  - 1.3.6.1.2.1.25.3   

  - 1.3.6.1.4.1.25461.2.1.2.1                                  

  - 1.3.6.1.4.1.25461.2.1.2.3

  - 1.3.6.1.4.1.25461.2.1.2.5

  get:                       

  - 1.3.6.1.2.1.1.3.0                                                                                                    

  metrics:     
........
 ```


The problem is that when I invoke this module I have the following return: 

~ # wget "http://localhost:9116/snmp?module=palo_alto&target=10.91.2.8"

Connecting to localhost:9116 (127.0.0.1:9116)

wget: server returned error: HTTP/1.1 500 Internal Server Error

Other modules are working fine. 
I have put the pod in debug mode and  it show no errors:

This what happens when I manually trigger the wget:

level=debug ts=2020-10-29T13:51:01.543Z caller=main.go:99 module=palo_alto target=10.91.2.8 msg="Starting scrape"

level=debug ts=2020-10-29T13:51:01.543Z caller=collector.go:131 module=palo_alto target=10.91.2.8 msg="Getting OIDs" oids=1

level=debug ts=2020-10-29T13:51:01.556Z caller=collector.go:140 module=palo_alto target=10.91.2.8 msg="Get of OIDs completed" oids=1 duration_seconds=12.076006ms

level=debug ts=2020-10-29T13:51:01.556Z caller=collector.go:164 module=palo_alto target=10.91.2.8 msg="Walking subtree" oid=1.3.6.1.2.1.2

level=debug ts=2020-10-29T13:51:02.597Z caller=collector.go:177 module=palo_alto target=10.91.2.8 msg="Walk of subtree completed" oid=1.3.6.1.2.1.2 duration_seconds=1.040800849s

level=debug ts=2020-10-29T13:51:02.597Z caller=collector.go:164 module=palo_alto target=10.91.2.8 msg="Walking subtree" oid=1.3.6.1.2.1.25.1

level=debug ts=2020-10-29T13:51:02.609Z caller=collector.go:177 module=palo_alto target=10.91.2.8 msg="Walk of subtree completed" oid=1.3.6.1.2.1.25.1 duration_seconds=12.430949ms

level=debug ts=2020-10-29T13:51:02.609Z caller=collector.go:164 module=palo_alto target=10.91.2.8 msg="Walking subtree" oid=1.3.6.1.2.1.25.2

level=debug ts=2020-10-29T13:51:02.634Z caller=collector.go:177 module=palo_alto target=10.91.2.8 msg="Walk of subtree completed" oid=1.3.6.1.2.1.25.2 duration_seconds=24.308584ms

level=debug ts=2020-10-29T13:51:02.634Z caller=collector.go:164 module=palo_alto target=10.91.2.8 msg="Walking subtree" oid=1.3.6.1.2.1.25.3

level=debug ts=2020-10-29T13:51:02.647Z caller=collector.go:177 module=palo_alto target=10.91.2.8 msg="Walk of subtree completed" oid=1.3.6.1.2.1.25.3 duration_seconds=13.810531ms

level=debug ts=2020-10-29T13:51:02.648Z caller=collector.go:164 module=palo_alto target=10.91.2.8 msg="Walking subtree" oid=1.3.6.1.4.1.25461.2.1.2.1

level=debug ts=2020-10-29T13:51:02.857Z caller=collector.go:177 module=palo_alto target=10.91.2.8 msg="Walk of subtree completed" oid=1.3.6.1.4.1.25461.2.1.2.1 duration_seconds=209.861654ms

level=debug ts=2020-10-29T13:51:02.870Z caller=collector.go:164 module=palo_alto target=10.91.2.8 msg="Walking subtree" oid=1.3.6.1.4.1.25461.2.1.2.3

level=debug ts=2020-10-29T13:51:03.078Z caller=collector.go:177 module=palo_alto target=10.91.2.8 msg="Walk of subtree completed" oid=1.3.6.1.4.1.25461.2.1.2.3 duration_seconds=207.255972ms

So, I'm starting / finishing the scrap without any errors, but the webserver is returning 500.  Do you have any ideas what could be happening here? 

Thank you in advance 
Best Regards
Carlos

level=debug ts=2020-10-29T13:51:03.080Z caller=collector.go:164 module=palo_alto target=10.91.2.8 msg="Walking subtree" oid=1.3.6.1.4.1.25461.2.1.2.5

level=debug ts=2020-10-29T13:51:03.094Z caller=collector.go:177 module=palo_alto target=10.91.2.8 msg="Walk of subtree completed" oid=1.3.6.1.4.1.25461.2.1.2.5 duration_seconds=13.937742ms

level=debug ts=2020-10-29T13:51:03.141Z caller=main.go:110 module=palo_alto target=10.91.2.8 msg="Finished scrape" duration_seconds=1.598034556

[Brian Candler]

Try using wget -O -  (or curl) to see what the body of the 500 error response says

[Carlos R Tasinafo]

Hi Brian

Thank you for the email

unfortunately the output is the same. :

wget: server returned error: HTTP/1.1 500 Internal Server Error

However I made further progress in troubleshooting 

I figured out that when I exclude the OID:  1.3.6.1.4.1.25461.2.1.2.3 from the config file it works fine

The config that does not work 

```  palo_alto:

    walk:

      - sysUpTime

      - interfaces

      - hrDevice

      - hrSystem

      - hrStorage

      - 1.3.6.1.4.1.25461.2.1.2.1 # panSys

      - 1.3.6.1.4.1.25461.2.1.2.3 # panSession

      - 1.3.6.1.4.1.25461.2.1.2.5 # panGlobalProtect

    timeout: 30s

    auth:

      community: xxxxx```

The config that works:

```  palo_alto:

    walk:

      - sysUpTime

      - interfaces

      - hrDevice

      - hrSystem

      - hrStorage

      - 1.3.6.1.4.1.25461.2.1.2.1 # panSys

     # - 1.3.6.1.4.1.25461.2.1.2.3 # panSession

      - 1.3.6.1.4.1.25461.2.1.2.5 # panGlobalProtect

    timeout: 30s

    auth:

      community: xxxxx```

Somehow the OID responsible to retrieve the information from PA sessions is the causing the issue

Thank you

Regards

Carlos

On Thu, 29 Oct 2020 at 15:44, Brian Candler <b.ca...@pobox.com> wrote:

Try using wget -O -  (or curl) to see what the body of the 500 error response says

--
You received this message because you are subscribed to a topic in the Google Groups "Prometheus Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/prometheus-users/7CkbDw6Aip8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to prometheus-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/8ad71278-d681-4797-84f3-392b3b2b60e7o%40googlegroups.com.