server returned HTTP status 500 Internal Server Error
175 views
Skip to first unread message
sabarish narayanan
Oct 13, 2020, 8:52:34 AM10/13/20
to Prometheus Users
Hi, I am trying to monitor a fortigate firewall but I'm getting "server returned HTTP status 500 Internal Server Error".
The snmpwalk works fine.
snmpwalk -v3 -l authPriv -u username -a MD5 -A ********* -x AES -X ********* XX.XX.XX.XX ifXTable
Did not find 'zeroDotZero' in module SNMPv2-SMI (/auto/mibs/v2/IP-MIB.my)
Did not find 'zeroDotZero' in module SNMPv2-SMI (/auto/mibs/v2/EVENT-MIB.my)
Did not find 'zeroDotZero' in module SNMPv2-SMI (/usr/share/snmp/mibs/DISMAN-SCHEDULE-MIB.txt)
IF-MIB::ifName.1 = STRING: dmz
IF-MIB::ifName.2 = STRING: wan1
IF-MIB::ifName.3 = STRING: wan2
IF-MIB::ifName.4 = STRING: modem
IF-MIB::ifName.5 = STRING: internal3
IF-MIB::ifName.6 = STRING: internal4
IF-MIB::ifName.7 = STRING: ssl.root
IF-MIB::ifName.8 = STRING: internal
IF-MIB::ifName.9 = STRING: VLAN2
IF-MIB::ifName.10 = STRING: internal6
IF-MIB::ifName.11 = STRING: internal7
IF-MIB::ifName.12 = STRING: vpn
IF-MIB::ifName.13 = STRING: ipsecvpn
IF-MIB::ifName.14 = STRING: ipsec2
IF-MIB::ifName.15 = STRING: XXXVPN
IF-MIB::ifInMulticastPkts.1 = Counter32: 0
IF-MIB::ifInMulticastPkts.2 = Counter32: 0
IF-MIB::ifInMulticastPkts.3 = Counter32: 0
IF-MIB::ifInMulticastPkts.4 = Counter32: 0
IF-MIB::ifInMulticastPkts.5 = Counter32: 0
IF-MIB::ifInMulticastPkts.6 = Counter32: 0
IF-MIB::ifInMulticastPkts.7 = Counter32: 0
^C
generator.yml file :-
modules:
XPPC-MIB:
walk:
- upsThreePhaseBatteryTimeRemain
- upsThreePhaseBatteryTemperature
- upsThreePhaseOutputFrequency
- upsThreePhaseOutputVoltageR
- upsThreePhaseOutputVoltageS
- upsThreePhaseOutputVoltageT
- upsThreePhaseOutputLoadPercentageR
- upsThreePhaseOutputLoadPercentageS
- upsThreePhaseOutputLoadPercentageT
- upsConfigOutputVA
version: 3
max_repetitions: 25
retries: 3
timeout: 10s
auth:
username: username
security_level: authNoPriv
password: ***********
auth_protocol: MD5
lookups:
- source_indexes: [upsThreePhaseBatteryTimeRemain]
lookup: TimeRemain
- source_indexes: [upsThreePhaseBatteryTemperature]
lookup: BatteryTemperature
- source_indexes: [upsThreePhaseOutputFrequency]
lookup: OutputFrequency
- source_indexes: [upsThreePhaseOutputVoltageR]
lookup: OutputVoltageR
- source_indexes: [upsThreePhaseOutputVoltageS]
lookup: OutputVoltageS
- source_indexes: [upsThreePhaseOutputVoltageT]
lookup: OutputVoltageT
- source_indexes: [upsThreePhaseOutputLoadPercentageR]
lookup: OutputLoadPercentageR
- source_indexes: [upsThreePhaseOutputLoadPercentageS]
lookup: OutputLoadPercentageS
- source_indexes: [upsThreePhaseOutputLoadPercentageT]
lookup: OutputLoadPercentageT
- source_indexes: [upsConfigOutputVA]
lookup: ApparentPowerVA
fortigate_snmp:
walk:
- ifXTable
# - fgSystem
# - fgVpn
# - fgIntf
# - fgInetProto
version: 3
max_repetitions: 25
timeout: 10s
auth:
username: username # Required, no default. -u option to NetSNMP.
security_level: authPriv # Defaults to noAuthNoPriv. -l option to NetSNMP.
# Can be noAuthNoPriv, authNoPriv or authPriv.
password: ******** # Has no default. Also known as authKey, -A option to NetSNMP.
# Required if security_level is authNoPriv or authPriv.
auth_protocol: MD5 # MD5 or SHA, defaults to MD5. -a option to NetSNMP.
# Used if security_level is authNoPriv or authPriv.
priv_protocol: AES # DES or AES, defaults to DES. -x option to NetSNMP.
# Used if security_level is authPriv.
priv_password: ******** # Has no default. Also known as privKey, -X option to NetSNMP.
# Required if security_level is authPriv.
cisco_switch_snmp:
walk:
- sysName
- sysLocation
- sysUpTimeInstance
# - .1.3.6.1.4.1.9.9.402.1.3.1.2.1
- cpeExtMainPseDescr
# - .1.3.6.1.4.1.9.9.500.1.2.1.1.8.1001
- cswSwitchSoftwareImage
# - .1.3.6.1.4.1.9.9.13.1.3.1.2
- ciscoEnvMonTemperatureStatusDescr
# - .1.3.6.1.4.1.9.9.13.1.3.1.3
- ciscoEnvMonTemperatureStatusValue
# - .1.3.6.1.4.1.9.9.13.1.3.1.4
- ciscoEnvMonTemperatureThreshold
# - .1.3.6.1.4.1.9.9.13.1.3.1.5
- ciscoEnvMonTemperatureLastShutdown
# - .1.3.6.1.4.1.9.9.13.1.3.1.6
- ciscoEnvMonTemperatureState
# - .1.3.6.1.4.1.9.9.13.1.5.1.2
- ciscoEnvMonSupplyStatusDescr
# - .1.3.6.1.4.1.9.9.13.1.5.1.3
- ciscoEnvMonSupplyState
# - .1.3.6.1.4.1.9.9.13.1.5.1.4
- ciscoEnvMonSupplySource
# - .1.3.6.1.4.1.9.9.109.1.1.1.1.6
- cpmCPUTotal5secRev
# - .1.3.6.1.4.1.9.9.109.1.1.1.1.7
- cpmCPUTotal1minRev
# - .1.3.6.1.4.1.9.9.109.1.1.1.1.8
- cpmCPUTotal5minRev
- ifHCInOctets
- ifHCOutOctets
- ifDescr
- ifAlias
- ifHighSpeed
- ifOperStatus
- ifLastChange
- ifInErrors
- ifOutErrors
version: 3
max_repetitions: 25
timeout: 180s
auth:
username: username # Required, no default. -u option to NetSNMP.
security_level: authPriv # Defaults to noAuthNoPriv. -l option to NetSNMP.
# Can be noAuthNoPriv, authNoPriv or authPriv.
password: ********* # # Has no default. Also known as authKey, -A option to NetSNMP.
# Required if security_level is authNoPriv or authPriv.
auth_protocol: SHA # MD5 or SHA, defaults to MD5. -a option to NetSNMP.
# Used if security_level is authNoPriv or authPriv.
priv_protocol: AES # DES or AES, defaults to DES. -x option to NetSNMP.
# Used if security_level is authPriv.
priv_password: ******* # Has no default. Also known as privKey, -X option to NetSNMP.
# Required if security_level is authPriv.
lookups:
- source_indexes: [sysName]
lookup: sysName
- source_indexes: [sysLocation]
lookup: sysLocation
- source_indexes: [sysUpTimeInstance]
lookup: uptime
- source_indexes: [cpeExtMainPseDescr]
lookup: ciscoProduct
- source_indexes: [cswSwitchSoftwareImage]
lookup: ciscoSoftware
- source_indexes: [ciscoEnvMonTemperatureStatusDescr]
lookup: ciscoEnvMonTemperatureStatusDescr
- source_indexes: [ciscoEnvMonTemperatureStatusValue]
lookup: ciscoEnvMonTemperatureStatusValue
- source_indexes: [ciscoEnvMonTemperatureThreshold]
lookup: ciscoEnvMonTemperatureThreshold
- source_indexes: [ciscoEnvMonTemperatureLastShutdown]
lookup: ciscoEnvMonTemperatureLastShutdown
- source_indexes: [ciscoEnvMonTemperatureState]
lookup: ciscoEnvMonTemperatureState
- source_indexes: [ciscoEnvMonSupplyStatusDescr]
lookup: ciscoEnvMonSupplyStatusDescr
- source_indexes: [ciscoEnvMonSupplyState]
lookup: ciscoEnvMonSupplyState
- source_indexes: [ciscoEnvMonSupplySource]
lookup: ciscoEnvMonSupplySource
- source_indexes: [cpmCPUTotal5secRev]
lookup: cpmCPUTotal5secRev
- source_indexes: [cpmCPUTotal1minRev]
lookup: cpmCPUTotal1minRev
- source_indexes: [cpmCPUTotal5minRev]
lookup: cpmCPUTotal5minRev
- source_indexes: [ifDescr]
lookup: ifDescr
prometheus.yml file :-
# Global config
global:
scrape_interval: 15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
# scrape_timeout: 15s # scrape_timeout is set to the global default (10s).
# A scrape configuration containing exactly one endpoint to scrape:# Here it's Prometheus itself.
rule_files:
- prometheus_rules.yml
scrape_configs:
# The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
# - job_name: 'prometheus'
# # metrics_path defaults to '/metrics'
# # scheme defaults to 'http'.
# static_configs:
# - targets: ['localhost:9090']
- job_name: 'apc3p-snmp'
static_configs:
- targets:
- XX.XX.XX.XX # SNMP device.
scrape_interval: 5s
scrape_timeout : 5s
metrics_path: /snmp
params:
module: [XPPC-MIB]
relabel_configs:
- source_labels: [__address__]
target_label: __param_target
- source_labels: [__param_target]
target_label: instance
- target_label: __address__
replacement: 127.0.0.1:9116 # The SNMP exporter's real hostname:port.
# replacement: 192.168.0.75:9116 # The SNMP exporter's real hostname:port.
- job_name: 'fortigate-snmp'
static_configs:
- targets:
- XX.XX.XX.XX # fortigate device.
scrape_interval: 10m
scrape_timeout : 10m
metrics_path: /snmp
params:
module: [fortigate_snmp]
relabel_configs:
- source_labels: [__address__]
target_label: __param_target
- source_labels: [__param_target]
target_label: instance
- target_label: __address__
replacement: 127.0.0.1:9116 # SNMP exporter.
- job_name: 'cisco-switch-snmp'
static_configs:
- targets:
- XX.XX.XX.XX # cisco C3650-24TS device.
scrape_interval: 3m
scrape_timeout : 3m
metrics_path: /snmp
params:
module: [cisco_switch_snmp]
relabel_configs:
- source_labels: [__address__]
target_label: __param_target
- source_labels: [__param_target]
target_label: instance
- target_label: __address__
replacement: 127.0.0.1:9116 # SNMP exporter.
snmp_exporter status :-
Loaded: loaded (/etc/systemd/system/snmp_exporter.service; enabled; vendor preset: disabled)
Active: active (running) since Tue 2020-10-13 16:02:58 IST; 2h 11min ago
Main PID: 1741643 (snmp_exporter)
Tasks: 8 (limit: 23823)
Memory: 31.0M
CGroup: /system.slice/snmp_exporter.service
└─1741643 /usr/local/bin/snmp_exporter/snmp_exporter --config.file /etc/prometheus/snmp.yml
Oct 13 16:02:58 user systemd[1]: Stopped SNMP Exporter.
Oct 13 16:02:58 user systemd[1]: Started SNMP Exporter.
Oct 13 16:02:58 user snmp_exporter[1741643]: level=info ts=2020-10-13T10:32:58.098Z caller=main.go:149 msg="Starting snmp_exporter" version="(version=0.19.0, branch=HEAD, revision=9dcbc02f59648b21fcf632de1b62a30df70f4649)"
Oct 13 16:02:58 user snmp_exporter[1741643]: level=info ts=2020-10-13T10:32:58.098Z caller=main.go:150 build_context="(go=go1.14.7, user=root@387afaad41d6, date=20200831-12:07:03)"
Oct 13 16:02:58 user snmp_exporter[1741643]: level=info ts=2020-10-13T10:32:58.105Z caller=main.go:243 msg="Listening on address" address=:9116
Oct 13 16:06:47 user snmp_exporter[1741643]: level=info ts=2020-10-13T10:36:47.613Z caller=collector.go:224 module=fortigate_snmp target=XX.XX.XX.XX msg="Error scraping target" err="scrape canceled (possible timeout) walking target XX.XX.XX.XX"
Oct 13 16:09:20 user snmp_exporter[1741643]: level=info ts=2020-10-13T10:39:20.073Z caller=collector.go:224 module=fortigate_snmp target=XX.XX.XX.XX msg="Error scraping target" err="scrape canceled (possible timeout) walking target XX.XX.XX.XX"The prometheus rules are not related to the firewall.
I tried changing the scrape interval and scrape timeout. If I reduced them, I got "context deadline exceeded". What am I doing wrong and how do I fix it? Any help is appriciated. Thanks in advance.
I tried changing the scrape interval and scrape timeout. If I reduced them, I got "context deadline exceeded". What am I doing wrong and how do I fix it? Any help is appriciated. Thanks in advance.
Reply all
Reply to author
Forward
0 new messages