"collected metric was collected before with the same name and label values" is label_value size limited?

101 views

Skip to first unread message

Florent Lourme

unread,

Jul 31, 2020, 4:25:51 AM7/31/20

to Prometheus Users

Hi all,

I have an issue of duplicated metric from snmp_exporter for this OID

ltmVirtualServStatClientCurConns | 1.3.6.1.4.1.3375.2.2.10.2.3.1.12 from F5-BIGIP-LOCAL-MIB

http://www.oidview.com/mibs/3375/F5-BIGIP-LOCAL-MIB.html

When i check the /metrics output from snmp_exporter for that target i can see this error.

collected metric "ltmVirtualServStatClientCurConns" { label:<name:"ltmVirtualServStatName" value:"/Common/ref-si-silo-cs-pdct-rp2web2app.private.sfr.com.app/vip-ref-si-silo-cs-pdct-rp2web2app.pri" > counter:<value:0 > } was collected before with the same name and label values

I should have two different metrics with two different "ltmVirtualServStatName" label value below:

/Common/ref-si-silo-cs-pdct-rp2web2app.private.sfr.com.app/vip-ref-si-silo-cs-pdct-rp2web2app.private.sfr.com_HTTP_extra
/Common/ref-si-silo-cs-pdct-rp2web2app.private.sfr.com.app/vip-ref-si-silo-cs-pdct-rp2web2app.private.sfr.com_HTTP_intra

but it seems that the "ltmVirtualServStatName" label_value is limited to "/Common/ref-si-silo-cs-pdct-rp2web2app.private.sfr.com.app/vip-ref-si-silo-cs-pdct-rp2web2app.pri" in the /metric output so the snmp_exporter is seeing the same metric two times and drop the error.

I have another F5 LTM in the cluster with the same configuration (but another version) and i can get two different metrics

ltmVirtualServStatClientCurConns{ltmVirtualServStatName="/Common/ref-si-silo-cs-pdct-rp2web2app.private.sfr.com.app/vip-ref-si-silo-cs-pdct-rp2web2a780a7faf"} 0
ltmVirtualServStatClientCurConns{ltmVirtualServStatName="/Common/ref-si-silo-cs-pdct-rp2web2app.private.sfr.com.app/vip-ref-si-silo-cs-pdct-rp2web2ac8889aed"} 0

This time it seems that the snmp_exporter is putting a random string at the end of the label ltmVirtualServStatName to make it unique

Do you have any idea why this is happening ? The MIB is the same on both devices.

Thanks!

Flo

Brian Brazil

unread,

Jul 31, 2020, 5:22:21 AM7/31/20

to Florent Lourme, Prometheus Users

Can you share your generator config?

Brian

Thanks!
Flo

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/c1e04ea8-eb5d-487d-8d7b-c741aa7ffd89o%40googlegroups.com.

Brian Brazil

www.robustperception.io

Florent Lourme

unread,

Jul 31, 2020, 6:00:00 AM7/31/20

to Prometheus Users

Hi Brian,

Thanks for your reply, i've found the issue...

The target (Y.Y.Y.Y) on which i get the doublon is truncating the index string. So it gives multiple metric for the same index value.

root@sys-d063323c ~ # snmpwalk -v2c -m all -c XXXXXXX Y.Y.Y.Y ltmVirtualServStatClientCurConns 2>/dev/null | grep -i 'vip-ref-si-silo-gp-pdct-rp2app.priv'

F5-BIGIP-LOCAL-MIB::ltmVirtualServStatClientCurConns."/Common/ref-si-silo-gp-pdct-rp2app.private.sfr.com.app/vip-ref-si-silo-gp-pdct-rp2app.private.sfr".64.1734 = Counter64: 0

F5-BIGIP-LOCAL-MIB::ltmVirtualServStatClientCurConns."/Common/ref-si-silo-gp-pdct-rp2app.private.sfr.com.app/vip-ref-si-silo-gp-pdct-rp2app.private.sfr".64.1735 = Counter64: 0

F5-BIGIP-LOCAL-MIB::ltmVirtualServStatClientCurConns."/Common/ref-si-silo-gp-pdct-rp2app.private.sfr.com.app/vip-ref-si-silo-gp-pdct-rp2app.private.sfr".64.1736 = Counter64: 0

This is not the case for the other target X.X.X.X which is generating a 8 characters hash to generate a unique value

1 root@sys-d063323c ~ # snmpwalk -v2c -m all -c XXXXXXX X.X.X.X. ltmVirtualServStatClientCurConns 2>/dev/null | grep -i 'vip-ref-si-silo-gp-pdct-rp2app.priv'

F5-BIGIP-LOCAL-MIB::ltmVirtualServStatClientCurConns."/Common/ref-si-silo-gp-pdct-rp2app.private.sfr.com.app/vip-ref-si-silo-gp-pdct-rp2app.priva490090bc" = Counter64: 0

F5-BIGIP-LOCAL-MIB::ltmVirtualServStatClientCurConns."/Common/ref-si-silo-gp-pdct-rp2app.private.sfr.com.app/vip-ref-si-silo-gp-pdct-rp2app.priva688454f2" = Counter64: 0

F5-BIGIP-LOCAL-MIB::ltmVirtualServStatClientCurConns."/Common/ref-si-silo-gp-pdct-rp2app.private.sfr.com.app/vip-ref-si-silo-gp-pdct-rp2app.privad806b1b0" = Counter64: 0

After looking into both devices configuration, an hotfix is installed on X.X.X.X but not on Y.Y.Y.Y ...

Sorry for the inconveniance

Le vendredi 31 juillet 2020 11:22:21 UTC+2, Brian Brazil a écrit :

On Fri, 31 Jul 2020 at 09:25, Florent Lourme <floren...@gmail.com> wrote:

Hi all,

I have an issue of duplicated metric from snmp_exporter for this OID

ltmVirtualServStatClientCurConns | 1.3.6.1.4.1.3375.2.2.10.2.3.1.12 from F5-BIGIP-LOCAL-MIB
http://www.oidview.com/mibs/3375/F5-BIGIP-LOCAL-MIB.html

When i check the /metrics output from snmp_exporter for that target i can see this error.
collected metric "ltmVirtualServStatClientCurConns" { label:<name:"ltmVirtualServStatName" value:"/Common/ref-si-silo-cs-pdct-rp2web2app.private.sfr.com.app/vip-ref-si-silo-cs-pdct-rp2web2app.pri" > counter:<value:0 > } was collected before with the same name and label values
I should have two different metrics with two different "ltmVirtualServStatName" label value below:
/Common/ref-si-silo-cs-pdct-rp2web2app.private.sfr.com.app/vip-ref-si-silo-cs-pdct-rp2web2app.private.sfr.com_HTTP_extra
/Common/ref-si-silo-cs-pdct-rp2web2app.private.sfr.com.app/vip-ref-si-silo-cs-pdct-rp2web2app.private.sfr.com_HTTP_intra
but it seems that the "ltmVirtualServStatName" label_value is limited to "/Common/ref-si-silo-cs-pdct-rp2web2app.private.sfr.com.app/vip-ref-si-silo-cs-pdct-rp2web2app.pri" in the /metric output so the snmp_exporter is seeing the same metric two times and drop the error.

I have another F5 LTM in the cluster with the same configuration (but another version) and i can get two different metrics
ltmVirtualServStatClientCurConns{ltmVirtualServStatName="/Common/ref-si-silo-cs-pdct-rp2web2app.private.sfr.com.app/vip-ref-si-silo-cs-pdct-rp2web2a780a7faf"} 0
ltmVirtualServStatClientCurConns{ltmVirtualServStatName="/Common/ref-si-silo-cs-pdct-rp2web2app.private.sfr.com.app/vip-ref-si-silo-cs-pdct-rp2web2ac8889aed"} 0
This time it seems that the snmp_exporter is putting a random string at the end of the label ltmVirtualServStatName to make it unique
Do you have any idea why this is happening ? The MIB is the same on both devices.
Can you share your generator config?

Brian

Thanks!
Flo

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to promethe...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/c1e04ea8-eb5d-487d-8d7b-c741aa7ffd89o%40googlegroups.com.