Sflow-rt not showing metrics for one of two machines
12 views
Skip to first unread message
Stefano R
Mar 19, 2026, 7:44:29 AM (4 days ago) Mar 19
to sFlow-RT
Hello Guys,
I have two machines sending the same data to the sflow-rt collector, however one of them is showing all of the available metrics from the packets received:
while for the other one I only see few of them, and not for example the if specific stats:
The packet from the non working machine shows the counter samples and the flow samples being present for the interfaces, but sflow doesn't have the metric for them:
Do you know how and what logs I can enable to debug it? I cannot figure it out by just looking at the packets itself, and they seem identical between the working and not working device.
Thanks in advance for your help
S
Peter Phaal
Mar 19, 2026, 11:04:46 AM (4 days ago) Mar 19
to sFlow-RT
What is the difference between the two agents? Are they different network operating systems / vendors?
Check the /agents/json query to see if there are issues with sequence numbers / decoding (you can add a filter ?agent=<agent IP> to select just one agent).
Use the /metric/<agent IP>/json to see metrics exported by each agent.
The sFlow-Test application, https://github.com/sflow-rt/sflow-test, is useful for identifying issues with an sFlow agent.
Given that the decode of sFlow packets from the agent indicates that it is sending interface counters, I suspect that there is an issue with sequence numbers. sFlow-RT ignores out of order / duplicate records to avoid generating incorrect metric values.
Stefano R
Mar 19, 2026, 12:10:31 PM (4 days ago) Mar 19
to sFlow-RT
What is the difference between the two agents? Are they different network operating systems / vendors?
Same vendor, same configuration and same version. The even stranger thing is that only on peak traffic I don't see the stats. in low condition traffic I have no issues and the sflow metrics are presents.
Check the /agents/json query to see if there are issues with sequence numbers / decoding (you can add a filter ?agent=<agent IP> to select just one agent).
just obfuscating the IP :
-> curl -sX GET "http://localhost:8008/agents/json?agent={notWorkingIP}" -H 'accept: */*'
{"{notWorkingIP}": {
"sFlowDatagramsLost": 93,
"sFlowDatagramSource": ["{notWorkingIP}"],
"sFlowNotificationSamples": 0,
"sFlowFlowDuplicateSamples": 0,
"sFlowDatagramsReceived": 367412,
"sFlowNotificationDrops": 0,
"sFlowFlowSamples": 172,
"sFlowDatagramsOutOfOrder": 27775, <<<< This is the main difference from the working one
"sFlowNotificationDuplicateSamples": 0,
"sFlowFlowDrops": 0,
"sFlowFlowLostSamples": 0,
"sFlowNotificationLostSamples": 0,
"sFlowCounterSamples": 0, <<<< This is also 0, but I do see them in the wireshark
"sFlowFlowDatasources": 5,
"sFlowCounterOutOfOrderSamples": 0,
"sFlowNotificationOutOfOrderSamples": 0,
"firstSeen": 20308516,
"sFlowCounterDatasources": 1,
"sFlowFlowOutOfOrderSamples": 0,
"sFlowNotificationDatasources": 0,
"uptime": 18518000,
"sFlowCounterDuplicateSamples": 0,
"lastSeen": 0,
"sFlowDatagramsDuplicates": 0,
"sFlowCounterLostSamples": 0
}}
{"{notWorkingIP}": {
"sFlowDatagramsLost": 93,
"sFlowDatagramSource": ["{notWorkingIP}"],
"sFlowNotificationSamples": 0,
"sFlowFlowDuplicateSamples": 0,
"sFlowDatagramsReceived": 367412,
"sFlowNotificationDrops": 0,
"sFlowFlowSamples": 172,
"sFlowDatagramsOutOfOrder": 27775, <<<< This is the main difference from the working one
"sFlowNotificationDuplicateSamples": 0,
"sFlowFlowDrops": 0,
"sFlowFlowLostSamples": 0,
"sFlowNotificationLostSamples": 0,
"sFlowCounterSamples": 0, <<<< This is also 0, but I do see them in the wireshark
"sFlowFlowDatasources": 5,
"sFlowCounterOutOfOrderSamples": 0,
"sFlowNotificationOutOfOrderSamples": 0,
"firstSeen": 20308516,
"sFlowCounterDatasources": 1,
"sFlowFlowOutOfOrderSamples": 0,
"sFlowNotificationDatasources": 0,
"uptime": 18518000,
"sFlowCounterDuplicateSamples": 0,
"lastSeen": 0,
"sFlowDatagramsDuplicates": 0,
"sFlowCounterLostSamples": 0
}}
{"{workingIP}": {
"sFlowDatagramsLost": 19,
"sFlowDatagramSource": ["{workingIP}"],
"sFlowNotificationSamples": 0,
"sFlowFlowDuplicateSamples": 0,
"sFlowDatagramsReceived": 2842677,
"sFlowNotificationDrops": 0,
"sFlowFlowSamples": 16722945,
"sFlowDatagramsOutOfOrder": 0,
"sFlowNotificationDuplicateSamples": 0,
"sFlowFlowDrops": 0,
"sFlowFlowLostSamples": 114,
"sFlowNotificationLostSamples": 0,
"sFlowCounterSamples": 16320,
"sFlowFlowDatasources": 7,
"sFlowCounterOutOfOrderSamples": 0,
"sFlowNotificationOutOfOrderSamples": 0,
"firstSeen": 20420730,
"sFlowCounterDatasources": 8,
"sFlowFlowOutOfOrderSamples": 0,
"sFlowNotificationDatasources": 0,
"uptime": 190142520,
"sFlowCounterDuplicateSamples": 0,
"lastSeen": 1,
"sFlowDatagramsDuplicates": 0,
"sFlowCounterLostSamples": 0
}}
"sFlowDatagramsLost": 19,
"sFlowDatagramSource": ["{workingIP}"],
"sFlowNotificationSamples": 0,
"sFlowFlowDuplicateSamples": 0,
"sFlowDatagramsReceived": 2842677,
"sFlowNotificationDrops": 0,
"sFlowFlowSamples": 16722945,
"sFlowDatagramsOutOfOrder": 0,
"sFlowNotificationDuplicateSamples": 0,
"sFlowFlowDrops": 0,
"sFlowFlowLostSamples": 114,
"sFlowNotificationLostSamples": 0,
"sFlowCounterSamples": 16320,
"sFlowFlowDatasources": 7,
"sFlowCounterOutOfOrderSamples": 0,
"sFlowNotificationOutOfOrderSamples": 0,
"firstSeen": 20420730,
"sFlowCounterDatasources": 8,
"sFlowFlowOutOfOrderSamples": 0,
"sFlowNotificationDatasources": 0,
"uptime": 190142520,
"sFlowCounterDuplicateSamples": 0,
"lastSeen": 1,
"sFlowDatagramsDuplicates": 0,
"sFlowCounterLostSamples": 0
}}
and this is from a tcpdump I just took :
tshark -r notWorkingIP.pcap -Y "sflow_245.sampletype == 2" -V | grep -A20 "Counters sample"
Counters sample, seq 1926
0000 0000 0000 0000 0000 .... .... .... = Enterprise: standard sFlow (0)
.... .... .... .... .... 0000 0000 0010 = sFlow sample type: Counters sample (2)
Sample length (byte): 108
Sequence number: 1926
0000 0000 .... .... .... .... .... .... = Source ID type: 0
.... .... 0000 1111 0100 0110 0101 1100 = Source ID index: 1001052
Counters records: 1
Generic interface counters
0000 0000 0000 0000 0000 .... .... .... = Enterprise: standard sFlow (0)
.... .... .... .... .... 0000 0000 0001 = Format: Generic interface counters (1)
Flow data length (byte): 88
Interface index: 1001052
Interface Type: 161
Interface Speed: 100000000000
Interface Direction: Full-Duplex (1)
.... .... .... .... .... .... .... ...1 = IfAdminStatus: Up
.... .... .... .... .... .... .... ..1. = IfOperStatus: Up
Input Octets: 1186698636074949
Input Packets: 2799359716
Input Multicast Packets: 877742075
Input Broadcast Packets: 990198356
Input Discarded Packets: 0
Counters sample, seq 1926
0000 0000 0000 0000 0000 .... .... .... = Enterprise: standard sFlow (0)
.... .... .... .... .... 0000 0000 0010 = sFlow sample type: Counters sample (2)
Sample length (byte): 108
Sequence number: 1926
0000 0000 .... .... .... .... .... .... = Source ID type: 0
.... .... 0000 1111 0100 0110 0101 1100 = Source ID index: 1001052
Counters records: 1
Generic interface counters
0000 0000 0000 0000 0000 .... .... .... = Enterprise: standard sFlow (0)
.... .... .... .... .... 0000 0000 0001 = Format: Generic interface counters (1)
Flow data length (byte): 88
Interface index: 1001052
Interface Type: 161
Interface Speed: 100000000000
Interface Direction: Full-Duplex (1)
.... .... .... .... .... .... .... ...1 = IfAdminStatus: Up
.... .... .... .... .... .... .... ..1. = IfOperStatus: Up
Input Octets: 1186698636074949
Input Packets: 2799359716
Input Multicast Packets: 877742075
Input Broadcast Packets: 990198356
Input Discarded Packets: 0
Use the /metric/<agent IP>/json to see metrics exported by each agent.
Yes, this was in the screenshot, but I can do the same via cli :
-> curl -sX GET "http://localhost:8008/metric/{workingIP}/json" -H 'accept: */*'
{
"1001029.ifoutdiscards": 0,
"1001028.ifinerrors": 0,
"1001032.ifspeed": 100000000000,
"1001032.ifoutoctets": 2.404787327207351E8,
"1001029.prometheus_12": 1.7502649410866306E9,
"1001104.ifoutucastpkts": 1631866.60680435,
"1001029.ifoututilization": 8.202547930834584,
"24001.ifinunknownprotos": 0,
"1001032.ifinmulticastpkts": 231.02277267279266,
"1001014.ifoperstatus": "up",
"27001.eth_alignmenterrors": 0,
"27001.ifoutbroadcastpkts": 0,
"1001028.arm_bytes": 2.2630598928033912E8,
"1001032.ifinerrors": 0,
"1001029.arm_dstas": 2.239511529955379E8,
"1001052.arm_bytes": 4.494110158223985E9,
"27001.eth_carriersenseerrors": 0,
"1001014.prometheus_28": 3.2217111564166945E8,
"1001052.ifoperstatus": "up",
"1001029.ifinutilization": 6.206686667066467,
"1001052.ifoutdiscards": 0,
"1001028.ifinmulticastpkts": 0,
"1001014.iftype": "ieee8023adLag",
"27001.ifoutdiscards": 0,
"1001029.ifoutoctets": 1.0253184913543229E9,
"1001028.ifoutoctets": 8.47914633E7,
"1001052.ifinerrors": 0,
"24001.flowgraph-pair": 89639.77066553601,
"1001029.iftype": "ieee8023adLag",
"1001029.ifinmulticastpkts": 0.09995002498750626,
"27001.eth_latecollisions": 0,
"1001014.ifspeed": 100000000000,
"24001.ifindiscards": 0,
"1001032.ifinutilization": 9.53404968677587,
"1001104.ifoperstatus": "up",
"1001104.ifinbroadcastpkts": 0,
"1001028.arm_dstas": 2748302.587485982,
"1001029.ifinucastpkts": 1867789.7051474263,
"1001104.arm_dstpeer": 2.234021957935419E8,
"27001.ifindiscards": 0,
"1001014.ddos_protect_tcp_flood": 2.05786441158796E-6,
"1001028.arm_srcas": 2.289233500490104E7,
"1001104.iftype": "ieee8023adLag",
"1001104.ifinunknownprotos": 0,
"1001104.ifouterrors": 0,
"1001028.ifoutucastpkts": 177816.5,
"24001.ifinbroadcastpkts": 0,
"1001052.ifadminstatus": "up",
"1001052.prometheus_28": 1.517154982664847E9,
"1001014.ifinoctets": 3.1066233852917665E8,
"1001029.ifinerrors": 0,
"1001032.ifinpkts": 1948947.263284059,
"24001.arm_dstas": 2.766321784545277E8,
"1001052.prometheus_24": 162773.55139098343,
"1001032.arm_srcas": 6.552027917708361E8,
"24001.ifinerrors": 0,
"1001032.ifoutucastpkts": 377590.4914103076,
"27001.eth_internalmacrxerrors": 0,
"1001052.prometheus_5": 608632.6003069924,
"1001014.ifinunknownprotos": 0,
"1001104.ifoutbroadcastpkts": 0,
"27001.ifinunknownprotos": 0,
"1001052.prometheus_8": 2.2619949913002174E7,
"1001052.prometheus_9": 3.2803474920990605E9,
"1001052.arm_dstas": 1.3590992122387356E8,
"1001029.ifspeed": 100000000000,
"24001.eth_singlecollisions": 0,
"1001029.ifinpkts": 1867789.8050974514,
"1001032.ifoutpkts": 377591.29045145825,
"27001.ifinucastpkts": 0.09995002498750626,
"24001.eth_internalmactxerrors": 0,
"1001028.ifoutbroadcastpkts": 0,
"1001052.prometheus_12": 3.4198234805599065E9,
"1001029.arm_bytes": 2.1039952415661578E9,
"24001.ifdirection": "full-duplex",
"1001014.ddos_protect_ip_flood": 2.05786441158796E-6,
"1001029.flowgraph-pair": 68672.5897304701,
"1001032.ifdirection": "full-duplex",
"24001.ifoutoctets": 1.7100662788605697E9,
"1001029.ifoutpkts": 1660047.0764617692,
"1001052.ifoutmulticastpkts": 0,
"1001104.ifinucastpkts": 2328175.9952110145,
"24001.eth_sqetesterrors": 0,
"1001028.ifoperstatus": "up",
"1001052.ifoutucastpkts": 4989444.4,
"1001029.ifadminstatus": "up",
"1001032.ifindiscards": 0,
"1001014.ifinucastpkts": 389521.8824940048,
"1001052.ifoutpkts": 4989445.5,
"1001014.prometheus_8": 1.0401234111108901E7,
"24001.prometheus_5": 638482.8314757305,
"1001014.prometheus_9": 3.266339770795697E8,
"1001014.prometheus_5": 199.57605731690742,
"24001.prometheus_1": 3351.3277436209987,
"1001052.flowgraph-pair": 99504.92737321918,
"1001052.ifinpkts": 3043419,
"1001014.prometheus_1": 653.0365301171651,
"27001.ifoutpkts": 0.09995002498750626,
"1001052.arm_dstaspath": 1.3590992122387356E8,
"1001032.prometheus_12": 1.2636259617753892E9,
"1001029.ifoutucastpkts": 1660047.0764617692,
"1001104.ifdirection": "full-duplex",
"24001.eth_carriersenseerrors": 0,
"1001028.ifspeed": 100000000000,
"1001052.ifouterrors": 0,
"1001052.ifinbroadcastpkts": 110.10000000000001,
"24001.ifinoctets": 2.8750284779610195E9,
"27001.eth_symbolerrors": 0,
"24001.ifouterrors": 0,
"1001032.iftype": "ieee8023adLag",
"1001028.ifinucastpkts": 288332.7,
"1001032.prometheus_24": 238217.6312796434,
"24001.eth_excessivecollisions": 0,
"24001.eth_toolongs": 0,
"1001032.prometheus_28": 7.869731597316215E8,
"1001032.ifoutdiscards": 0,
"1001052.arm_srcas": 9.734401849866718E8,
"1001028.ifindex": "1001028",
"27001.eth_toolongs": 0,
"1001028.ifoutpkts": 177817.5,
"1001052.ifoutbroadcastpkts": 1.1,
"27001.ifinoctets": 8.195902048975512,
"24001.ifadminstatus": "up",
"1001014.ifoutucastpkts": 279820.4436450839,
"1001104.arm_dstaspath": 2.234021957935419E8,
"1001104.prometheus_12": 1.8031002089612768E9,
"1001014.ifadminstatus": "up",
"1001029.ifoperstatus": "up",
"1001032.ifoutmulticastpkts": 0.0998801438274071,
"27001.ifspeed": 10000000000,
"1001104.ifinutilization": 9.823018451960492,
"27001.ifoututilization": 7.756121939030484E-7,
"1001014.ifoutmulticastpkts": 0.9992006394884093,
"27001.eth_fcserrors": 0,
"1001014.ifinpkts": 389521.8824940048,
"27001.ifindex": "27001",
"24001.eth_symbolerrors": 0,
"27001.iftype": "ethernetCsmacd",
"27001.ifadminstatus": "up",
"1001052.arm_dstpeer": 1.3590992122387356E8,
"1001032.ifindex": "1001032",
"27001.eth_singlecollisions": 0,
"27001.eth_internalmactxerrors": 0,
"1001104.ifspeed": 100000000000,
"1001014.arm_dstpeer": 3.6598104761438474E7,
"1001052.ifinucastpkts": 3043187.1,
"1001014.ifouterrors": 0,
"27001.ifinbroadcastpkts": 0,
"1001029.arm_srcas": 3.951635808022792E8,
"1001028.prometheus_1": 2.0669861613132214,
"1001032.arm_bytes": 1.4660513111372392E9,
"24001.eth_fcserrors": 0,
"1001028.prometheus_5": 106547.60496541545,
"24001.eth_internalmacrxerrors": 0,
"1001104.ifindex": "1001104",
"27001.ifoutoctets": 9.695152423788105,
"1001028.prometheus_8": 5010460.782084496,
"24001.arm_bytes": 4.302376807677212E9,
"1001028.prometheus_9": 1.1640109226897946E8,
"1001029.ifoutmulticastpkts": 0,
"1001028.ifoutmulticastpkts": 1,
"27001.ifinmulticastpkts": 0,
"1001028.ifinpkts": 288332.7,
"27001.eth_multiplecollisions": 0,
"1001014.ifdirection": "full-duplex",
"1001028.ifinbroadcastpkts": 0,
"27001.ifdirection": "full-duplex",
"24001.ifoututilization": 13.680530230884559,
"1001104.prometheus_28": 7.624217338714464E8,
"1001104.prometheus_27": 4.469658923745612,
"27001.ifinpkts": 0.09995002498750626,
"27001.ifoperstatus": "up",
"24001.ifoperstatus": "up",
"1001104.ifoutoctets": 1.0123366882171007E9,
"24001.ifinpkts": 4817530.934532734,
"1001052.iftype": "ieee8023adLag",
"1001032.ifouterrors": 0,
"27001.eth_excessivecollisions": 0,
"1001028.ifinoctets": 8.97161617E7,
"1001028.ifadminstatus": "up",
"1001032.ifinbroadcastpkts": 338.39392728725534,
"1001032.prometheus_9": 9.925713023946667E8,
"1001014.arm_dstaspath": 3.6598104761438474E7,
"1001032.prometheus_8": 1.6967779336312674E7,
"24001.ifoutucastpkts": 3945302.748625687,
"1001104.prometheus_24": 166222.9569503179,
"27001.ifinerrors": 0,
"1001014.arm_bytes": 5.633695192167859E8,
"1001014.ifindex": "1001014",
"1001014.ifindiscards": 0,
"1001028.flowgraph-pair": 26867.361670006627,
"1001029.ifinbroadcastpkts": 0,
"1001028.arm_dstaspath": 2748302.587485982,
"1001104.ifinerrors": 0,
"1001104.arm_srcas": 5.112489531820186E8,
"1001104.ifinmulticastpkts": 0.09977052778609198,
"1001028.ifinunknownprotos": 0,
"1001052.ifinunknownprotos": 0,
"1001032.arm_dstas": 1.0012613019925371E8,
"1001032.ifoututilization": 1.9238298617658807,
"1001029.prometheus_8": 1.9618134731423803E7,
"1001029.prometheus_9": 7.643961272781603E8,
"1001014.ifinmulticastpkts": 0,
"1001104.ifoutdiscards": 0,
"1001029.prometheus_5": 32.10642954145778,
"1001052.ifindiscards": 0,
"1001032.ifoutbroadcastpkts": 0.6991610067918498,
"1001029.ifinunknownprotos": 0,
"1001029.prometheus_1": 2840.8276923232515,
"24001.ifoutbroadcastpkts": 0,
"1001029.ifdirection": "full-duplex",
"27001.eth_sqetesterrors": 0,
"24001.arm_srcas": 7.75961332457563E8,
"24001.ifoutpkts": 3945302.748625687,
"1001014.flowgraph-pair": 47302.29964176728,
"1001032.ifoperstatus": "up",
"1001028.ifdirection": "full-duplex",
"24001.arm_dstpeer": 2.766321784545277E8,
"1001104.flowgraph-pair": 70034.1639348969,
"1001014.arm_dstas": 3.6598104761438474E7,
"1001052.ifinutilization": 13.3860554192,
"1001104.ifindiscards": 0,
"24001.eth_latecollisions": 0,
"1001104.ifoutpkts": 1631866.60680435,
"24001.prometheus_8": 1.9653329170323733E7,
"24001.prometheus_9": 3.136866296340246E9,
"24001.ifinucastpkts": 4817530.934532734,
"27001.ifoutmulticastpkts": 0,
"27001.ifoutucastpkts": 0.09995002498750626,
"1001028.ifouterrors": 0,
"1001029.arm_dstpeer": 2.239511529955379E8,
"1001104.ifoututilization": 8.098693505736806,
"1001028.iftype": "ieee8023adLag",
"1001029.ifouterrors": 0,
"1001028.arm_dstpeer": 2748302.587485982,
"1001029.ifindex": "1001029",
"1001104.arm_dstas": 2.234021957935419E8,
"1001032.ifadminstatus": "up",
"1001029.ifoutbroadcastpkts": 0,
"1001032.arm_dstaspath": 1.0012613019925371E8,
"1001052.ifinoctets": 1.6732569273999999E9,
"1001032.ifinoctets": 1.1917562108469837E9,
"24001.eth_multiplecollisions": 0,
"27001.eth_deferredtx": 0,
"1001028.ifinutilization": 0.7177292936,
"1001104.ifoutmulticastpkts": 0,
"1001052.ifinmulticastpkts": 121.80000000000001,
"1001029.ifinoctets": 7.758358333833084E8,
"1001052.ifoutoctets": 2.8283535103E9,
"1001014.ifoutbroadcastpkts": 0,
"1001029.arm_dstaspath": 2.239511529955379E8,
"27001.ifouterrors": 0,
"1001014.ifoutpkts": 279821.4428457234,
"27001.ifinutilization": 6.556721639180409E-7,
"24001.ifinutilization": 23.000227823688153,
"1001028.ifoututilization": 0.6783317064,
"1001052.prometheus_1": 2196.438882678829,
"1001014.ifinutilization": 2.485298708233413,
"1001028.prometheus_12": 1.2668087226865657E8,
"24001.arm_dstaspath": 2.766321784545277E8,
"1001104.ifinpkts": 2328176.0949815423,
"1001014.ifinerrors": 0,
"1001104.prometheus_1": 2374.613670718827,
"1001014.ifoutoctets": 1.3935600099920064E8,
"1001014.arm_srcas": 2.9273486799976754E8,
"1001104.prometheus_8": 2.6456008652777843E7,
"1001104.prometheus_9": 1.293214755431997E9,
"1001052.ifoututilization": 22.6268280824,
"1001028.ifindiscards": 0,
"1001032.ifinunknownprotos": 0,
"1001104.prometheus_5": 461792.39462930395,
"1001029.ifindiscards": 0,
"24001.eth_deferredtx": 0,
"1001014.prometheus_12": 4.819254680831177E8,
"1001028.prometheus_24": 144.6890312919255,
"1001032.ifinucastpkts": 1948377.846584099,
"1001032.flowgraph-pair": 59525.1682980244,
"24001.ifindex": "24001",
"24001.ifinmulticastpkts": 0,
"24001.iftype": "ethernetCsmacd",
"1001029.prometheus_28": 4.4790194232373273E8,
"1001032.prometheus_1": 3403.1090182806197,
"1001032.prometheus_5": 676802.2707205323,
"1001029.prometheus_24": 221203.49926168352,
"24001.ifoutmulticastpkts": 0,
"24001.prometheus_27": 2.1602168498648386E-13,
"1001014.ifoutdiscards": 0,
"24001.prometheus_24": 255361.01844273822,
"1001052.ifindex": "1001052",
"1001104.arm_bytes": 2.248789469171914E9,
"24001.ifoutdiscards": 0,
"24001.prometheus_29": 220.6051219954789,
"1001014.ifinbroadcastpkts": 0,
"1001052.ifspeed": 100000000000,
"24001.prometheus_28": 8.39733552237979E8,
"1001032.arm_dstpeer": 1.0012613019925371E8,
"24001.ifspeed": 100000000000,
"1001014.prometheus_24": 45712.55710820156,
"1001104.ifinoctets": 1.2278773064950614E9,
"1001014.ifoututilization": 1.1148480079936052,
"1001029.prometheus_18": 1.0428210772062423,
"24001.prometheus_12": 2.9769024932843075E9,
"1001104.ifadminstatus": "up",
"1001028.prometheus_28": 4.082748353412452E7,
"24001.eth_alignmenterrors": 0,
"1001028.ifoutdiscards": 0,
"24001.prometheus_18": 594616.3324231681,
"1001052.ifdirection": "full-duplex"
}
-> curl -sX GET "http://localhost:8008/metric/{notWorkingIP}/json" -H 'accept: */*'
{
"23001.agg_attachedaggid": "1001014",
"23001.ifdirection": "full-duplex",
"23001.agg_actorsystemid": "B492FE7B7963",
"1001009.arm_srcas": 2134241.3003626335,
"1000011.prometheus_9": 310361.1739900747,
"1000011.prometheus_8": 310361.1739900747,
"23001.agg_actoradminstate": "00000000",
"1000011.prometheus_12": 310361.1739900747,
"1001009.prometheus_12": 9849570.901089504,
"1001009.flowgraph-pair": 19761.493521876233,
"1001028.arm_bytes": 8951956.565409934,
"23001.ifspeed": 100000000000,
"23001.agg_partneroperstate": "00111111",
"1001009.arm_bytes": 8.403767118673168E7,
"23001.ifadminstatus": "up",
"1000011.flowgraph-pair": 19225.588868304796,
"23001.ifindex": "23001",
"23001.iftype": "ethernetCsmacd",
"1001028.prometheus_12": 1206068.8097364784,
"23001.agg_actoroperstate": "00111101",
"1000011.prometheus_28": 310361.1739900747,
"23001.agg_partneradminstate": "00000000",
"23001.agg_partneropersystemid": "D4EB68D47145",
"1001028.flowgraph-pair": 19761.493521876233,
"1001009.prometheus_28": 4052178.2084302874,
"1001028.prometheus_28": 1206068.8097364784,
"23001.ifoperstatus": "up",
"1001028.prometheus_8": 1206068.8097364784,
"1001028.prometheus_9": 1206068.8097364784,
"1000011.arm_bytes": 2249393.8975916617,
"1001009.prometheus_9": 1.1417546372496428E7,
"1001009.prometheus_8": 4052178.2084302874
}
{
"1001029.ifoutdiscards": 0,
"1001028.ifinerrors": 0,
"1001032.ifspeed": 100000000000,
"1001032.ifoutoctets": 2.404787327207351E8,
"1001029.prometheus_12": 1.7502649410866306E9,
"1001104.ifoutucastpkts": 1631866.60680435,
"1001029.ifoututilization": 8.202547930834584,
"24001.ifinunknownprotos": 0,
"1001032.ifinmulticastpkts": 231.02277267279266,
"1001014.ifoperstatus": "up",
"27001.eth_alignmenterrors": 0,
"27001.ifoutbroadcastpkts": 0,
"1001028.arm_bytes": 2.2630598928033912E8,
"1001032.ifinerrors": 0,
"1001029.arm_dstas": 2.239511529955379E8,
"1001052.arm_bytes": 4.494110158223985E9,
"27001.eth_carriersenseerrors": 0,
"1001014.prometheus_28": 3.2217111564166945E8,
"1001052.ifoperstatus": "up",
"1001029.ifinutilization": 6.206686667066467,
"1001052.ifoutdiscards": 0,
"1001028.ifinmulticastpkts": 0,
"1001014.iftype": "ieee8023adLag",
"27001.ifoutdiscards": 0,
"1001029.ifoutoctets": 1.0253184913543229E9,
"1001028.ifoutoctets": 8.47914633E7,
"1001052.ifinerrors": 0,
"24001.flowgraph-pair": 89639.77066553601,
"1001029.iftype": "ieee8023adLag",
"1001029.ifinmulticastpkts": 0.09995002498750626,
"27001.eth_latecollisions": 0,
"1001014.ifspeed": 100000000000,
"24001.ifindiscards": 0,
"1001032.ifinutilization": 9.53404968677587,
"1001104.ifoperstatus": "up",
"1001104.ifinbroadcastpkts": 0,
"1001028.arm_dstas": 2748302.587485982,
"1001029.ifinucastpkts": 1867789.7051474263,
"1001104.arm_dstpeer": 2.234021957935419E8,
"27001.ifindiscards": 0,
"1001014.ddos_protect_tcp_flood": 2.05786441158796E-6,
"1001028.arm_srcas": 2.289233500490104E7,
"1001104.iftype": "ieee8023adLag",
"1001104.ifinunknownprotos": 0,
"1001104.ifouterrors": 0,
"1001028.ifoutucastpkts": 177816.5,
"24001.ifinbroadcastpkts": 0,
"1001052.ifadminstatus": "up",
"1001052.prometheus_28": 1.517154982664847E9,
"1001014.ifinoctets": 3.1066233852917665E8,
"1001029.ifinerrors": 0,
"1001032.ifinpkts": 1948947.263284059,
"24001.arm_dstas": 2.766321784545277E8,
"1001052.prometheus_24": 162773.55139098343,
"1001032.arm_srcas": 6.552027917708361E8,
"24001.ifinerrors": 0,
"1001032.ifoutucastpkts": 377590.4914103076,
"27001.eth_internalmacrxerrors": 0,
"1001052.prometheus_5": 608632.6003069924,
"1001014.ifinunknownprotos": 0,
"1001104.ifoutbroadcastpkts": 0,
"27001.ifinunknownprotos": 0,
"1001052.prometheus_8": 2.2619949913002174E7,
"1001052.prometheus_9": 3.2803474920990605E9,
"1001052.arm_dstas": 1.3590992122387356E8,
"1001029.ifspeed": 100000000000,
"24001.eth_singlecollisions": 0,
"1001029.ifinpkts": 1867789.8050974514,
"1001032.ifoutpkts": 377591.29045145825,
"27001.ifinucastpkts": 0.09995002498750626,
"24001.eth_internalmactxerrors": 0,
"1001028.ifoutbroadcastpkts": 0,
"1001052.prometheus_12": 3.4198234805599065E9,
"1001029.arm_bytes": 2.1039952415661578E9,
"24001.ifdirection": "full-duplex",
"1001014.ddos_protect_ip_flood": 2.05786441158796E-6,
"1001029.flowgraph-pair": 68672.5897304701,
"1001032.ifdirection": "full-duplex",
"24001.ifoutoctets": 1.7100662788605697E9,
"1001029.ifoutpkts": 1660047.0764617692,
"1001052.ifoutmulticastpkts": 0,
"1001104.ifinucastpkts": 2328175.9952110145,
"24001.eth_sqetesterrors": 0,
"1001028.ifoperstatus": "up",
"1001052.ifoutucastpkts": 4989444.4,
"1001029.ifadminstatus": "up",
"1001032.ifindiscards": 0,
"1001014.ifinucastpkts": 389521.8824940048,
"1001052.ifoutpkts": 4989445.5,
"1001014.prometheus_8": 1.0401234111108901E7,
"24001.prometheus_5": 638482.8314757305,
"1001014.prometheus_9": 3.266339770795697E8,
"1001014.prometheus_5": 199.57605731690742,
"24001.prometheus_1": 3351.3277436209987,
"1001052.flowgraph-pair": 99504.92737321918,
"1001052.ifinpkts": 3043419,
"1001014.prometheus_1": 653.0365301171651,
"27001.ifoutpkts": 0.09995002498750626,
"1001052.arm_dstaspath": 1.3590992122387356E8,
"1001032.prometheus_12": 1.2636259617753892E9,
"1001029.ifoutucastpkts": 1660047.0764617692,
"1001104.ifdirection": "full-duplex",
"24001.eth_carriersenseerrors": 0,
"1001028.ifspeed": 100000000000,
"1001052.ifouterrors": 0,
"1001052.ifinbroadcastpkts": 110.10000000000001,
"24001.ifinoctets": 2.8750284779610195E9,
"27001.eth_symbolerrors": 0,
"24001.ifouterrors": 0,
"1001032.iftype": "ieee8023adLag",
"1001028.ifinucastpkts": 288332.7,
"1001032.prometheus_24": 238217.6312796434,
"24001.eth_excessivecollisions": 0,
"24001.eth_toolongs": 0,
"1001032.prometheus_28": 7.869731597316215E8,
"1001032.ifoutdiscards": 0,
"1001052.arm_srcas": 9.734401849866718E8,
"1001028.ifindex": "1001028",
"27001.eth_toolongs": 0,
"1001028.ifoutpkts": 177817.5,
"1001052.ifoutbroadcastpkts": 1.1,
"27001.ifinoctets": 8.195902048975512,
"24001.ifadminstatus": "up",
"1001014.ifoutucastpkts": 279820.4436450839,
"1001104.arm_dstaspath": 2.234021957935419E8,
"1001104.prometheus_12": 1.8031002089612768E9,
"1001014.ifadminstatus": "up",
"1001029.ifoperstatus": "up",
"1001032.ifoutmulticastpkts": 0.0998801438274071,
"27001.ifspeed": 10000000000,
"1001104.ifinutilization": 9.823018451960492,
"27001.ifoututilization": 7.756121939030484E-7,
"1001014.ifoutmulticastpkts": 0.9992006394884093,
"27001.eth_fcserrors": 0,
"1001014.ifinpkts": 389521.8824940048,
"27001.ifindex": "27001",
"24001.eth_symbolerrors": 0,
"27001.iftype": "ethernetCsmacd",
"27001.ifadminstatus": "up",
"1001052.arm_dstpeer": 1.3590992122387356E8,
"1001032.ifindex": "1001032",
"27001.eth_singlecollisions": 0,
"27001.eth_internalmactxerrors": 0,
"1001104.ifspeed": 100000000000,
"1001014.arm_dstpeer": 3.6598104761438474E7,
"1001052.ifinucastpkts": 3043187.1,
"1001014.ifouterrors": 0,
"27001.ifinbroadcastpkts": 0,
"1001029.arm_srcas": 3.951635808022792E8,
"1001028.prometheus_1": 2.0669861613132214,
"1001032.arm_bytes": 1.4660513111372392E9,
"24001.eth_fcserrors": 0,
"1001028.prometheus_5": 106547.60496541545,
"24001.eth_internalmacrxerrors": 0,
"1001104.ifindex": "1001104",
"27001.ifoutoctets": 9.695152423788105,
"1001028.prometheus_8": 5010460.782084496,
"24001.arm_bytes": 4.302376807677212E9,
"1001028.prometheus_9": 1.1640109226897946E8,
"1001029.ifoutmulticastpkts": 0,
"1001028.ifoutmulticastpkts": 1,
"27001.ifinmulticastpkts": 0,
"1001028.ifinpkts": 288332.7,
"27001.eth_multiplecollisions": 0,
"1001014.ifdirection": "full-duplex",
"1001028.ifinbroadcastpkts": 0,
"27001.ifdirection": "full-duplex",
"24001.ifoututilization": 13.680530230884559,
"1001104.prometheus_28": 7.624217338714464E8,
"1001104.prometheus_27": 4.469658923745612,
"27001.ifinpkts": 0.09995002498750626,
"27001.ifoperstatus": "up",
"24001.ifoperstatus": "up",
"1001104.ifoutoctets": 1.0123366882171007E9,
"24001.ifinpkts": 4817530.934532734,
"1001052.iftype": "ieee8023adLag",
"1001032.ifouterrors": 0,
"27001.eth_excessivecollisions": 0,
"1001028.ifinoctets": 8.97161617E7,
"1001028.ifadminstatus": "up",
"1001032.ifinbroadcastpkts": 338.39392728725534,
"1001032.prometheus_9": 9.925713023946667E8,
"1001014.arm_dstaspath": 3.6598104761438474E7,
"1001032.prometheus_8": 1.6967779336312674E7,
"24001.ifoutucastpkts": 3945302.748625687,
"1001104.prometheus_24": 166222.9569503179,
"27001.ifinerrors": 0,
"1001014.arm_bytes": 5.633695192167859E8,
"1001014.ifindex": "1001014",
"1001014.ifindiscards": 0,
"1001028.flowgraph-pair": 26867.361670006627,
"1001029.ifinbroadcastpkts": 0,
"1001028.arm_dstaspath": 2748302.587485982,
"1001104.ifinerrors": 0,
"1001104.arm_srcas": 5.112489531820186E8,
"1001104.ifinmulticastpkts": 0.09977052778609198,
"1001028.ifinunknownprotos": 0,
"1001052.ifinunknownprotos": 0,
"1001032.arm_dstas": 1.0012613019925371E8,
"1001032.ifoututilization": 1.9238298617658807,
"1001029.prometheus_8": 1.9618134731423803E7,
"1001029.prometheus_9": 7.643961272781603E8,
"1001014.ifinmulticastpkts": 0,
"1001104.ifoutdiscards": 0,
"1001029.prometheus_5": 32.10642954145778,
"1001052.ifindiscards": 0,
"1001032.ifoutbroadcastpkts": 0.6991610067918498,
"1001029.ifinunknownprotos": 0,
"1001029.prometheus_1": 2840.8276923232515,
"24001.ifoutbroadcastpkts": 0,
"1001029.ifdirection": "full-duplex",
"27001.eth_sqetesterrors": 0,
"24001.arm_srcas": 7.75961332457563E8,
"24001.ifoutpkts": 3945302.748625687,
"1001014.flowgraph-pair": 47302.29964176728,
"1001032.ifoperstatus": "up",
"1001028.ifdirection": "full-duplex",
"24001.arm_dstpeer": 2.766321784545277E8,
"1001104.flowgraph-pair": 70034.1639348969,
"1001014.arm_dstas": 3.6598104761438474E7,
"1001052.ifinutilization": 13.3860554192,
"1001104.ifindiscards": 0,
"24001.eth_latecollisions": 0,
"1001104.ifoutpkts": 1631866.60680435,
"24001.prometheus_8": 1.9653329170323733E7,
"24001.prometheus_9": 3.136866296340246E9,
"24001.ifinucastpkts": 4817530.934532734,
"27001.ifoutmulticastpkts": 0,
"27001.ifoutucastpkts": 0.09995002498750626,
"1001028.ifouterrors": 0,
"1001029.arm_dstpeer": 2.239511529955379E8,
"1001104.ifoututilization": 8.098693505736806,
"1001028.iftype": "ieee8023adLag",
"1001029.ifouterrors": 0,
"1001028.arm_dstpeer": 2748302.587485982,
"1001029.ifindex": "1001029",
"1001104.arm_dstas": 2.234021957935419E8,
"1001032.ifadminstatus": "up",
"1001029.ifoutbroadcastpkts": 0,
"1001032.arm_dstaspath": 1.0012613019925371E8,
"1001052.ifinoctets": 1.6732569273999999E9,
"1001032.ifinoctets": 1.1917562108469837E9,
"24001.eth_multiplecollisions": 0,
"27001.eth_deferredtx": 0,
"1001028.ifinutilization": 0.7177292936,
"1001104.ifoutmulticastpkts": 0,
"1001052.ifinmulticastpkts": 121.80000000000001,
"1001029.ifinoctets": 7.758358333833084E8,
"1001052.ifoutoctets": 2.8283535103E9,
"1001014.ifoutbroadcastpkts": 0,
"1001029.arm_dstaspath": 2.239511529955379E8,
"27001.ifouterrors": 0,
"1001014.ifoutpkts": 279821.4428457234,
"27001.ifinutilization": 6.556721639180409E-7,
"24001.ifinutilization": 23.000227823688153,
"1001028.ifoututilization": 0.6783317064,
"1001052.prometheus_1": 2196.438882678829,
"1001014.ifinutilization": 2.485298708233413,
"1001028.prometheus_12": 1.2668087226865657E8,
"24001.arm_dstaspath": 2.766321784545277E8,
"1001104.ifinpkts": 2328176.0949815423,
"1001014.ifinerrors": 0,
"1001104.prometheus_1": 2374.613670718827,
"1001014.ifoutoctets": 1.3935600099920064E8,
"1001014.arm_srcas": 2.9273486799976754E8,
"1001104.prometheus_8": 2.6456008652777843E7,
"1001104.prometheus_9": 1.293214755431997E9,
"1001052.ifoututilization": 22.6268280824,
"1001028.ifindiscards": 0,
"1001032.ifinunknownprotos": 0,
"1001104.prometheus_5": 461792.39462930395,
"1001029.ifindiscards": 0,
"24001.eth_deferredtx": 0,
"1001014.prometheus_12": 4.819254680831177E8,
"1001028.prometheus_24": 144.6890312919255,
"1001032.ifinucastpkts": 1948377.846584099,
"1001032.flowgraph-pair": 59525.1682980244,
"24001.ifindex": "24001",
"24001.ifinmulticastpkts": 0,
"24001.iftype": "ethernetCsmacd",
"1001029.prometheus_28": 4.4790194232373273E8,
"1001032.prometheus_1": 3403.1090182806197,
"1001032.prometheus_5": 676802.2707205323,
"1001029.prometheus_24": 221203.49926168352,
"24001.ifoutmulticastpkts": 0,
"24001.prometheus_27": 2.1602168498648386E-13,
"1001014.ifoutdiscards": 0,
"24001.prometheus_24": 255361.01844273822,
"1001052.ifindex": "1001052",
"1001104.arm_bytes": 2.248789469171914E9,
"24001.ifoutdiscards": 0,
"24001.prometheus_29": 220.6051219954789,
"1001014.ifinbroadcastpkts": 0,
"1001052.ifspeed": 100000000000,
"24001.prometheus_28": 8.39733552237979E8,
"1001032.arm_dstpeer": 1.0012613019925371E8,
"24001.ifspeed": 100000000000,
"1001014.prometheus_24": 45712.55710820156,
"1001104.ifinoctets": 1.2278773064950614E9,
"1001014.ifoututilization": 1.1148480079936052,
"1001029.prometheus_18": 1.0428210772062423,
"24001.prometheus_12": 2.9769024932843075E9,
"1001104.ifadminstatus": "up",
"1001028.prometheus_28": 4.082748353412452E7,
"24001.eth_alignmenterrors": 0,
"1001028.ifoutdiscards": 0,
"24001.prometheus_18": 594616.3324231681,
"1001052.ifdirection": "full-duplex"
}
-> curl -sX GET "http://localhost:8008/metric/{notWorkingIP}/json" -H 'accept: */*'
{
"23001.agg_attachedaggid": "1001014",
"23001.ifdirection": "full-duplex",
"23001.agg_actorsystemid": "B492FE7B7963",
"1001009.arm_srcas": 2134241.3003626335,
"1000011.prometheus_9": 310361.1739900747,
"1000011.prometheus_8": 310361.1739900747,
"23001.agg_actoradminstate": "00000000",
"1000011.prometheus_12": 310361.1739900747,
"1001009.prometheus_12": 9849570.901089504,
"1001009.flowgraph-pair": 19761.493521876233,
"1001028.arm_bytes": 8951956.565409934,
"23001.ifspeed": 100000000000,
"23001.agg_partneroperstate": "00111111",
"1001009.arm_bytes": 8.403767118673168E7,
"23001.ifadminstatus": "up",
"1000011.flowgraph-pair": 19225.588868304796,
"23001.ifindex": "23001",
"23001.iftype": "ethernetCsmacd",
"1001028.prometheus_12": 1206068.8097364784,
"23001.agg_actoroperstate": "00111101",
"1000011.prometheus_28": 310361.1739900747,
"23001.agg_partneradminstate": "00000000",
"23001.agg_partneropersystemid": "D4EB68D47145",
"1001028.flowgraph-pair": 19761.493521876233,
"1001009.prometheus_28": 4052178.2084302874,
"1001028.prometheus_28": 1206068.8097364784,
"23001.ifoperstatus": "up",
"1001028.prometheus_8": 1206068.8097364784,
"1001028.prometheus_9": 1206068.8097364784,
"1000011.arm_bytes": 2249393.8975916617,
"1001009.prometheus_9": 1.1417546372496428E7,
"1001009.prometheus_8": 4052178.2084302874
}
Neil McKee
Mar 19, 2026, 2:46:19 PM (4 days ago) Mar 19
to Stefano R, sFlow-RT
One way to extract the relevant sFlow fields so we can check that the sequence numbers are implemented correctly is to stop sFlow-RT and run this:
docker run -p 6343:6343/udp sflow/sflowtool -L "datagramSourceIP,agent,agentSubId,packetSequenceNo,sampleType,sourceId,sampleSequenceNo"
or, if you have compiled sflowtool (https://github.com/sflow/sflowtool) it is just this:
sflowtool -L "datagramSourceIP,agent,agentSubId,packetSequenceNo,sampleType,sourceId,sampleSequenceNo"
You should see output like this:
172.17.0.1,10.0.0.117,100000,701290,COUNTERSSAMPLE,3:140917,25931
10.0.0.30,10.0.0.117,100000,701290,COUNTERSSAMPLE,3:140917,25931
10.0.0.30,10.0.0.252,0,4943354,FLOWSAMPLE,0:10,2672239
10.0.0.30,10.0.0.252,0,4943354,FLOWSAMPLE,0:10,2672240
10.0.0.30,10.0.0.252,0,4943354,FLOWSAMPLE,0:10,2672241
10.0.0.30,10.0.0.30,100000,2455656,DISCARD,0:0,10981484
10.0.0.30,10.0.0.117,100000,701290,COUNTERSSAMPLE,3:140917,25931
10.0.0.30,10.0.0.252,0,4943354,FLOWSAMPLE,0:10,2672239
10.0.0.30,10.0.0.252,0,4943354,FLOWSAMPLE,0:10,2672240
10.0.0.30,10.0.0.252,0,4943354,FLOWSAMPLE,0:10,2672241
10.0.0.30,10.0.0.30,100000,2455656,DISCARD,0:0,10981484
So we have pulled out just the fields that matter for checking the sequence numbers.
Notes:
(1) The datagramSourceIP is irrelevant and will be ignored. I included it here just to make that point(!)
(2) The packetSequenceNo should increment monotonically for unique combinations of agent + agentSubId
(3) The sampleSequenceNo should increment monotonically for unique combinations of agent + agentSubId + sampleType + sourceId
Your two agents could have the same issue, but the traffic patterns may be exposing the issue on one agent more than on the other. For example, if a run of packet samples are all on the same interface and there is nothing in between, then it may look as though the packetSequenceNo and sampleSequenceNo are both incrementing correctly, at least for a little while. So sFlow-RT might allow those samples to be processed.
Can you tell from this whether the agent implementation is incrementing the sequence numbers correctly?
------
Neil McKee
InMon Corp.
Neil McKee
InMon Corp.
--
You received this message because you are subscribed to the Google Groups "sFlow-RT" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sflow-rt+u...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/sflow-rt/53b0388f-0b21-492e-b859-6d5fe808b3b2n%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages