[P4-discuss] Huge fluctuations in calculated link latency information

[Bibek Shrestha]

Hi,

I am trying to calculate the link latency between the switches. Below
is my bare minimal topology.

(h1)------(s1)------(s2)-----(h2)

Link latency is calculated in p4 as,
link_latency = s2.standard_metadata.ingress_timestamp -
s1.standard_metadata.egress_timestamp

I am using the bmv2 switch and made a change in the code to sync the
time with the system time as suggested. My setup is all done in the
mininet.

My expectation was that the latency would mostly be stable around some
values with some jitter. But what I am seeing is huge fluctuations in
the link latency whenever there is a change in queue occupancy in the
switches. If the switches are congested then the link latencies would
go up by a very high number.

Is my expectation of having relatively stable link latency with some
jitter correct in terms of how real networks work? If so, why am I
seeing such behavior with the bmv2 switch? Or is my way of calculating
the link latency is flawed? In that case, how should I calculate the
link latency?

Thank you

[John Grant]

ingress_timestamp is defined as "Time that packet began processing in
IngressParser" which I guess would include time waiting for previous
packets to be parsed; anyway it definitely wouldn't be the time the PHY
received the packet. What you're seeing suggests the parser isn't able
to work at line speed.

--
John Grant
Nine Tiles, Cambridge, England
+44 1223 862599 and +44 1223 511455
http://www.ninetiles.com

> _______________________________________________
> P4-discuss mailing list
> P4-di...@lists.p4.org
> http://lists.p4.org/mailman/listinfo/p4-discuss_lists.p4.org

[John Grant]

It's reasonable in this situation where the incoming packets are being
inspected by software and "ingress" means getting to the front of the
queue for inspection (a bit like queueing for immigration at an
airport). It wouldn't be reasonable if "ingress" meant the packet
arriving at the PHY in the case of a point-to-point wire.

J.

On 31/10/2020 20:29, Bibek Shrestha wrote:
> Does that mean the huge latency fluctuations that I am seeing is reasonable?

[Andy Fingerhut]

I think it might be good if there was an article written about the pitfalls of trying to use mininet and bmv2 simple_switch for performance simulations, so that we could point people at it whenever they ask questions like this.

bmv2 simple_switch has several properties in how it queues things internally, and schedules packets for ingress and egress processing, that are not good fits for how a switch ASIC does those things in real time.

Similarly, mininet links with maximum bit rates configured on them do not behave the same as constant bit rate physical Ethernet links.  They allow packets to be enqueued arbitrarily quickly, and then have a FIFO queue with packet buffer "inside" of them that only lets them reach the other end of the link at the configured maximum bit rate.  Physical Ethernet links do not behave that way -- they only allow packets to be transmitted on them at the maximum bit rate, and any queues before them must be in the device that is transmitting on the link.

Andy