Invalid a.MinRTT value for NDT5 measurements?

[Maxime Mouchet]

Hi everyone,

I noticed that the values of a.MinRTT for NDT5 measurements seem too low.

For example 45.237.70.122 is an IP behind a satellite link that can be pinged at around ~500 ms from Sao Paulo.

Looking at the raw data it seems that a.MinRTT is equal to raw.S2C.TCPInfo.MinRTT divided by 1,000,000, which gives an (impossible) value of 0.599ms:

SELECT date, raw.ClientIP, a.MinRTT, raw.S2C.MinRTT, raw.S2C.TCPInfo.MinRTT

FROM `measurement-lab`.ndt.ndt5

WHERE date = '2023-11-01' AND raw.ClientIP = '45.237.70.122'

[
   {
      "date":"2023-11-01",
      "ClientIP":"45.237.70.122",
      "MinRTT":"0.599",
      "MinRTT_1":"716000000",
      "MinRTT_2":"599000"
   }
]

However we get much more likely values if we divide by 1000 instead. Here are the quantiles on the whole dataset for a single day:

SELECT

  APPROX_QUANTILES(a.MinRTT, 10) a_min_rtt,

  APPROX_QUANTILES(raw.S2C.TCPInfo.MinRTT / 1000, 10) raw_min_rtt

FROM `measurement-lab`.ndt.ndt5

WHERE date = '2023-01-01'

[
   {
      "a_min_rtt":[
         "-1.0",
         "-1.0",
         "-1.0",
         "-1.0",
         "-1.0",
         "0.0010680000000000002",
         "0.008627000000000001",
         "0.019338",
         "0.041866999999999994",
         "0.11881699999999999",
         "4294.967295"
      ],
      "raw_min_rtt":[
         "0.07",
         "5.294",
         "8.57",
         "12.981",
         "19.206",
         "27.687",
         "41.666",
         "79.015",
         "119.876",
         "143.79",
         "4294967.295"
      ]
   }
]

The values divided by 1000 look more plausible than the a.MinRTT value ( raw.S2C.TCPInfo.MinRTT / 1,000,000).

Best,

Maxime

[Stephen Soltesz]

Thank you, Maxime, I'll take a closer look this week.

This was the last associated issue related to MinRTT & ndt5: https://github.com/m-lab/etl/issues/1094 

I'll update that issue.

--
You received this message because you are subscribed to the Google Groups "discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss+u...@measurementlab.net.
To view this discussion on the web visit https://groups.google.com/a/measurementlab.net/d/msgid/discuss/52c10142-95e7-4828-8f97-2bf219c239b4n%40measurementlab.net.

[Matt Mathis]

Performance Enhancing Proxies sometimes do local ACKing which confuses the measurements.    PEPs are particularly popular on satellite links and other long paths that might be the concatenation of two subpaths with different properties.   This can make a huge performance difference for example if there are high loss path concatenated with a high jitter path.

See https://datatracker.ietf.org/doc/html/rfc3135 for a survey of techniques and their implications.

To view this discussion on the web visit https://groups.google.com/a/measurementlab.net/d/msgid/discuss/CAOd-A3T49wfOKLnXMUBLYai7kFCd96Q9avakHt4W_F0nOayo_A%40mail.gmail.com.

--

Thanks,

--MM--

Evil is defined by mortals who think they know "The Truth" and use force to apply it to others. 

[Maxime Mouchet]

Thanks for the link @Matt, that’s a good point.

However I think the issue applies to all IPs, as evidenced by the quantiles.

90% of the a.MinRTT values are below 0.11ms.

In comparison the “fixed” values have p50=27ms and p90=143ms which looks much more reasonable (though it might be pure luck :-)).

Here is another example with a non-satellite IP in Thailand, where a.MinRTT gives an impossible 0.07ms from Hong Kong, whereas the fixed value would give 70ms:

SELECT date, server.Site, raw.ClientIP, a.MinRTT, raw.S2C.MinRTT, raw.S2C.TCPInfo.MinRTT

FROM `measurement-lab`.ndt.ndt5

WHERE date = '2023-11-01' AND raw.ClientIP = '1.10.132.195'

[{

"date": "2023-11-01",

"Site": "hkg02",

"ClientIP": "1.10.132.195",

"MinRTT": "0.073052",

"MinRTT_1": "74000000",

"MinRTT_2": "73052"

}]

[Phillipa Gill]

If I recall correctly, the NDT5/NDT7 raw tables are in microseconds (so dividing by 1000 would get you milliseconds). 

The unit is also not consistent between these tables and the unified downloads tables. It'd be great if this was documented more clearly somewhere. 

To view this discussion on the web visit https://groups.google.com/a/measurementlab.net/d/msgid/discuss/5EB0A46D-EF02-4CD5-BCA9-7E8955334A67%40ipinfo.io.

[Stephen Soltesz]

Maxime, thank you for reporting this. There was a bug, due likely to the unfortunate combination of MinRTT values and units in the ndt5 dataset. There is data represented in milliseconds, microseconds, and nanoseconds in the archived data and BigQuery tables. The summary a.MinRTT value should all be in milliseconds now. The historical ndt5 data has been reprocessed with this fix; can you confirm things appear to be WAI now?

Phillipa, I've updated the available documentation for the https://www.measurementlab.net/tests/ndt/ndt5/ https://www.measurementlab.net/tests/ndt/ndt7/ schema descriptions with the correct units on the MinRTT values (milli, micro, or nano seconds). These pages are generated from schema descriptions here: https://github.com/m-lab/etl/tree/main/schema/descriptions. Updates or suggestions are always welcome; just create a PR or issue. We do not have a means of generating schema documentation for view schemas today (e.g. unified_downlaods). However, the unified tables use the summary records (a.*) directly.

[Maxime Mouchet]

LGTM, thanks a lot @Stephen!

SELECT

  APPROX_QUANTILES(a.MinRTT, 10) a_min_rtt,

  APPROX_QUANTILES(raw.S2C.TCPInfo.MinRTT / 1000, 10) raw_min_rtt

FROM `measurement-lab`.ndt.ndt5

WHERE date = '2023-01-01'

AND a.MinRTT > 0

[
   {
      "a_min_rtt":[
         "0.07",
         "5.266",
         "8.5",
         "12.882",
         "19.249",
         "27.959",
         "41.538",
         "79.808",
         "119.917",
         "143.491",
         "4294967.295"
      ],
      "raw_min_rtt":[
         "0.07",
         "5.266",
         "8.5",
         "12.882",
         "19.249",
         "27.959",
         "41.538",
         "79.808",
         "119.917",
         "143.491",
         "4294967.295"
      ]
   }
]