consistent time offset of 5ms

[Thomas Hamer]

Hey Team,

I am monitoring time for several well trusted stratum 1, 2, and 3 NTP services, but I have observed that the google time servers at 216.239.35.0, and 216.239.35.8 are consistently out of zero by around +5ms. .4, and .12 are seemingly fine.

my other upstream time sources are national measurements institute, Apple, Cloudflare, and, AWS Time Sync, and some internal gps clocks. I can organise a screenshot if thats interesting, but corp vpn does not permit at this time. 

is there any reason for this offset, or is this actually in error?

[Edward McGuire]

On Tuesday, June 11, 2024 at 8:42:50 AM UTC-5 Thomas Hamer wrote:
> I am monitoring time for several well trusted stratum 1, 2, and 3
> NTP services, but I have observed that the google time servers at
> 216.239.35.0, and 216.239.35.8 are consistently out of zero by
> around +5ms. .4, and .12 are seemingly fine.

From my place I see
216.239.35.0 .GOOG. 1 u 28 64 7 7.084 +0.054 0.045
216.239.35.4 .GOOG. 1 u 29 64 7 24.203 -4.344 0.202
216.239.35.8 .GOOG. 1 u 25 64 7 24.534 -4.243 0.091
216.239.35.12 .GOOG. 1 u 26 64 7 1.916 +0.020 0.059

I can't explain the seeming disagreement. I also can't explain seeming disagreements between NIST clocks as measured from my place, that appeared after the February AT&T outage and are still ongoing.

The NTP manual says synchronization distance is based on one-half the delay. Naively, I wonder if that means NTP assumes the request packet trip time and the response packet trip time are symmetric, and mis-estimates the offset when they are asymmetric. Someone who understands the clock filtering protocol can correct me on this.

[Thomas Hamer]

[Thomas Hamer]

seems like this got resolved. Thanks.

[Dave Hart]

On Tuesday, June 11, 2024 at 3:25:19 PM UTC Edward McGuire wrote:

The NTP manual says synchronization distance is based on one-half the delay. Naively, I wonder if that means NTP assumes the request packet trip time and the response packet trip time are symmetric, and mis-estimates the offset when they are asymmetric. Someone who understands the clock filtering protocol can correct me on this.

It's true that NTP assumes the network path is symmetric and asymmetry results in clock estimation error.  It would be lovely to know the asymmetry but there's no way given the way the protocol works. 

[Edward McGuire]

I made a study of the actual asymmetry I'm seeing with NIST clocks, outside the NTP protocol. The enclosed chart is the result.

What's going on here is, an external program compares each peer offset to the System Peer offset. For the purpose of the study, whatever difference it finds is assumed to be caused by asymmetry, because these are NIST clocks. The external program then eliminates asymmetry, by introducing either outbound delay or inbound delay.

For example, consider a circuit where the outbound is 10 ms and the inbound is 20 ms. NTP calculates the round trip as 30 ms, half of the round trip as 15 ms, "discovers" that the peer timestamp taken at T+10ms is 15ms - 10ms = 5 ms fast, and excludes it from the cluster.

Now the external program introduces 2 x 5 ms = 10 ms outbound delay. This eliminates the asymmetry and the peer joins the cluster. But from time to time, a peer's asymmetry changes abruptly. So the program is run every couple of hours, recalculates, and rebalances.

You can see on the chart that the peers are not in lock step. Single peers are going wrong at random times. This is not what you would expect if the asymmetry is due to a simple problem either at my end or at their end. So the cause of the observed assymetry has to be somewhere between.

[Edward McGuire]

Late last week, the time offsets between NIST clocks went away. Whatever causes them has apparently gone quiet for now.

[Thomas Hamer]

The time offsets for the google clocks is back again. Anyone at google we can talk to about that? We are not dependent on this time, but we use it as a reference in the context of other clocks for validity. 

Im seeing that the 216.239.35.8 server is reporting times that are about 5ms ahead of real time. This started at 7pm AEST on the 30th of may.