Oncore leap second bug at 11/29/2003 62:28:15
Tom Van Baak
bug showed up as predicted by Motorola. It was
a little different than we thought; this is how an
Oncore VP GPS receiver reported time yesterday:
11/27/2003 23:59:58
11/27/2003 23:59:59
11/29/2003 62:28:15
11/28/2003 00:00:00
11/28/2003 00:00:01
If any of you saw NTP anomalies I'd be interested
to hear about them. Presumably the invalid hour
saved the day (the time stamp would be rejected).
For details see:
http://www.leapsecond.com/notes/leapsec256.htm
Tapio Sokura
> Sure enough, the Oncore 256 week leap second
> bug showed up as predicted by Motorola. It was
> a little different than we thought; this is how an
> Oncore VP GPS receiver reported time yesterday:
>
> 11/27/2003 23:59:58
> 11/27/2003 23:59:59
> 11/29/2003 62:28:15
> 11/28/2003 00:00:00
> 11/28/2003 00:00:01
>
> If any of you saw NTP anomalies I'd be interested
> to hear about them. Presumably the invalid hour
> saved the day (the time stamp would be rejected).
I'm running an Oncore VP (6 channels) with version 8.4 firmware and ntpd
4.1...@1.791 in Linux with nanokernel (2.2.19). My ntpd clockstats file
had these entries in it at the change:
(last line of 2003-11-27)
52970 86399.100 127.127.30.1 3278966398.999990798 2003 331 23 59 59 58
rstat 08 dop 0.0 nsat 7,2 traim 2 sigma 65535 neg-sawtooth -13 sat
5000051470
(first line of 2003-11-28)
52971 1.100 127.127.30.1 3278966400.999997669 2003 332 0 0 0 0 rstat
08 dop 0.0 nsat 7,2 traim 2 sigma 65535 neg-sawtooth -15 sat 5000052100
Compared to normal it looks like one timestamp was completely left out
of the log (the zero second stamp). loopstats shows a jump in offset
just after the UTC day change from the regular +10 us to about -2 ms for
a while:
52970 86324.106 0.000014667 -264.099197 0.000000821 0.005368 4
52970 86340.099 0.000012934 -264.099197 0.000001389 0.004648 4
52970 86356.100 0.000010806 -264.099197 0.000003048 0.004026 4
52970 86373.100 0.000012878 -264.099197 0.000001656 0.003486 4
52970 86390.107 0.000012850 -264.099197 0.000003494 0.003019 4
52971 7.099 -0.000662972 -264.099197 0.000002537 0.002615 8
52971 36.310 -0.002153100 -264.108124 0.000001896 0.005005 8
52971 477.106 0.000026050 -264.149780 0.000003183 0.021274 4
52971 495.101 0.000027275 -264.149780 0.000000662 0.018424 4
52971 511.106 0.000026562 -264.149780 0.000001356 0.015956 4
I guess ntpd lost the ball for a while because there are no loopstats
lines between seconds 36 and 477. This might also have something to do
with the Oncore losing satellites; according to clockstats it was locked
to zero satellites for about 200 seconds starting six seconds after the
day change. Oncore losing the satellites every now and then is normal in
my setup because the antenna is placed inside the house.
Nothing was output to the system log when this happened. I wasn't around
when this event actually took place, just browsed through the logs
afterwards.
Jens Rosenboom
Our Stratacom GPS clocks also seem to have been affected, but not
now, but already 4 weeks earlier! On 11/01/2003 00:00:00 they started
setting the "leap second ahead" indicator in the time string output,
which in itself would have been too bad.
The bad thing was, that this action seems to have taken so much of
their available CPU power, that the rate at which the timestrings
(which were still being output once per second) were updated to the
correct time, dropped from once per second to about twice per minute.
The obvious result being ntpd loosing sync on the GPS as prefer peer
and disabling PPS, which otherwise would have continued correctly.
We have four receivers, all did exactly the same thing, and they got
back to normal only after power cycling them.
Hal Murray
>now, but already 4 weeks earlier! On 11/01/2003 00:00:00 they started
>setting the "leap second ahead" indicator in the time string output,
>which in itself would have been too bad.
Thanks for the report.
My old HP Z3801A wasn't working as well as I expected/remembered,
but I hadn't made time to work on it. (I was thinking I had
fatfingered something when upgrading to 4.2.0.)
Looks OK now.
--
The suespammers.org mail server is located in California. So are all my
other mailboxes. Please do not send unsolicited bulk e-mail or unsolicited
commercial e-mail to my suespammers.org address or any of my other addresses.
These are my opinions, not necessarily my employer's. I hate spam.