We're having an issue with an NTPD whereby it's defaulting (or whatever the correct terminology is) to the LOCAL clock, this is occurring when one of our servers loses connectivity. We have 4 server's setup and the local clock is also configured:
server 127.127.1.0 # local clock
fudge 127.127.1.0 stratum 10
ntpq -p output:
remote refid st t when poll reach delay offset jitter
==============================================================================
hostname .INIT. 16 u - 1024 0 0.000 0.000 0.000
hostname .INIT. 16 u - 1024 0 0.000 0.000 0.000
hostname .INIT. 16 u - 1024 0 0.000 0.000 0.000
hostname.INIT. 16 u - 1024 0 0.000 0.000 0.000
*LOCAL(0) .LOCL. 10 l 9 64 377 0.000 0.000 0.001
The issue with this is that once it defaults to the LOCAL, it doesn't sync with an external source again, until we manually restart ntpd. I'm sure this is something simple, but I'm hoping someone can assist.
Thanks,
Stephen
This email and any attachments may contain confidential and proprietary information of Blackboard that is for the sole use of the intended recipient. If you are not the intended recipient, disclosure, copying, re-distribution or other use of any of this information is strictly prohibited. Please immediately notify the sender and delete this transmission if you received this email in error.
Patient: Doctor, it hurts when I do this!
Doctor: Then don't do that. Next patient!
Why do you have LOCAL in your ntp.conf? It is almost always a REALLY bad
idea because it leaves the clock free-running.
It is oft discussed on this list why so many software distributions
include LOCAL in the default ntp.conf. They really, really should stop
doing it and so should you.
The real question is why you are not getting to any of the named servers
in ntp.conf.
--
R. Kevin Oberman, Network Engineer
Energy Sciences Network (ESnet)
Ernest O. Lawrence Berkeley National Laboratory (Berkeley Lab)
E-mail: obe...@es.net Phone: +1 510 486-8634
Key fingerprint:059B 2DDF 031C 9BA3 14A4 EADA 927D EBB3 987B 3751
I'm not sure why we have it in there, as you say it might have been part of the default config in Redhat's EL ntp package. And yep, I don't understand why it's reverting to the LOCAL clock and then just deciding to stick with it until we restart ntp.
Cheers,
Stephen
This email and any attachments may contain confidential and proprietary information of Blackboard that is for the sole use of the intended recipient. If you are not the intended recipient, disclosure, copying, re-distribution or other use of any of this information is strictly prohibited. Please immediately notify the sender and delete this transmission if you received this email in error.
Reach = 0, not getting responses from "hostname" servers?
Poll = 1024, it will try again in 17 minutes?
>> The issue with this is that once it defaults to the LOCAL, it doesn't
>> sync with an external source again, until we manually restart
>> ntpd. I'm sure this is something simple, but I'm hoping someone can
>> assist.
What else does your NTP Conf contain? What version of NTP? Which OS?
My guess would be that NTP is not seeing any answers
from the "hostname" servers.
Is DNS up when NTPd starts?
{I think I recall two or three people who said they had
a issue on boot; due to dynamic IPs? ... and DNS not
resolving the host names for far too long after NTP
was started?}
Do you see the NTP query packets going out?
Do you see NTP response packets coming back?
If restarting NTP solves the issue the following
are unlikely to be a cause of the issue:
Perhaps NTP conf restrict issues?
What version of NTP? (restrict source nomodify ?)
Are the "hostname" servers on your LAN? internet?
Access policies? / Firewall?
Something else has port 123 open already?
--
E-Mail Sent to this address <Blac...@Anitech-Systems.com>
will be added to the BlackLists.
Until about a year ago I had always included the lines in
my ntp.conf. I've never noticed any resulting disasters.
server 127.127.1.0
fudge 127.127.1.0 stratum 10
Since update to 4.2.6p2 I've changed to using tos orphan.
What happens without the lines for local clock?
Did you use ntpdate or ntpq -g -q to set the clock before
starting ntpd?
What is the frequency?
I've found that bad things happen at much above 50ppm.
Just now the seven systems here have:
1 2 3 4 5 6 7
offset(us) 92 715 3 51 53 399 525
frequency(ppm) 0.8 -48.5 -36.2 -18.2 -10.4 9.6 0.4
Offsets vary a lot (maybe +/- 500u) except (3) which is
from GPS but frequency usually varies less than 1ppm
David
I have lots of systems that work fine at well over 50 ppm.
With a good PPS, the drift will track temperature and you
can use the system as a thermometer.
--
These are my opinions, not necessarily my employer's. I hate spam.
Get rid of the local clock.
There is no reason for it.
>
> server 127.127.1.0 # local clock
> fudge 127.127.1.0 stratum 10
>
> ntpq -p output:
>
> remote refid st t when poll reach delay offset jitter
>==============================================================================
> hostname .INIT. 16 u - 1024 0 0.000 0.000 0.000
> hostname .INIT. 16 u - 1024 0 0.000 0.000 0.000
> hostname .INIT. 16 u - 1024 0 0.000 0.000 0.000
> hostname.INIT. 16 u - 1024 0 0.000 0.000 0.000
> *LOCAL(0) .LOCL. 10 l 9 64 377 0.000 0.000 0.001
Sinced it cannot get at any of the other servers, it needs to use Local
and the only one it can contact-- totally uselessly, but it c an contact
it.
>
> The issue with this is that once it defaults to the LOCAL, it doesn't sync with an external source again, until we manually restart ntpd. I'm sure this is something simple, but I'm hoping someone can assist.
No, your external sources are not reachable. Fix that problem. Firewall,
network, whatever.
Is anyone able to shed some further light on this issue? I plan to remove the LOCAL clock from ntp's configuration, but I'm still keen to know why it's defaulting to the LOCAL clock once network connectivity is down, and then ignoring any of the public NTP servers configured when network connectivity returns. It stays locked to the LOCAL clock for weeks until we actually restart it manually.
Cheers,
Stephen
-----Original Message-----
From: Kevin Oberman [mailto:obe...@es.net]
Sent: Wednesday, November 03, 2010 3:00 PM
To: Stephen Vaughan
Cc: ques...@lists.ntp.org
Subject: Re: Local clock - sync issue
This email and any attachments may contain confidential and proprietary information of Blackboard that is for the sole use of the intended recipient. If you are not the intended recipient, disclosure, copying, re-distribution or other use of any of this information is strictly prohibited. Please immediately notify the sender and delete this transmission if you received this email in error.
Your billboard output shows that ntpd isn't able to reach any of the
servers you have listed in the configuration file. See if you can ping
them, and check your firewall settings.
Cheers,
David
I have no explanation, but one thing I am not clear about is whether the
other serves start responding after network connectivity is restored or
whether they respond but local remains the preferred peer.
Once the network is restored, does 'ntpq -p SERVER' work?
http://www.novell.com/coolsolutions/feature/15345.html
Cheers,
Stephen
-----Original Message-----
From: questions-bounces+stephen.vaughan=blackbo...@lists.ntp.org [mailto:questions-bounces+stephen.vaughan=blackbo...@lists.ntp.org] On Behalf Of David J Taylor
Sent: Monday, November 08, 2010 3:01 PM
To: ques...@lists.ntp.org
Subject: Re: Local clock - sync issue
Cheers,
David
_______________________________________________
questions mailing list
ques...@lists.ntp.org
http://lists.ntp.org/listinfo/questions
Cheers,
Stephen
This email and any attachments may contain confidential and proprietary information of Blackboard that is for the sole use of the intended recipient. If you are not the intended recipient, disclosure, copying, re-distribution or other use of any of this information is strictly prohibited. Please immediately notify the sender and delete this transmission if you received this email in error.
This advises some very anti-social practice, namely using burst on
public servers. This is likely to get you banned from those servers.
Also, it doesn't explain why you have zero reachability.
Getting the detailed diagnostics, by using rv on the association
numbers, may give a better clue.
I do have some concern that a local clock could result in all the others
being false tickers, because the error band on the local clock is too
small, but ntpd is supposed to discriminate against the local clock. In
any case, false tickers still have a non-zero reachability.
Actually, one possibility, is that the error statistics on the remote
servers are wrong, resulting in their confidence intervals not
overlapping. That doesn't really work for the Novell example, as they
only have one server. Unfortunately, they've used ntpdc peers, rather
than nptq peers, so one can't really tell how the servers are being treated.
However, in all cases, the basic problem is using a local clock driver
for no valid reason.
> Yep it does work, I think that article I just posted is the fix.
That should go in the Wiki.
--
Steve Kostecke <kost...@ntp.org>
NTP Public Services Project - http://support.ntp.org/
> Yep it does work, I think that article I just posted is the fix.
Actually, that belongs in the Wiki as an example of what _not_ to do.
"burst" makes your ntpd poll the remote time server 8 times in rapid
sucession at each poll interval. Use of 'burst' against time servers
which you do not control (or don't have explicit permission from) is
considered to be unfriendly.
If you must use 'burst' to maintain sync then you have some network
issues which need to be addressed.
Cheers,
Stephen
-----Original Message-----
From: questions-bounces+stephen.vaughan=blackbo...@lists.ntp.org [mailto:questions-bounces+stephen.vaughan=blackbo...@lists.ntp.org] On Behalf Of Steve Kostecke
Sent: Monday, November 08, 2010 6:20 PM
To: ques...@lists.ntp.org
Subject: Re: Local clock - sync issue
_______________________________________________
questions mailing list
ques...@lists.ntp.org
http://lists.ntp.org/listinfo/questions
This email and any attachments may contain confidential and proprietary information of Blackboard that is for the sole use of the intended recipient. If you are not the intended recipient, disclosure, copying, re-distribution or other use of any of this information is strictly prohibited. Please immediately notify the sender and delete this transmission if you received this email in error.
If I recall correctly his problem was that he had 4 outside sources plus
the local time. Then something disconnected him from those outside
sources for a long time ( hours) After that was fixed, the system never
again went back to those outside sources-- gave zero reachability even
after they were reconnected, as if ntpd gave up on them after it could
not reach them for a while.
> If I recall correctly his problem was that he had 4 outside sources plus
> the local time. Then something disconnected him from those outside
> sources for a long time ( hours) After that was fixed, the system never
> again went back to those outside sources-- gave zero reachability even
> after they were reconnected, as if ntpd gave up on them after it could
> not reach them for a while.
>
It had fallen back to a 1024 second poll interval, so it might be as
simple as he failed to wait until a successful poll was made. He was
possibly expecting it to find the servers within tens of seconds.
One could possibly argue that a response from another server, after the
local clock became selected, ought to force the poll interval down.
I don't know if ntpd handles the poll interval differently when there is
a local clock selected as against when there is no source, but it should
probably not.
That is possible. Only he can tell us how long he waited.
>
> One could possibly argue that a response from another server, after
> the
> local clock became selected, ought to force the poll interval down.
I think that ntp degrades the poll interval if there is not response, to
prevent the situation where ntpd suddenly floods a server that has gone
down. I think it used to decrease the poll interval which resulted in a
server coming up being bombarded by thousands of ntp requests/sec,
because
everyone had decreased their poll interval.
Thanks, Stephen. I don't like that solution, as it imposes more load than
necessary on the remote servers. What version of NTP are you running -
the output from:
ntpq -c rv
David
Network connectivity goes down, ntp reverts to LOCAL clock, then if you look at the logs and check ntpq, it remains bound to the LOCAL clock, several weeks after the connectivity went down. A simple restart of ntpd brings the external clocks back online.
Cheers,
Stephen
-----Original Message-----
From: questions-bounces+stephen.vaughan=blackbo...@lists.ntp.org [mailto:questions-bounces+stephen.vaughan=blackbo...@lists.ntp.org] On Behalf Of unruh
Sent: Tuesday, November 09, 2010 3:36 AM
To: ques...@lists.ntp.org
Subject: Re: Local clock - sync issue
_______________________________________________
Perhaps he meant "IBURST"? That's certain allowable, particularly if he
does restarts at long (weeks or months) intervals. Given that NTPD
needs up to ten hours to get a tight lock on the correct time, most
users will want to minimize restarts.
> Perhaps he meant "IBURST"? That's certain allowable, particularly if he
> does restarts at long (weeks or months) intervals. Given that NTPD
> needs up to ten hours to get a tight lock on the correct time, most
> users will want to minimize restarts.
>
If you read the article, you would have seen that it explicitly said use
both.
Do you, by any chance, have an unstable IP address? Some ISPs generate
these. One theories is that it is to break attempts to run servers.
What "article" are you talking about? If it was posted here, could you
tell me the date/time stamp on the message?
I don't recall seeing any reference to the use of *both* BURST and IBURST.
IBURST causes NTPD to send eight request packets at intervals of two
seconds. This fills NTPD's "filter pipeline" and is a "one time"
operation.
BURST mode sends packets at two second intervals in perpetuity. ISTR
that BURST mode was intended for use with dialup connections and should
never be used otherwise.
(Shrug)
<http://groups.google.com/group/comp.protocols.time.ntp/msg/ac4e5ac7682ba4a7>
From: Stephen Vaughan <Stephen...@blackboard.com>
Newsgroups: comp.protocols.time.ntp
Subject: Re: Local clock - sync issue
Date: Mon, 8 Nov 2010 22:24:10 GMT
Message-ID: <F415B494EA44D6478B23E...@AZEX07.bbbb.net>
See Link: <http://www.novell.com/coolsolutions/feature/15345.html>
<BlockQuote>
The solution is to make NTP more persistent when probing
the remote time server.
This is done by appending the flags burst and iburst to
the remote server;
burst tells NTP to send a burst of eight packets to the
remote server instead of one when the server is reachable,
and iburst tells it to do the same when the server is not
reachable.
The result is faster and more reliable synchronizations.
</BlockQuote>
Stephen Vaughan would likely benefit from "iburst";
However if Stephen Vaughan turns on "burst",
his NTP clients may be treated as a source of abuse.
Cheers,
Stephen
-----Original Message-----
From: questions-bounces+stephen.vaughan=blackbo...@lists.ntp.org [mailto:questions-bounces+stephen.vaughan=blackbo...@lists.ntp.org] On Behalf Of David J Taylor
Sent: Tuesday, November 09, 2010 5:21 AM
To: ques...@lists.ntp.org
Subject: Re: Local clock - sync issue
David
_______________________________________________
Cheers,
Stephen
-----Original Message-----
From: questions-bounces+stephen.vaughan=blackbo...@lists.ntp.org [mailto:questions-bounces+stephen.vaughan=blackbo...@lists.ntp.org] On Behalf Of unruh
Sent: Tuesday, November 09, 2010 1:43 AM
To: ques...@lists.ntp.org
Subject: Re: Local clock - sync issue
_______________________________________________
Cheers,
Stephen
-----Original Message-----
From: questions-bounces+stephen.vaughan=blackbo...@lists.ntp.org [mailto:questions-bounces+stephen.vaughan=blackbo...@lists.ntp.org] On Behalf Of David Woolley
Sent: Tuesday, November 09, 2010 5:13 PM
To: ques...@lists.ntp.org
Subject: Re: Local clock - sync issue
_______________________________________________
That's a little behind the current version, Stephen - 4.2.6p2. See:
http://www.ntp.org/downloads.html
Whether there has been anything fixed which might affect your issue I
don't know.
ntpd 4.2.2p1 may date back to July 2006, if my understanding of the
following page is correct:
http://www.eecis.udel.edu/~ntp/ntp_spool/ntp4/ntp-4.2/
Cheers,
David
Cheers,
Stephen
-----Original Message-----
From: Ukos...@ntp.org [mailto:Ukos...@ntp.org] On Behalf Of Steve Kostecke
Sent: Wednesday, November 10, 2010 11:13 AM
To: Stephen Vaughan
Subject: Re: Local clock - sync issue
In comp.protocols.time.ntp, you wrote:
> version="ntpd 4.2...@1.1570-o Thu May 14 13:01:41 UTC 2009 (1)",
When you compare the versions of the release you're using and the
current stable release (4.2.2p1 vs 4.2.6p2) you might think there's not
much difference. But 4.2.2* is actually 2 development cycles behind
4.2.6* We only provide patches and updates for the current stable
release (4.2.6*) and the current ntp-dev (4.2.7*).
4.2.2p1 looks like v4.2.2.1 but is really Protocol 4 v2.2.1
4.2.6p2 looks like v4.2.6.2 but is really Protocol 4 v2.6.2
--
Steve Kostecke <kost...@ntp.org>
NTP Public Services Project - http://support.ntp.org/
This email and any attachments may contain confidential and proprietary information of Blackboard that is for the sole use of the intended recipient. If you are not the intended recipient, disclosure, copying, re-distribution or other use of any of this information is strictly prohibited. Please immediately notify the sender and delete this transmission if you received this email in error.
> That's a little behind the current version, Stephen - 4.2.6p2. See:
>
> http://www.ntp.org/downloads.html
The current release versions are widely published:
http://www.ntp.org (in the side bar)
http://www.ntp.org/downloads.html
http://support.ntp.org (in the side bar)
http://support.ntp.org/download
http://bugs.ntp.org (near the bottom of the home-page)
http://support.ntp.org/rss/releases.xml (RSS) feed
http://freshmeat.net/projects/ntp/
http://www.facebook.com/pages/Network-Time-Protocol/193495410535
Please see http://support.ntp.org/bin/view/Main/ReleaseNotifications for
a comprehensive list of the distribution channels for our Release
Notifications.
> Whether there has been anything fixed which might affect your issue I
> don't know.
>
> ntpd 4.2.2p1 may date back to July 2006, if my understanding of the
> following page is correct:
>
> http://www.eecis.udel.edu/~ntp/ntp_spool/ntp4/ntp-4.2/
Yes. The 4.2.2p1 distribution tarball is dated:
ntp-4.2.2p1.tar.gz 08-Jul-2006 07:24 2.4M
I don't know, Stephen, but a lot has been fixed in over four years of
development.
(Thanks for the pointers to the version dates.)
Cheers,
David
If anybody has updates or corrections, please let me know.
H
Cheers,
Stephen
-----Original Message-----
From: questions-bounces+stephen.vaughan=blackbo...@lists.ntp.org [mailto:questions-bounces+stephen.vaughan=blackbo...@lists.ntp.org] On Behalf Of David J Taylor
Sent: Wednesday, November 10, 2010 3:32 PM
To: ques...@lists.ntp.org
Subject: Re: Local clock - sync issue
Cheers,
David
_______________________________________________
questions mailing list
ques...@lists.ntp.org
http://lists.ntp.org/listinfo/questions
This email and any attachments may contain confidential and proprietary information of Blackboard that is for the sole use of the intended recipient. If you are not the intended recipient, disclosure, copying, re-distribution or other use of any of this information is strictly prohibited. Please immediately notify the sender and delete this transmission if you received this email in error.
It may be, if you are using your more than four year old
version 4.2.2p1 circa Jul of 2006,
and the issue was fixed by 4.2.4p8 circa 2009.
> version="ntpd 4.2...@1.1570-o Thu May 14 13:01:41 UTC 2009 (1)",
I think there have been some more recent changes
to 4.2.7 (& 4.2.6 ?) {related to pool server use?}
that might improve recovery of transient unreachability.
Although I'd guess 4.2.4 stable or later might resolve
your issue, why not try the latest stable, or even
a more recent 4.2.7 dev that fixes many other issues.
<http://archive.ntp.org/ntp4/ntp-4.2/ntp-4.2.6p2.tar.gz>
<http://support.ntp.org/bin/view/Main/SoftwareDownloads>
<http://ntp.org/downloads.html>
Certainly worth a try.
IIRC there a a few people here who run NTP on laptops,
as well as others on dynamic IP networks, that don't
always have connectivity that have found workarounds
for older versions of NTP.
Some flavor of monitoring app, that queries NTP
and check the status of peer associations,
restarting NTP if necessary.
e.g. parsing the output from "ntpq -c as"
or perhaps "ntpq -p"
Ref: Bug 622 (dynamic-if) Target Milestone: 4.2.4
<Block Quote>
Functionality:
- automatically bind to newly configured network interfaces
- allow associations to be configured even when the
current network configuration would not support that
- keep (configured) associations even when the interface
goes away
- remove hard coded limit on number of interfaces
(now only limited by libisc limits)
- check for interface changes via routing socket
on systems that support it.
- add ntpdc commands ifstats (display current interface list),
ifreload (re-check interfaces)
...
to be integrated in ntp-dev after release of 4.2.2
See also: Bugs 51, 244, 310, 314, 330, 549
(Shrug)
You may want to consider REHL4 & REHL5 recommendations
to update to 4.2.4p8:
CVE-2009-3563 RHSA-2009:1648 Severity M Fixed 20091208
ntp_request.c in ntpd in NTP before 4.2.4p8, and 4.2.5,
allows remote attackers to cause a denial of service
(CPU and bandwidth consumption) by using MODE_PRIVATE
to send a spoofed (1) request or (2) response packet
that triggers a continuous exchange of MODE_PRIVATE
error responses between two NTP daemons.
Mentioned in RHSA-2009-1651 for RHEL3 also.
(4.2.4p8) Which would also cover:
CVE-2009-0159 RHSA-2009:1039 Severity L Fixed 20090518
Stack-based buffer overflow in the cookedprint function
in ntpq/ntpq.c in ntpq in NTP before 4.2.4p7-RC2
allows remote NTP servers to execute arbitrary code
CVE-2009-1252 RHSA-2009:1039 Severity I Fixed 20090129
Stack-based buffer overflow in the crypto_recv function
in ntp_crypto.c in ntpd in NTP before 4.2.4p7 and 4.2.5
before 4.2.5p74, when OpenSSL and autokey are enabled,
allows remote attackers to execute arbitrary code via
a crafted packet containing an extension field.
CVE-2009-0021 RHSA-2009:0046 Severity M Fixed 20090518
NTP 4.2.4 before 4.2.4p5 and 4.2.5 before 4.2.5p150 does
not properly check the return value from the OpenSSL
EVP_VerifyFinal function, which allows remote attackers
to bypass validation of the certificate chain via a
malformed SSL/TLS signature for DSA and ECDSA keys,
a similar vulnerability to CVE-2008-5077.
... and issues even older than those.
I'd suggest that you are getting a new address each time the connection
recovers, but ntpd will have bound to the old addresses.
Incidentally, one of the changes since 2006 is to re-resolve external
servers occasionally. That might just rebind the local address.
The IP concept is based on addresses being constant.
Really???
I get a new address, potentially, every time I boot.
If IPv6 ever gets off the ground, that may change.
> I get a new address, potentially, every time I boot.
Properly configured, DHCP will give you back your old address, although
ntpd will also restart in that case.
NAT is a workround for a limited address space; it's a bolt on for
getting round limited address space, although even then, it is the
router and external address that should change, not the address of the
workstation.
The original intent of IP was to provide a stable, global, address
space, relatively independent of local delivery considerations.
Yes, if you enter an IP address of a timeserver into the config, you are indicating that such timeserver is not going to move until you change the config again. If you enter a DNS hostname into the config, it should be paying attention to the DNS TTL and periodically re-resolving it to see whether the A/CNAME record changed to a new IP (clamping the TTL to a sane range, perhaps, if needed).
> I get a new address, potentially, every time I boot.
That's perfectly OK as a client. However, someone in a dynamic IP pool typically shouldn't be providing / advertising NTP services....
Regards,
--
-Chuck
Cheers,
Stephen
-----Original Message-----
From: questions-bounces+stephen.vaughan=blackbo...@lists.ntp.org [mailto:questions-bounces+stephen.vaughan=blackbo...@lists.ntp.org] On Behalf Of E-Mail Sent to this address will be added to the BlackLists
Sent: Wednesday, November 10, 2010 5:02 PM
To: ques...@lists.ntp.org
Subject: Re: Local clock - sync issue
(Shrug)
_______________________________________________