Local NTP master
Andy Gray
I have a question about use of a local master to keep an NTP hierarchy
going in the event of loss of contact with an authoritative time
source.
We have 6 symmetrically peered Cisco routers at the core of our
network which act as NTP servers to the rest of it. They themselves
get time from two firewalls, which get time from a stratum 2 server at
an ISP. Normally this works really well, we have something like 450
routers synched.
Unfortunately we've had some problems with the firewalls and overnight
they've stopped serving NTP. The time on the 6 core routers has
eventually become non-authoritative, and they can therefore no longer
give time to the other 400+ routers. Everything ends up with a reach
value of zero.
My question is, can I configure the 6 core routers as NTP masters in
such a way that the authoritative time source from the ISP is normally
the reference, but if access to that fails they remain authoritative
time servers themselves such that our network remains synchronised,
even if not to absolute UTC?
Any thoughts most welcome
Cheers
Andy Gray
David Schwartz
> My question is, can I configure the 6 core routers as NTP masters in
> such a way that the authoritative time source from the ISP is normally
> the reference, but if access to that fails they remain authoritative
> time servers themselves such that our network remains synchronised,
> even if not to absolute UTC?
ntp master 8
ntp update-calendar
clock calendar-valid
DS
Jerome Limozin
what you need is to configure one of the 6 routers with 1 more time
source : itself.
The base ntp distribution contains a driver to synchronise to the host
clock. I don't know if this driver is supported on cisco's version of
ntp.
Also, make sure that all 6 routers peer to each other. i.e ntp.conf one
these 6 routers should look like :
server fw1
server fw2
peer router1
peer router2
peer router3
peer router4
peer router5
choose one router, and add following lines to ntp.conf:
server 127.127.1.0
fudge 127.127.1.0 stratum 10
This instructs the router to synchronise to local clock, with
stratum=10. in normal operation, the router will be synchronised to
higher stratum time source (ISP), and will not use the local time
source. If the stratum 2 time source disappears for some reason, the
router will start to sync to local clock. all other routers will
synchronise to it.
This is not too good, as all routers will drift together with the
undisciplined clock of your chosen router. If the stratum 2 time source
is off the air for some time, your whole network will have a slightly
wrong time.
Then, when the ISP time source comes back, all clients will suddenly
jump time to the correct one! (depending on ntp configuration..).. That
may cause undesirable effects.
A better option would be to purchase a local stratum-1 time source, a
gps based one for example. I myself using a truetime appliance, which
was no too expensive and extremely easy to configure. Now I have
extremely stable time source and I do not depend on an ISP or Internet.
Rob MacGregor
NTP isn't designed to work this way - check the white papers on the Sun
site. If you need to always have authoritative time then purchase a radio
clock/GPS NTP unit as backup.
--
Rob MacGregor (MCSE) [PGP key ID 0x1F5239DD]
The light at the end of the tunnel is an oncoming dragon.
Question intelligently:
http://www.tuxedo.org/~esr/faqs/smart-questions.html
Andy Gray
>
> NTP isn't designed to work this way - check the white papers on the Sun
> site. If you need to always have authoritative time then purchase a radio
> clock/GPS NTP unit as backup.
Well, thank-you all for your replies, though they are slightly
contradictory! I searched the Sun site and didn't find anything
useful, unfortunately. I agree the use of an internal clock as a
backup isn't ideal, but it's much cheaper than a GPS NTP server (been
trying to pursuade my colleagues that we should get a couple of those,
but I don't think we've got the budget)
I had imagined, perhaps naively, that in the event of loss of contact
with an authoritative source then our mesh of 6 routers would keep
reasonably accurate time as an aggregate of their own reasonably
disciplined clocks. I hadn't anticipated a situation where they all
gave up trusting each other completely.
It seems to me, that though NTP isn't really designed with this in
mind, it does work! The local high stratum master is ignored when a
good time source is available but becomes a reference if contact is
lost. All our strata shift about a bit as a result, but internal
synchronisation is maintained, which is more important than running at
absolute UTC. Agreed things could get a little ugly when authoritative
time source is restored.
Cheers
Andy
David L. Mills
Happiness = documentation -> reference clock drivers -> local clock
driver. This is the hard way. The easy way is to read the faq at
www.ntp.org.
Dave
Devon Caines
1) to peer with each other
2) as stratum 5 servers such that when they loose connection with the
rest of the network they use their internal clocks as a time reference..
Devon_C
Andy Gray
> Andy,
>
> Happiness = documentation -> reference clock drivers -> local clock
> driver. This is the hard way. The easy way is to read the faq at
> www.ntp.org.
>
> Dave
>
Dave
Sometimes it's like trying to get a useful answer out of Tony Blair at
Prime Ministers Question Time... "I refer the honourable gentleman to
the answer I gave just a few moments ago," he says, almost regardless
of the question asked or the answer recently given.
Whilst I agree it's important to refer people to the excellent
resources at www.ntp.org, I do think it's only really fair to do that
if it does actually provide a useful answer to the question asked.
www.ntp.org and the FAQ are my first port of call with any query, I
wouldn't have been able to achieve what I have with NTP without them,
but they don't, as far as I can tell, actually answer my question.
Unless, that is, you count Q. "6.1.2.1. Can I use my system clock as
reference clock?" A."In short: You can, but you should not." I could
take that answer without thinking, but I was trying to be creative and
maintain the credibility of NTP here, such that it keeps working
sufficiently well, even when the firewalls do not.
Cheers
Andy
Ronald Thornton
I assume by "use their internal clocks" you mean to add the LOCAL
clock to the ntp.conf. There appear to be a couple of problems here.
If the local network becomes isolated, the servers will quickly shift
to LOCAL rather than coasting at their current stratum level using the
latest drift number. It seems that you would really like to update
LOCAL fudge time2 with the current ntp.drift (or the latest number
inside ntpd) before you start using it as reference.
Second, if all the servers list LOCAL as stratum 5 (or whatever) won't
they all choose their LOCAL over their peers? Maybe you should
stagger the LOCAL stratum values, giving the most stable time server
the higher stratum.
If your isolated network can't have the correct time, you at least
want to minimize the drift and keep everyone together.
I don't think this level of graceful degradation during network
isolation is really addressed in the FAQ.
-Ron-
On Thu, 01 Nov 2001 11:53:49 -0600, Devon Caines <dev...@inetnebr.com>
wrote:
Per Hedeland
<Thor...@Aerostation.org> writes:
>
>If the local network becomes isolated, the servers will quickly shift
>to LOCAL rather than coasting at their current stratum level using the
>latest drift number. It seems that you would really like to update
>LOCAL fudge time2 with the current ntp.drift (or the latest number
>inside ntpd) before you start using it as reference.
Good thinking, and the ntpd authors already thought it - ntpd does this
automatically.
>Second, if all the servers list LOCAL as stratum 5 (or whatever) won't
>they all choose their LOCAL over their peers? Maybe you should
>stagger the LOCAL stratum values, giving the most stable time server
>the higher stratum.
Absolutely, and note that the values need to differ by 2 to have the
desired effect. Of course you should do this even if you can't be
bothered to determine which server is "most stable", for most people
having them agree is the most important thing.
--Per Hedeland
p...@bluetail.com
Ronald Thornton
Hedeland) wrote:
>In article <mfn2ut44u2tclicg5...@4ax.com> Ronald Thornton
><Thor...@Aerostation.org> writes:
>>
>>If the local network becomes isolated, the servers will quickly shift
>>to LOCAL rather than coasting at their current stratum level using the
>>latest drift number. It seems that you would really like to update
>>LOCAL fudge time2 with the current ntp.drift (or the latest number
>>inside ntpd) before you start using it as reference.
>
>Good thinking, and the ntpd authors already thought it - ntpd does this
>automatically.
I see "calibrate" code that updates fudgetime1 values of ref clocks
that are not the current reference, but I couldn't see where the
fudgetime2 value in LOCAL gets updated automatically. It seems like
the current drift_comp will head toward zero (or what ever might be in
LOCAL fudgetime2) when LOCAL becomes the selected reference.
>>Second, if all the servers list LOCAL as stratum 5 (or whatever) won't
>>they all choose their LOCAL over their peers? Maybe you should
>>stagger the LOCAL stratum values, giving the most stable time server
>>the higher stratum.
>
>Absolutely, and note that the values need to differ by 2 to have the
>desired effect. Of course you should do this even if you can't be
>bothered to determine which server is "most stable", for most people
>having them agree is the most important thing.
Good points, thanks.
>--Per Hedeland
>p...@bluetail.com
Mark Martinec
> I have a question about use of a local master to keep an NTP hierarchy
> going in the event of loss of contact with an authoritative time source.
I'll let somebody else answer this (aren't you just asking
how to specify stratum for a local clock? - which is a FAQ)
but on a sidenote:
> Unfortunately we've had some problems with the firewalls and overnight
> they've stopped serving NTP. The time on the 6 core routers has
> eventually become non-authoritative,
This reminds me of two machines at our institute where ntpd (4.1.x)
died last weekend when daylight saving time period ended.
One was Linux, the other a HPUX (details intentionally left out).
In both cases it turned out there was some kind of fight between
broken time library or environment variable settings, and the
real time clock. A reboot only made things worse, letting RTC
set the time wrong by 1 hour, and ntpd voluntarily dying
as the time offset was greater that 1000 seconds.
I know, pilot error, ntp did its job correctly, but worth
looking at your logs for the last weekend, and thinking againg
what time zone your RTC is ticking on.
--
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!! Mark Martinec (system manager) tel +386 1 4773-575 !!
!! J. Stefan Institute, Jamova 39 fax +386 1 2519-385 !!
!! SI-1000 Ljubljana, Slovenia mark.m...@ijs.si !!
!!!!!!!!!!!!!!!!!!!!!!!!!! http://www.ijs.si/people/mark/ !!!!
David L. Mills
Geeze, to be compared to slick Tony. Awesome! Fortunately, we get the
Beeb here and, having lived in Britain, I love to see the players have a
rollicking good time. PMQT is the most entertaining program on the
planet.
I make no apologies; I take no prisoners. Every time I answer The Local
Clock Question, I try to be more concise, crisp and to the point.
However, your concern about the ambiguity in the advice is well taken,
even if it does reflect my very real fear that somebody may come up a
broken local clock configured at stratum 1 on the public Internet and
cause very real damage.
Dave
David L. Mills
Probably not a good idea to configure all machines with the local clock
driver at the same stratum number. If you do and the clocks drift apart,
the clients will get into a pissing contest and degenerate to what might
be called a clockhopping frenzy. Better to configure two or three of
them with the local clock driver, but at different stratum levels. You
need to configure the clients so that they can see the selected servers
directly or indirectly.
If a machine loses all its sources, whether or not the local clock
driver is configured, it continues to coast at the frequency last
determined before the loss, so you don't need to do any fudge at all. A
client will not believe a high-stratum server running the local clock
driver until all other sources time out, which can take a long time if
the poll interval has ramped up to maxpoll (1024 s).
The protocol does have the well known characteristics of the
Bellman-Ford routing algorithm and, if a lot of machines are in the mix,
a timing island can form and it might not even have a local clock
driver. Once formed, it would have to "count to infinity", where
infinity is fortunately a small number, here 15. I've never seen this
happen, but if you have a lot of machines, timing islands could last for
days without external sources.
Ulrich and friends put a lot of work in the faq and it really is a good
place to start for hints and common Q&A. I try very hard to tell the
unvarnished truth in the html pages, even if they have much more boring
detail than you might want. If you need further details beyond the faq,
try the local clock driver page at the end of my arrows.
Dave
David L. Mills
Past evil deeds are returning to haunt me. Years ago some folks beat me
up to provide a calibration feature where the local clock driver could
introduce fixed time and frequency offsets without ever depending on an
external source. That's what the fudge time1 (time) and time2
(frequency) parameters are for. If you have an external source to
initially discipline the server time and frequency, you don't need the
fudge.
Andy Gray
> Andy,
>
> Geeze, to be compared to slick Tony. Awesome! Fortunately, we get the
> Beeb here and, having lived in Britain, I love to see the players have a
> rollicking good time. PMQT is the most entertaining program on the
> planet.
>
> I make no apologies; I take no prisoners. Every time I answer The Local
> Clock Question, I try to be more concise, crisp and to the point.
> However, your concern about the ambiguity in the advice is well taken,
> even if it does reflect my very real fear that somebody may come up a
> broken local clock configured at stratum 1 on the public Internet and
> cause very real damage.
>
> Dave
>
Somehow I knew you'd know what I was on about, it is fascinating
watching 'democracy' at work isn't it?
I will, however, apologise. Though the FAQ doesn't really address my
question, your documentation pages do, albeit a little indirectly.
Sorry for my moment of petulance.
Marc Martinec wrote, "aren't you just asking how to specify stratum
for a local clock? - which is a FAQ".
As it happens, in a way I was, though I didn't exactly know it. Now
I've made one of my core routers an NTP master, I understand a little
better how it works.
Part of my problem is that the Cisco implementation of NTP only
resembles ntpd. There are no such things as fudge, ntp.conf or drift
files, for example, though there are the Cisco equivalents (except for
fudge). Reading the documentation is consequently a matter of constant
interpretation, so I will admit I've missed a few things here and
there.
Regards
Andy
Ronald Thornton
That "past evil" was put to good use several years ago in a small
office network with no direct Internet connection and a phone bill
that would not allow regular dialup updates. Every week or 2 I would
run a dialup program to get the current offset from NIST, and tweak
the time2 for my server to refine its drift.
-Ron-
On Fri, 02 Nov 2001 03:04:04 +0000, "David L. Mills" <mi...@udel.edu>
wrote:
Per Hedeland
<Thor...@Aerostation.org> writes:
>On Thu, 1 Nov 2001 20:14:55 +0000 (UTC), p...@bluetail.com (Per
>Hedeland) wrote:
>
>>In article <mfn2ut44u2tclicg5...@4ax.com> Ronald Thornton
>><Thor...@Aerostation.org> writes:
>>>
>>>If the local network becomes isolated, the servers will quickly shift
>>>to LOCAL rather than coasting at their current stratum level using the
>>>latest drift number. It seems that you would really like to update
>>>LOCAL fudge time2 with the current ntp.drift (or the latest number
>>>inside ntpd) before you start using it as reference.
>>
>>Good thinking, and the ntpd authors already thought it - ntpd does this
>>automatically.
>
>I see "calibrate" code that updates fudgetime1 values of ref clocks
>that are not the current reference, but I couldn't see where the
>fudgetime2 value in LOCAL gets updated automatically. It seems like
>the current drift_comp will head toward zero (or what ever might be in
>LOCAL fudgetime2) when LOCAL becomes the selected reference.
Well, I couldn't point you to where in the code this magic is happening,
but hopefully Dave Mills agreeing that it will happen is enough for
you.:-) I.e. "it" being that the local clock will have the previously
determined drift compensation applied to it also when it becomes the
selected reference, which doesn't necessarily involve modifying the
fudgetime2 value I guess.
--Per Hedeland
p...@bluetail.com
David L. Mills
David Katz of Cisco was the original Cisco kid; he rather faithfully
copied the NTPv3 code available about ten years ago. I still have a
couple of now retired Cisco routers with David's code. I had it running
here with no problems at all.
Dave
David L. Mills
David Katz of Cisco was the original Cisco kid; he rather faithfully
copied the NTPv3 code available about ten years ago. I still have a
couple of now retired Cisco routers with David's code. I had it running
here with no problems at all.
Dave