ntpd 4.2.6p5 will not synchronize to broadcast servers

[Gary Johnson]

We have a system that uses ntpd to synchronize the system clock. We
allow the user to run ntpd in polling mode with a user-specified set of
servers, or in broadcast mode, accepting any broadcast NTP messages
received.

This used to work fine with ntpd 4.2.4p7. Since upgrading to 4.2.6p5,
polling mode continues to work fine but broadcast mode does not work at
all--ntpd never sees any broadcast messages.

The reason seems to be that ntpd 4.2.6p4 tries to initialize the
broadcast interface before initializing its list of available
interfaces. Here is a simple invocation of ntpd that exhibits the
problem. Note the message about "no broadcast interfaces available" on
the seventh line of output below.

$ ntpd -4 -b -D2 -c /dev/null
ntpd 4.2...@1.2349-o Fri Apr 19 01:23:45 UTC 2013 (1)
24 Apr 23:58:35 ntpd[17867]: set_process_priority: Leave priority alone: priority_done is <2>
24 Apr 23:58:35 ntpd[17867]: proto: precision = 5.380 usec
loop_config: item 1 freq 0.000000
event at 0 0.0.0.0 c01d 0d kern kernel time sync enabled
proto_config: code 1 value 1 dvalue 0.000000
24 Apr 23:58:35 ntpd[17867]: Unable to listen for broadcasts, no broadcast interfaces available
24 Apr 23:58:35 ntpd[17867]: line 1999510120 column 0 syntax error, unexpected $end
Finished Parsing!!
create_sockets(123)
24 Apr 23:58:35 ntpd[17867]: ntp_io: estimated max descriptors: 1024, initial socket boundary: 16
24 Apr 23:58:35 ntpd[17867]: Listen and drop on 0 v4wildcard 0.0.0.0 UDP 123
created interface #0: fd=16, bfd=-1, name=v4wildcard, flags=0x89, ifindex=0, sin=0.0.0.0, bcast=0.0.0.0, mask=255.255.255.255, Disabled:
create_interface(127.0.0.1#123)
24 Apr 23:58:35 ntpd[17867]: Listen normally on 1 lo 127.0.0.1 UDP 123
restrict: op 1 addr 127.0.0.1 mask 255.255.255.255 mflags 00003000 flags 00000001
created interface #1: fd=17, bfd=-1, name=lo, flags=0x5, ifindex=0, sin=127.0.0.1, mask=255.0.0.0, Enabled:
create_interface(10.10.143.200#123)
24 Apr 23:58:35 ntpd[17867]: Listen normally on 2 eth0 10.10.143.200 UDP 123
restrict: op 1 addr 10.10.143.200 mask 255.255.255.255 mflags 00003000 flags 00000001
created interface #2: fd=18, bfd=-1, name=eth0, flags=0x19, ifindex=0, sin=10.10.143.200, bcast=10.10.143.255, mask=255.255.255.0, Enabled:
create_interface(10.27.50.174#123)
24 Apr 23:58:35 ntpd[17867]: Listen normally on 3 remote 10.27.50.174 UDP 123
restrict: op 1 addr 10.27.50.174 mask 255.255.255.255 mflags 00003000 flags 00000001
created interface #3: fd=19, bfd=-1, name=remote, flags=0x19, ifindex=0, sin=10.27.50.174, bcast=10.27.50.255, mask=255.255.255.0, Enabled:
24 Apr 23:58:35 ntpd[17867]: peers refreshed
create_sockets: Total interfaces = 4
24 Apr 23:58:35 ntpd[17867]: Listening on routing socket on fd #20 for interface updates
auth_setkey: key 65535 type 4 len 4 6ee30717
event at 0 0.0.0.0 c016 06 restart
loop_config: item 2 freq 1000000000.000000
event at 0 0.0.0.0 c012 02 freq_set kernel 0.000 PPM
local_clock: mu 0 state 1 poll 3 count 0
event at 0 0.0.0.0 c011 01 freq_not_set
auth_agekeys: at 1 keys 1 expired 0
timer: interface update

After waiting long enough for a broadcast message to have been sent by
ntpd running on another host,

$ ntpq -crv
receive: at 96 127.0.0.1<-127.0.0.1 flags 5 restrict 000
sendpkt(17, dst=127.0.0.1, src=127.0.0.1, ttl=-6, len=372)
associd=0 status=c011 leap_alarm, sync_unspec, 1 event, freq_not_set,
version="ntpd 4.2...@1.2349-o Fri Apr 19 01:23:45 UTC 2013 (1)",
processor="mips64", system="Linux/3.4.34", leap=11, stratum=16,
precision=-17, rootdelay=0.000, rootdisp=1.440, refid=INIT,
reftime=00000000.00000000 Thu, Feb 7 2036 6:28:16.000,
clock=d522f08b.f70b1c35 Thu, Apr 25 2013 0:00:11.965, peer=0, tc=3,
mintc=3, offset=0.000, frequency=0.000, sys_jitter=0.000,
clk_jitter=0.008, clk_wander=0.000

I looked at the source code for both versions of ntpd and saw that in
4.2.4p7, the list of interfaces was initialized by the following cascade
of function calls,

ntpdmain() --> init_io() --> create_sockets()

In 4.2.6p4, however, most of the contents of the function init_io() have
been stripped out, including the call to create_sockets(). In 4.2.6p4,
create_sockets() is not called until later, after the attempt to create
the broadcast interface.

Is ntpd 4.2.6p4 really broken w.r.t. receiving broadcasts, or do we have
something configured incorrectly?

Regards,
Gary

[Harlan Stenn]

Gary,

The short answer is "I don't know". I believe we have successfully used
broadcast clients in 4.2.6.

Please see if ntp-dev works for you though - if there is a problem there
we want to fix that before releasing 4.2.8.

And having written the above:

http://bugs.ntp.org/show_bug.cgi?id=2261

[Brian Utterback]

I think this is a duplicate of bug 629.

> _______________________________________________
> questions mailing list
> ques...@lists.ntp.org
> http://lists.ntp.org/listinfo/questions

[Gary Johnson]

Brian Utterback <brian.u...@oracle.com> wrote:
> I think this is a duplicate of bug 629.
>
> On 4/24/2013 9:54 PM, Harlan Stenn wrote:
> > Gary,
> >
> > The short answer is "I don't know". I believe we have successfully used
> > broadcast clients in 4.2.6.
> >
> > Please see if ntp-dev works for you though - if there is a problem there
> > we want to fix that before releasing 4.2.8.
> >
> > And having written the above:
> >
> > http://bugs.ntp.org/show_bug.cgi?id=2261

Thank you both! I had found those bug reports earlier, but I didn't
think that wildcards and "255.255.255.255" applied in my case, so I
didn't look closely at either one. I read the discussions associated
with both last night and applied Dave Hart's suggestion at the bottom of
the bug 629 discussion to add

interface listen wildcard
interface listen all

to my /etc/ntp.conf. Broadcast now works!

I changed the simple test case I posted originally to

$ ntpd -4 -b -D2 -c /etc/ntp.conf.test &

where /etc/ntp.conf.test contains only those two interface commands. It
still emits the message

25 Apr 16:50:38 ntpd[31987]: Unable to listen for broadcasts, no broadcast interfaces available

but broadcast works anyway.

I then added those two interface commands to the top of our normal
/etc/ntp.conf file and broadcast now works in the context of our
application as well.

Does it matter where in /etc/ntp.conf those commands are put?

I will download and install ntp-dev later today and let you know how
that goes.

Thanks again,
Gary

[Charles Elliott]

> interface listen wildcard
> interface listen all

I tried these on a Windows 8, 64-bit machine. It does not solve any
problems; NTP still puts the following messages into the Event Log on
startup:

bind(308) AF_INET 192.168.1.255#123 flags 0x419 failed: Can't assign
requested address
failed to listen for broadcasts to 192.168.1.255 on interface #3 Ethernet 5

bind(308) AF_INET 192.168.0.255#123 flags 0x419 failed: Can't assign
requested address
failed to listen for broadcasts to 192.168.0.255 on interface #5 Ethernet 4

Unable to listen for broadcasts, no broadcast interfaces available

However, broadcasts still work fine, with the exception that now, with the
above two copied lines in the conf file, NTPD now puts the following two
lines in the Windows Event Log every 16 seconds:

192.168.1.88 local addr 192.168.1.100 -> 0.0.0.0
192.168.1.88 local addr 0.0.0.0 -> 192.168.1.100

192.168.1.88 is the broadcast server, and 192.168.1.100 is this machine, one
of the recipients.

Charles Elliott

[Gary Johnson]

Gary Johnson <gary...@eskimo.com> wrote:
> Brian Utterback <brian.u...@oracle.com> wrote:
> > I think this is a duplicate of bug 629.
> >
> > On 4/24/2013 9:54 PM, Harlan Stenn wrote:
> > > Gary,
> > >
> > > The short answer is "I don't know". I believe we have successfully used
> > > broadcast clients in 4.2.6.
> > >
> > > Please see if ntp-dev works for you though - if there is a problem there
> > > we want to fix that before releasing 4.2.8.
> > >
> > > And having written the above:
> > >
> > > http://bugs.ntp.org/show_bug.cgi?id=2261
>
> Thank you both! I had found those bug reports earlier, but I didn't
> think that wildcards and "255.255.255.255" applied in my case, so I
> didn't look closely at either one. I read the discussions associated
> with both last night and applied Dave Hart's suggestion at the bottom of
> the bug 629 discussion to add
>
> interface listen wildcard
> interface listen all
>
> to my /etc/ntp.conf. Broadcast now works!
>
> I changed the simple test case I posted originally to
>
> $ ntpd -4 -b -D2 -c /etc/ntp.conf.test &
>
> where /etc/ntp.conf.test contains only those two interface commands.

> I will download and install ntp-dev later today and let you know how
> that goes.

We had some issues with getting ntp-dev-4.2.7p367 compiled and installed
on our system but we now have it running. When given a list of servers,
it chooses and syncs to a system peer just fine.

The bad news is that this version does not work at all with broadcast
messages, not even with the interface commands added to /etc/ntp.conf.

$ ntpq -crv
associd=0 status=c016 leap_alarm, sync_unspec, 1 event, restart,
version="ntpd 4.2....@1.2483-o Mon Apr 29 21:01:02 UTC 2013 (1)",

processor="mips64", system="Linux/3.4.34", leap=11, stratum=16,

precision=-17, rootdelay=0.000, rootdisp=7.575, refid=INIT,

reftime=00000000.00000000 Thu, Feb 7 2036 6:28:16.000,

clock=d5297b65.cf1fce0d Mon, Apr 29 2013 23:06:13.809, peer=0, tc=3,
mintc=3, offset=0.000000, frequency=0.000, sys_jitter=0.000000,
clk_jitter=0.008, clk_wander=0.000

The debug output is now a little chatty with all the messages about
select().

$ ntpd -4 -b -D2 -c /etc/ntp.conf.test

29 Apr 23:10:45 ntpd[3361]: ntpd 4.2....@1.2483-o Mon Apr 29 21:01:02 UTC 2013 (1): Starting
29 Apr 23:10:45 ntpd[3361]: Command line: ntpd -4 -b -D2 -c /etc/ntp.conf.test
ntp_rlimit: STACK: 50 4k pages
ntp_rlimit: MEMLOCK: 32 MB
29 Apr 23:10:45 ntpd[3361]: set_process_priority: Leave priority alone: priority_done is <2>
29 Apr 23:10:45 ntpd[3361]: proto: precision = 6.919 usec (-17)

proto_config: code 1 value 1 dvalue 0.000000

29 Apr 23:10:45 ntpd[3361]: Unable to listen for broadcasts, no broadcast interfaces available
Finished Parsing!!
create_sockets(123)
move_fd: estimated max descriptors: 1024, initial socket boundary: 16
29 Apr 23:10:45 ntpd[3361]: Listen normally on 0 v4wildcard 0.0.0.0:123
created interface #0: fd=16, bfd=-1, name=v4wildcard, flags=0x89, ifindex=0, sin=0.0.0.0, bcast=0.0.0.0, mask=255.255.255.255, Enabled:
create_interface(127.0.0.1#123)
29 Apr 23:10:45 ntpd[3361]: Listen normally on 1 lo 127.0.0.1:123

restrict: op 1 addr 127.0.0.1 mask 255.255.255.255 mflags 00003000 flags 00000001
created interface #1: fd=17, bfd=-1, name=lo, flags=0x5, ifindex=0, sin=127.0.0.1, mask=255.0.0.0, Enabled:
create_interface(10.10.143.200#123)

29 Apr 23:10:45 ntpd[3361]: Listen normally on 2 eth0 10.10.143.200:123

restrict: op 1 addr 10.10.143.200 mask 255.255.255.255 mflags 00003000 flags 00000001
created interface #2: fd=18, bfd=-1, name=eth0, flags=0x19, ifindex=0, sin=10.10.143.200, bcast=10.10.143.255, mask=255.255.255.0, Enabled:
create_interface(10.27.50.174#123)

29 Apr 23:10:45 ntpd[3361]: Listen normally on 3 remote 10.27.50.174:123

restrict: op 1 addr 10.27.50.174 mask 255.255.255.255 mflags 00003000 flags 00000001
created interface #3: fd=19, bfd=-1, name=remote, flags=0x19, ifindex=0, sin=10.27.50.174, bcast=10.27.50.255, mask=255.255.255.0, Enabled:

29 Apr 23:10:45 ntpd[3361]: peers refreshed

create_sockets: Total interfaces = 4

29 Apr 23:10:45 ntpd[3361]: Listening on routing socket on fd #20 for interface updates

loop_config: item 1 freq 0.000000
event at 0 0.0.0.0 c01d 0d kern kernel time sync enabled

event at 0 0.0.0.0 c012 02 freq_set kernel 0.000 PPM
local_clock: mu 0 state 1 poll 3 count 0
event at 0 0.0.0.0 c011 01 freq_not_set

event at 0 0.0.0.0 c016 06 restart

select() returned -1: Interrupted system call
auth_agekeys: at 1 keys 0 expired 0
select() returned -1: Interrupted system call
timer: interface update
select() returned -1: Interrupted system call
select() returned -1: Interrupted system call
select() returned -1: Interrupted system call
select() returned -1: Interrupted system call
select() returned -1: Interrupted system call
...

To top it off, this version of ntpdc is not working for us, either. It
seems that anything it can answer by itself works, but anything it needs
to query ntpd for times out.

$ ntpdc --version
ntpdc 4.2....@1.2483-o Mon Apr 29 21:01:06 UTC 2013 (1)
$ ntpdc -c kerninfo
localhost: timed out, nothing received
***Request timed out

In summary, our solution for now will be to go with 4.2.6p5 with those
two interface commands added to our /etc/ntp.conf. This is really going
to put us in a bind in the future, however, if this is not resolved in
4.2.8.

Please let me know if there is any other debug output I can provide or
experiments I can perform to help identify the cause of this or to help
verify a solution.

Regards,
Gary