unable to obtain neither an IPv4 nor an IPv6 dispatch

[Matthew Calder]

I'm completely new to BIND configuration. I'm running on BIND 9.9.5-3-Ubuntu (Extended Support Version) on Ubuntu 14.04 and seeing this error on startup/reload.

Jul 15 19:20:57 named[1978]: server.c:2772: unexpected error: 
Jul 15 19:20:57 named[1978]: unable to obtain neither an IPv4 nor an IPv6 dispatch

This happens if I do rndc reload/reconfig. My named.conf.local has many around 500 views and large match-client ACLs because I am experimenting with a global split-horizon configuration. I would appreciate any help.

[Mark Andrews]

You need to increase ISC_SOCKET_MAXSOCKETS.

e.g.

./configure STC_CDEFINES=-DISC_SOCKET_MAXSOCKETS=21000

In message <CANcFfBvV2wKtdc-dtZZcecC4...@mail.gmail.com>
, Matthew Calder writes:
> --===============1218088129802327245==
> Content-Type: multipart/alternative; boundary=089e013a044a72db8004fe551784
>
> --089e013a044a72db8004fe551784
> Content-Type: text/plain; charset=UTF-8

> --089e013a044a72db8004fe551784
> Content-Type: text/html; charset=UTF-8
> Content-Transfer-Encoding: quoted-printable
>
> <div dir=3D"ltr"><div><span style=3D"color:windowtext;line-height:17px;font=
> -family:Calibri,Sans-Serif;font-size:11pt"><span style=3D"background-color:=
> inherit">I&#39;m completely new to BIND configuration. I&#39;m running on <=
> span style=3D"color:windowtext;line-height:17px;font-family:Calibri,Sans-Se=
> rif;font-size:11pt"><span style=3D"background-color:inherit">BIND 9.9.5-3-U=
> buntu (Extended Support Version) on Ubuntu 14.04 and seeing this error on s=
> tartup/reload.</span></span></span></span></div>
> <div><span style=3D"color:windowtext;line-height:17px;font-family:Calibri,S=
> ans-Serif;font-size:11pt"><span style=3D"background-color:inherit"></span><=
> /span><br></div><div><span style=3D"color:windowtext;line-height:17px;font-=
> family:Calibri,Sans-Serif;font-size:11pt"><span style=3D"background-color:i=
> nherit">Jul 15 19:20:57</span><span style=3D"background-color:inherit"> nam=
> ed[1978]: server.c:2772: unexpected error:</span></span><span style=3D"line=
> -height:17px;font-family:Calibri,Sans-Serif;font-size:11pt"><span>=C2=A0</s=
> pan><br>
> </span><span style=3D"color:windowtext;line-height:17px;font-family:Calibri=
> ,Sans-Serif;font-size:11pt"><span style=3D"background-color:inherit">Jul 15=
> 19:20:57 </span><span style=3D"background-color:inherit">named[1978]: unab=
> le to obtain neither an IPv4 nor an IPv6 dispatch</span></span></div>
> <div><span style=3D"color:windowtext;line-height:17px;font-family:Calibri,S=
> ans-Serif;font-size:11pt"><span style=3D"background-color:inherit"></span><=
> /span><br></div><div><span style=3D"color:windowtext;line-height:17px;font-=
> family:Calibri,Sans-Serif;font-size:11pt"><span style=3D"background-color:i=
> nherit"><span style=3D"color:windowtext;line-height:17px;font-family:Calibr=
> i,Sans-Serif;font-size:11pt"><span style=3D"background-color:inherit">This =
> happens if I do rndc reload/reconfig. </span></span></span></span><span sty=
> le=3D"color:windowtext;line-height:17px;font-family:Calibri,Sans-Serif;font=
> -size:11pt"><span style=3D"background-color:inherit">My named.conf.local ha=
> s many=C2=A0around=C2=A0500 views=C2=A0and large match-client ACLs because =
> I am experimenting with a global split-horizon configuration. I would appre=
> ciate any help.</span></span></div>
> </div>
>
> --089e013a044a72db8004fe551784--
>
> --===============1218088129802327245==
> Content-Type: text/plain; charset="us-ascii"
> MIME-Version: 1.0
> Content-Transfer-Encoding: 7bit
> Content-Disposition: inline
>
> _______________________________________________
> Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe
> from this list
>
> bind-users mailing list
> bind-...@lists.isc.org
> https://lists.isc.org/mailman/listinfo/bind-users
> --===============1218088129802327245==--
--
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742 INTERNET: ma...@isc.org

[Matthew Calder]

Thanks Mark. I tried building with increased ISC_SOCKET_MAXSOCKETS with BIND 9.9.5 but I kept seeing the same error. Eventually, I tried BIND 9.10.0 and the error no longer happens. 

[Cathy Almond]

It might have something to do with the number of CPUs that named detects
when it starts, which (by default) drives how many listening tasks it
starts per listening interface.

BIND 9.10 changed the defaults slightly, but you can also control how
many listening tasks per interface using the -U option.

Is there anything else unusual in your test set-up? I notice you said
that you have 500 views - do you also have a large number of virtual
interfaces, each with their own listen-on?

On 18/07/2014 07:06, Matthew Calder wrote:
> Thanks Mark. I tried building with increased ISC_SOCKET_MAXSOCKETS with
> BIND 9.9.5 but I kept seeing the same error. Eventually, I tried BIND
> 9.10.0 and the error no longer happens.
>
>
> On Wed, Jul 16, 2014 at 3:56 PM, Mark Andrews <ma...@isc.org
> <mailto:ma...@isc.org>> wrote:
>
>
> You need to increase ISC_SOCKET_MAXSOCKETS.
>
> e.g.
>
> ./configure STC_CDEFINES=-DISC_SOCKET_MAXSOCKETS=21000
>
> In message
> <CANcFfBvV2wKtdc-dtZZcecC4...@mail.gmail.com

> <mailto:CANcFfBvV2wKtdc-dtZZcecC4giF1nfVGqVQ-KXQGpTEx6%2B%2B...@mail.gmail.com>>

> > bind-...@lists.isc.org <mailto:bind-...@lists.isc.org>

> > https://lists.isc.org/mailman/listinfo/bind-users
> > --===============1218088129802327245==--
> --
> Mark Andrews, ISC
> 1 Seymour St., Dundas Valley, NSW 2117, Australia

> PHONE: +61 2 9871 4742 <tel:%2B61%202%209871%204742>
> INTERNET: ma...@isc.org <mailto:ma...@isc.org>

[Matthew Calder]

Hi Cathy,

I think you may be correct. I am running these in cloud hosted VMs. I found that the problem is difficult to reproduce with a 2 core instance. With a 4 core instance it only happens with reload/reconfig and with 8 cores it doesn't get past start up. From the VM's perspective, I only have 2 interfaces, lo and eth0.

Here is what shows up in the logs for BIND 9.9.5 on start.

Jul 18 18:42:39 named[11861]: adjusted limit on open files from 4096 to 1048576
Jul 18 18:42:39 named[11861]: found 4 CPUs, using 4 worker threads
Jul 18 18:42:39 named[11861]: using 4 UDP listeners per interface
Jul 18 18:42:39 named[11861]: using up to 4096 sockets
Jul 18 18:42:39 named[11861]: loading configuration from '/etc/bind/named.conf'
Jul 18 18:42:45 named[11861]: reading built-in trusted keys from file '/etc/bind/bind.keys'
Jul 18 18:42:45 named[11861]: using default UDP/IPv4 port range: [1024, 65535]
Jul 18 18:42:45 named[11861]: using default UDP/IPv6 port range: [1024, 65535]
Jul 18 18:42:45 named[11861]: listening on IPv6 interfaces, port 53
Jul 18 18:42:45 named[11861]: listening on IPv4 interface lo, 127.0.0.1#53
Jul 18 18:42:45 named[11861]: listening on IPv4 interface eth0, 100.67.236.8#53

[Matthew Calder]

At the moment I'm limited to using 2 UDP listeners per interface. When stress testing I can see that only 2 out of 4 CPUs are being used, I'm guessing because I'm limited to 2 listeners. Any suggestions for what could be limiting BIND from using a higher number of UDP listeners?

[Cathy Almond]

On 24/07/2014 01:35, Matthew Calder wrote:
> At the moment I'm limited to using 2 UDP listeners per interface. When
> stress testing I can see that only 2 out of 4 CPUs are being used, I'm
> guessing because I'm limited to 2 listeners. Any suggestions for what
> could be limiting BIND from using a higher number of UDP listeners?

named will tell you in its startup logs how many workers it is running
(either from auto-detecting #cpus or from -n when running named). It
will also tell you how many listeners per interface (default for 9.9 is
the same as the number of worker threads, but it's also controlled by -U
at start up).

The number of listeners per interface is also capped at the number of
worker threads.

>From BIND 9.10 we changed the default behaviour - it's this:

"If not specified, named will calculate a default value based on the
number of detected CPUs: 1 for 1 CPU, 2 for 2-4 CPUs, and the number of
detected CPUs divided by 2 for values higher than 4. If -n has been set
to a higher value than the number of detected CPUs, then -U may be
increased as high as that value, but no higher."

Also, working out what's going on is a bit of an 'art' because within
BIND, depending on how the OS works, you're probably going to get each
worker thread pegged to the same CPU most of the time (but not always),
but each worker thread itself is going to be running multiple tasks -
not just the listeners, so while it sometimes works out under stress
that we see the listener tasks always on the same threads and those
threads always on the same CPU, it's not guaranteed to split that way
(particularly if -n and -U are the same and/or -n is set larger than the
number of available CPUs).

Cathy