Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

networking.service: start operation timed out

838 views
Skip to first unread message

Ross Boylan

unread,
Aug 26, 2022, 6:10:06 PM8/26/22
to
In Debian 11/bullseye my system keeps reporting timeouts while trying
to bring up the first non-loopback interface. According to ip, the
interface actually is up, but ifup/down do not know that. My 2nd
interface is down, and there is no mention of attempting to bring it
up in the logs. I can bring up both interfaces after startup,
suggesting there may be something special about the initial
environment that is causing trouble, but I don't know what.

I'd appreciate any suggestions about how to diagnose or cure the
problem. I have set VERBOSE=yes in /etc/default/networking

When I started the errors were coming on a bridge interface;
suspecting a bridge-specific problem I disabled the bridge (comment
out the auto br0 in /etc/network/interface). Now I have the exact
same problem with a vanilla declaration for a lan.

My machine has 2 physical ethernet ports, resolvconf, KDE,
network-manager, bridge-utils, tor, and nftables. I have systemctl
disable'd NetworkManager and tor, though with no visible effect.
NetworkManager was mentioning the interfaces controlled by
/etc/network/interfaces, but it wasn't clearly doing anything to them,
and the default options do say to leave them alone. I have customized
scripts for networking and resolvconf.

------------------------------------- /etc/network/interfaces
----------------------------
# The next directory is empty
# many posts about similar problems had an interfaces.d/setup file
causing trouble
source /etc/network/interfaces.d/*

# The loopback network interface
auto lo
iface lo inet loopback

rename enp5s0=ethlan
rename eno2=ethworld

auto ethlan
iface ethlan inet static
address 192.168.1.10/24
# not sure if it makes sense to be my own gateway
# gateway 192.168.1.10
broadcast 192.168.1.255
dns-domain betterworld.us
dns-search betterworld.us

#auto br0
iface br0 inet static
# details omitted, presumed irrelevant

auto ethworld
iface ethworld inet static
address 66.181.128.36/24 # IP obscured
gateway 66.181.128.1
# I am my own nameserver, but next is a fallback
dns-nameservers N.N.N.N
pre-up /etc/network/rb-iface-manage up # brings up firewall.
takes a few seconds.

post-down /etc/network/rb-iface-manage down
post-down ip route add default dev ethlan

--------------- journalctl -b -u networking
--------------------------------------------
-- Journal begins at Thu 2022-05-19 12:21:39 PDT, ends at Fri
2022-08-26 11:58:07 PDT. --
Aug 26 11:55:19 barley ifup[1563]: /sbin/ip link set enp5s0 name ethlan
Aug 26 11:55:19 barley ifup[1563]: /sbin/ip link set eno2 name ethworld
Aug 26 11:55:19 barley ifup[1563]: /bin/run-parts --exit-on-error
--verbose /etc/network/if-pre-up.d
Aug 26 11:55:19 barley ifup[1573]: run-parts: executing
/etc/network/if-pre-up.d/bridge
Aug 26 11:55:19 barley ifup[1573]: run-parts: executing
/etc/network/if-pre-up.d/vde2
Aug 26 11:55:19 barley ifup[1573]: run-parts: executing
/etc/network/if-pre-up.d/wpasupplicant
Aug 26 11:55:18 barley systemd[1]: Starting Raise network interfaces...
Aug 26 11:55:19 barley ifup[1563]: ifup: configuring interface lo=lo (inet)
Aug 26 11:55:19 barley ifup[1563]: /bin/run-parts --exit-on-error
--verbose /etc/network/if-pre-up.d
Aug 26 11:55:19 barley ifup[1583]: run-parts: executing
/etc/network/if-pre-up.d/bridge
Aug 26 11:55:19 barley ifup[1583]: run-parts: executing
/etc/network/if-pre-up.d/vde2
Aug 26 11:55:19 barley ifup[1583]: run-parts: executing
/etc/network/if-pre-up.d/wpasupplicant
Aug 26 11:55:19 barley ifup[1563]: /sbin/ip link set dev lo up
Aug 26 11:55:19 barley ifup[1563]: /bin/run-parts --exit-on-error
--verbose /etc/network/if-up.d
Aug 26 11:55:19 barley ifup[1590]: run-parts: executing
/etc/network/if-up.d/000resolvconf
Aug 26 11:55:19 barley ifup[1590]: run-parts: executing
/etc/network/if-up.d/bind9
Aug 26 11:55:20 barley ifup[1590]: run-parts: executing
/etc/network/if-up.d/chrony
Aug 26 11:55:20 barley ifup[1590]: run-parts: executing
/etc/network/if-up.d/wpasupplicant
Aug 26 11:55:20 barley ifup[1563]: ifup: configuring interface
ethlan=ethlan (inet)
Aug 26 11:55:20 barley ifup[1563]: /bin/run-parts --exit-on-error
--verbose /etc/network/if-pre-up.d
Aug 26 11:55:20 barley ifup[1658]: run-parts: executing
/etc/network/if-pre-up.d/bridge
Aug 26 11:55:20 barley ifup[1658]: run-parts: executing
/etc/network/if-pre-up.d/vde2
Aug 26 11:55:20 barley ifup[1658]: run-parts: executing
/etc/network/if-pre-up.d/wpasupplicant
Aug 26 11:55:20 barley ifup[1563]: /sbin/ip addr add
192.168.1.10/255.255.255.0 broadcast 192.168.1.255 dev
ethlan label ethlan
Aug 26 11:55:20 barley ifup[1563]: /sbin/ip link set dev ethlan up
Aug 26 11:55:20 barley ifup[1563]: /bin/run-parts --exit-on-error
--verbose /etc/network/if-up.d
Aug 26 11:55:20 barley ifup[1680]: run-parts: executing
/etc/network/if-up.d/000resolvconf
Aug 26 11:56:18 barley systemd[1]: networking.service: start operation
timed out. Terminating.
Aug 26 11:56:18 barley ifup[1563]: Got signal Terminated, terminating...
Aug 26 11:56:18 barley ifup[1563]: ifup: failed to bring up ethlan
Aug 26 11:56:18 barley ifup[1691]: Terminated
Aug 26 11:56:18 barley ifup[1691]: Terminated
Aug 26 11:56:18 barley systemd[1]: networking.service: Main process
exited, code=exited, status=1/FAILURE
Aug 26 11:56:18 barley systemd[1]: networking.service: Failed with
result 'timeout'.
Aug 26 11:56:18 barley systemd[1]: Failed to start Raise network interfaces.
---------------------------------------------

Comments.
1. The man page for ifup -a says interfaces will be brought up in the
order specified. OTOH, man interfaces says calls to different
interfaces can be run in parallel. Although this doesn't logically
imply ifup -a (which is what the networking.service uses) brings
things up in parallel, it kind of looks above as if that's what's
happening.
2. There are 3 if-pre-up.d sections (for lo, ethlan, ethworld?) and 2
if-up.d sections. None are explicitly labelled with which interface
they are being run for. Perhaps it can be inferred from earlier and
later loglines. The 2nd if-up.d section has only the first run-parts
script mentioned.
3. lo and ethlan are both brought up by ip
4. ifup doesn't really fail; it is killed by systemd after
networking.service's timeout (which I lowered to 1m from 5) expires.
It clearly doesn't do much to bring up ethworld.
5. Presumably the problem is that ifup -a never completes. The logs
suggest that if-up.d/000resolvconf is where it hangs up, which I
suppose means it might actually be in one of resolvconf's scripts that
it hangs up. However, the ip route for ethlan is set when the system
comes up.
6. Or maybe it's the next script, if-up.d/bind9 which is the problem.
000resolvconf is as packaged, while bind9 is a local creation, though
based on package resolvconf's advice. I.e., the bind9 script is more
likely to have a bug.
7. After all this, ifdown ethlan fails with the message the interface
is not up, despite what ip link says. ifdown --force ethlan; ifup -v
ethlan; ifup -v ethworld then bring both interfaces up. ifup returns
without error from each.
8. Some of the triggered scripts are about updating bind and
isc-dhcp-server. As networking services, I presume they are not yet
active when networking is being set up. The scripts seem to have
appropriate defenses in place, but maybe something is off. For
example, if-up.d/bind9 has `/usr/sbin/rndc reconfig >/dev/null 2>&1
|| :`. My assumption was that rndc would fail if bind9 were not
running, and that the `|| :` would conceal the failure. If rndc
blocks awaiting a response, that would hang things up. FWIW, if I
stop named and run rndc reconfig I get an immediate failure.

Andrew M.A. Cater

unread,
Aug 26, 2022, 6:40:05 PM8/26/22
to
On Fri, Aug 26, 2022 at 03:06:24PM -0700, Ross Boylan wrote:
> In Debian 11/bullseye my system keeps reporting timeouts while trying
> to bring up the first non-loopback interface. According to ip, the
> interface actually is up, but ifup/down do not know that. My 2nd
> interface is down, and there is no mention of attempting to bring it
> up in the logs. I can bring up both interfaces after startup,
> suggesting there may be something special about the initial
> environment that is causing trouble, but I don't know what.
>
> I'd appreciate any suggestions about how to diagnose or cure the
> problem. I have set VERBOSE=yes in /etc/default/networking
>

Cut /etc/network/interfaces down to the lo stanza

# The loopback network interface
auto lo
iface lo inet loopback

Remove EVERYTHING else.

Use Network Manager / nmtui / nmcli to build back what you need slowly

If you have interfaces mentioned in /etc/network/interfaces - the old way -
you will face problems.

Hand editing the files is probably not the way to do this any more.

>

Hope this helps,

With every good wish, as ever,

Andy Cater

Felix Miata

unread,
Aug 26, 2022, 7:10:05 PM8/26/22
to
Ross Boylan composed on 2022-08-26 15:06 (UTC-0700):

> I'd appreciate any suggestions about how to diagnose or cure the problem.

I switched most of my static installations to systemd-networkd.service last year:
# inxi -Snz
System:
Kernel: 5.10.0-17-amd64 arch: x86_64 bits: 64 Console: pty pts/0
Distro: Debian GNU/Linux 11 (bullseye)
Network:
Device-1: Intel Ethernet I219-V driver: e1000e
IF: eth0 state: up speed: 1000 Mbps duplex: full mac: <filter>
# systemctl list-unit-files | egrep 'netw|solv'
dbus-org.freedesktop.network1.service alias -
networking.service disabled enabled
systemd-network-generator.service disabled disabled
systemd-networkd-wait-online.service disabled disabled
systemd-networkd.service enabled enabled
systemd-resolved.service disabled enabled
systemd-networkd.socket enabled enabled
network-online.target static -
network-pre.target static -
network.target static -
# ls -gG /etc/systemd/network
total 4
-rw-r--r-- 1 147 Jun 13 2021 eth0.network
# ls -gG /etc/resolv.conf
-rw-r--r-- 1 233 Nov 30 2020 /etc/resolv.conf
#
--
Evolution as taught in public schools is, like religion,
based on faith, not based on science.

Team OS/2 ** Reg. Linux User #211409 ** a11y rocks!

Felix Miata

Jeremy Ardley

unread,
Aug 26, 2022, 8:20:05 PM8/26/22
to

On 27/8/22 6:06 am, Ross Boylan wrote:
> In Debian 11/bullseye my system keeps reporting timeouts while trying
> to bring up the first non-loopback interface. According to ip, the
> interface actually is up, but ifup/down do not know that. My 2nd
> interface is down, and there is no mention of attempting to bring it
> up in the logs. I can bring up both interfaces after startup,
> suggesting there may be something special about the initial
> environment that is causing trouble, but I don't know what.
>
> I'd appreciate any suggestions about how to diagnose or cure the
> problem. I have set VERBOSE=yes in /etc/default/networking
>
First of all ensure NetworkManager is really dead.

You should see

systemctl status NetworkManager
● NetworkManager.service
   Loaded: masked (Reason: Unit NetworkManager.service is masked.)
   Active: inactive (dead)

if not, use

systemctl disable NetworkManager

systemctl mask NetworkManager

Then ensure that only the really basics of networking are enabled

cat /etc/networki/interfaces

# This file describes the network interfaces available on your system

# and how to activate them. For more information, see interfaces(5).

# note subdirectory source statement is commented out

# source /etc/network/interfaces.d/*

# The loopback network interface
auto lo
iface lo inet loopback

At his stage the only thing you need to attend to is
/etc/systemd/network files

In my case I have a single file

/etc/systemd/network/20-wired.network

Getting network changes to fully take may require a reboot after
initially disabling NetworkManager and restricting /etc/network/interfaces

Then it is just a matter of configuring your network by editing your
/etc/systemd/network/<configuration file(s)>

systemd-networkd does work reasonably well. It just requires a bit of study

--

Jeremy

OpenPGP_signature

Anssi Saari

unread,
Aug 27, 2022, 4:20:05 AM8/27/22
to
Ross Boylan <rossb...@stanfordalumni.org> writes:

> In Debian 11/bullseye my system keeps reporting timeouts while trying
> to bring up the first non-loopback interface.

I wonder why it is that you have a script for wpa_supplicant if you
don't have wireless interfaces?

Assuming that's not the problem, I guess you'll need to iterate a lot,
enable or disable one script at a time until you find which one causes
the problem. Or mod all scripts to log when they exit too.

Brian

unread,
Aug 27, 2022, 6:10:05 AM8/27/22
to
On Fri 26 Aug 2022 at 22:37:08 +0000, Andrew M.A. Cater wrote:

> On Fri, Aug 26, 2022 at 03:06:24PM -0700, Ross Boylan wrote:
> > In Debian 11/bullseye my system keeps reporting timeouts while trying
> > to bring up the first non-loopback interface. According to ip, the
> > interface actually is up, but ifup/down do not know that. My 2nd
> > interface is down, and there is no mention of attempting to bring it
> > up in the logs. I can bring up both interfaces after startup,
> > suggesting there may be something special about the initial
> > environment that is causing trouble, but I don't know what.
> >
> > I'd appreciate any suggestions about how to diagnose or cure the
> > problem. I have set VERBOSE=yes in /etc/default/networking
> >
>
> Cut /etc/network/interfaces down to the lo stanza
>
> # The loopback network interface
> auto lo
> iface lo inet loopback
>
> Remove EVERYTHING else.

Actuall, the lo stanza can also be removed because ifupdown will
deal with it without any help. See the changelog.Debian.

--
Brian.

Curt

unread,
Aug 27, 2022, 8:00:05 AM8/27/22
to
On 2022-08-27, Jeremy Ardley <jer...@ardley.org> wrote:
>>
>> I'd appreciate any suggestions about how to diagnose or cure the
>> problem. I have set VERBOSE=yes in /etc/default/networking
>>
> First of all ensure NetworkManager is really dead.

Your advice and the advice of Andrew M.A. Cater appear
to be antithetical.

This is only an observation and not a criticism of any kind.

Maybe it's not a useful observation, though. I don't know.

to...@tuxteam.de

unread,
Aug 27, 2022, 8:30:05 AM8/27/22
to
On Sat, Aug 27, 2022 at 11:55:44AM -0000, Curt wrote:
> On 2022-08-27, Jeremy Ardley <jer...@ardley.org> wrote:
> >>
> >> I'd appreciate any suggestions about how to diagnose or cure the
> >> problem. I have set VERBOSE=yes in /etc/default/networking
> >>
> > First of all ensure NetworkManager is really dead.
>
> Your advice and the advice of Andrew M.A. Cater appear
> to be antithetical.
>
> This is only an observation and not a criticism of any kind.

There are different flavours around. I'm more the ifupdown type.
When I see Network Manager, I run away. When I see systemd...
no, I don't see systemd. Then there's the NM flavour. Then, the
systemd-networkd flavour.

There's room for all :)

Cheers
--
t
signature.asc

Jeremy Ardley

unread,
Aug 27, 2022, 8:40:05 AM8/27/22
to
My experience with linux networking has a history of banging my head
against poorly explained, often buggy networking configurations.

I was sort of O.K. with the /etc/network/interfaces approach and could
manage simple networks more or less reliably.

Then NetworkManager intruded and I could see for very simple uses it
more or less worked and was perhaps easier for newbies. NM worked in
addition to the /etc/network/interfaces.

My problems really came about when I started trying to use
NetworkManager and a bunch of add-on utilities to add IPv6 delegation to
my debian router. It became obvious NM was not my friend and did a lot
of things that were not really explicable. My networking was not
consistent or reliable.

I took a deep breath and dived into systemd-networkd and immediately
found many people did not trust systemd-netword and systemnd.
Nevertheless. I read the manual many times and I could get networkd to
do everything I wanted in a predictable manner. I did have to go through
eradicating NetworkManager and removing some of the add-ons such as
radvd. I was also using isc-dhcp client and it was buggy as well. It's
now deprecated. Going to systemd-networkd removed that as one of several
sources of problems.


So from my point of view: networkd pretty good. NetworkManager and
friends - nah!

The only thing I did do out of the ordinary was disable systemd-resolved
which is still problematic. I replaced it with bind as a forwarder and
reconfigured the resolve mechanism.

To make up for my going to the dark side with networkd I set up one of
my servers with Devuan (actually it was Armbian and I did a risky but
successful in-place conversion). Networking on the devuan system is
straight forward (though they still allow NetworkManager). And the lack
of systemd makes it seem familiar and easy to use)

Getting back to the OP. I recall he was complaining about NetworkManager
still appearing in the logs. I gave him advice to make absolutely sure
it couldn't be started by some obscure systemd process.

I still don't really trust systemd, but I'm quite happy with the subset
systemd-networkd.


--
Jeremy

OpenPGP_signature

Jeremy Ardley

unread,
Aug 27, 2022, 9:10:05 AM8/27/22
to
Further to my longer reply on this list, there are three separate
network configuration & management services that can be running at the
same time on a debian system

* networking.service
* systemd-networkd.service
* NetworkManager.service

The usual mode of operation under systemd has

* networking.service always enabled
* NetworkManager.service is usually enabled by default
* systemd-networkd.service may or may not be enabled by default, but
usually has no meaningful configuration

My experience is to leave networking.service handling loopback and then
have either systemd-networkd.service to manage the more complex stuff,
or have NetworkManager.service do this.

It's really bad news to have both NetworkManager.service and
systemd-networkd.service trying to work at the same time.

If you have a laptop and/or wireless connections and you are a relative
newbie, use NetworkManager.service and NetworkManager GUI to control
stuff and will mostly do what you want - but not always, and not
transparently. You may find rebooting the system is the only way to fix
problems.

If you want absolute reliable control then networking.service on its own
is a good simple choice. As an alternative, systemd-networkd-service may
give you more features and still be reliable.

--
Jeremy

OpenPGP_signature

Greg Wooledge

unread,
Aug 27, 2022, 10:00:05 AM8/27/22
to
On Sat, Aug 27, 2022 at 09:07:49PM +0800, Jeremy Ardley wrote:
> Further to my longer reply on this list, there are three separate network
> configuration & management services that can be running at the same time on
> a debian system
>
> * networking.service
> * systemd-networkd.service
> * NetworkManager.service
>
> The usual mode of operation under systemd has
>
> * networking.service always enabled
> * NetworkManager.service is usually enabled by default
> * systemd-networkd.service may or may not be enabled by default, but
> usually has no meaningful configuration
>
> My experience is to leave networking.service handling loopback and then have
> either systemd-networkd.service to manage the more complex stuff, or have
> NetworkManager.service do this.

For the record, NetworkManager.service is typically only installed if
you select a Desktop Environment (GNOME, KDE, etc.). On a system without
one of those beasts installed:

unicorn:~$ systemctl status NetworkManager.service
Unit NetworkManager.service could not be found.

In which case, network interface configuration is probably handled
exclusively by networking.service (a.k.a. /etc/network/interfaces).

None of these configurations is "right" or "wrong". All that matters
is getting one of them to work for you. If you prefer NM, then by all
means use NM -- even if you have to install it separately. If you
prefer interfaces(5), then use that.

Ross Boylan

unread,
Aug 29, 2022, 2:40:06 AM8/29/22
to
Thanks Andrew, Felix, Jeremy, Anssi, Curt, Tomas and Greg for your
suggestions and comments.

I followed several of them by trimming my network/interfaces file to
nothing and then slowly adding stuff back.

The blank file and the one with only the loopback worked:
networking.services reported successful completion, and everything
mentioned in interfaces was configured.

However, adding the ethlan stanza and auto declaration failed.
Removing the 2 dns-* lines allowed it to complete successfully. This
is consistent with the fact that the hangup seemed to be on
/etc/network/if.up/000resolvconf which acts on the dns-* lines.
However, I still can't see why it should have hung up.

Adding in ethworld with an auto again produces a failure and doing
ifdown --force ethworld; ifup -v ethworld did not entirely fix it: the
result of the second command was
#,,,,,
/sbin/ip link set dev ethworld up
/sbin/ip route add default via 66.181.128.1 dev ethworld onlink
RTNETLINK answers: File exists
ifup: failed to bring up ethworld

and I was never able to get DNS and routing in a functional state
(likely the routing was the key to the problem since the default route
was to ethlan, leaving no way to reach external DNS).
Note that the ethworld stanza also includes dns-* directives; I guess
these account for the hangup in ethworld, but I haven't tested that
theory.

I removed the auto for ethworld and networking.service was able to
complete without error. Once I manually brought up ethworld,
networking seemed to be OK.

I added the following 2 scripts to help trace what was going on; they
show that in addition to the interface-specific events the scripts are
also called once with special, generic values (e.g., IFACE=--all),
explaining why the scripts seemed to be called too many times. These
were in /etc/network/if-pre-up.d/ and /etc/network/if-up.d/:
------------------- 000debug----------------------
#! /bin/sh
# scrap to trace activity
LFILE=/root/000debug.log
echo "----------------------------------------------------" >> $LFILE
echo "$0 $$">> $LFILE
echo "$*" >> $LFILE
date >> $LFILE
printenv >> $LFILE
echo "----------------------------------------------------" >> $LFILE

--------------------- zzzdebug----------------------------------------
#! /bin/sh
# scrap to trace activity
LFILE=/root/000debug.log
echo "Finishing $0 $$">> $LFILE
date >> $LFILE
echo "----------------------------------------------------" >> $LFILE
------------------------ end------------------------------------------

On the whole ifupdown, systemd, NetworkManager question, I'm trying to
stick with ifupdown for several reasons
1. It's what I'm already using.
2. NetworkManager's intended use, as I understand it, is for a roaming
laptop, not a box that is the main server and router on a home
network. I'm not saying it can't fit my needs, just that I would
anticipate getting it to do so might be difficult (as @Jeremy
reports).
3. The parts of systemd I know are complex, and I'm frankly annoyed by
its apparent desire to take over my whole computer.
4. Some of the components I use for bridging work specifically with
the /etc/network/interfaces file.

I'll admit that ifup/down is not so easy either!

@Jeremy NetworkManager was already disabled. I wasn't aware of
masking; at your suggestion I masked it too. The fact that I still
had problems almost certainly means they are not NetworkManager's
fault.

@Ansi: I think I have wpa_supplicant as a result of a generic desktop
install; I do also have a wireless card in the machine, though I've
never used it and I don't think it works.

Ross Boylan

unread,
Aug 29, 2022, 10:00:05 PM8/29/22
to
The bind9 script I created in /etc/resolvconf/update.d executed
systemctl reload named
near the end (if systemd is active, which it is for me). Adding set
-x to the script showed that this was where the process of bringing up
the interfaces hung up.

named is obviously not active when the network is coming up; I thought
it would just fail. But, probably because named.service says
After=network.service (network, not networking, but I assume they are
related), it blocks. The solution was to test first:
systemctl -q is-active named.service && systemctl reload named >
/dev/null 2>&1 || :
The stuff after the reload command was there all along.

Tracing this back the chain, because the script blocks the resolvconf
-a invocation in /etc/network/if-up.d/000resolvconf, that was the
last script showing on my earlier posts. resolvconf -a is only
invoked if there was some dns information in the interface stanza,
which is why removing dns- lines from ethlan allowed the network to
come up, and why bringing up ethworld automatically failed.

Now everything just works.

Thanks again to everyone.

There are probably some general lessons, though I'm not sure what they
are. Clearly the systemd semantics tripped me up; it's kind of an odd
beast. I understand one of its major goals was to allow startup to
proceed in parallel, which is pretty asynchronous. But it has to
assure that certain things happen in a certain order, which results in
some things being synchronous and blocking. I'm surprised that a tool
intended for use from the command line (systemctl) is blocking.

Ross

Jeremy Ardley

unread,
Aug 30, 2022, 9:00:07 PM8/30/22
to

On 30/8/22 9:56 am, Ross Boylan wrote:
>
> Now everything just works.
>
> Thanks again to everyone.
>
> There are probably some general lessons, though I'm not sure what they
> are. Clearly the systemd semantics tripped me up; it's kind of an odd
> beast. I understand one of its major goals was to allow startup to
> proceed in parallel, which is pretty asynchronous. But it has to
> assure that certain things happen in a certain order, which results in
> some things being synchronous and blocking. I'm surprised that a tool
> intended for use from the command line (systemctl) is blocking.
>
> Ross
>

One of my problems with systemd is the that name resolution is by
default done by resolved. If resolved was bug free that might be O.K.
but it's not - and in a production environment it's not a safe option.

A result of the use of resolved is the start-up and dependency logic. If
you start doing things outside of the plan, you run into all sorts of
problems. I use bind9 on my various machines and have had to go to some
lengths to take resolved out of the equation.

On a similar but different topic. I have a router that connects to an
upstream server and also runs haproxy. The upstream connection uses DHCP
and IPv6 solicitation. The problem is haproxy fails to start when the
upstream connection is not established and configured quickly enough.
What would be very helpful is a systemd way to start haproxy when the
network is established 'as configured'. So far all I can do is run a
cron job to see if haproxy is running and if not, try and restart it.
There has to be a better way.

--
Jeremy

OpenPGP_signature

Greg Wooledge

unread,
Aug 30, 2022, 9:00:07 PM8/30/22
to
On Wed, Aug 31, 2022 at 08:49:29AM +0800, Jeremy Ardley wrote:
> One of my problems with systemd is the that name resolution is by default
> done by resolved.

Not in Debian.

unicorn:~$ systemctl status systemd-resolved
● systemd-resolved.service - Network Name Resolution
Loaded: loaded (/lib/systemd/system/systemd-resolved.service; disabled; ve>
Active: inactive (dead)
Docs: man:systemd-resolved.service(8)
man:org.freedesktop.resolve1(5)
https://www.freedesktop.org/wiki/Software/systemd/writing-network->
https://www.freedesktop.org/wiki/Software/systemd/writing-resolver>

That's the Debian default. I didn't have to disable it, although I
certainly *would* have, had the default been otherwise.

Tixy

unread,
Aug 31, 2022, 4:00:06 AM8/31/22
to
On Wed, 2022-08-31 at 08:49 +0800, Jeremy Ardley wrote:
> A result of the use of resolved is the start-up and dependency logic.

Another problem I had with systemd-resolved on an Ubuntu box was that
it refuses to forward single part names to the DNS server it got by
DHCP (names like 'printer1' as opposed 'printer1.domain'). Instead, it
wants to try and resolve them as Link-Local Multicast Name
Resolution. I tried disabling that feature [1] but it still doesn't
forward, so I disabled systemd-resolved and hard coded /etc/resolv.conf
point to my DNS server.

My Debian boxes don't have this problem because as Greg pointed out,
systemd-resolved isn't enabled on Debian by default (as of the current
Stable release.)

[1] https://bugs.launchpad.net/netplan/+bug/1777523/comments/8

--
Tixy

Anssi Saari

unread,
Aug 31, 2022, 9:20:05 AM8/31/22
to
I was wondering about the same thing, so far I've needed to explicitly
enable systemd-resolved on Debian when I've wanted it.

I wonder what bugs Jeremy has found and reported against
systemd-resolved though. I remember getting a big headache trying to get
interface specific DNS configuration going only to eventually find out
it really wasn't working in the version Debian packaged at the time.

Jeremy Ardley

unread,
Aug 31, 2022, 9:50:06 AM8/31/22
to

On 31/8/22 9:16 pm, Anssi Saari wrote:
>
> I wonder what bugs Jeremy has found and reported against
> systemd-resolved though. I remember getting a big headache trying to get
> interface specific DNS configuration going only to eventually find out
> it really wasn't working in the version Debian packaged at the time.
>

My main problem was unexplained systemd-resolved slowdowns and timeouts
on some DNS queries. It may have been related DNSSEC?
The same queries using named had no problems, so I switched to that for
the local resolver.

I've also just had a look at

man systemd-resolved.service

The configuration seems very complex, especially the multicast DNS and
the myriad variations relating to /etc/resolv.conf. If I have a spare
week and an urgent need for multicast DNS I'll work through it.


--
Jeremy

OpenPGP_signature

Jeremy Ardley

unread,
Aug 31, 2022, 11:10:05 AM8/31/22
to

On 31/8/22 10:45 pm, Chuck Zmudzinski wrote:
>
> I don't use haproxy but I see there is a package for it in the Debian
> repos. I think what you are seeing should be reported as a bug in
> haproxy if you are using the Debian packaged version. The haproxy
> package should start haproxy at the appropriate time during boot,
> and systemd provides the ability to make services such as haproxy
> depend on certain systemd targets being reached before it tries to
> start, such as the network-online target which I think would be
> enough for haproxy to start. But in any case, you might report a bug
> in haproxy and see if the package maintainers can help you out if
> you are using the Debian packaged version.
>
haproxy does retry three (?) times over a period. The problem is my upstream provider can take up to 10 minutes to provide a dhcp address and ipv6 RA.

The network service does start correctly, but lapses into a retry mode when it can't get the full delegation at once.

haproxy requires a configured interface for it to bind to. Typically this means bind to an IP address and port. If the solicitation to the upstream router hasn't happened, there is no IP and port to bind. haproxy does have an (undocumented?) retry feature to repeatedly try to bind over a period.

If any bug request is to be logged, perhaps it should be for haproxy to have configurable binding options including number of retries or time elapsed?

Jeremy

OpenPGP_signature

Anssi Saari

unread,
Sep 2, 2022, 6:40:05 AM9/2/22
to
Chuck Zmudzinski <brch...@netscape.net> writes:

> On 8/31/22 11:03 AM, Jeremy Ardley wrote:
> It also seems to be a ridiculously long time (ten minutes) for your provider
> to configure your interface. I would look for a different provider if they
> can't or won't fix it.

Or maybe just create an overlay for the haproxy systemd unit with
systemd-networkd-wait-online.service in requires= and after=.
0 new messages