docker0 bridge doesn't start at boot

[Mimmus]

 Hi,
I have a server with two network cards and statically assigned IPs by these two units (created by a cloud-config):

 $ cat /etc/systemd/network/static.network
[Match]
MACAddress=00:50:56:83:50:f6
[Network]
Gateway=10.57.1.254
Address=10.57.0.227/23
DNS=10.52.1.37
DNS=10.52.1.38

$ cat /etc/systemd/network/static-ens224.network
[Match]
MACAddress=00:50:56:83:63:86
[Network]
Address=192.168.99.107/24

(the second one is used for NFS).

On this server, docker0 bridge doesn't start at boot but only after a "sudo systemctl restart docker".

Any help to debug if I have any error on my cloud-config?

[Alex Crawford]

On 09/15, Mimmus wrote:
> On this server, docker0 bridge doesn't start at boot but only after a "sudo
> systemctl restart docker".

docker is socket activated by default on Container Linux. You will need
to explicitly enable the service if you are using Ignition to configure
your system. It sounds like your are using cloud-config (which we no
longer recommend) so you'll need to add a "command: start" under the
docker service in the config.

-Alex

--
This email and any files transmitted with it are confidential and intended
solely for the use of the individual or entity to whom they are addressed.
This message contains confidential information and is intended only for the
individual named. If you are not the named addressee you should not
disseminate, distribute or copy this e-mail. Please notify the sender
immediately by e-mail if you have received this e-mail by mistake and
delete this e-mail from your system. If you are not the intended recipient
you are notified that disclosing, copying, distributing or taking any
action in reliance on the contents of this information is strictly
prohibited.

[Mimmus]

Alex,
docker service starts but network is not working because docker0 bridge interface is down.
I suspect some little typo error in systemd unit or cloud-config script and I'm asking if there is a way to debug.

I'm still using cloud-config because I still had no much time to study Ignition and because our deploy system (Red Hat Satellite) offers a cloud-config-based provisioning script that works quite well by default.

[Alex Crawford]

On 09/16, Mimmus wrote:
> docker service starts but network is not working because docker0 bridge
> interface is down.
> I suspect some little typo error in systemd unit or cloud-config script and
> I'm asking if there is a way to debug.

Your best bet is journalctl and networkctl. Looking at my box, I see the
following from `networkctl`:

IDX LINK TYPE OPERATIONAL SETUP
1 lo loopback carrier configured
2 eth0 ether routable configured
3 eth1 ether routable configured
4 docker0 ether routable unmanaged

Are you writing a new docker.service from your cloud-config? If so, you
shouldn't do that and instead configure it via drop-ins. If you can
share your config, I can take a look for any issues.

-Alex

[Mimmus]

No, I'm only doing a basic setup to install Mesosphere DC/OS.
I need a second network interface to mount a common NFS share.

This is the relevant section of cloud-config:

    - name: systemd-networkd.service
      command: stop
    - name: static.network
      command: start
      content: |
        [Match]
        MACAddress=00:50:56:83:75:4b
        [Network]
        Gateway=10.192.1.254
        Address=10.192.0.225/23
        DNS=10.192.1.37
        DNS=10.192.1.38



    - name: static-ens224.network
      command: start
      content: |
        [Match]
        MACAddress=00:50:56:83:50:f0
        [Network]
        Address=192.168.99.105/24

    - name: down-interfaces.service
      command: start
      content: |
        [Service]
        Type=oneshot
        ExecStart=/usr/bin/bash -c 'for i in $(/usr/bin/ls /sys/class/net/); do [ $i == "lo" ] || (/usr/bin/ip link set $i down; /usr/bin/ip addr flush dev $i); done'
    - name: systemd-networkd.service
      command: restart

'networkctl' output:
 $ networkctl

 IDX LINK             TYPE               OPERATIONAL SETUP
  1 lo               loopback           carrier     configured

  2 ens192           ether              routable    configured
  3 ens224           ether              routable    configured

  4 docker0          ether              routable    unmanaged

Thanks a lot

[Alex Crawford]

On 09/21, Mimmus wrote:
> - name: down-interfaces.service
> command: start
> content: |
> [Service]
> Type=oneshot
> ExecStart=/usr/bin/bash -c 'for i in $(/usr/bin/ls
> /sys/class/net/); do [ $i == "lo" ] || (/usr/bin/ip link set $i down;
> /usr/bin/ip addr flush dev $i); done'
> - name: systemd-networkd.service
> command: restart
>
>
>
> 'networkctl' output:
> $ networkctl
> IDX LINK TYPE OPERATIONAL SETUP
> 1 lo loopback carrier configured
> 2 ens192 ether routable configured
> 3 ens224 ether routable configured
> 4 docker0 ether routable unmanaged

My guess is that your down-interfaces.service is taking down the bridge
but nothing brings it back up. Since docker0 is marked as unmanaged,
networkd completely ignores the interface. This assumes that docker is
started before down-interfaces.service. It's impossible to know from
your config though, so you'd have to look at your dependency graphs and
logs.

-Alex

[Mimmus]

It's a good hint!

That section is a more generalized version of what suggested in:
https://coreos.com/os/docs/latest/cloud-config-examples.html#networkd-and-dhcp-behavior-with-cloud-init

[Mimmus]

Thanks to your hint, we solved the problem!

The command ((used to reset network interfaces if static IP is used):

    /usr/bin/bash -c 'for i in $(/usr/bin/ls  /sys/class/net/); do [ $i == "lo" ] || (/usr/bin/ip link set $i down; /usr/bin/ip addr flush dev $i); done'

was too generic, only 'lo' was excluded, better using a logic to include only real interfaces.

Thanks again