D-Bus In Containers?

[vindicator]

Version:

  dbus-broker: 33

  systemd: 255

I didn't want to post this as an "issue" in github since I didn't know if it was dbus or systemd related, or functions as-intended.

I've been exploring "systemd-nspawn" and found that system dbus simply wasn't working.

It was hard to find anything meaningful in the Systemd debug log output, but setting dbus-broker as the default, it got me more information.

First, I found when I "--boot" using "--volatile", the system dbus service starts just fine (same if I use qemu or virtualbox).

But when I DON'T use "--volatile", it fails:

*****

...

Dec 16 17:37:26 containerName systemd[1]: dbus-broker.service: Got notification message from PID 138 (READY=1)
Dec 16 17:37:26 containerName systemd[1]: dbus-broker.service: Changed start -> running
Dec 16 17:37:26 containerName systemd[1]: dbus-broker.service: Job 628 dbus-broker.service/start finished, result=done
Dec 16 17:37:26 containerName systemd[1]: Started D-Bus System Message Bus.
Dec 16 17:37:26 containerName dbus-broker-launch[139]: ERROR launcher_run_child @ ../dbus-broker-33/src/launch/launcher.c +325: Permission denied
Dec 16 17:37:26 containerName dbus-broker-launch[138]: ERROR service_add @ ../dbus-broker-33/src/launch/service.c +1011: Transport endpoint is not connected
Dec 16 17:37:26 containerName dbus-broker-launch[138]:       launcher_add_services @ ../dbus-broker-33/src/launch/launcher.c +804
Dec 16 17:37:26 containerName dbus-broker-launch[138]:       launcher_run @ ../dbus-broker-33/src/launch/launcher.c +1415
Dec 16 17:37:26 containerName dbus-broker-launch[138]:       run @ ../dbus-broker-33/src/launch/main.c +152
Dec 16 17:37:26 containerName dbus-broker-launch[138]:       main @ ../dbus-broker-33/src/launch/main.c +178
Dec 16 17:37:26 containerName dbus-broker-launch[138]: Exiting due to fatal error: -107

...

*****

"Transport endpoint is not connected" looks very pertinent which seems to come from: https://github.com/bus1/dbus-broker/blob/6665abab581c2b223bf0a240a4941f00dbe7a365/src/launch/service.c#L999C9-L999C57

I don't know how I'd be able to test the bus controller from the CLI to see if it's running or whatever.

I know "--volatile" uses a constant/temporary /usr and everything else is recreated temporarily on-the-fly, so I don't know if there's something set (not-recreated) in /var, /etc, /run that would otherwise prevent it from running from a regular boot.

[vindicator]

Looks like it may have something to do with overlays being used...

Succeeds Fully:

*****

$ mkdir ./systemdRoot

$ sudo pacman -S --root ./systemdRoot systemd

$ systemd-nspawn --directory=./systemdRoot --boot

# systemctl status dbus

//"good". dbus broker would be fine too I assume

# systemctl status --user dbus

//"good" after started first

*****

Also:

*****

$ mkdir ./systemdRoot

$ mkdir ./systemdRootBound

$ sudo pacman -S --root ./systemdRoot systemd

$ sudo mount --bind ./systemdRoot ./systemdRootBound

$ systemd-nspawn --directory=./systemdRootBound --boot

# systemctl status dbus

//"good". dbus broker would be fine too I assume

# systemctl status --user dbus

//"good" after started first

*****

Succeeds Partially?:

*****

$ mkdir ./overlayGlibcGccLibs

$ sudo pacman -S --root ./overlayGlibcGccLibs

$ mkdir ./overlayWork

$ mkdir ./overlayMerged

$ mkdir ./overlayBash

$ sudo mount -t overlay overlay \
      -o index=off,metacopy=off,lowerdir=./overlayGlibcGccLibs,upperdir=./overlayBash,workdir=./overlayWork \
      ./overlayMerged

$ sudo pacman -S --root ./overlayMerged

$ sudo umount ./overlayMerged

$ mkdir ./overlaySystemd

$ sudo mount -t overlay overlay \
      -o index=off,metacopy=off,lowerdir=./overlayGlibcGccLibs:./overlayBash,upperdir=./overlaySystemd,workdir=./overlayWork \

      ./overlayMerged
$ sudo mount --bind /proc ./overlayMerged/proc

$ sudo pacman -S --root ./overlayMerged

$ systemd-nspawn --directory=/overlayMerged --boot --volatile

# systemctl status dbus

//"good"

# systemctl status --user dbus

Failed to connect to bus: No such file or directory

*****

Fail:

*****

//Same setup as above, except without "--volatile"

# systemctl status dbus

//"fail": "activating", then "failed (Result: timeout)"

*****

[vindicator]

I *THINK* it may be resolved now. After LOTS of testing of various scenarios, I *THINK* I may not have been clearing out the overlaySystemd and something had been made muddy.

I just now had come across the case of what happens when I wipe out overlayMerged and see the overlay<upperdir> files/dirs all change to "character special file".

So when I wiped out that overlay<upperdir>, remounted, reinstalled systemd, everthing *APPEARS* to be fine again. Learning moment for me here (always is, as I explore).

[vindicator]

Newer finding... Now it looks to be related to permissions. I normally "umask 077", and for no known reason, I decided to change to 022 for this creation session.

After I got everything working with the systemd layer, I started moving onto the next overlay layers where the directories were still 0700 and the weird dbus behavior was happening again after simply installing pacman in that layer.

Even after deleting the content in those folders so there were no chance of being "-type c" files, the overlay folder was still 0700.

I deleted those later layers that had residing and recreated them so they now had 755 perms and the issue was resolved after installing pacman again in that layer and testing it.

"Why?" dunno. It just seems to be more likely the cause than my previous "*THINK*" before.