"Failed to start Create System Users" when booting Debian 10 rootfs from NFS mount.

[Lie Rock]

Hi,

I'm trying to bring up the Debian 10 root file system on an ARM SoC board. When the rootfs was in an SD card the board worked well. When I put the rootfs on an NFS server and tried to boot the board through NFS mount, it reported error through serial port:

[FAILED] Failed to start Create System Users.
See 'systemctl status systemd-sysusers.service' for details.

And this is the only error message printed out. The board went all the way to login inputI, but I could not login with any of the preset accounts including root (because no users have been created as it suggested?), and I didn't see any way to run commands to check system status for details.

So how is the process "create system users" performed when Linux/Debian starts? What can be contributing to this error?

Any suggestions would be greatly appreciated.

Rock

[Greg Wooledge]

On Tue, Aug 16, 2022 at 03:58:30PM -0400, Lie Rock wrote:
> So how is the process "create system users" performed when Linux/Debian
> starts? What can be contributing to this error?

unicorn:~$ grep -ri 'create system users' /lib/systemd
/lib/systemd/system/systemd-sysusers.service:Description=Create System Users

unicorn:~$ systemctl cat systemd-sysusers.service
[...]
Documentation=man:sysusers.d(5) man:systemd-sysusers.service(8)
[...]
ExecStart=systemd-sysusers

unicorn:~$ man systemd-sysusers
[...]
systemd-sysusers creates system users and groups, based on the file
format and location specified in sysusers.d(5).

That's enough to get you started down the rabbit hole(s). You should
also definitely check the logs on your system (e.g.
journaltctl -u systemd-sysusers) to see what *exactly* went wrong.

[to...@tuxteam.de]

On Tue, Aug 16, 2022 at 04:20:36PM -0400, Greg Wooledge wrote:
> On Tue, Aug 16, 2022 at 03:58:30PM -0400, Lie Rock wrote:
> > So how is the process "create system users" performed when Linux/Debian
> > starts? What can be contributing to this error?
>
> unicorn:~$ grep -ri 'create system users' /lib/systemd
> /lib/systemd/system/systemd-sysusers.service:Description=Create System Users

[...]

Good research, and "thank you" from a systemd-abstainer, that's
my way to learn, after all :)

I'd contribute my hunch: perhaps systemd is trying to get sysusers
up "too early", before the root file system is pivoted-in?

Feeding my search engine with "NFS root" and +systemd turns up a
bunch of interesting suggestions (e.g. network has to be up before
NFS has to be mounted, etc:).

Good luck... and tell us what it was ;-)

Cheers
--
t

[mj]

Hi,

A suggestion: we've had issues in the past, where on NFS root the issue
was that setting "Linux Capabilities" (setcap) fails, because NFS does
not support the extended attributes to store them.

Perhaps that is your issue as well?

MJ

Op 16-08-2022 om 21:58 schreef Lie Rock: