error creating thin jail on 15.0

[Graham Menhennitt]

Hello,

I've recently upgraded my server from 14.3 to 15.0. Everything appeared
to work successfully in the upgrade.

I now want to create a thin jail using UFS. I've followed the handbook
section 17.5 exactly except that my jail is named "vaultwarden" instead
of "thinjail". Accordingly, I've replaced "thinjail" by "vaultwarden"
everywhere.

When I try to start the jail, I get an error:

# service jail start vaultwarden
Starting jails: cannot start jail  "vaultwarden":
jail: vaultwarden: mount.devfs:
/usr/local/jails/containers/vaultwarden/dev: No such file or directory
.
#

I tried creating directory /usr/local/jails/containers/vaultwarden/dev.
Then I get a different error:

# service jail start vaultwarden
Starting jails: cannot start jail  "vaultwarden":
1
jail: vaultwarden: /bin/sh /etc/rc: failed
.
#

I'm completely new to jails. Can somebody please give me a clue on how
to diagnose then problem.

Thanks in advance,

    Graham

[James Gritton]

Considering /dev didn't exist in the jail directory, I'm guessing that
nothing exist there. That leaves two possibilities: the files weren't
extracted into the master directory, or they weren't reflected into the
thin jail directory. What it look like when you run:

ls -l /usr/local/jails/templates/15.0-RELEASE
ls -l /usr/local/jails/containers/vaultwarden

Both of those directories should show a root directory listing. If the
first one isn't there, something went wrong in downloading and unpacking
the base file set. I would imagine there'd be some kind of error
message if that were the case. If the second one isn't there, the clone
didn't work. Again, I'd expect an error message. If both of them *do*
show files, there's probably something wrong in the jail.conf.

Another command with potentially useful output is "zfs list -t all". I
would expect to see a line showing the dataset named
zroot/jails/containers/vaultwarden at the mount point
/usr/local/jails/containers/vaultwarden.

- Jamie

[Graham Menhennitt]

Thanks very much for replying, Jamie.

I suspect that you're correct in saying that "files weren't extracted
into the master directory". There is no
directory /usr/local/jails/templates/15.0-RELEASE as you said in your
suggestions. There is only /usr/local/jails/templates/15.0-RELEASE-base
(and /usr/local/jails/templates/15.0-RELEASE-skeleton which is created
by the later mkdir command in the instructions).

I checked my command history and I'm fairly certain that I followed the
instructions correctly.

See below for the results of the "ll" commands, and the relevant extract
from the history.

I see that there is the "ezjail" utility. I may try using that.

Thanks again,

    Graham



# ll /usr/local/jails/templates/
total 8
drwxr-xr-x  15 root wheel 512 23 Dec 15:30 15.0-RELEASE-base
drwxr-xr-x   8 root wheel 512 23 Dec 15:28 15.0-RELEASE-skeleton

# ll /usr/local/jails/templates/15.0-RELEASE-base/
total 60
drwxr-xr-x   2 root wheel 1024 23 Dec 15:26 bin
drwxr-xr-x  15 root wheel 1536 23 Dec 15:26 boot
-r--r--r--   1 root wheel 6070 28 Nov 14:51 COPYRIGHT
dr-xr-xr-x   2 root wheel  512 28 Nov 14:42 dev
lrwxr-xr-x   1 root wheel   12 23 Dec 15:29 etc -> skeleton/etc
lrwxr-xr-x   1 root wheel   13 23 Dec 15:29 home -> skeleton/home
drwxr-xr-x   4 root wheel 2048 28 Nov 14:46 lib
drwxr-xr-x   3 root wheel  512 28 Nov 14:43 libexec
drwxr-xr-x   2 root wheel  512 28 Nov 14:42 media
drwxr-xr-x   2 root wheel  512 28 Nov 14:42 mnt
drwxr-xr-x   2 root wheel  512 28 Nov 14:42 net
dr-xr-xr-x   2 root wheel  512 28 Nov 14:42 proc
drwxr-xr-x   2 root wheel 2560 23 Dec 15:26 rescue
lrwxr-xr-x   1 root wheel   13 23 Dec 15:29 root -> skeleton/root
drwxr-xr-x   2 root wheel 3072 23 Dec 15:26 sbin
drwxr-xr-x   2 root wheel  512 23 Dec 15:29 skeleton
lrwxr-xr-x   1 root wheel   11 28 Nov 14:42 sys -> usr/src/sys
lrwxr-xr-x   1 root wheel   12 23 Dec 15:30 tmp -> skeleton/tmp
drwxr-xr-x  13 root wheel  512 23 Dec 15:29 usr
lrwxr-xr-x   1 root wheel   12 23 Dec 15:30 var -> skeleton/var

# ll /usr/local/jails/templates/15.0-RELEASE-skeleton/
total 24
drwxr-xr-x  30 root wheel 2560 23 Dec 15:27 etc
drwxr-xr-x   2 root wheel  512 23 Dec 15:27 home
drwxr-x---   2 root wheel  512 23 Dec 15:28 root
drwxrwxrwt   2 root wheel  512 23 Dec 15:28 tmp
drwxr-xr-x   3 root wheel  512 23 Dec 15:27 usr
drwxr-xr-x  24 root wheel  512 23 Dec 15:28 var

# ll /usr/local/jails/containers/vaultwarden
total 24
drwxr-xr-x  30 root wheel 2048 23 Dec 15:31 etc
drwxr-xr-x   2 root wheel  512 23 Dec 15:31 home
drwxr-x---   2 root wheel  512 23 Dec 15:31 root
drwxr-xr-t   2 root wheel  512 23 Dec 15:31 tmp
drwxr-xr-x   3 root wheel  512 23 Dec 15:31 usr
drwxr-xr-x  24 root wheel  512 23 Dec 15:31 var

# history
...
  441  mkdir /usr/local/jails/
  442  mkdir /usr/local/jails/{media,templates,containers}
  443  ll /usr/local/jails/
  444  vi /etc/jail.conf
  445  # mkdir /usr/local/jails/templates/15.0-RELEASE-base
  446  mkdir /usr/local/jails/templates/15.0-RELEASE-base
  447  fetch
https://download.freebsd.org/ftp/releases/amd64/amd64/15.0-RELEASE/base.txz
-o /usr/local/jails/media/15.0-RELEASE-base.txz
  448  tar -xf /usr/local/jails/media/15.0-RELEASE-base.txz -C
/usr/local/jails/templates/15.0-RELEASE-base --unlink
  449  cp /etc/resolv.conf
/usr/local/jails/templates/15.0-RELEASE-base/etc/resolv.conf
  450  cp /etc/localtime
/usr/local/jails/templates/15.0-RELEASE-base/etc/localtime
  451  freebsd-update -b /usr/local/jails/templates/15.0-RELEASE-base/
fetch install
  452  mkdir /usr/local/jails/templates/15.0-RELEASE-skeleton
  453  mkdir -p /usr/local/jails/templates/15.0-RELEASE-skeleton/home
  454  mkdir -p /usr/local/jails/templates/15.0-RELEASE-skeleton/usr
  455  mv /usr/local/jails/templates/15.0-RELEASE-base/etc
/usr/local/jails/templates/15.0-RELEASE-skeleton/etc
  456  mv /usr/local/jails/templates/15.0-RELEASE-base/usr/local
/usr/local/jails/templates/15.0-RELEASE-skeleton/usr/local
  457  mv /usr/local/jails/templates/15.0-RELEASE-base/tmp
/usr/local/jails/templates/15.0-RELEASE-skeleton/tmp
  458  mv /usr/local/jails/templates/15.0-RELEASE-base/var
/usr/local/jails/templates/15.0-RELEASE-skeleton/var
  459  mv /usr/local/jails/templates/15.0-RELEASE-base/root
/usr/local/jails/templates/15.0-RELEASE-skeleton/root
  460  cd /usr/local/jails/templates/15.0-RELEASE-base/
  461  mkdir skeleton
  462  ln -s skeleton/etc etc
  463  ln -s skeleton/home home
  464  ln -s skeleton/root root
  465  ln -s ../skeleton/usr/local usr/local
  466  ln -s skeleton/tmp tmp
  467  ln -s skeleton/var var
  468  cp -R /usr/local/jails/templates/15.0-RELEASE-skeleton
/usr/local/jails/containers/vaultwarden
  469  mkdir -p /usr/local/jails/vaultwarden-nullfs-base
  470  vi /usr/local/jails/vaultwarden-nullfs-base.fstab
  471  service jail start vaultwarden
...
#

[James Gritton]

I suspect a copy/paste problem: one of the commands is:
# tar -xf /usr/local/jails/media/15.0-RELEASE-base.txz -C
/usr/local/jails/templates/15.0-RELEASE --unlink

and it wouldn't be hard to turn it into
# tar -xf /usr/local/jails/media/15.0-RELEASE-base.txz -C
/usr/local/jails/templates/15.0-RELEASE0-base --unlink

since that's the name of the tar file without its extension. So you
might be almost there - just rename the directory, and proceed from
there.

I don't know if ezjail does that thin jail setup, but something like
that is certainly worth a try.

- Jamie

[Graham Menhennitt]

Thanks again for replying.

The command that I used is copied directly from section 17.5.2 of the
handbook (at
https://docs.freebsd.org/en/books/handbook/jails/#creating-thin-jail-nullfs).
Perhaps the handbook is wrong.

Anyway I tried your suggested command (without the "-base" in that tar
command and lots of subsequent commands), but it fails in the same way.

Thanks,

Graham

[James Gritton]

OK, I see where we've been going differnt directions. I was using the
ZFS section of the instructions, and you were using the NullFS section
(I just took a closer look at your command history). That way of doing
thing is a lot messier, but it seems things should still work out.

The essential part that seems to not be working is the fstab file,
whnere /usr/local/jails/templates/15.0-RELEASE-base (which has all the
files you've been missing) is mounted onto
/usr/local/jails/vaultwarden-nullfs-base/. With that, you should be
able to see all the regular files (including the dev directory that was
the first sign of something wrong). So perhaps something is going wrong
with that mount.

- Jamie

[Graham Menhennitt]

Yes, you're correct. It appears that I copied the jail.conf entry from
section 17.5.1 instead of 17.5.2. So, it was missing the mount.fstab
entry. Once I corrected that, it all started correctly.

Thanks very much for your help, Jamie.

Regards,

    Graham