[gentoo-user] rescue cd for zfs 2.1 or thereabouts

[John Covici]

Hi. I have been using 5.4 lts kernels for a while, but it seems I
need to change to 5.10 lts -- even Debian is now using 5.10, so it
seems time to do this.

Now, the problem is that I am using zfs and will not give it up, and
the version I have been using 0.8.6 is no longer supported in 5.10
versions of the kernel. So, I need a newer version of zfs and a
rescue cd in case I get into trouble. Sysresc seems to no longer be
compatible withgentoo linux, so what is available? I could use gentoo
catalyst to make something -- I have done that in the past, but its
quite a bit of work and I would prefer if there were something
available I could use out of the box.

Thanks in advance for any suggestions.

--
Your life is like a penny. You're going to lose it. The question is:
How do
you spend it?

John Covici wb2una
cov...@ccs.covici.com

[Miles Malone]

What is it makes you think system rescue cd isnt compatible with
gentoo? it's built on arch these days, but it's still the go-to
rescue and install CD, and I cannot imagine it being an issue. I
suggest giving it a go and make sure it mounts your ZFS array without
issues, and go from there.

It's still the USB boot image I use for absolutely everything in gentoo

[Vitor Hugo Nunes dos Santos]

[John Covici]

On Mon, 23 Aug 2021 05:51:38 -0400,
Miles Malone wrote:
>
> What is it makes you think system rescue cd isnt compatible with
> gentoo? it's built on arch these days, but it's still the go-to
> rescue and install CD, and I cannot imagine it being an issue. I
> suggest giving it a go and make sure it mounts your ZFS array without
> issues, and go from there.
>
> It's still the USB boot image I use for absolutely everything in gentoo
>
> On Mon, 23 Aug 2021 at 18:15, John Covici <cov...@ccs.covici.com> wrote:
> >
> > Hi. I have been using 5.4 lts kernels for a while, but it seems I
> > need to change to 5.10 lts -- even Debian is now using 5.10, so it
> > seems time to do this.
> >
> > Now, the problem is that I am using zfs and will not give it up, and
> > the version I have been using 0.8.6 is no longer supported in 5.10
> > versions of the kernel. So, I need a newer version of zfs and a
> > rescue cd in case I get into trouble. Sysresc seems to no longer be
> > compatible withgentoo linux, so what is available? I could use gentoo
> > catalyst to make something -- I have done that in the past, but its
> > quite a bit of work and I would prefer if there were something
> > available I could use out of the box.
> >
> > Thanks in advance for any suggestions.

Well, I cannot find any zfs packages and it does not say even what
kernel version it has -- but the whole point is I do need zfs and it
seems not to be there.

[Rich Freeman]

On Mon, Aug 23, 2021 at 8:13 AM John Covici <cov...@ccs.covici.com> wrote:
>
> Well, I cannot find any zfs packages and it does not say even what
> kernel version it has -- but the whole point is I do need zfs and it
> seems not to be there.
>

I've also been using the Ubuntu image. It is pretty easy to install
zfs on them (not sure if they have it out of the box, but they have a
working package manager).

--
Rich

[John Covici]

So, which image is it exactly and how to boot up to a virtual console,
rather than some kind of gui?

[Rich Freeman]

On Mon, Aug 23, 2021 at 10:13 AM John Covici <cov...@ccs.covici.com> wrote:
>
> So, which image is it exactly and how to boot up to a virtual console,
> rather than some kind of gui?

It has been a while, but I usually just download their desktop
installer. I suspect their server installer would make it easier to
get to a text console, but there might be a boot option for that on
the desktop version. I don't mind having a GUI - I can always fire up
a terminal and being able to copy/paste from a browser can be helpful.
Getting sshd running in any of them is usually a 1-2 liner.

--
Rich

[Neil Bothwick]

On Mon, 23 Aug 2021 04:15:10 -0400, John Covici wrote:

> Now, the problem is that I am using zfs and will not give it up, and
> the version I have been using 0.8.6 is no longer supported in 5.10
> versions of the kernel. So, I need a newer version of zfs and a
> rescue cd in case I get into trouble.

It doesn't answer your question, other have done that, but FWIW I went
from ZFS 0.8.4 to 2.0 a year ago and it went without a hitch. Just emerge
the later ZFS packages first, check everything works then update your
kernel.


--
Neil Bothwick

I have seen things you lusers would not believe.
I've seen Sun monitors on fire off the side of the multimedia lab.
I've seen NTU lights glitter in the dark near the Mail Gate.
All these things will be lost in time, like the root partition last
week. Time to die.

[Rich Freeman]

On Mon, Aug 23, 2021 at 11:11 AM Neil Bothwick <ne...@digimed.co.uk> wrote:
>
> On Mon, 23 Aug 2021 04:15:10 -0400, John Covici wrote:
>
> > Now, the problem is that I am using zfs and will not give it up, and
> > the version I have been using 0.8.6 is no longer supported in 5.10
> > versions of the kernel. So, I need a newer version of zfs and a
> > rescue cd in case I get into trouble.
>
> It doesn't answer your question, other have done that, but FWIW I went
> from ZFS 0.8.4 to 2.0 a year ago and it went without a hitch. Just emerge
> the later ZFS packages first, check everything works then update your
> kernel.

Yeah, I'm pretty conservative with zfs upgrades but have had no
issues. The one thing I wish is that there were better documentation
of grub compatibility. I have a root partition I haven't upgraded the
features on because I have no idea whether it would break grub, and
you can't reverse this. Sure, I have backups but I really don't want
to deal with the hassle of restoring a filesystem over some feature
I'd have to google to even know what it does.

--
Rich

[Vitor Hugo Nunes dos Santos]

Your root pool setup doesn't really matter for GRUB compatibility. What's important is that you have the proper features setup on your boot pool. That's the one GRUB loads.

[Rich Freeman]

On Mon, Aug 23, 2021 at 11:45 AM Vitor Hugo Nunes dos Santos
<vito...@teknik.io> wrote:
>
> Your root pool setup doesn't really matter for GRUB compatibility. What's important is that you have the proper features setup on your boot pool. That's the one GRUB loads.

Sure, and in this case /boot is on my root pool.

In any case, it would be nice if there was some way to tell what zpool
features grub requires, as it is a limiting factor for whatever pool
/boot happens to be on.

--
Rich

[Meik Frischke]

Am Mo., 23. Aug. 2021 um 10:15 Uhr schrieb John Covici <cov...@ccs.covici.com>:
>
> Hi. I have been using 5.4 lts kernels for a while, but it seems I
> need to change to 5.10 lts -- even Debian is now using 5.10, so it
> seems time to do this.
>
> Now, the problem is that I am using zfs and will not give it up, and
> the version I have been using 0.8.6 is no longer supported in 5.10
> versions of the kernel. So, I need a newer version of zfs and a
> rescue cd in case I get into trouble. Sysresc seems to no longer be
> compatible withgentoo linux, so what is available? I could use gentoo
> catalyst to make something -- I have done that in the past, but its
> quite a bit of work and I would prefer if there were something
> available I could use out of the box.
>
> Thanks in advance for any suggestions.
>

Hi,

LRS (Linux Recovery System) [1] may be an alternative. It is a
Gentoo-based live system with ZFS support and was (partly) inspired by
the move of SystemRescue to Arch.
Latest changelog states:

> ============ 20210820.17 ============
> [X] Multiple updates and rebuilds applied
> [X] Kernel updated to 5.10.60 + ZFS modules

It seems to be updated regularly and may be worth a try.

Regards,
Meik

[1] https://forums.gentoo.org/viewtopic-t-1106306.html

[Neil Bothwick]

On Mon, 23 Aug 2021 13:37:01 -0400, Rich Freeman wrote:

> > Your root pool setup doesn't really matter for GRUB compatibility.
> > What's important is that you have the proper features setup on your
> > boot pool. That's the one GRUB loads.
>
> Sure, and in this case /boot is on my root pool.
>
> In any case, it would be nice if there was some way to tell what zpool
> features grub requires, as it is a limiting factor for whatever pool
> /boot happens to be on.

Aren't new features only applied to newly created pools and datasets? If
/boot was compatible with GRUB it should still be. At least that's how I
read the einfo messages zfs spits out.


--
Neil Bothwick

Don't cry because it's over. Smile because it happened.

[Rich Freeman]

On Mon, Aug 23, 2021 at 1:57 PM Neil Bothwick <ne...@digimed.co.uk> wrote:
>
> On Mon, 23 Aug 2021 13:37:01 -0400, Rich Freeman wrote:
>
> > > Your root pool setup doesn't really matter for GRUB compatibility.
> > > What's important is that you have the proper features setup on your
> > > boot pool. That's the one GRUB loads.
> >
> > Sure, and in this case /boot is on my root pool.
> >
> > In any case, it would be nice if there was some way to tell what zpool
> > features grub requires, as it is a limiting factor for whatever pool
> > /boot happens to be on.
>
> Aren't new features only applied to newly created pools and datasets? If
> /boot was compatible with GRUB it should still be. At least that's how I
> read the einfo messages zfs spits out.

That is correct. I want to know when I can enable the new features.
Of course, it would also be nice to know which features must be
disabled when creating a new boot pool.

It is just frustrating that as far as I can tell, there is no
documentation about what features grub supports. It wouldn't be hard
for them to just say "here are the zpool features known to work in
grub version foo." The best I've found is the Arch wiki, which has
the disclaimer that it is probably out of date and to check the man
pages, but of course the man pages don't actually say anything.

--
Rich

[Neil Bothwick]

On Mon, 23 Aug 2021 14:02:29 -0400, Rich Freeman wrote:

> > Aren't new features only applied to newly created pools and datasets?
> > If /boot was compatible with GRUB it should still be. At least that's
> > how I read the einfo messages zfs spits out.
>
> That is correct. I want to know when I can enable the new features.
> Of course, it would also be nice to know which features must be
> disabled when creating a new boot pool.
>
> It is just frustrating that as far as I can tell, there is no
> documentation about what features grub supports. It wouldn't be hard
> for them to just say "here are the zpool features known to work in
> grub version foo." The best I've found is the Arch wiki, which has
> the disclaimer that it is probably out of date and to check the man
> pages, but of course the man pages don't actually say anything.

All I could find was this:

http://git.savannah.gnu.org/cgit/grub.git/tree/grub-core/fs/zfs/zfs.c#n276

For a program with so much documentation, GRUB seems sorely lacking in
this respect. It makes me glad I decided to keep /boot off my zpools.


--
Neil Bothwick

Is that "woof" feed me; "woof" walk me; "woof" there's a burglar? What??

[Rich Freeman]

On Mon, Aug 23, 2021 at 2:13 PM Neil Bothwick <ne...@digimed.co.uk> wrote:
>
> All I could find was this:
>
> http://git.savannah.gnu.org/cgit/grub.git/tree/grub-core/fs/zfs/zfs.c#n276
>
> For a program with so much documentation, GRUB seems sorely lacking in
> this respect. It makes me glad I decided to keep /boot off my zpools.
>

Even this seems lacking. For example, encryption is not read-only
compatible (which seems obvious), and it isn't listed as compatible in
the source code you linked. However, grub-mount supposedly uses the
grub drivers and it has a command line option to provide an encryption
key. Maybe it is only compatible with the grub-mount command and not
at boot time, but if so that seems like something worth pointing out
since one of the purposes of grub-mount is to test filesystem
compatibility.

--
Rich

[Vitor Hugo Nunes dos Santos]

You set yourself up for failure by sharing the same pool for /boot and root.
Here's the flags you're meant to use for your boot pool:

https://openzfs.github.io/openzfs-docs/Getting%20Started/Debian/Debian%20Buster%20Root%20on%20ZFS.html#step-2-disk-formatting

Under "Create boot pool".
GRUB purposefully has lacking documentation, as they are not friendly towards
ZFS as a whole, and also because most people doing ZFS nowadays do an EFISTUB
setup with no GRUB, exactly to avoid these issues.

[Vitor Hugo Nunes dos Santos]

To expand on this even further:

GRUB usage with ZFS, in my experience, requires the following patch:

https://vhns.com.br/pix/grub-zfs-patch.html

I took it from GRUB's mailling list. I don't really recall who wrote it.

August 23, 2021 4:51 PM, "Vitor Hugo Nunes dos Santos" <vito...@teknik.io> wrote:

> You set yourself up for failure by sharing the same pool for /boot and root.
> Here's the flags you're meant to use for your boot pool:
>
> https://openzfs.github.io/openzfs-docs/Getting%20Started/Debian/Debian%20Buster%20Root%20on%20ZFS.ht

[Neil Bothwick]

On Mon, 23 Aug 2021 19:50:38 +0000, Vitor Hugo Nunes dos Santos wrote:

> GRUB purposefully has lacking documentation, as they are not friendly
> towards ZFS as a whole

That's ridiculous, if they can support it with code, they can support it
with documentation.

> and also because most people doing ZFS nowadays
> do an EFISTUB setup with no GRUB, exactly to avoid these issues.

What about those for whom EFI is not an option?


--
Neil Bothwick

"Most problems go away if you just wait long enough. It might look like
I'm standing motionless but I'm actively waiting for our problems to go
away. I don't know why this works but it does."
Scott Adams, Dilbert comic

[Vitor Hugo Nunes dos Santos]

Don't complain to me on the Gentoo mailling list.

Go bother:

A.The ZFS mailling list
B.The GRUB mailling list

IIRC this has to do with the whole CDDL x GPL situation.

On the topic of not having UEFI as a possibility:

Either upgrade your machine or go back to the steps mentioned above.

I don't mean to be rude, it's just the situation of things right now.
I also wish GRUB properly supported ZFS, to the point of letting me set
all it's features on a single pool and use boot environments.

[John Covici]

On Mon, 23 Aug 2021 14:13:30 -0400,
Neil Bothwick wrote:
>
> [1 <text/plain; US-ASCII (quoted-printable)>]

I did exactly that -- when you emerge zfs, it gives you the create
command, so you can tell by that the features it wants, but there
seems to be no good reason to have a boot pool, I can afford the 1 or
2 gig space.
So back to my original question, I downloaded -- after a lot of
trouble finding it -- the Ubuntu 21.04 live server as they call it,
but I cannot find any documentation as to how to use it as a rescue
disk -- seems to be just an install disk. Am I missing something
here?

[John Covici]

I will definitely check this out -- Thanks.

[Neil Bothwick]

On Mon, 23 Aug 2021 16:57:11 -0400, John Covici wrote:

> So back to my original question, I downloaded -- after a lot of
> trouble finding it -- the Ubuntu 21.04 live server as they call it,
> but I cannot find any documentation as to how to use it as a rescue
> disk -- seems to be just an install disk. Am I missing something
> here?

Use the desktop disc, I don't think the server one is a live environment
(it wouldn't make much sense for a server). Either use an xterm on the
desktop or find the invocation to boot to a console, possibly adding
"systemd.unit=multi-user.target" to the kernel options.


--
Neil Bothwick

You are a completely unique individual, just like everybody else.

[John Covici]

On Mon, 23 Aug 2021 17:21:35 -0400,

Neil Bothwick wrote:
>
> [1 <text/plain; US-ASCII (quoted-printable)>]

> On Mon, 23 Aug 2021 16:57:11 -0400, John Covici wrote:
>
> > So back to my original question, I downloaded -- after a lot of
> > trouble finding it -- the Ubuntu 21.04 live server as they call it,
> > but I cannot find any documentation as to how to use it as a rescue
> > disk -- seems to be just an install disk. Am I missing something
> > here?
>
> Use the desktop disc, I don't think the server one is a live environment
> (it wouldn't make much sense for a server). Either use an xterm on the
> desktop or find the invocation to boot to a console, possibly adding
> "systemd.unit=multi-user.target" to the kernel options.

OK, I will check that out, thanks. It looks like that complete gentoo system
mentioned in the forums is looking decent, except its quite large, I
am checking that one also.

[Neil Bothwick]

On Tue, 24 Aug 2021 02:57:04 -0400, John Covici wrote:

> OK, I will check that out, thanks. It looks like that complete gentoo
> system mentioned in the forums is looking decent, except its quite
> large, I am checking that one also.

You can also add additional modules to sysrescd, so it may be easier to
stick with that and add ZFS to it.

Looking at https://www.system-rescue.org/Modules/ it looks like you just
install them after booting the live USB then run cowpacman2srm to put
everything you installed onto a file that you add to the USB stick. See
the section titled "Creating SRM modules out of pacman packages".


--
Neil Bothwick

Assassins do it from behind.

[Robert David]

Hi John,

my approach is to have EFI partition with staticly compiled grub, alpine
linux rescue system and kernel with tiny initramfs for zfs root.

It works really well, only thing you need to consider is when upgrading
root pool, you will not be able to boot to previous BE with old zfs.
Without upgrading the pool, the transition is just easy as recompiling
new kernel and upgrading the zfs userspace tools.

For the alpine I use script to put a new version on /boot
https://github.com/robertek/root-scripts/blob/master/alpine_recovery_update

and having grub entry:

menuentry "Alpine linux recovery" {
linux /boot/vmlinuz-lts modules=loop,squashfs,sd-mod,nvme quiet
nomodeset
initrd /boot/initramfs-lts
}

The alpine extended version contains zfs modules, so you only need to
"apk add zfs" and then modprobe zfs.

The extended version is little bit bigger, but I'm fine to live with 1G efi
partition.

Robert.

[John Covici]

Well, I never use a boot pool, I boot with ext4 and just do the root
on zfs. But, I was more interested in some external media, so I am
looking at that Linux Recovery file system to see if I can get that
working, or if there is a way to add zfs to the sysresc cd, I might do
that.

[Rich Freeman]

On Tue, Aug 31, 2021 at 9:33 AM John Covici <cov...@ccs.covici.com> wrote:
>
> Well, I never use a boot pool, I boot with ext4 and just do the root
> on zfs. But, I was more interested in some external media, so I am
> looking at that Linux Recovery file system to see if I can get that
> working, or if there is a way to add zfs to the sysresc cd, I might do
> that.
>

Yeah, if I had to use a separate boot partition of any sort, I
probably wouldn't make it zfs. There really isn't any benefit from
using it for that.

It might be FAT32 if I'm using EFI/etc, or it might be ext4.

If I'm concerned about mirroring for something like that I could just
rsync it once a day to zfs. We're talking about a tiny amount of data
that doesn't change much, and if you use a week-old kernel after a
restore it isn't the end of the world.

What I probably should get around to is grokking EFI+linux. I'm not
sure what the cleanest solution for that is these days - I've never
actually set up EFI on linux, mostly because I'm not sure what the
best practice is. I know at a high level that a few options exist. I
want a final experience that is closer to grub2 than
syslinux/lilo/etc, to use an analogy (not saying it should actually
use grub).

--
Rich

[Matt Connell (Gmail)]

On Tue, 2021-08-31 at 10:04 -0400, Rich Freeman wrote:
> What I probably should get around to is grokking EFI+linux.  I'm not
> sure what the cleanest solution for that is these days - I've never
> actually set up EFI on linux, mostly because I'm not sure what the
> best practice is.

I can't speak to best practices, and I'm a long way from an expert on
EFI, but the the instructions on the Gentoo handbook pages on disk
partitioning + grub setup worked as expected for me.

In a small nutshell, you have a small EFI+boot partition, set to type
'EFI System' and formatted FAT32, then tell grub to use it as an EFI
directory when calling grub-install.

[Neil Bothwick]

On Tue, 31 Aug 2021 09:33:06 -0400, John Covici wrote:

> or if there is a way to add zfs to the sysresc cd, I might do
> that.

There is, I posted a link last week.

>> You can also add additional modules to sysrescd, so it may be easier to
>> stick with that and add ZFS to it.
>>
>> Looking at https://www.system-rescue.org/Modules/ it looks like you just
>> install them after booting the live USB then run cowpacman2srm to put
>> everything you installed onto a file that you add to the USB stick. See
>> the section titled "Creating SRM modules out of pacman packages".


--
Neil Bothwick

Snacktrek, n.:
The peculiar habit, when searching for a snack, of constantly
returning to the refrigerator in hopes that something new will have
materialized.

[Michael]

In simple(r) systems where you only boot the same OS you can instead use the
kernel's EFI stub to get the UEFI firmware to load the latest OS kernel
directly from the ESP, without a 3rd party boot manager:

https://wiki.gentoo.org/wiki/EFI_stub

You'll use the efibootmgr to manage the kernel images stored on ESP, or your
UEFI configuration menu if it has this functionality.

https://wiki.gentoo.org/wiki/Efibootmgr


If you are multibooting frequently and getting into the UEFI boot menu to
change the boot order or running efibootmgr is too much hassle, then a 3rd
party boot manager will be useful. Your choice of GRUB, rEFInd, systemd-boot,
syslinux, EFI executable image will be installed and loaded/run by the UEFI
firwmare from the ESP, with which in turn you will select and load your
desired OS.

https://wiki.gentoo.org/wiki/GRUB2

http://www.rodsbooks.com/refind/

https://wiki.gentoo.org/wiki/Systemd-boot

https://wiki.syslinux.org/wiki/index.php?title=Install#UEFI

[Rich Freeman]

On Tue, Aug 31, 2021 at 1:51 PM Michael <confa...@kintzios.com> wrote:
>
> >
> > In a small nutshell, you have a small EFI+boot partition, set to type
> > 'EFI System' and formatted FAT32, then tell grub to use it as an EFI
> > directory when calling grub-install.
>
> In simple(r) systems where you only boot the same OS you can instead use the
> kernel's EFI stub to get the UEFI firmware to load the latest OS kernel
> directly from the ESP, without a 3rd party boot manager:
>
> https://wiki.gentoo.org/wiki/EFI_stub
>
> You'll use the efibootmgr to manage the kernel images stored on ESP, or your
> UEFI configuration menu if it has this functionality.
>
> https://wiki.gentoo.org/wiki/Efibootmgr
>
>
> If you are multibooting frequently and getting into the UEFI boot menu to
> change the boot order or running efibootmgr is too much hassle, then a 3rd
> party boot manager will be useful. Your choice of GRUB, rEFInd, systemd-boot,
> syslinux, EFI executable image will be installed and loaded/run by the UEFI
> firwmare from the ESP, with which in turn you will select and load your
> desired OS.
>

So, which (if any) of these options supports either:

1. An EFI partition plus /boot on zfs (with no limitations on pool
config, ie it can be a root pool).
2. An EFI partition that contains everything.

If I want to use grub+EFI with a zfs root it sounds like I'd need TWO
boot partitions - an EFI partition (FAT32), and a /boot partition
(anything, but if ZFS it needs to have controlled features). That
seems even more messy than what I'm doing now.

--
Rich

[Neil Bothwick]

On Tue, 31 Aug 2021 14:21:35 -0400, Rich Freeman wrote:

> > If you are multibooting frequently and getting into the UEFI boot
> > menu to change the boot order or running efibootmgr is too much
> > hassle, then a 3rd party boot manager will be useful. Your choice of
> > GRUB, rEFInd, systemd-boot, syslinux, EFI executable image will be
> > installed and loaded/run by the UEFI firwmare from the ESP, with
> > which in turn you will select and load your desired OS.
> >
>
> So, which (if any) of these options supports either:
>
> 1. An EFI partition plus /boot on zfs (with no limitations on pool
> config, ie it can be a root pool).
> 2. An EFI partition that contains everything.
>
> If I want to use grub+EFI with a zfs root it sounds like I'd need TWO
> boot partitions - an EFI partition (FAT32), and a /boot partition
> (anything, but if ZFS it needs to have controlled features). That
> seems even more messy than what I'm doing now.

systemd-boot and refind both support everything on EFI. I am pretty sure
GRUB does too, but I have no reason to use GRUB with EFI. My setup on
this box is /boot on FAT32 and / (and everything else) on btrfs. I've
also used the same setup with ZFS.


--
Neil Bothwick

If a book about failures doesn't sell, is it a success?

[Michael]

Any boot option on a UEFI MoBo requires an 'EFI System Partition' (ESP),
formatted as VFAT. The UEFI firmware boot loader will list/load/run any *.efi
software stored in the ESP compatible with the UEFI API, whether this is a
boot loader, a kernel with an EFI stub, or some .efi diagnostic application.

As long as your boot loader of choice, or kernel image and any initrd contains
the requisite fs drivers, there will be no problem mounting and accessing
whatever root fs needs to be accessed.

GRUB contains a number of ZFS modules to do this job (zfscrypt.mod,
zfsinfo.mod, zfs.mod) - not sure about the other boot managers.

Typical GRUB installations have /boot/efi mounted on the ESP, with the
grubx64.efi image on it, while the rest of the files, vmlinuz symlinks, etc.
are on the root partition.

Please beware, I have not used zfs to date, only btrfs, so the above merely
reflects my understanding rather than in depth experience of the difficulty in
managing such a setup.

[Rich Freeman]

On Tue, Aug 31, 2021 at 3:44 PM Michael <confa...@kintzios.com> wrote:
>
> Please beware, I have not used zfs to date, only btrfs, so the above merely
> reflects my understanding rather than in depth experience of the difficulty in
> managing such a setup.

To save you digging through the thread, the issue with zfs is that it
adds new features over time, and grub isn't necessarily compatible
with all of them. You can control which features are enabled on-disk
for compatibility, but grub doesn't do a great job documenting which
features are/aren't supported in any particular version. So, it is a
bit of a guessing game.

It has been pointed out that there are various guides online, but:

1. They don't all say the exact same thing.
2. They aren't official upstream docs.
3. They rarely specify what version of grub they're talking about.

The typical solution is to either use very conservative settings for
your root partition (which isn't ideal from a zfs standpoint), or have
a separate /boot pool which means that you don't have to encumber the
rest of the system with whatever grub's limitations might be. Then
you just never update that partition and it shouldn't break. That
basically is no different than just having /boot on ext4 or vfat or
whatever. With this solution you also can't just freely resize /boot
the way you could if it were part of the pool.

--
Rich

[Michael]

Yes, I understood this much, although I haven't red the various online guides
you refer to above.

Is there some particular need to resize boot? Is there even a need to have a
pool for it? It seems to me having a stand alone simple VFAT partition for /
boot which doubles up as ESP with a /boot/EFI subdirectory will do the job,
although I understand not all use cases are as simplistic as what I envisage
here.

[Jack]

On 2021.08.31 16:20, Michael wrote:
[snip....]

> Is there some particular need to resize boot? Is there even a need
> to have a pool for it? It seems to me having a stand alone simple
> VFAT partition for / boot which doubles up as ESP with a /boot/EFI
> subdirectory will do the job, although I understand not all use cases
> are as simplistic as what I envisage here.

My only need to resize my boot/ESP partition was when I decided to
increase the number of kernel/initrd combinations I wanted to keep
around at one time, and ran out of space.

Jack

[Neil Bothwick]

On Tue, 31 Aug 2021 20:44:50 +0100, Michael wrote:

> > systemd-boot and refind both support everything on EFI. I am pretty
> > sure GRUB does too, but I have no reason to use GRUB with EFI. My
> > setup on this box is /boot on FAT32 and / (and everything else) on
> > btrfs. I've also used the same setup with ZFS.
>
> Any boot option on a UEFI MoBo requires an 'EFI System Partition'
> (ESP), formatted as VFAT. The UEFI firmware boot loader will
> list/load/run any *.efi software stored in the ESP compatible with the
> UEFI API, whether this is a boot loader, a kernel with an EFI stub, or
> some .efi diagnostic application.
>
> As long as your boot loader of choice, or kernel image and any initrd
> contains the requisite fs drivers, there will be no problem mounting
> and accessing whatever root fs needs to be accessed.
>
> GRUB contains a number of ZFS modules to do this job (zfscrypt.mod,
> zfsinfo.mod, zfs.mod) - not sure about the other boot managers.
>
> Typical GRUB installations have /boot/efi mounted on the ESP, with the
> grubx64.efi image on it, while the rest of the files, vmlinuz symlinks,
> etc. are on the root partition.
>
> Please beware, I have not used zfs to date, only btrfs, so the above
> merely reflects my understanding rather than in depth experience of the
> difficulty in managing such a setup.

I find it simpler to make /boot a FAT partition, then /boot/efi is the
ESP and all boot-related files are on the same filesystem. Like you, I
have only used this with btrfs (and ext4 on LVM). I do use ZFS but that
system is not EFI.


--
Neil Bothwick

Theory is when you know everything, but nothing works.
Reality is when everything works, but you don't know why.
However, usually theory and reality are mixed together :
Nothing works, and nobody knows why not.