Bug#1031183: grub-installer: postinst fails if efivarfs cannot be mounted

[Emanuele Rocca]

Package: grub-installer
Version: 1.186
Severity: important

Hi,

On systems where efivarfs cannot be mounted, the grub installation step fails
even though it would have otherwise worked just fine skipping the mount
efivarfs command, i.e. system installation is successful with this preseed file:

d-i partman/early_command string sed -i 's/mountvirtfs efivarfs/#/' /var/lib/dpkg/info/grub-installer.postinst

The relevant code in d/postinst looks as follows, suggesting the intention to
ignore failures:

mountvirtfs efivarfs /target/sys/firmware/efi/efivars || true

However, mountvirtfs itself is exiting with 1 in case of mount errors:

mountvirtfs () {
fstype="$1"
path="$2"
if grep -q "[[:space:]]$fstype\$" /proc/filesystems && \
! grep -q "^[^ ]\+ \+$path " /proc/mounts; then
mkdir -p "$path" || \
die grub-installer/mounterr "Error creating $path"
mount -t "$fstype" "$fstype" "$path" || \
die grub-installer/mounterr "Error mounting $path"
trap "umount $path" HUP INT QUIT KILL PIPE TERM EXIT
fi
}

I'm unsure as to what the best course of action is here, but perhaps an idea is
to avoid calling "die" when mount fails for efivarfs, and log an error to
/var/log/syslog instead? Of course the relevant umount should be skipped too.

In any case, the "|| true" part in the mountvirtfs efivarfs call should
probably be dropped.

Please note that this issue is different from https://bugs.debian.org/933523.
In that case, installing grub fails *because* efivarfs does not get mounted
properly, and the surprising bit is that the mountvirtfs efivarfs call does
*not* fail for some reason. :-)

FTR here's the error I get trying to mount efivarfs manually:

~ # mount -t efivarfs efivarfs /target/sys/firmware/efi/efivars
mount: mounting efivarfs on /target/sys/firmware/efi/efivars failed: Operation not supported

Thanks!
ema

[Emanuele Rocca]

control: tag -1 patch

On Sun, Feb 12, 2023 at 09:46:34PM +0100, Emanuele Rocca wrote:
> I'm unsure as to what the best course of action is here, but perhaps an idea is
> to avoid calling "die" when mount fails for efivarfs, and log an error to
> /var/log/syslog instead? Of course the relevant umount should be skipped too.
>
> In any case, the "|| true" part in the mountvirtfs efivarfs call should
> probably be dropped.

Patch at https://salsa.debian.org/installer-team/grub-installer/-/merge_requests/11

[Arnaud Rebillout]

Hello,

I'm testing a Kali Linux mini.iso, in a BIOS virtual machine. Kali uses
the grub-installer package from Debian, no change and it's up-to-date.

The postinst still fails at this point. The error can be reproduced from
the console:

  mkdir -p /target/sys/firmware/efi/efivars
  mkdir: can't create directory '/target/sys/firmware/efi': Operation
not permitted

I suppose the mkdir call must also be allowed to fail when fstype is
efivarfs (following the same logic that is already used for mount).

Do you want me to submit a patch?

NB: the issue only affects a BIOS VM. If I boot a UEFI VM, everything
works fine.

Thanks

--
Arnaud Rebillout / OffSec / Kali Linux Developer

[Roland Clobus]

Control: affects 1039710 grub-installer

Hello Arnaud,

This looks to be very similar that I reported in the first part of
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1039710
On BIOS-boots the EFI /sys mount is not present.

With kind regards,
Roland Clobus

[Arnaud Rebillout]

On 12/07/2023 21:57, Roland Clobus wrote:
>
> This looks to be very similar that I reported in the first part of
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1039710
> On BIOS-boots the EFI /sys mount is not present.

I've read #1039710, I can confirm about the syslog issue,
/var/log/syslog has just one line in it.

Regarding GRUB failing to install, I forgot to mention how it actually
looks like: a error screen, red, stating "Failed to mount /target/proc".
The message is pretty misleading though, it's hardcoded in the template,
and it can show up in various situations: when mkdir fails, or when
mount fails, and for either /proc, /sys or /sys/firmware/efi/efivars...

Cheers,

[Arnaud Rebillout]

Following up with that, it seems that the failure is due to a change in
the kernel.

I'm testing 2 Kali netinst ISOs, one from last week (no problem), and a
daily iso from today (which fails).

Last weekly iso had kernel 6.1.0-kali9-amd64. In this iso, `cat
/proc/filesystems | grep efi` gives nothing, hence the grub-installer
postinst doesn't try to mount the efivarfs.

Today's iso has kernel 6.3.0-kali1-amd64. In this iso, `cat
/proc/filesystems | grep efi` gives `nodev efivarfs`, hence the
grub-installer postinst tries to mount efivarfs, and fails.

[Arnaud Rebillout]

Tentative fix, for what it's worth:

https://salsa.debian.org/installer-team/grub-installer/-/merge_requests/19

[Philip Hands]

Arnaud Rebillout <arn...@kali.org> writes:

> Tentative fix, for what it's worth:
>
> https://salsa.debian.org/installer-team/grub-installer/-/merge_requests/19

The original code there seems a bit tangled, and the need to check for
efivarfs in two places seemed suboptimal, so here's an attempt to
improve on it, including making the displayed error less misleading:

https://salsa.debian.org/philh/grub-installer/-/commit/f14c5e70

[ It didn't seem worth distinguishing between the mkdir or the mount
failing on-screen, so I've just logged that instead, and having done
that, since there would only be one call to die() and I'd need to pass
extra parameters for the error substutions, I just got rid of die() and
put the code inline instead. ]

To see what that looks like, I temporarily disabled the efivarfs test:

https://salsa.debian.org/philh/grub-installer/-/commit/fcb794f6

then one gets to see this error:

https://openqa.debian.net/tests/176310#step/grub/5

=-=-

However, I am now wondering whether we might not be better off using
`archdetect` to see if we're on an efi system, and then make the attempt
to call mountvirtfs for the efivarfs conditional on that.

Cheers, Phil.
--
Philip Hands -- https://hands.com/~phil

[Philip Hands]

Philip Hands <ph...@hands.com> writes:
...

> However, I am now wondering whether we might not be better off using
> `archdetect` to see if we're on an efi system, and then make the attempt
> to call mountvirtfs for the efivarfs conditional on that.

After a diversion[1], I had a look at the archdetect option, and have
discovered that this simple patch:

https://salsa.debian.org/philh/grub-installer/-/commit/6f33bd183d7d0ced76958440534407dc9d0ad141

fixes the UEFI boot, without breaking the BIOS boot (on amd64 at least,
while doing a minimal install):

https://openqa.debian.net/tests/overview?version=testing&build=--20230820_1958_salsa&distri=debian

I hope/assume that all the arches that need this have the good grace to
return `efi` as their subarch. If there's any risk that's not the case,
we could also apply the previous patch.

If you want to do your own tests, the mini.iso can be downloaded via:

https://salsa.debian.org/philh/grub-installer/-/jobs/4582667/artifacts/file/debian/output/debian-202306XX+salsaci+20230820+228-amd64-gtkmini.iso

BTW that's in the artifacts of the `mini-ISO` job, which can be found by
looking at the pipeline linked from the commit above[2], and looking to
the right for the Downstream pipelines -- further right you can find
links to the openQA jobs above.

Cheers, Phil.

[1] rewriting branch2repo so that it doesn't need a state repo. for
many use-cases, which will hopefully allow anyone to run these D-I tests
on salsa, without any special setup.

[2] https://salsa.debian.org/philh/grub-installer/-/pipelines/567684

[Pascal Hambourg]

On 21/08/2023 at 11:56, Arnaud Rebillout wrote:

> On 21/08/2023 04:03, Philip Hands wrote:
>> If you want to do your own tests, the mini.iso can be downloaded via:
>>
>> https://salsa.debian.org/philh/grub-installer/-/jobs/4582667/artifacts/file/debian/output/debian-202306XX+salsaci+20230820+228-amd64-gtkmini.iso
>

> Hello, I download it and tested it in a QEMU VM. I tested it two times:
> with and without UEFI enabled. I can report that installation succeeded
> in both cases.

Did anyone test with EFI boot and fallback to legacy mode in partman
("do not force EFI installation") ?

[Pascal Hambourg]

FTR, this is due to the following change in linux 6.3:

commit 301de9a2055357375a4e1053d9df0f8f3467ff00
Author: Johan Hovold <johan+...@kernel.org>
Date: Thu Jan 19 17:42:53 2023 +0100

efivarfs: always register filesystem

The efivar ops are typically registered at subsys init time so that
they are available when efivarfs is registered at module init time.

Other efivars implementations, such as Google SMI, exist and can
currently be built as modules which means that efivar may not be
available when efivarfs is initialised.

Move the efivar availability check from module init to when the
filesystem is mounted to allow late registration of efivars.