Bug#933523: debian-installer: efivarfs isn't mounted, install on efi system fails at grub dummy installation

[Leah Oswald]

Package: debian-installer
Version: 20190702
Severity: important
Tags: d-i

Hey, we tried to install debian buster with our preseed file (see: https://paste.debian.net/hidden/ecde0369/) that was working with debian stretch.
Now the installation fails at installing the grub dummy ("Unable to install GRUB in dummy | Executing "grub-install dummy failed."").

The syslog of the installer gives a hint to a missing access to the efi variables:
Jul 30 15:40:04 grub-installer: info: Installing grub on 'dummy'
Jul 30 15:40:04 grub-installer: info: grub-install does not support --no-floppy
Jul 30 15:40:04 grub-installer: info: Running chroot /target grub-install --force "dummy"
Jul 30 15:40:04 grub-installer: Installing for x86_64-efi platform.
Jul 30 15:40:06 grub-installer: grub-install: warning: Cannot read EFI Boot* variables.
Jul 30 15:40:06 grub-installer: grub-install: warning: read_file: could not read from file: Input/output error.
Jul 30 15:40:06 grub-installer: grub-install: warning: vars_get_variable: read_file(/sys/firmware/efi/vars/Boot0001-8be4df61-93ca-11d2-aa0d-00e098032b8c/raw_var) failed: Input/output error.
Jul 30 15:40:06 grub-installer: grub-install: warning: efi_get_variable: ops->get_variable failed: Input/output error.
Jul 30 15:40:06 grub-installer: grub-install: error: failed to register the EFI boot entry: Input/output error.
Jul 30 15:40:06 grub-installer: error: Running 'grub-install --force "dummy"' failed.

After manually mounting the efivarfs the installation of the grub dummy works. We could work arround this issue by adding the following line to our preseed file, but this shouldn't be the solution rather than a workarround.

d-i preseed/late_command string in-target sh -c "mount -t efivarfs efivarfs /sys/firmware/efi/efivars; grub-install; update-grub;"

-- System Information:
Debian Release: 10.0
APT prefers stable
APT policy: (500, 'stable')
Architecture: amd64 (x86_64)

Kernel: Linux 4.19.0-5-amd64 (SMP w/2 CPU cores)
Locale: LANG=de_DE.utf8, LC_CTYPE=de_DE.UTF-8 (charmap=UTF-8), LANGUAGE=de_DE.utf8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

[Steve McIntyre]

Hi Leah,

On Wed, Jul 31, 2019 at 10:19:19AM +0200, Leah Oswald wrote:
>Package: debian-installer
>Version: 20190702
>Severity: important
>Tags: d-i
>
>Hey, we tried to install debian buster with our preseed file (see: https://paste.debian.net/hidden/ecde0369/) that was working with debian stretch.
>Now the installation fails at installing the grub dummy ("Unable to install GRUB in dummy | Executing "grub-install dummy failed."").
>
>The syslog of the installer gives a hint to a missing access to the efi variables:
>Jul 30 15:40:04 grub-installer: info: Installing grub on 'dummy'
>Jul 30 15:40:04 grub-installer: info: grub-install does not support --no-floppy
>Jul 30 15:40:04 grub-installer: info: Running chroot /target grub-install --force "dummy"
>Jul 30 15:40:04 grub-installer: Installing for x86_64-efi platform.
>Jul 30 15:40:06 grub-installer: grub-install: warning: Cannot read EFI Boot* variables.
>Jul 30 15:40:06 grub-installer: grub-install: warning: read_file: could not read from file: Input/output error.
>Jul 30 15:40:06 grub-installer: grub-install: warning: vars_get_variable: read_file(/sys/firmware/efi/vars/Boot0001-8be4df61-93ca-11d2-aa0d-00e098032b8c/raw_var) failed: Input/output error.
>Jul 30 15:40:06 grub-installer: grub-install: warning: efi_get_variable: ops->get_variable failed: Input/output error.
>Jul 30 15:40:06 grub-installer: grub-install: error: failed to register the EFI boot entry: Input/output error.
>Jul 30 15:40:06 grub-installer: error: Running 'grub-install --force "dummy"' failed.
>
>After manually mounting the efivarfs the installation of the grub dummy works. We could work arround this issue by adding the following line to our preseed file, but this shouldn't be the solution rather than a workarround.
>
>d-i preseed/late_command string in-target sh -c "mount -t efivarfs efivarfs /sys/firmware/efi/efivars; grub-install; update-grub;"

Hmmm, that's odd. I'd expect the efivarfs stuff to Just Work (TM) for
you. Can you share a full syslog please? Curious to see what's going
on.

--
Steve McIntyre, Cambridge, UK. st...@einval.com
Into the distance, a ribbon of black
Stretched to the point of no turning back

[Leah Oswald]

Hey Steve,

forgot to mention that you need to disable the automatic grub installation for the workaround.

I've run the installation with the preseed file mentioned above and this is the syslog: https://drop.leah.is/pJqPJ62E/+inline

Regards Leah

[ZHENGYUAN]

Hi Steve,

I'm encountering the same problem here as Leah. In my case, my attempt was manual installation of Debian Buster (10.2) amd64 on my Supermicro X9DRL-3F running UEFI mode. My situation also followed the exact circumstances of OP in this StackExchange thread (including same X9 motherboard family).

Do let me know how I can help to provide more information to identify the cause of this issue.

Best Regards,

ZY

[Sebastian Neuser]

Hello there!


I ran into what seems to be the same problem as well and I think I may
have found a solution. Interestingly, the problem occurs only on one in
six/seven (identical, at least in theory) devices.

I stumbled upon https://salsa.debian.org/installer-team/grub-installer/-/commit/5eada0008eede06c97d55adca1a9eb1eb9447aee
and noticed that the actual script which is called by debian-installer
was not touched, so I came up with the following patch:

diff --git a/grub-installer b/grub-installer
index b824cab8..3380e6fc 100755
--- a/grub-installer
+++ b/grub-installer
@@ -64,6 +64,11 @@ if [ "$(udpkg --print-os)" = "linux" ] && [ ! -d "$ROOT/run/udev" ]; then
mount --bind /run $ROOT/run
fi

+# Sometimes we need to manually mount efivarfs: https://bugs.debian.org/933523
+if [ "$(udpkg --print-os)" = "linux" ] && [ -z $(ls "$ROOT/sys/firmware/efi/efivars/") ]; then
+ mount -t efivarfs efivarfs $ROOT/sys/firmware/efi/efivars || error "Mounting efivars failed"
+fi
+
get_serial_console() {
# Get the last 'console=' entry (if none, the whole string is returned)
local defconsole="$(sed -e 's/.*\(console=[^ ]*\).*/\1/' /proc/cmdline)"
@@ -1429,6 +1434,10 @@ fi
db_progress STEP 1
db_progress STOP

+if [ "$(udpkg --print-os)" = "linux" ] && ! umount $ROOT/sys/firmware/efi/efivars; then
+ info "Failed to unmount /sys/firmware/efi/efivars in $ROOT"
+fi
+
if [ "$(udpkg --print-os)" = "linux" ] && ! umount $ROOT/run; then
info "Failed to unmount /run in $ROOT"
fi

I'll create a merge request as soon as my account on salsa.debian.org is
approved. :-)


Cheers!

PizZaKatZe

[John Paul Adrian Glaubitz]

Hello!

On 7/21/21 12:33 PM, Sebastian Neuser wrote:
> I ran into what seems to be the same problem as well and I think I may
> have found a solution. Interestingly, the problem occurs only on one in
> six/seven (identical, at least in theory) devices.
>
> I stumbled upon https://salsa.debian.org/installer-team/grub-installer/-/commit/5eada0008eede06c97d55adca1a9eb1eb9447aee
> and noticed that the actual script which is called by debian-installer
> was not touched, so I came up with the following patch:

That's more like a workaround than a proper fix. The question is why doesn't the
postinst succeed in mounting efivarfs all the time.

Are you seeing any log messages indicating that mountvirtfs() failed during
the installation of the grub-installer udeb?

Adrian

--
.''`. John Paul Adrian Glaubitz
: :' : Debian Developer - glau...@debian.org
`. `' Freie Universitaet Berlin - glau...@physik.fu-berlin.de
`- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913

[John Paul Adrian Glaubitz]

Hi Sebastian!

On 7/21/21 2:55 PM, Sebastian Neuser wrote:

> On Wed, 2021-07-21 at 12:51 +0200, John Paul Adrian Glaubitz wrote:
>> Are you seeing any log messages indicating that mountvirtfs() failed
>> during the installation of the grub-installer udeb?
>

> So *that*'s what the postinst file is for! ;-)

Yes. It gets executed when the udeb is installed inside the running d-i
system.

> No, I see no messages indicating any errors. I just double-checked by
> monitoring `tail -f /var/log/syslog | grep efivars` during installation.
> I did see a debug message I added right before the call to mountvirtfs
> but no error message.
> Also, if I understand correctly, in case of an error the script would
> `exit 1` and the installation process would stop.

That's probably because of the "|| true" that Steve added to the mount call
which means that this line will always succeed even when the mount attempt
actually failed.

[Sebastian Neuser]

Hello!

On Wed, 2021-07-21 at 12:51 +0200, John Paul Adrian Glaubitz wrote:

> Are you seeing any log messages indicating that mountvirtfs() failed
> during the installation of the grub-installer udeb?

So *that*'s what the postinst file is for! ;-)

No, I see no messages indicating any errors. I just double-checked by
monitoring `tail -f /var/log/syslog | grep efivars` during installation.
I did see a debug message I added right before the call to mountvirtfs
but no error message.
Also, if I understand correctly, in case of an error the script would
`exit 1` and the installation process would stop.

Kind regards,

Sebastian

[Sebastian Neuser]

Hello Adrian,

On Wed, 2021-07-21 at 14:59 +0200, John Paul Adrian Glaubitz wrote:
> > Also, if I understand correctly, in case of an error the script
> > would `exit 1` and the installation process would stop.
>
> That's probably because of the "|| true" that Steve added to the mount
> call which means that this line will always succeed even when the
> mount attempt actually failed.

I don't think so:
mountvirtfs() mounts efivarfs with `mount ... || die ...` and die() ends
with `exit 1`, so `|| true` after the call to mountvirtfs() is actually
dead code.

Unless I still don't get shell script logic, which is of course entirely
possible. :-)


Best wishes,

Sebastian

[Emanuele Rocca]

Hi,

Just to confirm your understanding of the logic, indeed in case of mount
efivarfs errors the script should fail because of the die() calls, see
https://bugs.debian.org/1031183.

Why did the mount command seemingly succeed in this case and yet the FS did not
get mounted I guess is the open question.

Is this bug still reproducible with a recent installer image?

[Sebastian Neuser]

Hello there!

On Sun, 2023-02-12 at 22:03 +0100, Emanuele Rocca wrote:
> Why did the mount command seemingly succeed in this case and yet the
> FS did not get mounted I guess is the open question.

Indeed.

> Is this bug still reproducible with a recent installer image?

Unfortunately, we just returned our customer's hardware earlier today,
so I'm unable to test this. :-/


Greetings

PizZaKatZe