Debian installation using debootstrap and grub-install - no entry in ESC boot menu

[Valentin Caracalla]

Hello everyone,

I'm trying to install Debian on my Asus UX31A using command line utilities like debootstrap and grub-install. However, the installed system is not bootable. The problem is that the internal drive (which I install the system to) doesn't show up in the boot menu (which is what the user sees when pressing ESC during power-on).

I created a minimalist recipe demonstrating the issue:

sudo parted /dev/sda mklabel gpt
sudo parted /dev/sda mkpart init 0% 50%
sudo parted /dev/sda mkpart root 50% 100%
sudo parted /dev/sda set 1 bios_grub on
sudo mkfs.ext4 /dev/sda2
sudo mount /dev/sda2 /mnt
sudo debootstrap stable /mnt
sudo mount --bind /sys /mnt/sys
sudo mount --bind /proc /mnt/proc
sudo mount --bind /dev /mnt/dev
sudo mount --bind /dev/pts /mnt/dev/pts
sudo mount --bind /run /mnt/run
sudo chroot /mnt apt install grub-pc
sudo chroot /mnt grub-install /dev/sda
sudo umount /mnt/run
sudo umount /mnt/dev/pts
sudo umount /mnt/dev
sudo umount /mnt/proc
sudo umount /mnt/sys
sudo umount /mnt

I've intentionally stripped the parts concerning installation of a kernel and creating configuration files like grub.cfg and fstab, these things work already. For now, all I want to see is that the user can get a GRUB command line after power-on.

The grub-install command outputs "Installation finished. No error reported." and therefore I expect being able to boot into the GRUB command line. But again, the problem is that the ESC boot menu doesn't show an entry for (the model name of) /dev/sda, so I can't boot into it.

Previously, I've successfully installed Debian using official installation media on this machine (also using BIOS boot interface), so I know that it works in principle. But now I want to do it using command line utilities like debootstrap and grub-install.

Any help would be very appreciated.

Kind regards,
Valentin Caracalla

[Anssi Saari]

Valentin Caracalla <vorube...@tutanota.com> writes:

> Previously, I've successfully installaed Debian using official

> installation media on this machine (also using BIOS boot interface),
> so I know that it works in principle.

I can't see anything wrong with the script. Did that installation use
GPT and a BIOS Boot Partition though?

I guess I have to ask, why not just use UEFI?

[Valentin Caracalla]

There are a few things I forgot to say:

The recipe I posted earlier is executed on a system installed on the external drive /dev/sdb, which I call the installer system. It is also a Debian system, with the recipe's dependencies installed. To reproduce the issue (if you want), I suggest using a Debian Live-CD.

After installation and before attempting to boot I unplug the external drive to make sure it doesn't interfere with the boot process. With the external drive unplugged, there should be exactly one entry in the ESC boot menu, but there is none: It only offers me to enter setup, and that is what it does when I boot without pressing ESC.

Instead of booting the computer directly, I also tried booting the internal drive in a VM executed on the installer system using the following command:

sudo qemu-system-x86_64 -accel kvm -smp 2 -m 2G /dev/sda

This will show a GRUB command line as I expected. It just doesn't work on the real system, but in the VM it works (I hate that).

[Valentin Caracalla]

> I can't see anything wrong with the script. Did that installation use> GPT and a BIOS Boot Partition though?The successful installation (with official installation media) used aBIOS partition table, but I prefer GPT.> I guess I have to ask, why not just use UEFI?I also tried that and I considered posting a similar recipe for EFI in thefirst message. But it doesn't work either, so I thought it is better toask the question with BIOS, because it seemed easier to me.Here's the recipe for EFI:
sudo parted /dev/sda mklabel gpt sudo parted /dev/sda mkpart init 0% 50% sudo parted /dev/sda mkpart root 50% 100% sudo mkfs.vfat /dev/sda1 sudo mkfs.ext4 /dev/sda2 sudo mount /dev/sda2 /mnt sudo mkdir /mnt/boot sudo mkdir /mnt/boot/efi sudo mount /dev/sda1 /mnt/boot/efi sudo debootstrap stable /mnt sudo mount --bind /sys /mnt/sys sudo mount --bind /proc /mnt/proc sudo mount --bind /dev /mnt/dev sudo mount --bind /dev/pts /mnt/dev/pts sudo mount --bind /run /mnt/run sudo chroot /mnt apt install grub-efi sudo chroot /mnt grub-install /dev/sda sudo umount /mnt/run sudo umount /mnt/dev/pts sudo umount /mnt/dev sudo umount /mnt/proc sudo umount /mnt/sys sudo umount /mnt/boot/efi sudo umount /mntBut this doesn't work either. Same problem here. However I can make such anEFI installation using official installation media on the same machine andthat does work.

[Anssi Saari]

Valentin Caracalla <vorube...@tutanota.com> writes:

> But this doesn't work either. Same problem here. However I can make

> such an EFI installation using official installation media on the same

> machine and that does work.

That recipe (and the whole post) was hard to read but don't you need
some flags for the ESP partition, like esp and possibly boot as well?
The partition table on one EFI system I have looks like this, I think
it's probably what Debian installer created:

# parted /dev/nvme0n1 print
Model: KINGSTON SA2000M8250G (nvme)
Disk /dev/nvme0n1: 250GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:

Number Start End Size File system Name Flags
1 1049kB 538MB 537MB fat32 boot, esp
2 538MB 249GB 248GB ext4 Zippy root
3 249GB 250GB 1024MB linux-swap(v1) Zippy swap swap

[Valentin Caracalla]

I apologize for the formatting in my last post, I don't know what happened. And many thanks for your help!

I checked my partition table using "sudo parted /dev/sda print" and it didn't show any flags for partition 1 (the "init" partition). Therefore I manually set the flags using "sudo parted /dev/sda set 1 boot on" and now it shows both flags, "boot" and "esp":

Model: ATA ADATA XM11 256GB (scsi)
Disk /dev/sda: 256GB

Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:  

Number  Start   End    Size   File system  Name  Flags

1      1049kB  128GB  128GB  fat32        init  boot, esp
2      128GB   256GB  128GB  ext4         root

However, after reboot, the ESC boot menu still doesn't show an entry for the installed system.

[Nicolas George]

Valentin Caracalla (12023-04-25):
> The disadvantage of using EFI is that it doesn't work in QEMU, i.e. the following will not show a GRUB command line:

>
> sudo qemu-system-x86_64 -accel kvm -smp 2 -m 2G /dev/sda

Oh, I must check if the KVM virtual machine booting on UEFI I have been
toying with these lasts few weeks really exists then.

Regards,

--
Nicolas George

[Valentin Caracalla]

By the way:

The disadvantage of using EFI is that it doesn't work in QEMU, i.e. the following will not show a GRUB command line:

sudo qemu-system-x86_64 -accel kvm -smp 2 -m 2G /dev/sda

The same thing works for the BIOS boot interface, however (as in my original recipe).

[Steve McIntyre]

That's just qemu-system-x86_64 defaulting to using SeaBIOS for
firmware. I boot VMs in UEFI mode all the time, using the EDK2 binary
builds in the ovmf package.

--
Steve McIntyre, Cambridge, UK. st...@einval.com
< sladen> I actually stayed in a hotel and arrived to find a post-it
note stuck to the mini-bar saying "Paul: This fridge and
fittings are the correct way around and do not need altering"

[Max Nikulin]

On 25/04/2023 21:40, Valentin Caracalla wrote:
> I checked my partition table using "sudo parted /dev/sda print"
>

> Number  Start   End    Size   File system  Name  Flags
> 1      1049kB  128GB  128GB  fat32        init  boot, esp
> 2      128GB   256GB  128GB  ext4         root

Please, show

ls -lA EFI/BOOT
ls -lA EFI/debian

residing on /dev/sda1 (128GB sounds like unreasonable large ESP
partition, 0.5GB is usually enough, e.g. 550MiB recommended by
https://www.rodsbooks.com/gdisk/advice.html#esp_sizing)

efibootmgr -v

See https://wiki.debian.org/UEFI

[Nicolas George]

Max Nikulin (12023-04-25):

> 0.5GB is usually enough, e.g. 550MiB recommended by
> https://www.rodsbooks.com/gdisk/advice.html#esp_sizing)

If you do not intend to install a Microsoft bootloader or anything
besides GRUB, 16 megaoctets is plenty enough, probably can work with
less.

Regards,

--
Nicolas George

[Valentin Caracalla]

Hello Steve,

thanks a lot for the tip! However, I'm a complete novice when it comes to running custom firmware in QEMU. I just tried the following:

1.: Download the latest EDK2 release from Github:

https://github.com/tianocore/edk2/archive/refs/tags/edk2-stable202302.tar.gz

2.: Extract the OVMF firmware file:

tar xz edk2-edk2-stable202302/OvmfPkg/OvmfPkgX64.fdf <Downloads/edk2-edk2-stable202302.tar.gz
3.: Try to run it with QEMU:

sudo qemu-system-x86_64 -accel kvm -bios edk2-edk2-stable202302/OvmfPkg/OvmfPkgX64.fdf -smp 2 -m 2G /dev/sda
This will just give me a QEMU error message ("could not load PC BIOS"). Is it possible to boot an EFI system with QEMU without downloading a custom firmware file, i.e. is there a Debian package providing this functionality?

Kind regards,
Valentin Caracalla

[Nicolas George]

Valentin Caracalla (12023-04-25):

> thanks a lot for the tip! However, I'm a complete novice when it comes to running custom firmware in QEMU. I just tried the following:
>
> 1.: Download the latest EDK2 release from Github:

I could boot a VM with a UEFI bootloader using only files packaged by
the host operating system, an old Ubuntu LTS, 18 IIRC. I do not have my
notes near at hand, but I found the procedure easily either on some
stackoverflow question or possibly in the answers of an AI search
engine, as I am frequently toying with them these days.

HtH

Regards,

--
Nicolas George

[Nicolas George]

Steve McIntyre (12023-04-25):

> >If you do not intend to install a Microsoft bootloader or anything
> >besides GRUB, 16 megaoctets is plenty enough, probably can work with
> >less.

> Please STOP giving this advice to people!

That was not advice, that was information. Make your own advice with it.

--
Nicolas George

[Steve McIntyre]

My local test scripts for UEFI boot do other things too, but I add the
following:

-pflash <OVMF image> -pflash <vars image>

where I copy matching images from /usr/share/OVMF in the ovmf
package. See the file /usr/share/doc/ovmf/README.Debian for what the
different files offer (secure boot, etc.).

--
Steve McIntyre, Cambridge, UK. st...@einval.com

"Yes, of course duct tape works in a near-vacuum. Duct tape works
anywhere. Duct tape is magic and should be worshipped."
-― Andy Weir, "The Martian"

[Steve McIntyre]

Please STOP giving this advice to people!

Running out of space on the ESP may cause a lot of hassle
later. *Right now*, GRUB is small. But things do grow over time. Also,
if anybody wants to install an extra OS, or use fwupd to install
firmware updates (for example), saving a small amount of disk space
here could cause a massive PITA later.

--
Steve McIntyre, Cambridge, UK. st...@einval.com

[Valentin Caracalla]

Here's the output you requested:

user@host:~$ ls -dl $(find /mnt/boot/efi)
drwxr-xr-x 3 root root   32768 Jan  1  1970 /mnt/boot/efi
drwxr-xr-x 3 root root   32768 Apr 25 13:59 /mnt/boot/efi/EFI
drwxr-xr-x 2 root root   32768 Apr 25 13:59 /mnt/boot/efi/EFI/debian
-rwxr-xr-x 1 root root     108 Apr 25 13:59 /mnt/boot/efi/EFI/debian/BOOTX64.CSV
-rwxr-xr-x 1 root root   84648 Apr 25 13:59 /mnt/boot/efi/EFI/debian/fbx64.efi
-rwxr-xr-x 1 root root     126 Apr 25 13:59 /mnt/boot/efi/EFI/debian/grub.cfg
-rwxr-xr-x 1 root root 4150720 Apr 25 13:59 /mnt/boot/efi/EFI/debian/grubx64.efi
-rwxr-xr-x 1 root root  845480 Apr 25 13:59 /mnt/boot/efi/EFI/debian/mmx64.efi
-rwxr-xr-x 1 root root  934240 Apr 25 13:59 /mnt/boot/efi/EFI/debian/shimx64.efiuser@host:~$ efibootmgr -v               
EFI variables are not supported on this system.

[Nicolas George]

Valentin Caracalla (12023-04-26):

> EFI variables are not supported on this system.

To install GRUB in UEFI, you need to have booted the kernel in UEFI.

Try to find a live image that does, and you can reinstall GRUB from
there.

Regards,

--
Nicolas George

[Max Nikulin]

Unified Kernel Images are coming (kernel + initramfs e.g. to avoid
separate unencrypted /boot), so even 550MiB may become too small
partition in a couple of years.

P.S. Nicolas, it seems your mailer has issues with parsing or formatting
timestamps.

[Max Nikulin]

On 26/04/2023 05:02, Valentin Caracalla wrote:
>
> user@host:~$ ls -dl $(find /mnt/boot/efi)

find /mnt/boot/efi -print0 | xargs -0 ls -dl --

should be more resistant to peculiar file names, but it does not matter
in this case.

...

> -rwxr-xr-x 1 root root     126 Apr 25 13:59 /mnt/boot/efi/EFI/debian/grub.cfg
> -rwxr-xr-x 1 root root 4150720 Apr 25 13:59 /mnt/boot/efi/EFI/debian/grubx64.efi

...

Unless firmware is buggy and it requires EFI/BOOT/BOOTX64.EFI removable
layout, it should be enough (of course with other files that I removed
from the quote).

user@host:~$ efibootmgr -v
> EFI variables are not supported on this system.

Either you run it from qemu booted in BIOS mode or you did not mount to
chroot (I have never tried to manage EFI variables from chroot)

efivarfs on /sys/firmware/efi/efivars type efivarfs
(rw,nosuid,nodev,noexec,relatime)

Likely it is the reason why installer was not able to create a BootXXXX
entry and to adjust BootOrder.

[Greg Wooledge]

On Wed, Apr 26, 2023 at 09:34:11AM +0700, Max Nikulin wrote:
> On 26/04/2023 05:02, Valentin Caracalla wrote:
> >
> > user@host:~$ ls -dl $(find /mnt/boot/efi)
>
> find /mnt/boot/efi -print0 | xargs -0 ls -dl --
>
> should be more resistant to peculiar file names, but it does not matter in
> this case.

find /mnt/boot/efi -exec ls -dl {} +

Also, GNU find has a -ls action, which has a different format, but is
worth a look:

find /mnt/boot/efi -ls

[David Wright]

On Wed 26 Apr 2023 at 09:14:25 (+0700), Max Nikulin wrote:
> On 26/04/2023 00:42, Nicolas George wrote:
> > Steve McIntyre (12023-04-25):

[ … ]

> P.S. Nicolas, it seems your mailer has issues with parsing or
> formatting timestamps.

Don't knock it! The Human Era is much easier for us to parse than
the French Republican calendar (pre 2018). Fortunately, we didn't
have to deal with decimal time.

Cheers,
David.

[Nicolas George]

Greg Wooledge (12023-04-25):

> find /mnt/boot/efi -exec ls -dl {} +

zsh
ls -dl /mnt/boot/efi/**/*

Regards,

--
Nicolas George

[Nicolas George]

David Wright (12023-04-25):

> Don't knock it! The Human Era is much easier for us to parse than

;-)

> the French Republican calendar (pre 2018).

I had not realized I had fans devoted to the point of tracking the eras
of my mail attribution. ;-)²

Regards,

--
Nicolas George