kernel panic with new 4.19.43-1.pvops.qubes.x86_64 kernel

154 views
Skip to first unread message

haaber

unread,
May 19, 2019, 9:27:40 PM5/19/19
to qubes-users, marm...@invisiblethingslab.com
after last update my Dell runs in a kernel panic -- reboot spiral. I
retype 4 important lines from a "photo screenshot" :

Initramfs unpacking failed: read error

Kernel panic - not syncing: VFS: Unable to mount root fs on
unknown-block(0,0)

CPU: 3 PID: 1 Comm: swapper/0 Not tainted 4.19.43-1.pvops.qubes.x86_64

Hardware name: Dell Latitude 7390/09386V BIOS 1.5.1 7/12/2018


For the moment, I downgraded to stable 4.19.42-1 kernel, hope someone
can help me to understand / solve this issue ... cheers, Bernhard



Marek Marczykowski-Górecki

unread,
May 19, 2019, 10:26:37 PM5/19/19
to haaber, qubes-users
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Mon, May 20, 2019 at 11:27:32AM +1000, haaber wrote:
> after last update my Dell runs in a kernel panic -- reboot spiral. I
> retype 4 important lines from a "photo screenshot" :
>
> Initramfs unpacking failed: read error

This seems to be the problem. Check if you have enough space for
initramfs (/boot on legacy system, /boot/efi on UEFI). If this partition
is very full, initramfs wouldn't fit. You may remove older
kernel/initramfs to free up some space. Then regenerate initramfs with:

sudo dracut -f --kver KERNEL_VERSION INITRAMFS_PATH

Replace KERNEL_VERSION with actual kernel version - you can copy it from
relevant /boot/vmlinuz-* filename. And replace INITRAMFS_PATH with
actual path:
- for legacy: /boot/initramfs-KERNEL_VERSION.img (fill KERNEL_VERSION)
- for UEFI: /boot/efi/EFI/qubes/initramfs-KERNEL_VERSION.img

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAlziEFUACgkQ24/THMrX
1yxiuAf+OiwCg8B7mVbe8prB7aBEbm56jpAR9p68EHRDd22yvCVKCKHyWFEQDfeL
8/8tI2AKuXbiPNLpD/895CQiOrP4sn0uhQ0f6B5T2ci1qmJuOxUHITo6ex6ZWPkV
FEOvgO++LJDYh3zLUFSuO8S+TvtuIbpIeH/q0FBAzRy3LUK74yvqsceJVXVEejl2
SvaynQ+eNZ77AC7mSwJgldm9zt8T8pu38+lz5VW4XRF0nOxQpDDmxM0S5Sglj9TF
rgBuTb1dO5fCnMTzbXsJAkm8RwPkr8bYENbEQl3iKFi40sNRrgTFyupglSZzfg4Z
IprykmYvN5kBqqUAfccUAa9POSquvw==
=MDy4
-----END PGP SIGNATURE-----

haaber

unread,
May 20, 2019, 12:36:26 AM5/20/19
to qubes-users, Marek Marczykowski-Górecki
Right, thank you Marek for the quick answer. The bad thing is that /boot
in std UEFI install is 200M - which is definitely too small, if you need
to "double" all files - one copy in efi/EFI/qubes and another one in
efi/EFI/BOOT (I suggest changing the automatic-install partition scheme
to use rather 500M for /boot), since "doubling" makes 6-8 kernels to
hold instead of 3-4 ! What would be best remedy? I see two solutions.

a) enlarge /boot (but I only have /dev/nvme0n1p1 200M EFI System and
/dev/nvme0n1p2 238.3G Linux filesystem which means that I would have to
shrink the thinpool inside the second partition ... I am afraid that a
full re-install is faster than solving all issues that will arise from
such an attempt!)

b) are sym / hard links in /boot allowed ?? I guess only the content of
/efi/EFI/BOOT must be an actual file : the content of /efi/EFI/qubes/
should be handled only by the fully-booted system (when updating), but
never by the boot-loader, which, badly enough, insists on /EFI/BOOT. So
could files in efi/EFI/qubes be sym'linked ??

Thanks, best, Bernhard

Marek Marczykowski-Górecki

unread,
May 20, 2019, 7:54:41 AM5/20/19
to haaber, qubes-users
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Mon, May 20, 2019 at 02:36:15PM +1000, haaber wrote:
> Right, thank you Marek for the quick answer. The bad thing is that /boot
> in std UEFI install is 200M - which is definitely too small, if you need
> to "double" all files - one copy in efi/EFI/qubes and another one in
> efi/EFI/BOOT (I suggest changing the automatic-install partition scheme
> to use rather 500M for /boot), since "doubling" makes 6-8 kernels to
> hold instead of 3-4 ! What would be best remedy? I see two solutions.

Bigger /boot/efi is already the case for R4.0.1 installer. But initial
R4.0 indeed had 200M there.

> a) enlarge /boot (but I only have /dev/nvme0n1p1 200M EFI System and
> /dev/nvme0n1p2 238.3G Linux filesystem which means that I would have to
> shrink the thinpool inside the second partition ... I am afraid that a
> full re-install is faster than solving all issues that will arise from
> such an attempt!)

Yes, resizing in this partition layout is non-trivial, reinstall may be
simpler.
Alternatively, you can remove older kernels and reduce installonly_limit
option in /etc/dnf/dnf.conf (default 3) to keep fewer older kernels.

> b) are sym / hard links in /boot allowed ?? I guess only the content of
> /efi/EFI/BOOT must be an actual file : the content of /efi/EFI/qubes/
> should be handled only by the fully-booted system (when updating), but
> never by the boot-loader, which, badly enough, insists on /EFI/BOOT. So
> could files in efi/EFI/qubes be sym'linked ??

This unfortunately won't fly. EFI System Partition (ESP) is accessed
directly by UEFI and needs to be FAT32, which does not support symlinks.

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAlzilXoACgkQ24/THMrX
1ywlgQf/byJT0685151GAUO5BDaQQUGZCHquWNi8vp9rh0w4hGGz8wNHbggPIAxm
GD4htwb7cbkp9qktuICWkUiZsBPnXJ3HlEIRZRRjbrIdslkKHS9mX0OuqlvY0RAJ
flOl9DvrdVokrcPWbqg0q2aKDjffR+w7q2EE9maAJzTNd7OUk7Yk0fNpuPcYtB/x
4jWtwZPZhXV5x9QEvGihsh029JQzdyGKVgR/P43yXXT+9Befp4pXEt46bUYYBo0H
Ozhtr+mPa7F/jZ5xrRaYeCT7Q52idO18T8VGY4D7+8VW7cp4bl5Y4OBX3Wu+B/gF
RYWfYtgqUHDCmeBj8YECKnl1AR4tQw==
=z78S
-----END PGP SIGNATURE-----

haaber

unread,
May 20, 2019, 9:43:09 PM5/20/19
to Marek Marczykowski-Górecki, qubes-users
>> b) are sym / hard links in /boot allowed ?? I guess only the content of
>> /efi/EFI/BOOT must be an actual file : the content of /efi/EFI/qubes/
>> should be handled only by the fully-booted system (when updating), but
>> never by the boot-loader, which, badly enough, insists on /EFI/BOOT. So
>> could files in efi/EFI/qubes be sym'linked ??
>
> This unfortunately won't fly. EFI System Partition (ESP) is accessed
> directly by UEFI and needs to be FAT32, which does not support symlinks.

right. From my old-day knowledge of FAT, hardlinks are possible (though
not really intended) -- meaning that one could hack it to an "unclean"
fs. Probably a bad approach, I would have to re-hack it after each
fs-change.

More natural: is there a way to change the name /efi/EFI/qubes into
/efi/EFI/BOOT ? That would solve all issues most elegantly (and avoid
the annoying copy & rename procure after each xen-update). Couldn't find
"EFI/qubes" in any config file, though. Bernhard

Marek Marczykowski-Górecki

unread,
May 21, 2019, 4:06:42 AM5/21/19
to haaber, qubes-users
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

If that's internal disk, you should be able to configure UEFI to use
/boot/efi/EFI/qubes. In fact, installer should do that for you... You
can do that either in UEFI setup (some vendors have include it
somewhere near boot order setting), or using efibootmgr from within the
system. You want to configure it to boot xen.efi.
As for efibootmgr, see the last step here:
https://www.qubes-os.org/doc/uefi-troubleshooting/#installation-finished-but-qubes-boot-option-is-missing-and-xencfg-is-empty

If that doesn't work with your UEFI, your option is to move EFI/qubes/*
into EFI/BOOT/ after each update. The path is included in relevant
packages, so you can't just configure it different. But you can move
bigger files (xen.efi, vmlinuz, initramfs) instead of copying to save
some space.

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAlzjsYoACgkQ24/THMrX
1yynVggAlT5YRHvBk+zAvWi3I4aIakjwgTl3BOfs8jIHOZhR+IqESW/i0aQTFKgR
cnXj00PvrV1Y0IoCRIzNYpJQAU1nSN9NgI8g/m+FJWPkQ9KKgZvulC39Eh4eQB4e
MBmqB2Uzu1w3bepygVh7w02IfOaNtlUOWbe18dWOhXdlPtnZ6Y/O+zeW43Y+djAY
2VsbNlKeuh2y7P0l2/qUbMYYN7Y4Me9mEmFvJG1qNdHD7ErExJdre23LXHr0GGsI
cX91O/E08ogIp2cbTKkoaOUQm3HPSbo41926k/SJBsbjmlXw1+tiILxbjSJMXjKp
S0e52ptKrDZ4uX9Di3lXub3rootENg==
=39sz
-----END PGP SIGNATURE-----

haaber

unread,
May 22, 2019, 12:10:37 AM5/22/19
to qubes...@googlegroups.com

> If that's internal disk, you should be able to configure UEFI to use
> /boot/efi/EFI/qubes. In fact, installer should do that for you... You
> can do that either in UEFI setup (some vendors have include it
> somewhere near boot order setting), or using efibootmgr from within the
> system. You want to configure it to boot xen.efi.
> As for efibootmgr, see the last step here:
> https://www.qubes-os.org/doc/uefi-troubleshooting/#installation-finished-but-qubes-boot-option-is-missing-and-xencfg-is-empty
>

Thank you, Marek. I used efibootmgr, that seems alright now. One last
thing worries me: while the initramfs for versions up to and including
4.19.42 where about 23M, the 4.19.43 file is only 8.4M ! Is this maybe a
short write (disk full overlooked/ignored in update script??).
That would explain a kernel panic even better :)) Can I checksum
4.19.43 manually? Or should I reinstall it? Bernhard
Reply all
Reply to author
Forward
0 new messages