Dual booting Qubes and Qubes?

[Claudia]

I'm wondering what the best way is to go about installing two instances
of Qubes 4.1 on the same machine, which are more or less independent of
each other.

It's fairly easy to dual boot different OSes because they each have
their own EFI directory, e.g. /boot/efi/EFI/{qubes,fedora,ubuntu}, but
what happens when you want to dual boot two of the same OS? (Or two
different releases of the same OS?)

My original thought was to just give each one its own directory in
/boot/efi/EFI/, but the /boot/efi/EFI/qubes/ path seems to be hard coded
somewhere, probably either in the grub2-efi package (which installs a
precompiled efi binary to that directory) or in a grub2-install hook
somewhere. Not sure which of those methods Qubes uses. At least, I
couldn't figure out any way to persistently change the name of the EFI
directory. Of course you could make your own directories by copying the
files. e.g. /boot/efi/EFI/qubes{0,1}/, but when it updates (or you
reinstall one of them), they would both try to install to
/boot/efi/EFI/qubes/ again. And you'd have to manually change the path
each time with efibootmgr.

My other thought is to give each instance its own EFI partition, /boot
partition, and root partition. efibootmgr allows you to specify which
partition, by index, the path resides on for a given boot menu entry.
However it's technically out of spec to have more than one EFI partition
on the same device and I don't know if UEFI implementations know how to
handle multiple EFI partitions correctly. Additionally, since EFI
partitions all use the same magic UUID, the OS wouldn't know which one
to mount at /boot/efi, so they could easily get mixed up. I guess you
could reconfigure /etc/fstab to mount based on partition number, e.g.
/dev/sda1, /dev/sda4, but that brings its own set of problems.
Nevertheless it seems like this might be the best option.

Another option is to uninstall grub on one of them, so that only one
instance ever touches /boot/efi/EFI/qubes. And then write a custom
grub.cfg with a menuentry for each instance. However I don't know how
you'd prevent the installer ISO from trying to install a boot loader
initially. A major reason for having two separate instances is so that I
can reinstall one or the other from ISO without much trouble. So I don't
want the installer clobbering my boot loader every time. Additionally
you'd have to manually modify your grub config with the new UUIDs every
time you reinstall.

I'm still new to UEFI systems so I don't know if any of these would
work. Anyone have any ideas or insights? (Funny how UEFI redesigned the
entire boot process, and it's still just as messy as it was before.)

-------------------------------------------------
This free account was provided by VFEmail.net - report spam to ab...@vfemail.net

ONLY AT VFEmail! - Use our Metadata Mitigator to keep your email out of the NSA's hands!
$24.95 ONETIME Lifetime accounts with Privacy Features!
15GB disk! No bandwidth quotas!
Commercial and Bulk Mail Options!

[awokd]

Claudia:

> My original thought was to just give each one its own directory in
> /boot/efi/EFI/, but the /boot/efi/EFI/qubes/ path seems to be hard coded
> somewhere, probably either in the grub2-efi package (which installs a
> precompiled efi binary to that directory) or in a grub2-install hook
> somewhere. Not sure which of those methods Qubes uses. At least, I
> couldn't figure out any way to persistently change the name of the EFI
> directory. Of course you could make your own directories by copying the
> files. e.g. /boot/efi/EFI/qubes{0,1}/, but when it updates (or you
> reinstall one of them), they would both try to install to
> /boot/efi/EFI/qubes/ again. And you'd have to manually change the path
> each time with efibootmgr.

This is partly what I need to do anyways on an older UEFI system. I
wrote a script that copies the updated files over to BOOTX64, and run it
after every update that touches EFI. Shouldn't be too hard to add
efibootmgr to it, and an edit to the .cfg file pointing at the
appropriate root=. Ignore the redundant .efi copies at the end; I'm
still not entirely sure which one my half broken UEFI implementation uses.

rm /boot/efi/EFI/BOOT/init*
rm /boot/efi/EFI/BOOT/vmlinuz*
cp /boot/efi/EFI/qubes/init* /boot/efi/EFI/BOOT/
cp /boot/efi/EFI/qubes/vmlinuz* /boot/efi/EFI/BOOT/
cp /boot/efi/EFI/qubes/xen.cfg /boot/efi/EFI/BOOT/BOOTX64.cfg
ls /boot/efi/EFI/qubes/xen-*.efi
cp /boot/efi/EFI/qubes/xen-*.efi /boot/efi/
cp /boot/efi/EFI/qubes/xen-*.efi /boot/efi/EFI/BOOT/
cp /boot/efi/EFI/qubes/xen-*.efi /boot/efi/EFI/BOOT/BOOTX64.efi

--
- don't top post
Mailing list etiquette:
- trim quoted reply to only relevant portions
- when possible, copy and paste text instead of screenshots

[Claudia]

I see. I'm guessing your firmware only attempts to read from /boot/efi/EFI/BOOT/BOOTX64.efi, even if the menu entry has a different path? I think an approach like this could work for managing EFI directories manually. Automated by a script of course, but still manual in the sense it's not handled by the OS.

I was really hoping there was a config setting or variable somewhere that could change the EFI directory, e.g. to /boot/efi/EFI/<whatever>/. On systems that use grub-install, this is specified by --bootloader-id= argument. So in that case you may be able replace /usr/bin/grub-install with a wrapper script that forces your desired --bootloader-id argument. However Fedora doesn't use grub-install, it uses a package which installs a prebaked grubx64.efi binary which is installed to the hardcoded path /boot/efi/EFI/fedora. This is in order to support Secure Boot. I'm assuming Qubes does it the same way. I'm also assuming that "fedora" gets changed to "qubes" by a patch or a compile-time variable but is not configurable at runtime. Maybe there's an option in Yum/dnf similar to apt's redirect option, to configure a file to be installed at a different path than what it's packaged as. I have no idea.

I did some research, and apparently this is a very common flaw. In fact, Mint uses the hardcoded directory /boot/efi/EFI/ubuntu/, meaning you can't dual boot Mint and Ubuntu from the same ESP. But the good news is that you can use more than one ESP, as long as your firmware supports it (it sounds like most of them do). So that's also an option. I was originally worried it would cause problems that they're using the same UUID, but actually they don't. I was confusing the filesystem UUID, which is unique, with the GPT partition identifier (GUID), which is a magic value. And I remembered you could also mount them by label anyway. https://www.zdnet.com/article/hands-on-linux-uefi-multi-boot-part-three-problem-solving/

So I guess I could use /boot/efi/EFI/qubes/ as a sort of staging area for both instances of Qubes, and then religiously after every update, run a script to move the directory to /boot/efi/EFI/qubes{0,1,...}, and then delete the newly-created efibootmgr entry for \EFI\qubes\grubx64.efi and (if necessary) re-create the one for \EFI\qubesN\grubx64.efi. Or, perhaps there's some way to prevent modification of efibootmgr entries so the package install hook (or whatever) can't mess with it.

I decided I'm going to try the dual-ESP approach first and see if it works. If not, then I'll try the EFI directory hack.

I formatted my disk like: ESP, /boot, root, ESP, /boot, root, (swap); and installed Qubes into the first "slot". I still have to install another Qubes instance into the second "slot" and make sure they both work. I'll follow up when I do.

[Claudia]

December 20, 2019 2:19 PM, "Claudia" <clau...@disroot.org> wrote:
>
> I decided I'm going to try the dual-ESP approach first and see if it works. If not, then I'll try
> the EFI directory hack.
>
> I formatted my disk like: ESP, /boot, root, ESP, /boot, root, (swap); and installed Qubes into the
> first "slot". I still have to install another Qubes instance into the second "slot" and make sure
> they both work. I'll follow up when I do.

The dual ESP approach seems to be working fine for me, but you do have to manually fiddle around with efibootmgr. The installer overwrites existing Qubes entries, although I'm not sure what exactly it looks for. Maybe changing the label would be sufficient to preserve it.

Dual booting R4.1 and R4.0, both using btrfs on dm-crypt. I can't speak to how LVM or anything else might be affected.

[trueriver]

On Sunday, 15 December 2019 17:04:52 UTC, Claudia wrote:

I'm wondering what the best way is to go about installing two instances
of Qubes 4.1 on the same machine, which are more or less independent of
each other.

It's fairly easy to dual boot different OSes because they each have
their own EFI directory, e.g. /boot/efi/EFI/{qubes,fedora,ubuntu}, but
what happens when you want to dual boot two of the same OS? (Or two
different releases of the same OS?)

First point

IF you have got UEFI booting working on your machine, and the Qubes installer

detects that is so, then grub does not come into it. So if you are on 4.0.2rc3 or

later and you still have a grub.cfg file being created at install, then the installer

is not fully recognising your UEFI and is adding the grub2 stuff for legacy boot.

Now to the real point. The way to do this is to create a triple boot within the EFI

partition, one for ecah of your two systems, and one for the Qubes updater. As you

say the updater wants to put things into /boot/efi/EFI/Qubes so we'd better let it do

that.

So after updating the first one, create an OS that is named something different

cd /boot/efi/EFI

cp -rv Qubes Q-foo

and you have the EFI setup that UEFI will believe is called Q-foo.

Go into the machine setup, and figure out how to register this with UEFI. Every

motherboard maker seems to do it differently. On my system I get to choose

what to call it (ie it is not fixed to the directory name)

Add it as a new choice, don't remove the Qubes directory.

Now install or update the other Qubes.

cd /boot/efi/EFI

cp -rv Qubes Q-bar

Go back into setup and register the Q-bar directory.

Now, when you boot, you may get the menu showing three choices,

or you may need to press esc or one of the function keys to see that menu.

After each update, which will go into Qubes, do the copy again

A couple of final points

The EFI filesystem will grow quite fast, as each boot option contains

a Xen loader and at least one kernel and initram for Linux. The EFI

filesystem won't grow with any Linux tool I know of, so you will

need to create a new, bigger, partition and copy the files across.

The 100M partition seemsto only fit around three Qubes options.

Create a new partition to hold the EFI, I suggest at least 256M and if

your disk can afford it, 1G.

If using gdisk, make it type EF00, if using

other partition tools find out how to mark it as an EFI partition.

If you contemplate booting Windows [noises off] then you also need the

EFI partition to be first on the disk (in logical or physical order, or both;

I am not sure) so you will need to tar the files to reinstall them later.

UEFI doesn't seem to care about where the partition is (at least mine

doesn't) but mine does care about there being only one EFI partition

and it must be on a GPT disk. So if you create a new partition elsewhere

on the disk, mark the old one as a FAT16 partition instead.

I hope this is useful: I have only just got this scheme working myself,

and it seems to work for me. I have mentioned all the pitfalls I fell

into, but I am not promising you won't find any new ones...

Please le me know how you get on