[Qubes R3] How to build "UEFI-compatible" ISO ?

[Vladimir Shipovalov]

On Monday, January 5, 2015 at 12:36:28 AM UTC, Eric Shelton wrote:

On Sunday, January 4, 2015 6:48:52 PM UTC-5, Vladimir Shipovalov wrote:

I burned Qubes R2 ISO image to DVD, at lowest speed and with checksum verification.

( had to use DVD - if I would simply "dd" this ISO to USB, it will not be discovered by Mac UEFI )

Qubes DVD shows as "Windows" (???) disc at Mac's UEFI. I choose this disc in UEFI menu,

then see this note for about a second:

ISOLINUX 4.05 2011-12-09 ETCD Copyright (C) 1994-2011 H. Peter Anvin et al

After that, it switches to this message, with black screen background as well :

- Press the <ENTER> key to begin the installation process.

_ <--- flickering '_'

I press the Enter key, but the installation is not continuing. Initially I thought that ISOLINUX

has trouble at detecting Macbook's keyboard, but then I tried using the external USB keyboard,

which works in UEFI selection menu, and it still does not seem to work in ISOLINUX prompt

I know that Qubes R2 ISO is based on outdated version of Fedora (17 or 18, probably)

At this time, its support for Mac UEFI booting has been poor, somewhere I've seen a thread :

another Mac user has experienced the same "Press the <Enter> key" problem

with some older Fedora ISO.

Then, just for experiment, I tried dd'ing the latest Fedora image (21.5) to USB.

UEFI instantly recognized my USB, and I've seen nice Fedora icon with "Fedora Media" label

( not that icon-less "Windows" label of Qubes ISO ) I tried booting Fedora and it worked perfectly!

( except Wi-Fi, but this site helped to get it working in 5 mins - http://chaidarun.com/fedora-mbp )

All the newest versions of popular linux distributions, like Ubuntu, could be booted as well.

There is a ticket about "Support for EFI boot" , planned for Release 3 : http://qubes-os.org/ticket/794

So my proposal to implement this feature is quite simple:

just check how UEFI support is implemented at latest Fedora 21.5, and port it to Qubes ISO image.

Meanwhile, I will be trying to boot Qubes R2 by one of the following ways:

1) Various EFI loaders for booting old Linuxes:

"Mac Linux EFI Loader", "ISO-2-USB EFI-Booter for Mac 0.01 beta" (with unclear origin), etc

2) Attempt to manually merge Qubes R2 ISO with Fedora 21.5 ISO

3) Try to upgrade old ISOLINUX 4.05 to new SYSLINUX 6.03 (EFI support added recently, but still unstable)

=== Hardware used for testing: ===

Early 2011 Macbook Pro 8,2 MC723xx/A 15" i7-2720QM 16 GB RAM

Boot ROM Version: MBP81.0047.B27 , SMC Version (system): 1.69f4

Please tell me your thoughts and experience about this subject

A number of the options you listed will likely not work out, because Xen 4.1 won't know what to do in an EFI environment - it expects to find and access things the old legacy ways.  Please see the following post, which describes my experiences getting a dual-boot OS X / Qubes up and running:

https://groups.google.com/d/msg/qubes-devel/uLDYGdKk_Dk/Zbsh2TXbXzUJ

The short of it is to use rEFInd (the best EFI bootloader I've encountered), and certain Apple-specific GPT formatting tricks to trigger a legacy boot environment.

As far as getting the EFI installer working, there are several things to note:

1) It will never work with the R2 codebase - Xen did not add EFI support until 4.3 or so.  Good news is that the R3 build process already generates xen.efi

2) The necessary dom0 Linux kernel changes were not incorporated until Linux 3.17, so a newer kernel is required.  This may introduce headaches with the patches being applied to the kernel; certainly by 3.18 there have been enough changes that some real work may need to be done to get the PVUSB patches to work - enough that USB over IP types of solutions may be preferable.

3) Linux needs to be build with EFI stub support (trivial: just a make config option)

4) As far as just porting over the Fedora 21.5 method, there are some differences booting up xen.efi.  Basically, instead of just linux.efi, you also need xen.efi and xen.cfg, which provides command line options for Xen and Linux.  I'm guessing it's not a big deal, but there is a difference.

5) The good news is that Fedora has worked out most of the EFI install and boot stuff.  The idea is supposed to be that if the boot media is started under EFI, and EFI install with automatically happen.

Best,

Eric

After studying Eric's message (quoted above), I decided to make Qubes R3 pre-release ISO in hope that it will be UEFI compatible.

However: despite I could see that UEFI-related files like xen-4.4.0.efi , ipxe.efi , etc. - have been generated during a build of sources :
(locations: qubes-builder/chroot-fc20/boot/efi/EFI/fedora/xen-4.4.0.efi | qubes-builder/chroot-fc20/usr/share/ipxe.efi , ......)
These "UEFI-related files" haven't been included to ISO - created by running a simple make iso command.

Instead, this command produced R3 ISO which has "the same style" as R2 ISO - in sense that it relies on the same legacy booting method which doesn't work on Macs.
I burned R3 ISO to DVD+R and got absolutely the same problem as I had with R2 - stuck at black screen with "Press the <ENTER> key to begin the installation process.")
Attachments contain a directory tree of my R3 ISO, just to show/prove that it doesn't contain any UEFI files...

Question: please tell, how to produce "UEFI-compatible" ISO after Qubes R3 build?

P.S. This message is a sequel to this post : https://groups.google.com/forum/#!topic/qubes-devel/NAUIW79Vfxc
( and also, to this "double post" created mistakengly from my second account - I honestly apologize for this inconvenience,
especially because it received a reply from Marek - https://groups.google.com/forum/#!topic/qubes-devel/-pg0Yd9Kodk )

[Eric Shelton]

Sorry, but right now there is no way to get it to work, even if we overlook the build issues (for example, copying xen.efi to the right destination, creating xen.cfg, and modifying grub.cfg to invoke xen.efi).  To boot under EFI + Xen, the dom0 Linux kernel must be at least version 3.17, as that is when the needed kernel patches were integrated.  I briefly tried swapping in 3.17.6, but my attempts did not seem to be fully compatible with Qubes needs.  I did not pursue it further, since I had no specific demand for 3.17.

If you're really driven to make this work, search for "Eric Shelton EFI" on the xen - devel mailing list.  I posted some patches (I don't remember if directly against 3.12, but probably close enough). You could use that to tweak the kernel build, and then resolve the build issue from there, even if you have to just do manual edits to a default install.

- Eric

[Marek Marczykowski-Górecki]

On Tue, Jan 13, 2015 at 03:50:22PM -0800, Vladimir Shipovalov wrote:
> On Monday, January 5, 2015 at 12:36:28 AM UTC, Eric Shelton wrote:
> > A number of the options you listed will likely not work out, because Xen
> > 4.1 won't know what to do in an EFI environment - it expects to find and
> > access things the old legacy ways. Please see the following post, which
> > describes my experiences getting a dual-boot OS X / Qubes up and running:
> >
> > https://groups.google.com/d/msg/qubes-devel/uLDYGdKk_Dk/Zbsh2TXbXzUJ
> >
> > The short of it is to use rEFInd (the best EFI bootloader I've
> > encountered), and certain Apple-specific GPT formatting tricks to trigger a
> > legacy boot environment.
> >
> > As far as getting the EFI installer working, there are several things to
> > note:
> > 1) It will never work with the R2 codebase - Xen did not add EFI support
> > until 4.3 or so. Good news is that the R3 build process already generates
> > xen.efi
> > 2) The necessary dom0 Linux kernel changes were not incorporated until
> > Linux 3.17, so a newer kernel is required. This may introduce headaches
> > with the patches being applied to the kernel; certainly by 3.18 there have
> > been enough changes that some real work may need to be done to get the
> > PVUSB patches to work - enough that USB over IP types of solutions may be
> > preferable.

Regarding PVUSB patch IMO we can simply drop it. Nobody is maintaining
PVUSB support, but USB over IP works pretty well and can be used in
Qubes without kernel modification. Take a look here:
https://wiki.qubes-os.org/ticket/531#comment:10

BTW There is "devel-3.17" branch in my linux-kernel repo. But from my
tests it needs some more work to be stable on Qubes. There was a
separate thread about the problems there:
https://groups.google.com/d/msg/qubes-devel/JVmIRRByrVM/uEMOyQ0D8BwJ
Perhaps switching to 3.18 solves some of those problems...

> > 3) Linux needs to be build with EFI stub support (trivial: just a make
> > config option)
> > 4) As far as just porting over the Fedora 21.5 method, there are some
> > differences booting up xen.efi. Basically, instead of just linux.efi, you
> > also need xen.efi and xen.cfg, which provides command line options for Xen
> > and Linux. I'm guessing it's not a big deal, but there is a difference.
> > 5) The good news is that Fedora has worked out most of the EFI install and
> > boot stuff. The idea is supposed to be that if the boot media is started
> > under EFI, and EFI install with automatically happen.
> >
> > Best,
> > Eric
> >
>
> After studying Eric's message (quoted above), I decided to make Qubes R3
> pre-release ISO in hope that it will be UEFI compatible.
>

> However: despite I could see that UEFI-related files like *xen-4.4.0.efi ,
> ipxe.efi* , etc. - have been generated during a build of sources :
> *(locations: qubes-builder/chroot-fc20/boot/efi/EFI/fedora/xen-4.4.0.efi |
> qubes-builder/chroot-fc20/usr/share/ipxe.efi , ......)*

> These "UEFI-related files" haven't been included to ISO - created by

> running a simple *make iso* command.

>
> Instead, this command produced R3 ISO which has "the same style" as R2 ISO
> - in sense that it relies on the same legacy booting method which doesn't
> work on Macs.
> I burned R3 ISO to DVD+R and got absolutely the same problem as I had with

> R2 - *stuck at black screen with "Press the <ENTER> key to begin the
> installation process.")*

> Attachments contain a directory tree of my R3 ISO, just to show/prove that
> it doesn't contain any UEFI files...
>

> *Question:* please tell, how to produce "UEFI-compatible" ISO after Qubes
> R3 build?

By implementing #794 ticket...
As I've said that stuff haven't been done in our installer yet.

If you want to contribute this work, take a look at installer-qubes-os
repo, especially lorax-templates-qubes/templates directory - this is
where bootloader stuff (for installation ISO) is prepared. You'll need
to fix x86.tmpl file (EFI support is commented out there) and edit files
in config_files/x86 to really use xen.efi, create xen.cfg and so on. You
can probably base on lorax package from Fedora
(https://git.fedorahosted.org/cgit/lorax.git).

Besides from the installer itself, installed system also needs
EFI-enabled boot configured. The code for that is in
anaconda/pyanaconda/bootloader.py - also needs to be adjusted to
configure xen.efi.

--
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

[Vladimir Shipovalov]

Thank you very much for these helpful guidelines, Eric and Marek !

I would try to use your advices soon, although my experience about EFI stuff is regrettably small...

[Eric Shelton]

My other suggestion is to see if there is SOME way to boot a legacy/BIOS OS on your system.  There may not be - we may be hitting the point where newer systems are pure EFI.  Explore things like all of your BIOS settings, swapping your DVD drive with a caddy holding a second hard drive (on some systems, this allows EFI boot on one drive, and legacy on the other), or all of the weird partitioning stuff I detailed for Apple systems (which triggers a compatibility mode).  The Hackintosh folks came up with Chameleon as a solution to boot an EFI OS on legacy systems; I have no idea if something has been hacked together for the reverse.

Best of luck,
Eric