Compiling an ISO with kernel-latest and modifications

[Dylanger Daly]

Hey Guys,

Apologies if someone has already asked this, is it possible to build a Qubes ISO with Kernel-Latest (Being 5.3.11-1 at time of writing) for both dom0 and appVMs?

It looks like the linux-kernel folder in qubes-builder/qubes-src is using the stable-4.14 branch, can I simply checkout v5.4.2-1 for example and build away?

TL;DR I'm trying to add Microsoft Surface Laptop 3 patches into the Qubes Installer and dom0 (https://github.com/qzed/linux-surfacegen5-acpi)

Any help is appreciated, cheers!

[Dylanger Daly]

If I run make qubes-os-iso it appears to override my changes in linux-kernel.

Anyone know how to make that not happen?

[Frédéric Pierret]

Hi,

Can you detail a little bit more exactly? If you want to build only the kernel and using other packages of qubes by using "USE_QUBES_REPO_VERSION=4.0"?

If so, in your local builder (not the chroot),

1) checkout the branch you want,

2) do the changes you want,

3) increment the release (rel file) else it will use certainly the qubes repo one.

Build the iso. That's it.

Best,

--
You received this message because you are subscribed to the Google Groups "qubes-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to qubes-devel...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/qubes-devel/3415ecdd-80f8-45cf-8c79-1d2e6e19d768%40googlegroups.com.

[Dylanger Daly]

Currently I'm trying to build an ISO with Microsoft Surface ACPI Patches (https://groups.google.com/forum/#!topic/qubes-users/4iswU7cfJHY)

Issue seems to be related to ACPI, I think it's possible there's no support for 10th Gen Intel on Stable 4.14 that comes in the installation media, so I'm trying to compile an ISO with the patches on 5.3.11-1.

I'm not using USE_QUBES_REPO_VERSION anywhere in builder, I'll add it now and try again.

On Wednesday, December 11, 2019 at 6:57:31 PM UTC+11, Frédéric Pierret wrote:

Hi,

Can you detail a little bit more exactly? If you want to build only the kernel and using other packages of qubes by using "USE_QUBES_REPO_VERSION=4.0"?

If so, in your local builder (not the chroot),

1) checkout the branch you want,

2) do the changes you want,

3) increment the release (rel file) else it will use certainly the qubes repo one.

Build the iso. That's it.

Best,

On 2019-12-11 08:46, 'Dylanger Daly' via qubes-devel wrote:

If I run make qubes-os-iso it appears to override my changes in linux-kernel.

Anyone know how to make that not happen?

On Monday, December 9, 2019 at 3:20:06 PM UTC+11, Dylanger Daly wrote:

Hey Guys,

Apologies if someone has already asked this, is it possible to build a Qubes ISO with Kernel-Latest (Being 5.3.11-1 at time of writing) for both dom0 and appVMs?

It looks like the linux-kernel folder in qubes-builder/qubes-src is using the stable-4.14 branch, can I simply checkout v5.4.2-1 for example and build away?

TL;DR I'm trying to add Microsoft Surface Laptop 3 patches into the Qubes Installer and dom0 (https://github.com/qzed/linux-surfacegen5-acpi)

Any help is appreciated, cheers!

--
You received this message because you are subscribed to the Google Groups "qubes-devel" group.

To unsubscribe from this group and stop receiving emails from it, send an email to qubes...@googlegroups.com.

[Frédéric Pierret]

Check this config: https://github.com/QubesOS/qubes-release-configs/blob/master/R4.0/qubes-os-iso-full-online.conf

We use this one for creating only latest iso without rebuilding Qubes packages. It only uses mirrors. In your case, add linux-kernel in COMPONENTS and do like I suggested in point 1-3.

Best,

To unsubscribe from this group and stop receiving emails from it, send an email to qubes-devel...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/qubes-devel/38d15c2f-513d-4165-bad9-145623f1831e%40googlegroups.com.

[Dylanger Daly]

Awesome, thanks for that, I think I'll stick to stock and _just_ update Kernel (On LTE)

If you comment out linux-kernel from COMPONENTS builder won't over write my branch in linux-kernel.

If I up the rel file, builder still reverts to stable-4.14 branch.

[Dylanger Daly]

-> Updating sources for linux-kernel...
--> Fetching from https://github.com/QubesOS/qubes-linux-kernel.git master...
--> Verifying tags...
--> Switching branch from surface-acpi branch to master
Switched to branch 'master'
--> Merging...

I can't seem to get my branch to stick

[Dylanger Daly]

Is this post still relevant? https://groups.google.com/forum/#!searchin/qubes-devel/linux-kernel|sort:date/qubes-devel/NfVQi0HXWEY/AECwbKZ0AQAJ

 - clear COMPONENTS (for the build time at least)
 - clear DISTS_VM (unless you want to use local template builds)
 - set USE_QUBES_REPO_VERSION=4.0
 - optionally set USE_QUBES_REPO_TESTING=1
 - set INSTALLER_KICKSTART=/tmp/
qubes-installer/conf/travis-iso-full.ks
   (file is in qubes-src/installer-qubes-os/conf/, but it's in that path inside build chroot)

To be clear I'm trying to do exactly the same thing as that post

[Dylanger Daly]

Okay, so I've switched back to using your link 'live' config

If I apply my commit over the top of master, it appears to keep it there

However after compiling, I appear to get stopped here

--> Done:
      qubes-src/linux-kernel/pkgs/dom0-fc25/x86_64/kernel-latest-5.4.2-2.qubes.x86_64.rpm
      qubes-src/linux-kernel/pkgs/dom0-fc25/x86_64/kernel-latest-devel-5.4.2-2.qubes.x86_64.rpm
      qubes-src/linux-kernel/pkgs/dom0-fc25/x86_64/kernel-latest-qubes-vm-5.4.2-2.qubes.x86_64.rpm
make[2]: Leaving directory '/home/user/Apps/qubes-builder'
make[1]: Leaving directory '/home/user/Apps/qubes-builder'
-> Preparing for ISO build...
--> Removing old rpms from the installer repos...
---> Cleaning up repo: dom0-updates...
---> Cleaning up repo: installer...
---> Cleaning up repo: qubes-dom0...
--> Copying installer-qubes-os RPMs...
--> Copying linux-yum RPMs...
--> Copying builder-rpm RPMs...
--> Copying linux-kernel RPMs...
Qubes OS Builder Repository                                                                                                   242 MB/s | 544 kB     00:00    
Dependencies resolved.
Nothing to do.
Complete!
-> Installing installer-qubes-os build dependencies in fc25 environment
Qubes OS Builder Repository                                                                                                   246 MB/s | 544 kB     00:00    
No package pungi-legacy available.
Error: Unable to find a match.
make: *** [Makefile:532: iso] Error 1

Appears to be a simple package missing?

It appears to be an issue with `pungi-legacy`

Last metadata expiration check: 0:00:37 ago on Wed Dec 11 23:28:32 2019.
Error:
 Problem: conflicting requests
  - nothing provides python2-functools32 needed by pungi-legacy-4.1.35-1.fc30.noarch
(try to add '--skip-broken' to skip uninstallable packages)

If I try to install it myself.

Any help is greatly appreciated.

Cheers!

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Wed, Dec 11, 2019 at 04:31:40AM -0800, 'Dylanger Daly' via qubes-devel wrote:
> Okay, so I've switched back to using your link 'live' config
>
> If I apply my commit over the top of master, it appears to keep it there

Just a little comment about your previous tries: when you modify
sources, you can simply avoid calling 'make get-sources' again. This
way, whatever changes you'll make, qubes-builder won't revert it.

BTW make qubs-os-iso is equivalent to:

make get-sources # download sources
make qubes # build individual packages
make sign-all # sign packages (if enabled)
make iso # compose installation image from the packages

Even more, you can replace 'make qubes' with make calls for individual
packages - see 'make build-info' and 'make help' for available options.

Are you sure you use release4.0 branch? If you use 'setup' script to
configure qubes-builder, that's one of the first prompts. Otherwise,
check BRANCH setting in builder.conf.

> Appears to be a simple package missing?
>
> It appears to be an issue with `pungi-legacy`
>
> Last metadata expiration check: 0:00:37 ago on Wed Dec 11 23:28:32 2019.
> Error:
> Problem: conflicting requests
> - nothing provides python2-functools32 needed by pungi-legacy-4.1.35-
> 1.fc30.noarch
> (try to add '--skip-broken' to skip uninstallable packages)

'fc30' sounds fishy here. Note the build is done within chroot
environment with Fedora 25 inside, not directly on your host, so
call like 'sudo dnf install pungi-legacy' won't help. What you may try
is:

sudo chroot chroot-dom0-fc25 dnf install pungi-legacy

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAl3w5iAACgkQ24/THMrX
1yydywgAksOkMX0zGwQ7rS7cHD0ovPQZZiRlQcuiq+mqgMB/VtiSXFPcetrpZeP+
0VMDdhywqTZbK6OAekNnK3p8Bz0Fol3IhAqn+KYxjUHCyTZ6bD5mYuI+kBTJLvWU
qX2vOPsXVMTFI1rIUtjfiWgcS2tLIZhorbihkqxZIx/Ql4DmSV8CAfIGgwdBEYgP
ZcTBSSIT2wbKbhCeKfwDdhgClbZ8y1buN2z+EdB+QPbvG+tngX/QyzFUubDT8jd3
iui2YtSJOJJpTaahPNwLEFWs/0h2IFxcHBskxmpvomn8q8BYOhdtg+F6Nh0TXLbC
UHEHtybNkAYtug6YBkHSRkwtVgSYiw==
=MBfa
-----END PGP SIGNATURE-----

[Dylanger Daly]

Thank you Marek!

I've managed to `make qubes` this has compile my kernel successfully.

However when trying to compile the iso with `make iso` I appear to be getting the same issue

[user@work qubes-builder]$ make iso

-> Preparing for ISO build...
--> Removing old rpms from the installer repos...
---> Cleaning up repo: dom0-updates...
---> Cleaning up repo: installer...
---> Cleaning up repo: qubes-dom0...
--> Copying installer-qubes-os RPMs...
--> Copying linux-yum RPMs...
--> Copying builder-rpm RPMs...
--> Copying linux-kernel RPMs...

Qubes OS Builder Repository                                                                                                                                                        256 MB/s | 544 kB     00:00    

Dependencies resolved.
Nothing to do.
Complete!
-> Installing installer-qubes-os build dependencies in fc25 environment

Qubes OS Builder Repository                                                                                                                                                        262 MB/s | 544 kB     00:00    

No package pungi-legacy available.
Error: Unable to find a match.
make: *** [Makefile:532: iso] Error 1

[user@work qubes-builder]$ sudo chroot chroot-dom0-fc25 dnf install pungi-legacy
Qubes OS Builder Repository                                                                                                                                                        257 MB/s | 544 kB     00:00    

No package pungi-legacy available.
Error: Unable to find a match.

I actually didn't have the BRANCH defined in my builder, after adding it I seem to still get the same error.

Here's my builder.conf

# vim: ft=make

VERBOSE ?= 2

BACKEND_VMM ?= xen

NO_SIGN="1"

GIT_BASEURL ?= https://github.com
GIT_PREFIX ?= QubesOS/qubes-

BRANCH ?= release4.0

RELEASE ?= 4.0

DIST_DOM0 ?= fc25
DISTS_VM ?=

COMPONENTS = \
    installer-qubes-os \
    linux-yum \
    builder-rpm \
    linux-kernel

BUILDER_PLUGINS ?= builder-rpm

USE_QUBES_REPO_VERSION=4.0

INSTALLER_KICKSTART=/home/user/qubes-src/installer-qubes-os/conf/iso-full-online.ks

It's my understanding this should build me a brand new latest-latest ISO with my compile kernel.

I'm not sure how I can install this pungi-legacy package, that appears to be the hold up.

Cheers!

[Dylanger Daly]

Quick update:

I forced release4.0 for installer-qubes-os, after doing that I had issues with symlinks, turns out I had like 40 mounts for chroot. Rebooting my appVM seems to have fixed it.

I'm successfully running `make iso`

/me crosses fingers

[Dylanger Daly]

--> Done:
The ISO can be found in iso/ subdirectory.
Thank you for building Qubes. Have a nice day!

Okay so I managed to successfully build an ISO, however, it appears to have totally ignored my kernel changes

[user@sys-usb BOOT]$ file vmlinuz
vmlinuz: Linux kernel x86 boot executable bzImage, version 4.19.86-1.pvops.qubes.x86_64 (user@build-fedora4) #1 SMP Sun Dec 1 07:16:00 UTC 2019, RO-rootFS, swap_dev 0x6, Normal VGA

This should read 5.4.2 (master at time of writing)

https://pastebin.com/z4hLjdBf

This is my builder, can someone please let me know if I need to specify that I want the kernel to be used for Installation as well?

Thank you for your help Marek & Frédéric!

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Thu, Dec 12, 2019 at 03:28:54AM -0800, 'Dylanger Daly' via qubes-devel wrote:
> --> Done:
> The ISO can be found in iso/ subdirectory.
> Thank you for building Qubes. Have a nice day!
>
> Okay so I managed to successfully build an ISO, however, it appears to have
> totally ignored my kernel changes
> [user@sys-usb BOOT]$ file vmlinuz
> vmlinuz: Linux kernel x86 boot executable bzImage, version 4.19.86-1.pvops.
> qubes.x86_64 (user@build-fedora4) #1 SMP Sun Dec 1 07:16:00 UTC 2019,
> RO-rootFS, swap_dev 0x6, Normal VGA
>
>
> This should read 5.4.2 (master at time of writing)
>
> https://pastebin.com/z4hLjdBf
>
> This is my builder, can someone please let me know if I need to specify
> that I want the kernel to be used for Installation as well?

I think this is because master branch build 'kernel-latest' package, but
installer use 'kernel'. Frédéric, do you remember how to configure it to
use 'kernel-latest'? Was it about different INSTALLER_KICKSTART setting?

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAl3yNzEACgkQ24/THMrX
1yx0/wf/VHs6iA8xzKhx+DLl+P3PEfpVIwm1zaZJdtisN3Z4Ajqg7aomX+N7SlKs
yLvDPP6PJgwJKIitV+phO3ihdDDKpGO/9n5z2oIFCUbk7VY40c3Ly3g2bHtYn4D2
bCJRD3rzMvvSowo3T5/WzgyoA2kWofyuLZJyebUbiNOvys4B19wiJ6l5TuYond7w
+KpXb0DkpbPBctNTYMEmyecE8nd48j2WlIL6LzXU3vcnPmsRj4ren+4Kweiq+OxS
zQlxvL0SOwwxKPMZBpsDmusfKVeimNcDQJrg9REQfI/EwW/Sbewf4KvFUqYvPQFr
Zej41r9cLYy5SUDfB+ZToOXcG7Godg==
=j7Jm
-----END PGP SIGNATURE-----

[Frédéric Pierret]

Yes, it's about modifying the kickstart. In my pipeline it's what I'm doing.

--
Envoyé de mon appareil Android avec Courriel K-9 Mail. Veuillez excuser ma brièveté.