QubesOS weekly builds
124 views
Skip to first unread message
Frédéric Pierret
Mar 21, 2021, 6:33:20 PM3/21/21
to qubes-users, qubes-devel, Marek Marczykowski-Górecki
Hi,
As some of you may know, months(years?) ago, I've setup a pipeline that is automatically PR latest kernels for Qubes OS and more recently, pulseaudio headers too. This is done every week.
At some point, I added the build of ISO including kernel-latest for users who were having issues with latest hardware. I stopped it quickly because we were merging more and more kernel versions thank to the help of automatic PR and Qubes point releases.
Due to recent troubles with kernels 5.4.X and 5.10.X, I've decided to add again to this weekly pipeline, the build of a fresh Qubes R4.1 ISO. I don't build any package or any template. It uses only Qubes OS repositories. The qubes-builder conf is: https://github.com/QubesOS/qubes-release-configs/blob/master/R4.1/qubes-os-iso-full-online.conf and the kickstart can be found here: https://github.com/QubesOS/qubes-installer-qubes-os/blob/master/conf/iso-full-online.ks.
Please note that, contrary to my first attempt, I don't include kernel-latest kernels. It's a standard R4.1 ISO as if Marek would release one. It is built in a dedicated AppVM together with Split GPG. The ISOs are signed by "fepitre-bot" 1C8714D640F30457EC953050656946BA873DDEC1. Some of you already download latest R4.1 devel ISOs in openQA but they are not signed and not necessary built in a safe environment because it's only for CI purposes. That's a solution between CI ISOs and R4.1 alpha release.
That said, the ISO(s) can be found on my self hosted server: https://qubes.notset.fr/iso/.
Best regards,
Frédéric
As some of you may know, months(years?) ago, I've setup a pipeline that is automatically PR latest kernels for Qubes OS and more recently, pulseaudio headers too. This is done every week.
At some point, I added the build of ISO including kernel-latest for users who were having issues with latest hardware. I stopped it quickly because we were merging more and more kernel versions thank to the help of automatic PR and Qubes point releases.
Due to recent troubles with kernels 5.4.X and 5.10.X, I've decided to add again to this weekly pipeline, the build of a fresh Qubes R4.1 ISO. I don't build any package or any template. It uses only Qubes OS repositories. The qubes-builder conf is: https://github.com/QubesOS/qubes-release-configs/blob/master/R4.1/qubes-os-iso-full-online.conf and the kickstart can be found here: https://github.com/QubesOS/qubes-installer-qubes-os/blob/master/conf/iso-full-online.ks.
Please note that, contrary to my first attempt, I don't include kernel-latest kernels. It's a standard R4.1 ISO as if Marek would release one. It is built in a dedicated AppVM together with Split GPG. The ISOs are signed by "fepitre-bot" 1C8714D640F30457EC953050656946BA873DDEC1. Some of you already download latest R4.1 devel ISOs in openQA but they are not signed and not necessary built in a safe environment because it's only for CI purposes. That's a solution between CI ISOs and R4.1 alpha release.
That said, the ISO(s) can be found on my self hosted server: https://qubes.notset.fr/iso/.
Best regards,
Frédéric
Beto HydroxyButyrate
Mar 21, 2021, 9:03:21 PM3/21/21
to qubes-users
Trying to use the info provided to kick off my own ISO build.
This posted links to does not use the `QubesBuilder` approach, as it references stuff apparently checked out to ~/qubes-src, rather than ~/qubes-builder/qubes-src.
Is this different approach to building documented? What else should check out to ~/qubes-src? How does this fit into the "Development Workflow"?
Per those docs:
Qubes is split into a bunch of git repos. This are all contained in the
qubes-src directory under qubes-builder. Subdirectories there are separate
components, stored in separate git repositories.
Just making ~/qubes-src a symbolic link to ~/qubes-builder/qubes-src does not fix things as, despite the claim that this are all contained in ~/qubes-builder/qubes-src, the two new git repos referenced are not contained there.
[user@qubes-build qubes-src]$ pwd
/home/user/qubes-builder/qubes-src
[user@qubes-build qubes-src]$ ls qubes-release-configs
ls: cannot access 'qubes-release-configs': No such file or directory
[user@qubes-build qubes-src]$ ls qubes-installer-qubes-os
ls: cannot access 'qubes-installer-qubes-os': No such file or directory
[user@qubes-build qubes-src]$ ls ~/qubes-src
ls: cannot access '/home/user/qubes-src': No such file or directory
/home/user/qubes-builder/qubes-src
[user@qubes-build qubes-src]$ ls qubes-release-configs
ls: cannot access 'qubes-release-configs': No such file or directory
[user@qubes-build qubes-src]$ ls qubes-installer-qubes-os
ls: cannot access 'qubes-installer-qubes-os': No such file or directory
[user@qubes-build qubes-src]$ ls ~/qubes-src
ls: cannot access '/home/user/qubes-src': No such file or directory
Frédéric Pierret
Mar 22, 2021, 3:11:32 AM3/22/21
to Beto HydroxyButyrate, qubes-users
Le 3/22/21 à 2:03 AM, Beto HydroxyButyrate a écrit :
> Trying to use the info provided to kick off my own ISO build.
> This posted links to does not use the `QubesBuilder` approach, as it references stuff apparently checked out to ~/qubes-src, rather than ~/qubes-builder/qubes-src.
>
> Is this different approach to building documented? What else should check out to ~/qubes-src? How does this fit into the "Development Workflow"?
>
> Due to recent troubles with kernels 5.4.X and 5.10.X, I've decided to add again to this weekly pipeline, the build of a fresh Qubes R4.1 ISO. I don't build any package or any template. It uses only Qubes OS repositories. The qubes-builder conf is: https://github.com/QubesOS/qubes-release-configs/blob/master/R4.1/qubes-os-iso-full-online.conf <https://github.com/QubesOS/qubes-release-configs/blob/master/R4.1/qubes-os-iso-full-online.conf> and the kickstart can be found here: https://github.com/QubesOS/qubes-installer-qubes-os/blob/master/conf/iso-full-online.ks <https://github.com/QubesOS/qubes-installer-qubes-os/blob/master/conf/iso-full-online.ks>.
> This posted links to does not use the `QubesBuilder` approach, as it references stuff apparently checked out to ~/qubes-src, rather than ~/qubes-builder/qubes-src.
>
> Is this different approach to building documented? What else should check out to ~/qubes-src? How does this fit into the "Development Workflow"?
>
It's written "qubes-builder" conf. So use this as builder.conf and that's all: make iso. The kickstart reference provided is the one used by this conf: https://github.com/QubesOS/qubes-release-configs/blob/master/R4.1/qubes-os-iso-full-online.conf#L24. Meaning the installer will use the file in installer-qubes-os sources: https://github.com/QubesOS/qubes-installer-qubes-os/blob/master/conf/iso-full-online.ks
Frédéric
Frédéric Pierret
Mar 22, 2021, 3:21:43 AM3/22/21
to Beto HydroxyButyrate, qubes-users
Le 3/22/21 à 2:03 AM, Beto HydroxyButyrate a écrit :
> Trying to use the info provided to kick off my own ISO build.
> This posted links to does not use the `QubesBuilder` approach, as it references stuff apparently checked out to ~/qubes-src, rather than ~/qubes-builder/qubes-src.
>
> Is this different approach to building documented? What else should check out to ~/qubes-src? How does this fit into the "Development Workflow"?
>
> Per those docs:
>
> /Qubes is split into a bunch of git repos. This are *_all _*contained in the qubes-src directory under qubes-builder. Subdirectories there are separate components, stored in separate git repositories./
> This posted links to does not use the `QubesBuilder` approach, as it references stuff apparently checked out to ~/qubes-src, rather than ~/qubes-builder/qubes-src.
>
> Is this different approach to building documented? What else should check out to ~/qubes-src? How does this fit into the "Development Workflow"?
>
> Per those docs:
>
> /
> /
> Just making ~/qubes-src a symbolic link to ~/qubes-builder/qubes-src does not fix things as, despite the claim that /this are all contained/ in ~/qubes-builder/qubes-src, the two new git repos referenced are not contained there.
>
I forgot to mention also that the reference used in builder.conf for INSTALLER_KICKSTART=ZZZ is a path under the build chroot and not to your local qubes-builder. The build chroot has sources from qubes-builder/qubes-src copied to chroot-dom0-fcXX/home/user/qubes-src. The value for the kickstart is then used when build is ran under the chroot. That probably deserves a note in the doc if it's not the case.
Frédéric
Holger Levsen
Mar 30, 2021, 6:29:44 PM3/30/21
to Frédéric Pierret, qubes-users, qubes-devel
Hi Frédéric,
On Sun, Mar 21, 2021 at 11:33:05PM +0100, Frédéric Pierret wrote:
> Due to recent troubles with kernels 5.4.X and 5.10.X, I've decided to add
> again to this weekly pipeline, the build of a fresh Qubes R4.1 ISO. I don't
> build any package or any template. It uses only Qubes OS repositories.
yay, that's very nice and useful! thank you!
> Please note that, contrary to my first attempt, I don't include kernel-latest kernels.
So do they have 5.4.x or 5.10.x?
> The ISOs are signed by "fepitre-bot" 1C8714D640F30457EC953050656946BA873DDEC1.
nice!
I'll give them a try in the next days on some new hardware which doesn't
work with the iso from December but should be working now...
I guess you have ran diffoscope on two builds, how is the result? Do you
already have this in CI too? (this is for testing for reproducible builds...)
--
cheers,
Holger
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ holger@(debian|reproducible-builds|layer-acht).org
⢿⡄⠘⠷⠚⠋⠀ PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C
⠈⠳⣄
People call vaccine mandates "Orwellian" even though Orwell died at 46 of
tuberculosis, which is now preventable with a vaccine.
On Sun, Mar 21, 2021 at 11:33:05PM +0100, Frédéric Pierret wrote:
> Due to recent troubles with kernels 5.4.X and 5.10.X, I've decided to add
> again to this weekly pipeline, the build of a fresh Qubes R4.1 ISO. I don't
> build any package or any template. It uses only Qubes OS repositories.
> Please note that, contrary to my first attempt, I don't include kernel-latest kernels.
> The ISOs are signed by "fepitre-bot" 1C8714D640F30457EC953050656946BA873DDEC1.
I'll give them a try in the next days on some new hardware which doesn't
work with the iso from December but should be working now...
I guess you have ran diffoscope on two builds, how is the result? Do you
already have this in CI too? (this is for testing for reproducible builds...)
--
cheers,
Holger
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ holger@(debian|reproducible-builds|layer-acht).org
⢿⡄⠘⠷⠚⠋⠀ PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C
⠈⠳⣄
People call vaccine mandates "Orwellian" even though Orwell died at 46 of
tuberculosis, which is now preventable with a vaccine.
Frédéric Pierret
Mar 31, 2021, 4:14:04 AM3/31/21
to qubes-users, qubes-devel
Hi Holger,
Le 3/31/21 à 12:29 AM, Holger Levsen a écrit :
>> Please note that, contrary to my first attempt, I don't include kernel-latest kernels.
>
> So do they have 5.4.x or 5.10.x?
R4.1 has switched to 5.10.X as default LTS that's a very good point for new hardware.
>> The ISOs are signed by "fepitre-bot" 1C8714D640F30457EC953050656946BA873DDEC1.
>
> nice!
>
>> That said, the ISO(s) can be found on my self hosted server: https://qubes.notset.fr/iso/.
>
> I'll give them a try in the next days on some new hardware which doesn't
> work with the iso from December but should be working now...
>
> I guess you have ran diffoscope on two builds, how is the result? Do you
> already have this in CI too? (this is for testing for reproducible builds...)
>
Not yet but I've discussed few days ago with Marek on how to do the build integration in order to reproduce the ISO. I'm finishing few Fedora related reproducible things then I guess I would do this, depending on what Marek has in mind for the schedule.
Additionally, I've added few days ago the automatic openQA trigger for each ISO I build: https://openqa.qubes-os.org/group_overview/1. It's jobs corresponding to "BUILD20XXYYZZ-4.1" where in the settings, for example this one: https://openqa.qubes-os.org/tests/16829#settings, it downloads from my hosting repository the built ISO.
Best regards,
Frédéric
Le 3/31/21 à 12:29 AM, Holger Levsen a écrit :
> Hi Frédéric,
>
> On Sun, Mar 21, 2021 at 11:33:05PM +0100, Frédéric Pierret wrote:
>> Due to recent troubles with kernels 5.4.X and 5.10.X, I've decided to add
>> again to this weekly pipeline, the build of a fresh Qubes R4.1 ISO. I don't
>> build any package or any template. It uses only Qubes OS repositories.
>
> yay, that's very nice and useful! thank you!
You are welcome.
>
> On Sun, Mar 21, 2021 at 11:33:05PM +0100, Frédéric Pierret wrote:
>> Due to recent troubles with kernels 5.4.X and 5.10.X, I've decided to add
>> again to this weekly pipeline, the build of a fresh Qubes R4.1 ISO. I don't
>> build any package or any template. It uses only Qubes OS repositories.
>
> yay, that's very nice and useful! thank you!
>> Please note that, contrary to my first attempt, I don't include kernel-latest kernels.
>
> So do they have 5.4.x or 5.10.x?
>> The ISOs are signed by "fepitre-bot" 1C8714D640F30457EC953050656946BA873DDEC1.
>
> nice!
>
>> That said, the ISO(s) can be found on my self hosted server: https://qubes.notset.fr/iso/.
>
> I'll give them a try in the next days on some new hardware which doesn't
> work with the iso from December but should be working now...
>
> I guess you have ran diffoscope on two builds, how is the result? Do you
> already have this in CI too? (this is for testing for reproducible builds...)
>
Additionally, I've added few days ago the automatic openQA trigger for each ISO I build: https://openqa.qubes-os.org/group_overview/1. It's jobs corresponding to "BUILD20XXYYZZ-4.1" where in the settings, for example this one: https://openqa.qubes-os.org/tests/16829#settings, it downloads from my hosting repository the built ISO.
Best regards,
Frédéric
Frédéric Pierret
Jun 10, 2021, 5:38:53 PM6/10/21
to qubes-users, qubes-devel, Marek Marczykowski-Górecki
Hi,
Le 3/21/21 à 11:33 PM, Frédéric Pierret a écrit :
I've added support to qubes-builder the possibility to build an ISO having the installer running kernel-latest and the installed QubesOS too. For documentation: https://github.com/QubesOS/qubes-builder/blob/master/doc/Configuration.md#iso_use_kernel_latest (pretty simple, isn't it?)
I'm pleased to announce you that I've added that to my weekly build pipeline where I will still build both versions: the standard and the one with kernel-latest embedded. Same as previously, you can find signed ISOs here https://qubes.notset.fr/iso/ and also result of openQA tests too (see openqa.qubes-os.org with build tag having -kernel-latest. For example: https://openqa.qubes-os.org/tests/overview?build=20210610-kernel-latest-4.1&distri=qubesos&version=4.1&groupid=1).
The goal is still the same: providing testing QubesOS images built in a sane environment for latest drivers support by Linux until LTS kernels would have enough backports for very recent hardware.
A final remark like on the Discourse thread, I do recurrent cleaning for space consideration. I keep only ISOs for the current month now.
Best regards,
Frédéric
Le 3/21/21 à 11:33 PM, Frédéric Pierret a écrit :
I'm pleased to announce you that I've added that to my weekly build pipeline where I will still build both versions: the standard and the one with kernel-latest embedded. Same as previously, you can find signed ISOs here https://qubes.notset.fr/iso/ and also result of openQA tests too (see openqa.qubes-os.org with build tag having -kernel-latest. For example: https://openqa.qubes-os.org/tests/overview?build=20210610-kernel-latest-4.1&distri=qubesos&version=4.1&groupid=1).
The goal is still the same: providing testing QubesOS images built in a sane environment for latest drivers support by Linux until LTS kernels would have enough backports for very recent hardware.
A final remark like on the Discourse thread, I do recurrent cleaning for space consideration. I keep only ISOs for the current month now.
Best regards,
Frédéric
Reply all
Reply to author
Forward
0 new messages