Worth keeping EOL Fedora repos in dom0 by default?

[deeplow]

Hey all!

I know Qubes assumes trust on the Fedora maintainers, but I'm wondering if there is value in keeping Fedora repositories and its signing key enabled in dom0 after the respective version's EOL, especially by default.

As I understand it, the TCB code installed in dom0 is kept up to date via Qubes-controlled repositories. From what I've seen, by the time Qubes 4.2 (and likely also 4.3) comes out, its Fedora version is already EOL. The only use-case I see for wanting this repo after install is to be able to add additional dom0 packages (which is generally discouraged).

Qubes' trust on the Fedora maintainers is not being put into question. The problem I'd like to highlight is that key holders may not have the same level of care about assumed-EOL singing keys: there could be a perceived risk difference between EOL keys and current keys. This could be as simple as not considering it necessary to do a key compromize disclosure for EOL keys (should it ever happen), under the assumption that nobody would be affected, where in fact, for Qubes users even more affected than for active fedora release keys.

To summarize, I see the risks as (1) having dom0 compromized due to a mismanaged EOL key and the benefits: (2) installing extra dom0 packages and (3) building the Qubes ISO using fedora repo's directly (which I have not checked if it's actually needed). Unless I'm missing a use-case for dom0's Fedora repos, I think this is a risk worth mitigating.

Here are some mutually-exclusive proposed solutions:

• Disable Fedora repo in dom0 for 4.3 and its key by default — advanced users who want to install additional dom0 software would be able to, by enabling the repo and importing its (already-present) key. This would still expose risks during the ISO building process (assuming packages are installed from fedora and not built from source).
• Mirror dom0 Fedora repository, but re-sign all artifacts with a Qubes key —  Under the assumption that Qubes releases always ship an EOL Fedora this would be a once-per minor release operation: essentially a trusted snapshot of every package's state.

Would this be something that makes sense for Qubes?

Cheers,

deeplow

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Sat, Oct 11, 2025 at 01:20:40PM +0000, 'deeplow' via qubes-devel wrote:
> Hey all!
>

> I know Qubes [assumes trust on the Fedora maintainers](https://doc.qubes-os.org/en/latest/developer/system/security-critical-code.html#buggy-code-vs-backdoored-code), but I'm wondering if there is value in keeping Fedora repositories and its signing key enabled in dom0 after the respective version's EOL, especially by default.

>
> As I understand it, the TCB code installed in dom0 is kept up to date via Qubes-controlled repositories. From what I've seen, by the time Qubes 4.2 (and likely also 4.3) comes out, its Fedora version is already EOL. The only use-case I see for wanting this repo after install is to be able to add additional dom0 packages (which is generally discouraged).

They are enabled for this very reason - to enable installing new
packages. This applies both to user installing something (for example
different desktop environment, or maybe some xfce plugin to show CPU
temperature or something). But also, sometimes updates gain new
dependency that need to be installed too.

> Qubes' trust on the Fedora maintainers is not being put into question. The problem I'd like to highlight is that key holders may not have the same level of care about assumed-EOL singing keys: there could be a perceived risk difference between EOL keys and current keys. This could be as simple as not considering it necessary to do a key compromize disclosure for EOL keys (should it ever happen), under the assumption that nobody would be affected, where in fact, for Qubes users even more affected than for active fedora release keys.

While I understand your concern, TBH I'd expect EOL keys to be destroyed
not long after they are not needed anymore. But also, I don't really see
a risk in keys handling that would apply to EOL keys but not the current
ones - build infrastructure is the same, they don't setup separate
instances for different versions.

> To summarize, I see the risks as (1) having dom0 compromized due to a mismanaged EOL key and the benefits: (2) installing extra dom0 packages and (3) building the Qubes ISO using fedora repo's directly (which I have not checked if it's actually needed). Unless I'm missing a use-case for dom0's Fedora repos, I think this is a risk worth mitigating.
>
> Here are some mutually-exclusive proposed solutions:
>

> - Disable Fedora repo in dom0 for 4.3 and its key by default— advanced users who want to install additional dom0 software would be able to, by enabling the repo and importing its (already-present) key. This would still expose risks during the ISO building process (assuming packages are installed from fedora and not built from source).
> - Mirror dom0 Fedora repository, but re-sign all artifacts with a Qubes key — Under the assumption that Qubes releases always ship an EOL Fedora this would be a once-per minor release operation: essentially a trusted snapshot of every package's state.

>
> Would this be something that makes sense for Qubes?

As listed above, option 1 is not really possible.
While option 2 technically might be possible, it would put quite a bit
of load on our infrastructure (repository size for our server and all mirrors).
Right now repository is already almost 1TB (in large part thanks to
templates, but firmware and kernel packages take also significant role).
Another several hundreds of GB per release is an issue.

In the past we did something else - we frozen repository metadata (which
includes hashes of all packages):
https://github.com/QubesOS/qubes-builder-rpm/pull/97
But we did it not because we were afraid about key compromise, but
because qubes-builder used package manager from inside chroot (so, for
dom0 packages - the one that is EOL) to download packages as build
dependencies. The issue doesn't apply to live qubes instance (thanks to
updatevm, and rpmcanon tool doing verification before feeding packages
to dom0's dnf), and also builderv2 doesn't do that anymore either.

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAmjqYZ0ACgkQ24/THMrX
1yzSKwf+NnhRxNR1GiowjxzZ99pYhFwmdJpLk32nBtCEi0xcH5vGA57C4cCWV8rZ
K+L5oJBMc56cJQIusAd2aXJxGJl+NvJwaS/BmHD8gTYyEe0bYnfGBu8hJ4Jn7WJz
3LQjdYHfZVOY5Xv6ET69sCmHkXVYC8+PMtIbe0043b+W1UpA9W5wu1ToXlCPnJzw
xVIMf20DgB4pnU2QgKvbdvZ1GmpuqY/GQHjHD2GB8rSyJYI2RnA0y2adECPh8zJq
zB6iPZgwmtx+ifFFqlUnXZZNcna6UB/zAln0+Xq50Yyzh1LJwsJtmKDHNRCaau0l
yVKiv5oS3qT/to4RYdL064ejpqCWcw==
=xWgX
-----END PGP SIGNATURE-----

[deeplow]

On Saturday, October 11th, 2025 at 2:54 PM, Marek Marczykowski-Górecki <marm...@invisiblethingslab.com> wrote:
> While I understand your concern, TBH I'd expect EOL keys to be destroyed
> not long after they are not needed anymore. But also, I don't really see
> a risk in keys handling that would apply to EOL keys but not the current
> ones - build infrastructure is the same, they don't setup separate
> instances for different versions.
>

> [...]

>
> As listed above, option 1 is not really possible.
> While option 2 technically might be possible, it would put quite a bit
> of load on our infrastructure (repository size for our server and all mirrors).
> Right now repository is already almost 1TB (in large part thanks to
> templates, but firmware and kernel packages take also significant role).
> Another several hundreds of GB per release is an issue.
>
> In the past we did something else - we frozen repository metadata (which
> includes hashes of all packages):
> https://github.com/QubesOS/qubes-builder-rpm/pull/97
> But we did it not because we were afraid about key compromise, but
> because qubes-builder used package manager from inside chroot (so, for
> dom0 packages - the one that is EOL) to download packages as build
> dependencies. The issue doesn't apply to live qubes instance (thanks to
> updatevm, and rpmcanon tool doing verification before feeding packages
> to dom0's dnf), and also builderv2 doesn't do that anymore either.

The frozen repository metadata approach is a much more sane one than
what I suggested. And the metalinks seem to be designed with "rogue mirrors"
in mind, which is precisely the case.

The only downsides I can think of are:

1. mirrors would be hard-coded into the file
2. This verification could only happen in the updateVM, given that the logic
on dom0's side purposely deals only with packages verification and installation.
If we wanted to make this a dom0 check, it would be further exposing its dnf call
to further untrusted input, which is probably undesirable.
3. The pinning hashes of update files, adds another failure point to system
updates, should there ever be an unforeseen need to distribute a new package
version in the EOL repos. This would be beyond the Qubes team's control and there
would need to be a way to recover from this situation and avoid manually editing
the hashes to work around the issues.

But overall it's a more elegant approach that what I had suggested.

However, from your comments my understanding is that the key management expectations
from EOL keys is viewed the same level of risks as non-EOL keys. Therefore that
this is not a risk worth mitigating. Or would this be worth exploring further?

Cheers,
deeplow

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Yeah, as a protection against compromised EOL keys, I don't think we need
this for the reasons explained above.
At some point, maybe, finally, we'll have more generic protection
against compromised package signing keys, in form of reproducible builds
(which will mean somebody would need to rebuild all relevant packages).
We did PoC of that integration down to the yum/dnf level a couple of
years ago, but for production deployment it's still a long way.

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAmjs+EIACgkQ24/THMrX
1yyaSwf8DX4BEP/b1JLawXyS+ulJKP9jTOGBWE6GwfM/soqRa2yAmcSvFiKtS5/p
+i6O3s927BdyQ3gHi4eHj6PTbqXPMtCahOUGgyAstGCe0bdV6ZYIaHe2iOV/6ynx
FOS64hKumyIZ31jyr7P3Na+KMgStJmljtqPD5xDITJpPAAIEAjAx8Rr0F98MBoZB
gXSIb5+5MSlEEKDp9npu7GkzzTy8bc2iqRiUmPIakvJQKVN+4nZatLAgo9hsVrpz
amMfZdl8Q2vKzMXF00X83J9MIF3dbj+rTUMQR0mMjBrLBWqeX67+YwYqnTjeFvNN
ZWhEzqwDp5CoUEZ3bOHirO/NKP0LOg==
=QPyQ
-----END PGP SIGNATURE-----

[deeplow]

On Monday, October 13th, 2025 at 2:07 PM, Marek Marczykowski-Górecki <marm...@invisiblethingslab.com> wrote:
> Yeah, as a protection against compromised EOL keys, I don't think we need
> this for the reasons explained above.
> At some point, maybe, finally, we'll have more generic protection
> against compromised package signing keys, in form of reproducible builds
> (which will mean somebody would need to rebuild all relevant packages).
> We did PoC of that integration down to the yum/dnf level a couple of
> years ago, but for production deployment it's still a long way.

Makes sense. Thanks for weighing in.

Best regards,
deeplow