Groups keyboard shortcuts have been updated
Dismiss
See shortcuts

HOWTO: Compiling Kernels for dom0

3,742 views
Skip to first unread message

Reg Tiangha

unread,
Apr 14, 2017, 2:21:29 PM4/14/17
to qubes...@googlegroups.com
Here's my contribution to the project.

On my GitHub account, I've now got branches tracking kernels from 4.4
all the way to 4.10. My intent is to keep them up-to-date with upstream
as much as possible, but all I can really test is to see is if they
still compile and/or install/boot. If there are any issues with new
versions, let me know, but I make no guarantees that I can actually
*fix* any regressions that may be introduced by upstream. That said, if
some people want to compile the latest kernel in a supported branch
themselves on their own schedules optimized for their specific hardware
setups, I hope this makes things a little easier you.

https://github.com/rtiangha/qubes-linux-kernel/


HOWTO:

- You'll need at least 4GB of free space in /home for each kernel you
hope to compile.

- In a Fedora TemplateVM matching the version running in your dom0,
install git and the qubes-kernel-vm-support package:

sudo dnf install git qubes-kernel-vm-support

I believe that should pull in everything you need to compile a kernel.
At the moment, if you want to build a kernel higher than 4.8, you'll
need to temporarily enable the current-testing repository since the
version that's in stable right now is too old to work with kernels 4.9
and above. That'll probably change eventually.

- Download sources:

git clone https://github.com/rtiangha/qubes-linux-kernel.git

- Enter directory:

cd qubes-linux-kernel

- Switch to the branch that you'd like to compile. For example, to
switch to the 4.4 branch:

git checkout stable-4.4

You can also choose from devel-4.8, stable-4.9, and devel-4.10.

- Compile rpms:

make rpms

- The rpms will be stored in the rpms/x86_64 directory. Copy those to
dom0 using these instructions:

https://www.qubes-os.org/doc/copy-from-dom0/

- Install rpms. In dom0, run:

dnf install kernel-<version>.rpm kernel-qubes-vm-<version>.rpm

- Reboot and see if it works


TIPS:

By default, the kernel configuration is set up for a very generic build
to work with a variety of hardware. If you're going to go through the
hassle of compiling your own kernels, you might as well optimize for
your particular hardware configuration. For example, if all you have
are AMD machines and no Intel machines, rather than compiling a kernel
for a generic x86_64 CPU, you can set the kernel to optimize for AMD
CPUs specifically and you may net some performance improvements as a result.

- To do this, first download the kernel sources (make rpms automatically
does this for you):

make get-sources

- Then extract the source files:

tar Jxf linux-<version>.tar.xz

- Move into the directory:

cd linux-<version>.tar.xz

- Copy the default Qubes kernel configuration into the directory:

cp ../config .config

- Now, sometimes new drivers or kernel options will be introduced
in-between kernel versions. It is always useful to check for that and to
merge in anything new that you may find desirable. To do so, first run:

make oldconfig

What that will do is check the current kernel configuration file against
what's available in the new kernel version. If there's nothing new, then
it will exit gracefully. If there are some new things, it'll prompt you
on whether or not you want to include them. If you have no idea what to
do, you can probably just accept the default choices or just say No and
still be safe if the current kernel configuration works for you.

- Customize your kernel:

make menuconfig

- You'll be presented with a menu with a whole lot of options. The
easiest ones to play with if you're just starting out is the Processor
Type; if you compile for your specific CPU rather than a generic one,
you may notice some performance improvements. Navigate to:

Processor type and features -> Processor family

And choose the Processor Family that best meets the machine you're
compiling for. In my case, I've got machines based on an Intel Core 2
Quad Q6600, an Intel Core i7-980x, and an Intel Core i7-2720M that I run
Qubes on, and I install these kernels on all of them, so I select the
"Core 2/new Xeon" option when I compile kernels for myself.

There are many kernel options that you can toggle, so if you want to go
further, I *highly* suggest reading up on the ArchLinux or Gentoo kernel
docs as they go more in depth on how to work with kernel options. Some
interesting things to try would be to disable any hardware drivers for
hardware you don't have, don't use, or will never use. Not only will
that cut down on the attack surface, but it'll also save you on
compilation time, RAM usage and disk space, which may result in some
performance improvements too. Just make sure not to disable a driver for
hardware that you actually have, and make sure you have a working dom0
kernel installed already to boot back into, just in case the one you
created doesn't work.

- When you're done, keep hitting ESC until you're asked if you want to
exit and save your work. Select "Yes". Then, copy back your work back to
the main directory:

cp .config ../config

- And then you can test your new configuration by compiling it with make
rpms. If you ever need to start over, run:

make clean

and it'll delete all of the directories with compiled stuff in it (it
won't touch the rpm directory though so your output will still be saved).


Compiling and customizing Linux kernels isn't too difficult. The main
costs are in disk space and compilation time, especially if you're
working with older or slower hardware. But once you're empowered to do
this on your own, you'll be able to compile and install kernels at your
leisure as well as keep up with upstream, rather than having to wait for
newer versions of the official ones to be released.

Hope this helps!


Reg Tiangha

unread,
Apr 14, 2017, 2:34:56 PM4/14/17
to qubes...@googlegroups.com
Oh, and some other tips I forgot to add:

- If you're going to test a lot of kernels, it may be useful to up dnf's
limit of simultaneously installed packages from 3 to higher, just so
that dnf doesn't accidentally uninstall a kernel you don't want it to
uninstall. To do so, open up /etc/dnf/dnf.conf in a text editor and
change the installonly_limit option to something higher (I have mine set
to 7).

- If you want to keep up with upstream on your own, just change the
number in the 'version' file to match the current kernel version number
available at kernel.org, and 'make rpms' will download that version
instead. If you want to make incremental changes on a single kernel
version on your own (for example, you're testing different kernel
configurations on the same kernel version) and don't want dnf to
complain about packages already being installed, you can increment the
number in the 'rel' file and that way, the generated rpm will install
cleanly because dnf will think the package you're installing is newer
than what it already has.



Foppe de Haan

unread,
Apr 14, 2017, 4:22:51 PM4/14/17
to qubes-users, r...@reginaldtiangha.com
1. Thanks. :)
2. Starting from a vanilla f23-template, I also had to install rpm-build (non-obvious 'error 8').

Reg Tiangha

unread,
Apr 14, 2017, 4:29:34 PM4/14/17
to qubes...@googlegroups.com
On 04/14/2017 02:22 PM, Foppe de Haan wrote:
> 1. Thanks. :)
> 2. Starting from a vanilla f23-template, I also had to install rpm-build (non-obvious 'error 8').
>
Thanks. And I just remembered that you'll probably need to install the
latest version of qubes-utils in dom0 too if you want to run kernels
newer than 4.8, otherwise things may not work properly. Unfortunately,
right now, it can only be found in current-testing as it hasn't yet been
pushed into stable either:

sudo qubes-dom0-update --enablerepo=qubes-dom0-current-testing qubes-utils

I think that's it? If anyone catches any other missing packages, do let
the list know.



Reg Tiangha

unread,
Apr 14, 2017, 4:35:14 PM4/14/17
to qubes...@googlegroups.com
Oh, and one more tip: If your machine has multiple CPUs, assign *all*
of them to your kernel build VM. Your kernel compile time will speed up
exponentially.


Foppe de Haan

unread,
Apr 14, 2017, 5:25:41 PM4/14/17
to qubes-users, r...@reginaldtiangha.com
make menuconfig also requires ncurses-devel

Foppe de Haan

unread,
Apr 15, 2017, 6:44:03 AM4/15/17
to qubes-users, r...@reginaldtiangha.com
also needed: rpm-sign, sparse, openssl-devel

and creating a signing key, see e.g. here: http://fedoranews.org/tchung/gpg/

Reg Tiangha

unread,
Apr 15, 2017, 12:02:31 PM4/15/17
to qubes...@googlegroups.com
On 04/15/2017 04:44 AM, Foppe de Haan wrote:
> also needed: rpm-sign, sparse, openssl-devel
>
> and creating a signing key, see e.g. here: http://fedoranews.org/tchung/gpg/
>

Thanks for all of that.

The signing part isn't that important though, unless you were planning
on distributing the package. The rpms still get generated and stored in
the rpms directory even though it throws out that error at the end about
being unable to sign them. But it's good to know how to fix that for
those who want to know.


Marek Marczykowski-Górecki

unread,
Apr 16, 2017, 9:10:11 PM4/16/17
to Reg Tiangha, qubes...@googlegroups.com, Andrew David Wong
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Fri, Apr 14, 2017 at 12:21:14PM -0600, Reg Tiangha wrote:
> Here's my contribution to the project.

Thanks!

Andrew, maybe it would be good idea to at least link to this thread
somewhere in "Building" section of docs? Or copy this instruction there
(the part about actual building and customizing), including adjustments
here and in subsequent messages?

> On my GitHub account, I've now got branches tracking kernels from 4.4
> all the way to 4.10.

I assume you've also seen devel-* branches on my github account.
Some signature verification of downloaded code would be useful here. I
see you sign your commits, so it should be easy (look for "Good
signature" at the top, also check if the key is what you expect):

git show --show-signature

Or in machine readable format:

git show -s --format=%G?

(should output "G" for good signature made with trusted key, see `git
show --help` for details)

Of course you need to have appropriate public key in your keyring first.

> You can also choose from devel-4.8, stable-4.9, and devel-4.10.
>
> - Compile rpms:
>
> make rpms
>
> - The rpms will be stored in the rpms/x86_64 directory. Copy those to
> dom0 using these instructions:
>
> https://www.qubes-os.org/doc/copy-from-dom0/
>
> - Install rpms. In dom0, run:
>
> dnf install kernel-<version>.rpm kernel-qubes-vm-<version>.rpm

Some, probably obvious warning: this will also execute some
pre/post-installation scripts in the package. It means that if the
building VM is compromised, it can include some code in the rpm package,
that will compromise dom0 when you install it.

> - Reboot and see if it works
>
>
> TIPS:
>
> By default, the kernel configuration is set up for a very generic build
> to work with a variety of hardware. If you're going to go through the
> hassle of compiling your own kernels, you might as well optimize for
> your particular hardware configuration. For example, if all you have
> are AMD machines and no Intel machines, rather than compiling a kernel
> for a generic x86_64 CPU, you can set the kernel to optimize for AMD
> CPUs specifically and you may net some performance improvements as a result.
>
> - To do this, first download the kernel sources (make rpms automatically
> does this for you):
>
> make get-sources

Don't forget about 'make verify-sources' (check signature on downloaded
tarball). It's better to call:

make get-sources verify-sources

> - Then extract the source files:
>
> tar Jxf linux-<version>.tar.xz
>
> - Move into the directory:
>
> cd linux-<version>.tar.xz

cd linux-<version>
- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJY9BXvAAoJENuP0xzK19cs/lwH/A/cZl1oj/NRScHNpkRt4584
Po2rCFKj35rMVGWtnBy9ssOW5Tqt/w7roN2YXczvd5AsWIzZHJMOy80a47VuJBeF
4FREcE4a5hRv6sGHzrtqimIbx9UWCemjQBokvbtADWohwBLhxYXfK6XM6ONaZzZi
k4F9/aLVizdFpt16EAGbsrT8d4jU1zISWcMNJie+0g9sflro+EO4clDGgdzyKHRJ
i3bO6m5nBS5xkz1TjgKHZhqIKSncdkkcRmjHG7QBDYps7DYMsc1STOrugkpZGNls
kvO6G1zlCeLHxnkUL5veXkqh+NXbAuTTu/jvn5L8nRarQRoafFJat25vOO8yLVQ=
=kh6j
-----END PGP SIGNATURE-----

Andrew David Wong

unread,
Apr 16, 2017, 9:16:22 PM4/16/17
to Marek Marczykowski-Górecki, Reg Tiangha, qubes...@googlegroups.com
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

On 2017-04-16 18:10, Marek Marczykowski-Górecki wrote:
> On Fri, Apr 14, 2017 at 12:21:14PM -0600, Reg Tiangha wrote:
>> Here's my contribution to the project.
>
> Thanks!
>
> Andrew, maybe it would be good idea to at least link to this
> thread somewhere in "Building" section of docs? Or copy this
> instruction there (the part about actual building and customizing),
> including adjustments here and in subsequent messages?
>

Good idea. I've just linked to thread for now, in case more useful
replies are added in the near future. Maybe someday someone can submit
a nice Markdown version to replace it. :)

- --
Andrew David Wong (Axon)
Community Manager, Qubes OS
https://www.qubes-os.org
-----BEGIN PGP SIGNATURE-----

iQIcBAEBCgAGBQJY9BddAAoJENtN07w5UDAwnuUP/i2TtLmmDZk6J+Ll86rumDYw
qK4cDtzozWxG5figs6zd4WiguyFs/SfuEU1DxJ7e/4+G730t2Purfy3ZUp1fWEhc
SLurA+NpJCM/6dfkGXswmNNTYv/ncKtfX/7L7ntr7F8D8PSaE/I2+zpwGFRWmnol
rr0vHmp6pqtYZFaV+ByYCIYmNHESOYtbuO4+SAOfbwzijWDckZXtkAoNpJdD3ypJ
go72WZAZeDjiCw2dFi8mThcWylxQ7WRd21mRUJIf2lEkHfXVTBP6OMJa02pjfolp
/J0j+6G9OG2yk7iJ05bLf2b7Y5MjhRgsJdFWBBxgH1AlxgBChLO3EYqzxDyHUq88
rZo7ePGMY38nEZofIA4pIIgYV/87Vg6BVddUm0MEVHwNjMsxh05pGTv7OzkM+Hc6
N2uH7E0vmYZERWwX34ASgiMG1RZZIswat2HSm/GEBEb1vpieOMDNAKdj7/tqNzK/
Nx82FOYWnHwkDI1sxDh4hWGsd/WeKKyk3A3kMvzsGS8JAPoQeRA3bes4+UtOBCDw
mygAbvHzxmz+Kx1JptrNBFRfs9ULJ6T3+E8adqNxd31R6mzHTo3g4JDJ0hOGINRq
MdZ8tZyUFCrAGRDNVN8vhdNMtq3WIJezWUhm6RaVSRb/8Ep9lcx/xQcTjpMVj9+O
KlAmJ+Y+CAmxyEnpwXdK
=OBeX
-----END PGP SIGNATURE-----

Reg Tiangha

unread,
Apr 16, 2017, 9:21:32 PM4/16/17
to qubes...@googlegroups.com
On 04/16/2017 07:16 PM, Andrew David Wong wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA512
>
> On 2017-04-16 18:10, Marek Marczykowski-Górecki wrote:
>> On Fri, Apr 14, 2017 at 12:21:14PM -0600, Reg Tiangha wrote:
>>> Here's my contribution to the project.
>> Thanks!
>>
>> Andrew, maybe it would be good idea to at least link to this
>> thread somewhere in "Building" section of docs? Or copy this
>> instruction there (the part about actual building and customizing),
>> including adjustments here and in subsequent messages?
>>

Thanks for the comments you two; I'm new to the Git stuff but I'm trying
to learn fast.

Actually, now that Foppe de Haan found some extra packages to install
to make everything work, I wouldn't mind having a chance to re-write it
and submit it. What's the best way to do that? Fork some kind of repo,
create a new file, and then submit a pull request? If so, which repo?
If not, what's the best/easiest way to submit a re-written guide to you
guys?

Andrew David Wong

unread,
Apr 17, 2017, 5:37:08 PM4/17/17
to Reg Tiangha, qubes...@googlegroups.com
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

https://www.qubes-os.org/doc/doc-guidelines/

Thanks!

- --
Andrew David Wong (Axon)
Community Manager, Qubes OS
https://www.qubes-os.org
-----BEGIN PGP SIGNATURE-----

iQIcBAEBCgAGBQJY9TV9AAoJENtN07w5UDAwQIwP/2XMv0t0n1Zo6ceTQ8HqxS18
AxdaYD2WEHeIv55o1Ywg6+Bq4HbvKM7Ai0QYKjHFLhzVpGFHNlbrJx0BcoznC6w9
khIH4mvaWgF5UXSXs+D6IRMeBkdfQNTL0sLL4FFlQwGnET/hFIbk20QJwZMnwbb8
dnXE8rz4lw3MnelUBAQgC745fpYdqdHw2FjopuprKvUEtDlTmj6Dd9cWuOnbkVWP
cgGm5ZpuJyJ9pw+v60z9x299pJexBqZUpMxdNaqSViNk+YCsIj4gsnrRH134Ys9U
zmCn4TzjX/lAzc2xxugkietglGM5uYYZoYOANr+JJ7BVB0XpZvXzTtVrWAuqEAvg
T6x8hSOBV0J8l38oeg9VANgEuu0ZtPpEc9VPeIVnbhqMGGM/gKhCGLg27n8pLVQI
mnEaxkR7qVByUPNItUf0r5olH+ksfd0DSd+5gxlRHjgHYRSI/Cy0Y8dDR+erpvAB
eX2kxCG98Bi2KTuGMa6yDAnMTN5IkzYGCrHB7T8zJXyQoWAHkguDweRgTciQcN9R
NtJ0EHGxo59dJKuUM+GJcllB+J3Aoy/A4TYkqzI3KHyFSy2IdY4zEDXf9atytnhJ
XPre+LYiQCXWLCxO1YscMq2Od/rm+nPZYvZlqBSHnUTYgBgnp6yjgBbW2z7fQXfJ
V8o3BpveihcQdWV931nd
=6dE/
-----END PGP SIGNATURE-----

gho...@gmail.com

unread,
Apr 28, 2017, 11:03:32 PM4/28/17
to qubes-users, r...@reginaldtiangha.com
Hello,

I am following your instructions and trying to compile devel-4.10. I am getting the following error. This error also occurs on stable-4.9. Any idea how i can fix this.

Thanks.

/home/user/qubes-linux-kernel/u2mfn/u2mfn.c: In function 'u2mfn_ioctl':
/home/user/qubes-linux-kernel/u2mfn/u2mfn.c:80:23: error: passing argument 5 of 'get_user_pages' from incompatible pointer type [-Werror=incompatible-pointer-types]
(data, 1, 1, 0, &user_page, 0);
^
In file included from /home/user/qubes-linux-kernel/u2mfn/u2mfn.c:26:0:
/home/user/qubes-linux-kernel/kernel-4.10.13/linux-4.10.13/include/linux/mm.h:1271:6: note: expected 'struct vm_area_struct **' but argument is of type 'struct page **'
long get_user_pages(unsigned long start, unsigned long nr_pages,
^
/home/user/qubes-linux-kernel/u2mfn/u2mfn.c:79:9: error: too many arguments to function 'get_user_pages'
ret = get_user_pages
^
In file included from /home/user/qubes-linux-kernel/u2mfn/u2mfn.c:26:0:
/home/user/qubes-linux-kernel/kernel-4.10.13/linux-4.10.13/include/linux/mm.h:1271:6: note: declared here
long get_user_pages(unsigned long start, unsigned long nr_pages,
^
cc1: some warnings being treated as errors
/home/user/qubes-linux-kernel/kernel-4.10.13/linux-4.10.13/scripts/Makefile.build:300: recipe for target '/home/user/qubes-linux-kernel/u2mfn/u2mfn.o' failed
make[4]: *** [/home/user/qubes-linux-kernel/u2mfn/u2mfn.o] Error 1
/home/user/qubes-linux-kernel/kernel-4.10.13/linux-4.10.13/Makefile:1490: recipe for target '_module_/home/user/qubes-linux-kernel/u2mfn' failed
make[3]: *** [_module_/home/user/qubes-linux-kernel/u2mfn] Error 2
Makefile:150: recipe for target 'sub-make' failed
make[2]: *** [sub-make] Error 2
Makefile:24: recipe for target '__sub-make' failed
make[1]: *** [__sub-make] Error 2
make[1]: Leaving directory '/home/user/qubes-linux-kernel/kernel-4.10.13/linux-obj'
error: Bad exit status from /var/tmp/rpm-tmp.6UeD6a (%build)


RPM build errors:
Bad exit status from /var/tmp/rpm-tmp.6UeD6a (%build)
Makefile:90: recipe for target 'rpms-dom0' failed
make: *** [rpms-dom0] Error 1

Reg Tiangha

unread,
Apr 28, 2017, 11:59:09 PM4/28/17
to qubes...@googlegroups.com
On 04/28/2017 09:03 PM, gho...@gmail.com
wrote:
You need to update the qubes-kernel-vm-support package in the Fedora VM
that you're trying to compile this in. A compatible version (3.2.4)
should has been pushed out to the stable repositories so running sudo
dnf upgrade should pull it in (unless you've never installed it in the
first place, in which case you should run sudo dnf install
qubes-kernel-vm-support instead). Once you've installed the package, try
compiling one of those kernels again and it should work.


gho...@gmail.com

unread,
Apr 29, 2017, 12:25:17 AM4/29/17
to qubes-users, r...@reginaldtiangha.com

> >
> You need to update the qubes-kernel-vm-support package in the Fedora VM
> that you're trying to compile this in. A compatible version (3.2.4)
> should has been pushed out to the stable repositories so running sudo
> dnf upgrade should pull it in (unless you've never installed it in the
> first place, in which case you should run sudo dnf install
> qubes-kernel-vm-support instead). Once you've installed the package, try
> compiling one of those kernels again and it should work.

I have version 3.2.3 installed. I also have the "current", "current-testing", and "unstable" repositories enabled in qubes-r3.repo.

doing a search for qubes-kernel-vm-support only shows version 3.2.3. Doing upgrade says nothing to do. Doing a reinstall only installs version 3.2.3 Is there somewhere else to get the version 3.2.4

Thanks.

Reg Tiangha

unread,
Apr 29, 2017, 12:29:33 AM4/29/17
to qubes...@googlegroups.com
On 04/28/2017 10:25 PM, gho...@gmail.com
wrote:
I don't know why it wouldn't work for you, unless you're running a
version of Qubes older than R3.2 or using an unsupported Fedora template.

As a last resort, you can replace your /usr/src/u2mfn-3.2.3/u2mfn.c file
with this one here:

https://raw.githubusercontent.com/QubesOS/qubes-linux-utils/master/kernel-modules/u2mfn/u2mfn.c

and compiling should work.


gho...@gmail.com

unread,
Apr 29, 2017, 1:20:44 AM4/29/17
to qubes-users, r...@reginaldtiangha.com

> I don't know why it wouldn't work for you, unless you're running a
> version of Qubes older than R3.2 or using an unsupported Fedora template.
>
> As a last resort, you can replace your /usr/src/u2mfn-3.2.3/u2mfn.c file
> with this one here:
>
> https://raw.githubusercontent.com/QubesOS/qubes-linux-utils/master/kernel-modules/u2mfn/u2mfn.c
>
> and compiling should work.

It works now. Thanks for your help.

I started from scratch with a new clone of the fedora 23 template that was installed with R3.2. I replaced the u2mfn.c file with the one you linked to.

Reg Tiangha

unread,
Apr 29, 2017, 1:55:06 AM4/29/17
to qubes...@googlegroups.com
On 04/28/2017 11:20 PM, gho...@gmail.com
wrote:
Hmm, looks like you're right. Only 3.2.3 is in all the Fedora repos
(23-25). Which is weird, because I could have sworn it was pushed out in
the latest round of stable updates. It does show up in the Debian repos,
though.

Well, the important thing was updating that u2mfn.c file to work with
kernels newer than 4.8. Glad to hear it still works.



Foppe de Haan

unread,
Apr 29, 2017, 1:56:42 AM4/29/17
to qubes-users, r...@reginaldtiangha.com, gho...@gmail.com
the update wasn't built for the fc23-vm: https://github.com/QubesOS/updates-status/issues/17

gho...@gmail.com

unread,
Apr 29, 2017, 1:59:24 AM4/29/17
to qubes-users, r...@reginaldtiangha.com, gho...@gmail.com
On Saturday, April 29, 2017 at 1:56:42 AM UTC-4, Foppe de Haan wrote:
> the update wasn't built for the fc23-vm: https://github.com/QubesOS/updates-status/issues/17

Thanks.

somehow I never found that page during google searching.

Reg Tiangha

unread,
Apr 29, 2017, 2:00:13 AM4/29/17
to qubes...@googlegroups.com
On 04/28/2017 11:56 PM, Foppe de Haan wrote:
> the update wasn't built for the fc23-vm: https://github.com/QubesOS/updates-status/issues/17
>
That's really weird, since it came out for dom0, which is essentially
fc23. And it looks like the fc24 and 25 versions never transferred over
from current-testing either.


Message has been deleted

0spin...@gmail.com

unread,
Jun 27, 2017, 6:40:00 AM6/27/17
to qubes-users, r...@reginaldtiangha.com
Anyone have an idea why, since 4.11.7, I am always getting a "initramfs not in gzip format" error?

Epitre

unread,
Jun 27, 2017, 10:09:13 AM6/27/17
to qubes-users, r...@reginaldtiangha.com, 0spin...@gmail.com
Le mardi 27 juin 2017 12:40:00 UTC+2, 0spin...@gmail.com a écrit :
> Anyone have an idea why, since 4.11.7, I am always getting a "initramfs not in gzip format" error?

Hi, same problem for me with 4.11.7. I also tried to select only AMD family (my type of processor) and it results the same.

Reg Tiangha

unread,
Jun 27, 2017, 2:28:55 PM6/27/17
to qubes...@googlegroups.com
I just tried 4.11.7 for myself on my machine, and it works fine in both
dom0 and in VMs.

So, if you're using my development branch, make sure to run 'git pull'
to ensure everything is synced up (for example, if you haven't done it
in a while, then you may not have the XSA 216 security patches applied
to your kernel that were released last week):

https://github.com/rtiangha/qubes-linux-kernel/tree/devel-4.11

Otherwise, make sure your config file has these options set (use 'make
menuconfig' and search for them to ensure they're set correctly):

CONFIG_HAVE_KERNEL_GZIP=y

CONFIG_RD_GZIP=y

CONFIG_DECOMPRESS_GZIP=y


If that still doesn't work, then more information is needed. But for
now, try the above and see if that works.



0spin...@gmail.com

unread,
Jun 27, 2017, 3:37:03 PM6/27/17
to qubes-users, r...@reginaldtiangha.com
Thanks. Was already up to date, though, and all gzip-related options were enabled (as before). Only change was a new package req (elfutils-libelf-devel).
As for new info, I have frustratingly little to offer:
-------------
mkdir -p /home/user/rpmbuild/BUILDROOT/kernel-4.11.7-18.pvops.qubes.x86_64//var/lib/qubes/vm-kernels/4.11.7-18
+ PATH=/sbin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/user/.local/bin:/home/user/bin
+ dracut --nomdadmconf --nolvmconf --kmoddir /home/user/rpmbuild/BUILDROOT/kernel-4.11.7-18.pvops.qubes.x86_64/lib/modules/4.11.7-18.pvops.qubes.x86_64 --modules 'kernel-modules qubes-vm-simple' --conf /dev/null --confdir /var/empty -d 'xenblk xen-blkfront cdrom ext4 jbd2 crc16 dm_snapshot' /home/user/rpmbuild/BUILDROOT/kernel-4.11.7-18.pvops.qubes.x86_64//var/lib/qubes/vm-kernels/4.11.7-18/initramfs 4.11.7-18.pvops.qubes.x86_64
Kernel version 4.11.7-18.pvops.qubes.x86_64 has no module directory /lib/modules/4.11.7-18.pvops.qubes.x86_64
ldconfig: need absolute file name for configuration file when using -r
dracut: ldconfig might need uid=0 (root) for chroot()
++ lsinitrd /home/user/rpmbuild/BUILDROOT/kernel-4.11.7-18.pvops.qubes.x86_64//var/lib/qubes/vm-kernels/4.11.7-18/initramfs usr/lib/modules/4.11.7-18.pvops.qubes.x86_64/modules.dep
+ modules_dep=
+ '[' -z '' ']'
++ mktemp -d
+ tmpdir=/tmp/tmp.0U02gQXJIH
+ zcat /home/user/rpmbuild/BUILDROOT/kernel-4.11.7-18.pvops.qubes.x86_64//var/lib/qubes/vm-kernels/4.11.7-18/initramfs
+ cpio -imd -D /tmp/tmp.0U02gQXJIH

gzip: /home/user/rpmbuild/BUILDROOT/kernel-4.11.7-18.pvops.qubes.x86_64//var/lib/qubes/vm-kernels/4.11.7-18/initramfs: not in gzip format
cpio: premature end of archive
+ exit 1
error: Bad exit status from /var/tmp/rpm-tmp.MAZxNe (%install)
----------------
(If there is a way to get more (relevant) information, you'll have to tell me where to (start) look(ing), sorry.)

Reg Tiangha

unread,
Jun 27, 2017, 3:53:28 PM6/27/17
to qubes...@googlegroups.com
On 2017-06-27 1:37 PM,
0spin...@gmail.com wrote:

> Thanks. Was already up to date, though, and all gzip-related options were enabled (as before). Only change was a new package req (elfutils-libelf-devel).
> As for new info, I have frustratingly little to offer:
> -------------
> mkdir -p /home/user/rpmbuild/BUILDROOT/kernel-4.11.7-18.pvops.qubes.x86_64//var/lib/qubes/vm-kernels/4.11.7-18
> + PATH=/sbin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/user/.local/bin:/home/user/bin
> + dracut --nomdadmconf --nolvmconf --kmoddir /home/user/rpmbuild/BUILDROOT/kernel-4.11.7-18.pvops.qubes.x86_64/lib/modules/4.11.7-18.pvops.qubes.x86_64 --modules 'kernel-modules qubes-vm-simple' --conf /dev/null --confdir /var/empty -d 'xenblk xen-blkfront cdrom ext4 jbd2 crc16 dm_snapshot' /home/user/rpmbuild/BUILDROOT/kernel-4.11.7-18.pvops.qubes.x86_64//var/lib/qubes/vm-kernels/4.11.7-18/initramfs 4.11.7-18.pvops.qubes.x86_64
> Kernel version 4.11.7-18.pvops.qubes.x86_64 has no module directory /lib/modules/4.11.7-18.pvops.qubes.x86_64
> ldconfig: need absolute file name for configuration file when using -r
> dracut: ldconfig might need uid=0 (root) for chroot()
> ++ lsinitrd /home/user/rpmbuild/BUILDROOT/kernel-4.11.7-18.pvops.qubes.x86_64//var/lib/qubes/vm-kernels/4.11.7-18/initramfs usr/lib/modules/4.11.7-18.pvops.qubes.x86_64/modules.dep
> + modules_dep=
> + '[' -z '' ']'
> ++ mktemp -d
> + tmpdir=/tmp/tmp.0U02gQXJIH
> + zcat /home/user/rpmbuild/BUILDROOT/kernel-4.11.7-18.pvops.qubes.x86_64//var/lib/qubes/vm-kernels/4.11.7-18/initramfs
> + cpio -imd -D /tmp/tmp.0U02gQXJIH
>
> gzip: /home/user/rpmbuild/BUILDROOT/kernel-4.11.7-18.pvops.qubes.x86_64//var/lib/qubes/vm-kernels/4.11.7-18/initramfs: not in gzip format
> cpio: premature end of archive
> + exit 1
> error: Bad exit status from /var/tmp/rpm-tmp.MAZxNe (%install)
> ----------------
> (If there is a way to get more (relevant) information, you'll have to tell me where to (start) look(ing), sorry.)
>


Curious: What is your build environment for this kernel? I only ever
use/test FC23 because that's what my dom0 runs, but there have been
issues when the compile environment is different (ex. FC25).

I've only ever gotten gzip errors if I completely remove gzip support
from my config options, but I've only seen it appear at boot and not at
compile time.

Reg Tiangha

unread,
Jun 27, 2017, 4:42:51 PM6/27/17
to qubes...@googlegroups.com
Also, are you using the 4.11 patches from the Hardened Kernel project? I
remember rpm generation failing one time (can't remember the exact error
message) with one version of their patches (it patched in properly and
successfully compiled, but just died when it came to generating the
rpms), which eventually got fixed in a later revision. If you're using
them but haven't updated it recently, their latest version is 4.11.7.a:

https://github.com/copperhead/linux-hardened/releases

I also have a branch that tracks just that, which is essentially the
same as my devel-4.11 branch but just adds in and updates the
linux-hardened patches as those get released, mainly for convenience:

https://github.com/rtiangha/qubes-linux-kernel/tree/devel-4.11-hard



0spin...@gmail.com

unread,
Jun 27, 2017, 6:50:54 PM6/27/17
to qubes-users, r...@reginaldtiangha.com
On Tuesday, June 27, 2017 at 10:42:51 PM UTC+2, Reg Tiangha wrote:
> On 2017-06-27 1:53 PM, Reg Tiangha wrote:
> > On 2017-06-27 1:37 PM,

It seems building works fine on fc23. Wonder what changed between 6/17 and today that fc25 no longer compiles kernels, though.

Wasn't using any patches from the hardening project.

Reg Tiangha

unread,
Jun 28, 2017, 12:49:07 AM6/28/17
to qubes...@googlegroups.com
On 06/27/2017 04:50 PM,
0spin...@gmail.com wrote:
> It seems building works fine on fc23. Wonder what changed between 6/17 and today that fc25 no longer compiles kernels, though.
>
> Wasn't using any patches from the hardening project.

I just spun up a FC25 BuildVM and *no* kernels (I even tried 4.4 and
4.9) compile any more on that machine (but they do with the same config
on FC23).

There was a change to the kernel.spec file a few weeks ago to work
around a buggy dracut on FC25 for R4.0 and I think that's what's causing
it (the script seems to die in that code chunk) and maybe reverting it
to the old version might help, but I won't have time to look at this
again for another couple of days. I've noticed that Marek has continued
to do work on that file since the last time it was synchronized so maybe
if I sync up with that version, it might work again.

For now though, things seem to work fine with an FC23 build VM, so I'd
suggest sticking with that for now.


Foppe de Haan

unread,
Aug 12, 2017, 1:18:35 AM8/12/17
to qubes-users, r...@reginaldtiangha.com
Wrt (4.11) build dependencies: you'll also need elfutils-libelf-devel, gcc-plugin-devel.

as before, it's easiest to build these in a fc23-based VM.

Grzesiek Chodzicki

unread,
Aug 12, 2017, 12:41:33 PM8/12/17
to qubes-users, r...@reginaldtiangha.com
On my machine the compilation fails with a non obvious error
Makefile:90: recipe for target 'rpms-dom0' failed
make: *** [rpms-dom0] Error 1

This is in a f25 based machine. Would it be possible to have the entire thing in chroot or a docker container?

Foppe de Haan

unread,
Aug 12, 2017, 12:48:44 PM8/12/17
to qubes-users, r...@reginaldtiangha.com

Probably, but IDK how, because I'm stupid. :)

If you're running f25, the easiest way to get it to work is to enable the fc27/rawhide repo in an AppVM, and selectively update the packages related to dracut (to dracut-045). After that, it'll compile.

Grzesiek Chodzicki

unread,
Aug 12, 2017, 1:27:45 PM8/12/17
to qubes-users, r...@reginaldtiangha.com
Just tried that. Didn't help unfortunately.

Foppe de Haan

unread,
Aug 12, 2017, 1:41:36 PM8/12/17
to qubes-users, r...@reginaldtiangha.com
it should provide more info in the 10-50 lines preceding the exit message?

Grzesiek Chodzicki

unread,
Aug 12, 2017, 6:02:09 PM8/12/17
to qubes-users, r...@reginaldtiangha.com
/home/user/qubes-linux-kernel/kernel-4.9.35/linux-4.9.35/scripts/gcc-plugin.sh: line 19: g++: command not found
Cannot use CONFIG_GCC_PLUGINS: your gcc installation does not support plugins, perhaps the necessary headers are missing?
scripts/Makefile.gcc-plugins:51: recipe for target 'gcc-plugins-check' failed
make[2]: *** [gcc-plugins-check] Error 1
make[2]: Leaving directory '/home/user/qubes-linux-kernel/kernel-4.9.35/linux-obj'
Makefile:150: recipe for target 'sub-make' failed
make[1]: *** [sub-make] Error 2
make[1]: Leaving directory '/home/user/qubes-linux-kernel/kernel-4.9.35/linux-4.9.35'
error: Bad exit status from /var/tmp/rpm-tmp.380q8a (%prep)


RPM build errors:
Bad exit status from /var/tmp/rpm-tmp.380q8a (%prep)
Makefile:90: recipe for target 'rpms-dom0' failed
make: *** [rpms-dom0] Error 1

No idea where that comes from as I do have gcc installed.

Foppe de Haan

unread,
Aug 13, 2017, 12:35:53 AM8/13/17
to qubes-users, r...@reginaldtiangha.com
try installing gcc-c++?

Foppe de Haan

unread,
Aug 13, 2017, 1:24:29 AM8/13/17
to qubes-users, r...@reginaldtiangha.com
For any newcomers: can you tell me if this covers all the bases?
https://github.com/0spinboson/qubes-doc/blob/patch-1/managing-os/compiling-your-own-kernel.md
(or if not, what's missing?)
Message has been deleted

Epitre

unread,
Aug 13, 2017, 3:19:25 AM8/13/17
to qubes-users, r...@reginaldtiangha.com
Hi,

It seems right for me. Just a a comment for the version in devel-4.11, the last working version (at least for me, and need to be confirmed) is 4.11.8:

The 4.11.12 has a Xen bug which has to be fixed and prevent Xen to work.
The 4.12.5 has also the same bug but need to have also 3 patches updated.

In both cases, qubes-core status:

août 11 21:37:07 dom0 startup-misc.sh[2712]: xenstore-write: xs_open: No such file or directory
août 11 21:37:07 dom0 startup-misc.sh[2712]: xenstore-write: xs_open: No such file or directory
août 11 21:37:07 dom0 startup-misc.sh[2712]: xc: error: Could not obtain handle on privileged command interface (2 = No such file or directory): Internal error
août 11 21:37:07 dom0 startup-misc.sh[2712]: libxl: error: libxl.c:116:libxl_ctx_alloc: cannot open libxc handle: No such file or directory
août 11 21:37:07 dom0 startup-misc.sh[2712]: cannot init xl context

I will dig more into the problem in the next week but if someone would like to test to confirm or not, it would help.

Moreover, for those who have problem with NOUVEAU driver (see my first post asking help: https://groups.google.com/d/msg/qubes-devel/koDHzHJICEs/M5B19MBgCgAJ) and their GTX970 with 4G of VRAM, I patched the qubes kernel (https://github.com/fepitre/qubes-linux-kernel) for version 4.9 and 4.11. The major problem is in the computation of VRAM which has been completely remade and solved in kernel 4.12.

Epitre

unread,
Aug 13, 2017, 3:38:06 AM8/13/17
to qubes-users, r...@reginaldtiangha.com
Sorry for the quick updates but writing the message it came to mind that it would maybe something related to Grub...and yes...I boot the my dev machine and I don't know why but the grub conf was badly updated...

I can confirm that the lastest working version is 4.11.12. I will also update properly my repo for the patches in devel-4.12.