Slimming Down the dom0 Kernel

[Reg Tiangha]

OK, so I think I've taken this personal project of mine to see how much
I could trim down the dom0 kernel as far as I can on my own (or rather,
I've found a set of settings that work for the hardware I own, but I'm
not sure how it'll perform on other people's hardware) so I'm ready to
share the results of my work.

This is what I've managed to slim down/disable and still have a working
dom0:

- Disabled the vast majority of options under "Networking Support"
including Bluetooth (I had to leave TCP/IP support enabled as libvirt
needs it to create sockets; because of that, I left SYN cookie and
Security Marking enabled in case it uses it, but if it doesn't, those
two options could also be disabled and things would still work).
- All hardware networking driver support (including Wireless); dom0
shouldn't have access to the network because it's all done through a
NetVM; hence that driver support no longer needs to be in dom0.
- Various hardware drivers (Legacy stuff like PCMCIA, ISDN, Floppy
Disks, plus Dallas 1-wire support (was on the fence on that one), stuff
regarding fibre channel support and other things that Qubes would be
unlikely to run on, although it's theoretically possible that it could be).
- Slimmed down on some of the Security options, taking out Apparmor (it
never worked in Fedora dom0 anyways) and SELinux support (if you really
use it in dom0, you could compile it back in yourself).
- Implemented most of the KSPP's recommended options here:
https://kernsec.org/wiki/index.php/Kernel_Self_Protection_Project

All-in-all, around 3,000 various kernel options were disabled, although
you could theoretically take that further by stripping out support for
hardware you don't use, and I'd love to do the same sorts of things to
some of the other sections like Cryptographic Support.

That said, the hardware support was something I struggled with. The
Networking stuff was easy; that's all handled by a NetVM so it could all
go. And if you have a USBVM, then it's easy: Disable all USB hardware
support except keyboard, mouse and touchscreen. However, for the moment,
there's no guarantee that every Qubes OS user runs a sys-usb VM (for
various reasons, many probably valid) so I left all of that support in.
On my personal kernel, however, I did disable all of that and leave it
to a sys-usb VM to sort out, but my USB hardware needs are minimal;
yours may not be. And ditto if you use the PCI pass-through functionality.

And there's also a near-infinite variety of hardware configurations out
there that Qubes could theoretically run on too, some of them really
obscure, so it was difficult for me to figure out what to cut and what
to keep/add in.

So this is where the Qubes community can help. I've posted my slim
kernel branch, based on 4.9 here:

https://github.com/rtiangha/qubes-linux-kernel/tree/devel-4.9-slim

And a somewhat stock 4.9 kernel branch here:

https://github.com/rtiangha/qubes-linux-kernel/tree/stable-4.9

And I'd like people to try both and offer some feedback on it. Is there
stuff that used to work on the full 4.9 kernel or older kernels that
doesn't work with the slim version? If so, let me know what it is and
I'll add that support back in.

Do people run Qubes on obscure hardware or unique/different hardware
combinations that no longer works on the slim kernel (ex. does anyone
really use ATA over Ethernet?)? Or something related to PCI
pass-through that no longer works? If so, again, let me know so I can
figure out what hardware support to re-enable.

And of course, you can go much further if you're willing to customize
your own kernel, either taking out support for hardware you don't
have/will never use, or adding in anything that's missing yourself. If
you choose to go that route, please share your results.

I don't have access to detailed stats or how RAM usage is affected, but
anecdotally, a kernel with the stock configuration took up around 200MB
of disk space, and my personal slim kernel takes up about 95MB. I assume
there's a RAM savings as well, but I've never measured, nor have I
measured how performance is impacted (although maybe that's where
something like the Phoronix Test Suite could come in once things settle
down a bit more).

To test, all you need to install is the kernel package that gets
generated once you've compiled it and reboot into it. You could also
install the kernel-qubes-vm version and use that in a VM; I haven't
tested it thoroughly but there may be a RAM savings. I'd recommend using
it in a non-networked VM though; it'll still connect to the Internet
thanks to TCP/IP support still being included, however, I stripped out
stuff like Netfilter/Iptables support, so I don't know how that impacts
security.

So that's about it. I'm not sure when I'll have more free time again to
work on this in depth so there may or may not be more detailed work on
this project coming from my end for a few months (limited volunteer
hours to spend and all that), but maybe together we can figure out a
slimmed-down stock dom0 kernel configuration that would still work for a
majority of Qubes OS users out-of-the-box that could be shipped in the
future.

[Eva Star]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 05/01/2017 02:36 AM, Reg Tiangha wrote:
> OK, so I think I've taken this personal project of mine to see how
> much I could trim down the dom0 kernel as far as I can on my own
> (or rather, I've found a set of settings that work for the hardware
> I own, but I'm not sure how it'll perform on other people's
> hardware) so I'm ready to share the results of my work.
>

All of this sounds very good. But most of us not so advanced unix
users to compile kernel and install it. Maybe, somebody (as I) can
try, but there is no readme on your repository how to do this and
install it :)

p.s. Maybe you forget about table(wacom) on you description and remove
it? Currently, usb-vm does not support tablets. (Not a big problem)

- --
Regards
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBCAAGBQJZCC28AAoJEGSin3PC/C0ANNcP/0CWC53vQtLOOamJmchyveiI
+F5lxEExIVh5BolIZ0jzHb4L/USVurKLqx1WkOfW9uX+HJ1M8aSFDY57ijp3jDQ2
ovbtI97tasQ0GLbNs0z7pBtA8Xb5VGQ7oYB/1+Dbl/+DB4g+APkvoCW5lfTwyVnJ
XCHWgjRKIzmxekdCTiHeR0YmZ7Pqbn+tZHo9BS+DM/ffR0saqlafaurnwfvT/ufT
P+NBvxrBPBb4J/uHcjU5IMthNUWNYFWMmPJ80VX1RZkj9Kljzh3WxMqpcOAN3Ddz
CfkKyYW5L1F8is4Nc4ZwSbye2oliycgJICV1MSob4+CrEsYsUJprSw3XJzmLIvJT
tmWl9bAMVzOSsvVkix5bWegUflUA8qwpRpVTD45JoBNadS8QVbD1XMmSaH34MDZL
Uy+sR6l8dc4eGFN201F3By6T95JC4gFSXSi/lIFibJSpZ0YpcY0KU6BgUDPiEAo5
j9f940XIldrp1ZQxbjH9+frHnbO4nvpbKaqyIzaXirfpNbYhdG1DWQGyZXGFawWU
Wx1g4QtHJJju2SYF/z1wCg5mWJRr7Eb2ItfRnqHihOeR07zwK6d3Vk2xBdGVSBll
A0I7KpiPBJsalEmsEcXPRAnECqKvtOgtpqxBayPwBEqLyRoou8Qbb2o6qdqFGiVe
4ah83X5Y/cch15GnPUyv
=Yrjq
-----END PGP SIGNATURE-----

[Reg Tiangha]

On 05/02/2017 12:57 AM, Eva Star wrote:
> All of this sounds very good. But most of us not so advanced unix
> users to compile kernel and install it. Maybe, somebody (as I) can
> try, but there is no readme on your repository how to do this and
> install it :)
>
> p.s. Maybe you forget about table(wacom) on you description and remove
> it? Currently, usb-vm does not support tablets. (Not a big problem)
>

The reason why that repository has no readme is because the Qubes master
repository has no readme. You do make a good point though: All Qubes
packages should have some kind of write up on how to compile them. You
can kind of infer how to do it by reading through the Makefiles, but
still, it'd definitely be easier for anyone wanting to jump in to simply
test them, if not develop for them, if easier instructions were
available on how to compile them and what the software dependencies for
that repository are.

I did post some compile instructions in this thread here:

https://groups.google.com/d/msg/qubes-users/yBeUJPwKwHM/CFLgGsyKBAAJ

However, writing up a finalized and nicely formatted version for the
Qubes website is next on my TO-DO list.

As for things like Wacom tablets, I didn't mention it in my message in
this thread, but it, along with all other input devices supported in the
kernel, is still enabled in the current set of kernel options for this
version of the dom0 kernel. But if you were compiling this for yourself
and you don't use such devices, that's another category of kernel config
options that you could disable.

I've done some further refinement on my personal kernel, and it only
takes up around 50MB of disk space, down from about 200MB with the stock
kernel. And I still think it could be reduced even more. So the one
currently found in the repository definitely can still be cut down as
well, and there are probably some obscure kernel options that aren't
applicable to a dom0 kernel that I've missed.

You could probably go even further with a VM kernel. I simply no longer
have the time to test right now (and probably won't for a few months),
but some light testing on my end shows you can remove things such as
S/ATA support and native GPU drivers, simply because those aren't
normally needed in a VM environment (at long as you're not doing GPU
passthrough, although I don't know if that works with the open source
graphics drivers to begin with). A stripped down VM kernel would also
make for an interesting research project too.

[cooloutac]

I too would have trouble compiling kernel for fedora too. I only know how to do it with debian using make-kpkg which is much easier.

[Reg Tiangha]

On 05/02/2017 12:41 PM, cooloutac wrote:
> I too would have trouble compiling kernel for fedora too. I only know how to do it with debian using make-kpkg which is much easier.
>

The Qubes kernel build scripts actually make it very easy; assuming you
have all the software dependencies installed, you just need to run 'make
rpms.' The tricky part is the qubes-kernel-vm-support package has yet to
make it down to fc23 stable yet, so there's still some manual
configuration that needs to be done to get 4.9+ to compile on that
template. Hopefully, that'll fix itself within a week or whenever the
next batch of current-testing stuff migrates over to stable and things
will just work out-of-the-box. Of course, you can grab it yourself by
temporarily enabling the current-testing repo on an fc23 build template.

But my hope is to have that nicely-formatted kernel compile write-up
done by the end of this week; I'm not sure how much free time I'll have
past this week leading into July to work on this stuff so my hope is
that more people will feel empowered to compile their own kernels and to
share their results to see if everyone can find more kernel config
options that can be trimmed out while still having the kernel work on a
variety of hardware. For me, each compile-install-test cycle takes about
20-40 minutes on the laptop I use to test (longer if I actually compile
the kernel on it; my desktop is way faster), so there's diminishing
returns on my end in regards to time spent in trying to finesse more
options to disable by testing them out one-at-a-time. But if everyone
were to do a little bit on their own and compare their results, then
maybe things would move a bit faster. Personally, I'd like to know how
much of the SCSI subsystem or deprecated ciphers in the Cryptography
system could be cut out and still have things work, but again, I don't
have much more free time to work on this stuff.

[cooloutac]

you lost me SCSI subsystem, you mean like firmware drivers? and cryptopgraphy system? no idea. but sounds very interesting, I appreciate your time.

Also if I install all that stuff in my system. Shouldn't I then make sure to uninstall it all, or at least disable all those compilers I installed for security reasons? We might need a guide on how to do that too.

[Reg Tiangha]

On 05/02/2017 01:40 PM, cooloutac wrote:
> you lost me SCSI subsystem, you mean like firmware drivers? and cryptopgraphy system? no idea. but sounds very interesting, I appreciate your time.
>
> Also if I install all that stuff in my system. Shouldn't I then make sure to uninstall it all, or at least disable all those compilers I installed for security reasons? We might need a guide on how to do that too.

In the kernel options menu, there are various sections that you can configure. Things like Processor Type & Features, Device Drivers, Networking Support, Cryptography, etc. The SCSI stuff lives in the Device Drivers section (along with other things like PCI and USB support), while the Cryptography section lists all of the ciphers that the kernel supports. You could probably slim down the Cryptography section by taking out either ciphers that may be dangerous to continue using because they've been proven to be ineffective, or at the very least, take out ciphers that QubesOS doesn't use in dom0. Problem is, I just don't know what Qubes uses.

As for installing stuff to compile a kernel, that's why you do the kernel compilation in a Template VM, ideally one that matches the distro that's being used in dom0. That way, all the dependencies are installed in there and all you need to do is copy out the generated rpms into dom0 from the TemplateVM and just install those.

[cooloutac]

I never even looked a a cryptography section man tyvm! yes would be very awesome to know which ones to disable. very interesting. and what hardware etc of course, and we can then just copy our config over when building the next one. I think that is always the part that kills people but 20-40 mins aint bad.

I wouldn't mind compiling a kernel myself with good instructions. I'm just weary because I have tried in debian template and can't get the gui. It goes green and then to yellow on boot, can connect from terminal to it. I probably didn't do it the Qubes way properly. I think because of a missing module Marmarek said once so I tried to modprobe it not sure which one or how.

But ya if some expert can make a guide to compile kernel that would be awesome.

and thats right doh we do it all in a vm on Qubes!

[Reg Tiangha]

On 05/02/2017 02:27 PM, cooloutac wrote:
> I never even looked a a cryptography section man tyvm! yes would be very awesome to know which ones to disable. very interesting. and what hardware etc of course, and we can then just copy our config over when building the next one. I think that is always the part that kills people but 20-40 mins aint bad.
>
> I wouldn't mind compiling a kernel myself with good instructions. I'm just weary because I have tried in debian template and can't get the gui. It goes green and then to yellow on boot, can connect from terminal to it. I probably didn't do it the Qubes way properly. I think because of a missing module Marmarek said once so I tried to modprobe it not sure which one or how.
>
> But ya if some expert can make a guide to compile kernel that would be awesome.
>
> and thats right doh we do it all in a vm on Qubes!

Exactly! All of this research to find a generic, slimmed down kernel
configuration (for dom0 and/or maybe for VMs too) that can run on a
variety of Qubes machines would only need to be done once. Once done, it
could just be reused for every kernel update in that branch, and would
only need to be revisited whenever there's a major kernel update to
figure out which new features to integrate or reject. And ditto for a
personal kernel too that's tailored only to your hardware. That's how I
do it for my machines.

It's worthwhile doing, regardless, if only to save on disk space and
maybe RAM (I still can't believe I cut my personal kernel down to 1/4
the size of the stock kernel). I am interested in the Cryptography
stuff, though, because by default, *all* of the ciphers are enabled, and
I wonder if it's possible if an attacker could somehow downgrade the
encryption cipher in a VM or dom0 used by various operations to make
things easier to exploit, in which case, you could mitigate some of that
at the kernel level by taking out the weaker ciphers or those that Qubes
doesn't explicitly use. I don't know if that's a valid attack vector,
but if I thought of it, maybe someone smarter than me has too, and also
knows how to pull it off.

[cooloutac]

ya of course.