Qubes OS development update
205 views
Skip to first unread message
Marek Marczykowski-Górecki
Sep 19, 2018, 12:12:51 AM9/19/18
to qubes-devel
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Hi,
It's been a while since I've sent last update like this. So here is what
is currently happening:
Qubes OS 4.1
- -------------
= GUI domain =
The major new feature in Qubes 4.1 is GUI domain[1]. There are two
approaches to move GUI to a VM:
- using GPU passthrough[2]
- using "nested qubes-gui" protocol[3]
Both are quite challenging, but there is some progress on the second
one. I've managed to configure one machine to have fully functional
Xfce4 running in normal AppVM with Xephyr in fullscreen mode. This works
quite well in a single monitor setup, but when introducing secondary
monitor things become messy (leaving alone dynamic changes, which don't
work at all). We're also collecting things that needs attention after
moving GUI away from dom0[4]. For example battery state widget becomes
useless.
None of this is pushed anywhere yet. See linked github issues for more
details.
= Update things =
In the meantime, as usual, we're updating core components like Xen,
libvirt, etc. And upstreaming patches where applicable. This time, we've
managed to upstream suspend-related patches to both Xen and libvirt. And
we're in the process for libvirt PVH support and Linux-based stubdomain
for Xen.
Frederic is also working on Fedora 29 support - at least in templates
(almost done).
= New qrexec policy =
While locking down dom0 access, one of the things that normally require
dom0 access is policy modification. Our initial plan was to manage the
whole policy using (rpm or other) packages, which allows proper
authentication of the changes. Using tags in default policy, it
also allows for some flexibility, but it turned out it isn't enough. So,
we've got a plan to design an qrexec API to modify qrexec policy. The
main difference from direct dom0 access is that such API can be limited
(for example allow to modify qubes.FileCopy policy, but not
qubes.VMShell), and also all changes can be properly logged. But in the
process, it turned out the current policy format is not flexible enough for this,
especially when mixing package-maintained and API-maintained policy.
So, there is ongoing work on new qrexec policy format[10]. The major
change is ability to specify multiple services in services in a single
file (allowing to ship multi-service solutions with a single file
policy, easing audit), and also ability to split the policy into
multiple files (for example separating local changes from those shipped
with a package). Basically, the policy get classic ".d" directory
format.
The current state is the specification[10]. Also some of the policy
parser is already implemented. While at it, we've also moved all the
qrexec related parts into separate "qubes-core-qrexec" repository (which
right now is only on Wojtek's github account). This will ease adding
"just qrexec" to various systems (maybe including other non-Linux
systems), without all the crap^H^H^H^Hstuff that lives in
qubes-core-agent-linux (a lot of system configuration, including pam, fstab
etc).
This part is handled mostly by Wojtek.
Qubes OS 3.2.1 and 4.0.1
- -------------------------
Almost there...
Major changes for 3.2.1 are just about updating included components:
- Fedora 28 and Debian 9 templates
- Whonix 14
- Kernel 4.14 by default
As said earlier, the changes made the installation image too large for
single-layer DVD.
Keep in mind that if you're running R3.2, there is no need for
reinstallation, this is only about updated installation image to
incorporate all the updates we're released in the meantime. Applying the
updates using normal mechanisms (including template updates[7][8][9])
should result in the same system.
In fact I have already built something that I hope will become
R3.2.1-rc1, but my R3.2 test machine have died yesterday :/
Automated installation tests are positive, though[5]. If anyone want to
help testing, you can grab the image here for example:
https://openqa.qubes-os.org/tests/445/asset/iso/Qubes-DVD-x86_64-3.2-20180916-3.iso
(note it is a test build, not release build, so don't use for anything
serious)
Qubes OS 4.0
- -------------
Simultaneously to working on Qubes OS 4.1, we put a lot of effort in making
Qubes OS 4.0 more and more stable and user friendly. This include fixing
various bugs (there are still many of them[11]), but also making GUI
tools more friendly, after some regression here in initial 4.0 release.
You may want to test recent Qube Manager update[12], which include a lot
of fixes.
The GUI tools part is Marta's work.
= Qubes OS 4.0.1 =
Similar to Qubes OS 3.2.1, there will be mostly updated templates and
incorporate all the updates released to date. And similarly to 3.2.1,
installation tests are positive[6].
= btrfs =
Rusty Bird have done excellent work and contributed file-reflink storage
driver[20], making use of copy-on-write feature of btrfs, instead of
building device-mapper layer on plain files.
- From now on (including Qubes OS 4.0.1) if you choose btrfs partitioning
layout, by default file-reflink driver will be used. Existing
storage pools are not affected by this change.
= Automated tests =
Some of you have already seen, we've got OpenQA instance running at
https://openqa.qubes-os.org/! It helps a lot with testing installation
- - multiple configurations are now tested (btrfs installation finally
works!), no need to do that manually, with a paper checklist.
But there is more! Since the Qubes in KVM works surprisingly well
(almost[13]), I thought why not run system tests there, instead of doing
that from time to time on a physical machine. This way, the results are
publicly available, so for example we can link to them from github
issues etc. To integrate it, we needed to plug individual test results
into OpenQA results page. Our test runner output nicely formatted
(colorful!) log, but nothing machine readable. But nose2 do have
various plugins, including junit-xml format, which is supported by
OpenQA (well, almost[15], as usual...). So, Wojtek gone ahead and fixed
our tests to be compatible with nose2[14]. This, plus some automation,
made it possible to observe system tests results online[16].
The tests are divided into 6 groups, mostly to parallelize execution. If
any test in group fails, the whole group is marked as failed, but you
can review individual test results inside. This means that all the
groups currently are "red"... But if you look closely inside, most of
the tests passes! One group (system_tests_extra) fails as a whole
because of a timeout, I need to increase it further (it's already at 3h).
It may be also a good idea to split the groups differently, for example
to optimize for time (the whole thing already takes 12h+), or results
easier to interpret (for example keep rarely failing tests together so
at least some groups will be "green").
One of the things done there is automatically installing updates from
current-testing repository. This is done using salt[17], based on [18].
The initial update takes about 30min, which should be also considered
when choosing test groups (too many groups means more time spent on
preparatory updates).
Next steps:
- link test results to relevant issues in updates-status
repository[19], so it will be easier to spot bugs detected by tests
- (maybe?) send test results via email, possibly to qubes-devel ML
- automatically schedule builds and tests - right now it is still
manual thing (ssh, up, enter)
- (maybe?) collect packages built in Travis-CI, create isolated
repository and run tests against it, so problems will be even detected
before package lands in testing repo; but there are more questions
than answers in this direction
The list could go on, but I think the above are the major points
happening right now.
[1] https://github.com/QubesOS/qubes-issues/issues/833
[2] https://github.com/QubesOS/qubes-issues/issues/2618
[3] https://github.com/QubesOS/qubes-issues/issues/833#issuecomment-277015652
[4] https://github.com/QubesOS/qubes-issues/issues/4186
[5] https://openqa.qubes-os.org/tests/overview?distri=qubesos&version=3.2&build=3.2-20180916-3&groupid=1
[6] https://openqa.qubes-os.org/tests/overview?distri=qubesos&version=4.0&build=4.0-20180916&groupid=1
[7] https://www.qubes-os.org/doc/templates/fedora/
[8] https://www.qubes-os.org/doc/templates/debian/
[9] https://www.qubes-os.org/news/2018/08/07/whonix-14-has-been-released/
[10] https://github.com/woju/qubes-core-qrexec/blob/master/Documentation/multifile-policy.markdown
[11] https://github.com/QubesOS/qubes-issues/issues?q=is%3Aopen+is%3Aissue+milestone%3A%22Release+4.0+updates%22+label%3Abug
[12] https://github.com/QubesOS/updates-status/issues/694
[13] https://markmail.org/thread/jlj4cxz5e33ile43
[14] https://github.com/QubesOS/qubes-core-admin/pull/209
[15] https://github.com/marmarek/openqa-tests-qubesos/blob/master/extra-files/convert_junit.py
[16] https://openqa.qubes-os.org/tests/overview?distri=qubesos&version=4.0&build=4.0-20180918&groupid=6
[17] https://github.com/marmarek/openqa-tests-qubesos/tree/master/extra-files/update
[18] https://github.com/QubesOS/qubes-issues/issues/2718
[19] https://github.com/QubesOS/updates-status/issues
[20] https://github.com/QubesOS/qubes-core-admin/pulls?utf8=%E2%9C%93&q=is%3Apr+author%3Arustybird+reflink
- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAluhzL4ACgkQ24/THMrX
1yx6AAf/a5YAfPOLsAt5Z2/n1WzGsNnRkspPfZlvwDHhqoqb4suQBOjCYnhWbk4z
luC82USvLUqKhpkoVHpKM9lElDLHuMBax+qRq24GSHILrZ+smePsdGSWgOdBQlDs
DDkZEGsqk0jOy2m+lZen0n940DmagV0+bgxSarDoTGXtHyEKsVNjH91TEl4pOAot
UXCTQu110oGtpVeLo9Yvkt6uCFxvR1c65bWbkUk1ruPPXAqn1h6aynargQjx8apD
UnhHlCnZMCFSCjtztRUjf3kgRRNm5HSOhir9nAUJ3zq2RxKuyz5INyGGcK16ysAZ
mfEKbpwlHmQNHXC7y85LqkS7jdY6Pg==
=t+K4
-----END PGP SIGNATURE-----
Hash: SHA256
Hi,
It's been a while since I've sent last update like this. So here is what
is currently happening:
Qubes OS 4.1
- -------------
= GUI domain =
The major new feature in Qubes 4.1 is GUI domain[1]. There are two
approaches to move GUI to a VM:
- using GPU passthrough[2]
- using "nested qubes-gui" protocol[3]
Both are quite challenging, but there is some progress on the second
one. I've managed to configure one machine to have fully functional
Xfce4 running in normal AppVM with Xephyr in fullscreen mode. This works
quite well in a single monitor setup, but when introducing secondary
monitor things become messy (leaving alone dynamic changes, which don't
work at all). We're also collecting things that needs attention after
moving GUI away from dom0[4]. For example battery state widget becomes
useless.
None of this is pushed anywhere yet. See linked github issues for more
details.
= Update things =
In the meantime, as usual, we're updating core components like Xen,
libvirt, etc. And upstreaming patches where applicable. This time, we've
managed to upstream suspend-related patches to both Xen and libvirt. And
we're in the process for libvirt PVH support and Linux-based stubdomain
for Xen.
Frederic is also working on Fedora 29 support - at least in templates
(almost done).
= New qrexec policy =
While locking down dom0 access, one of the things that normally require
dom0 access is policy modification. Our initial plan was to manage the
whole policy using (rpm or other) packages, which allows proper
authentication of the changes. Using tags in default policy, it
also allows for some flexibility, but it turned out it isn't enough. So,
we've got a plan to design an qrexec API to modify qrexec policy. The
main difference from direct dom0 access is that such API can be limited
(for example allow to modify qubes.FileCopy policy, but not
qubes.VMShell), and also all changes can be properly logged. But in the
process, it turned out the current policy format is not flexible enough for this,
especially when mixing package-maintained and API-maintained policy.
So, there is ongoing work on new qrexec policy format[10]. The major
change is ability to specify multiple services in services in a single
file (allowing to ship multi-service solutions with a single file
policy, easing audit), and also ability to split the policy into
multiple files (for example separating local changes from those shipped
with a package). Basically, the policy get classic ".d" directory
format.
The current state is the specification[10]. Also some of the policy
parser is already implemented. While at it, we've also moved all the
qrexec related parts into separate "qubes-core-qrexec" repository (which
right now is only on Wojtek's github account). This will ease adding
"just qrexec" to various systems (maybe including other non-Linux
systems), without all the crap^H^H^H^Hstuff that lives in
qubes-core-agent-linux (a lot of system configuration, including pam, fstab
etc).
This part is handled mostly by Wojtek.
Qubes OS 3.2.1 and 4.0.1
- -------------------------
Almost there...
Major changes for 3.2.1 are just about updating included components:
- Fedora 28 and Debian 9 templates
- Whonix 14
- Kernel 4.14 by default
As said earlier, the changes made the installation image too large for
single-layer DVD.
Keep in mind that if you're running R3.2, there is no need for
reinstallation, this is only about updated installation image to
incorporate all the updates we're released in the meantime. Applying the
updates using normal mechanisms (including template updates[7][8][9])
should result in the same system.
In fact I have already built something that I hope will become
R3.2.1-rc1, but my R3.2 test machine have died yesterday :/
Automated installation tests are positive, though[5]. If anyone want to
help testing, you can grab the image here for example:
https://openqa.qubes-os.org/tests/445/asset/iso/Qubes-DVD-x86_64-3.2-20180916-3.iso
(note it is a test build, not release build, so don't use for anything
serious)
Qubes OS 4.0
- -------------
Simultaneously to working on Qubes OS 4.1, we put a lot of effort in making
Qubes OS 4.0 more and more stable and user friendly. This include fixing
various bugs (there are still many of them[11]), but also making GUI
tools more friendly, after some regression here in initial 4.0 release.
You may want to test recent Qube Manager update[12], which include a lot
of fixes.
The GUI tools part is Marta's work.
= Qubes OS 4.0.1 =
Similar to Qubes OS 3.2.1, there will be mostly updated templates and
incorporate all the updates released to date. And similarly to 3.2.1,
installation tests are positive[6].
= btrfs =
Rusty Bird have done excellent work and contributed file-reflink storage
driver[20], making use of copy-on-write feature of btrfs, instead of
building device-mapper layer on plain files.
- From now on (including Qubes OS 4.0.1) if you choose btrfs partitioning
layout, by default file-reflink driver will be used. Existing
storage pools are not affected by this change.
= Automated tests =
Some of you have already seen, we've got OpenQA instance running at
https://openqa.qubes-os.org/! It helps a lot with testing installation
- - multiple configurations are now tested (btrfs installation finally
works!), no need to do that manually, with a paper checklist.
But there is more! Since the Qubes in KVM works surprisingly well
(almost[13]), I thought why not run system tests there, instead of doing
that from time to time on a physical machine. This way, the results are
publicly available, so for example we can link to them from github
issues etc. To integrate it, we needed to plug individual test results
into OpenQA results page. Our test runner output nicely formatted
(colorful!) log, but nothing machine readable. But nose2 do have
various plugins, including junit-xml format, which is supported by
OpenQA (well, almost[15], as usual...). So, Wojtek gone ahead and fixed
our tests to be compatible with nose2[14]. This, plus some automation,
made it possible to observe system tests results online[16].
The tests are divided into 6 groups, mostly to parallelize execution. If
any test in group fails, the whole group is marked as failed, but you
can review individual test results inside. This means that all the
groups currently are "red"... But if you look closely inside, most of
the tests passes! One group (system_tests_extra) fails as a whole
because of a timeout, I need to increase it further (it's already at 3h).
It may be also a good idea to split the groups differently, for example
to optimize for time (the whole thing already takes 12h+), or results
easier to interpret (for example keep rarely failing tests together so
at least some groups will be "green").
One of the things done there is automatically installing updates from
current-testing repository. This is done using salt[17], based on [18].
The initial update takes about 30min, which should be also considered
when choosing test groups (too many groups means more time spent on
preparatory updates).
Next steps:
- link test results to relevant issues in updates-status
repository[19], so it will be easier to spot bugs detected by tests
- (maybe?) send test results via email, possibly to qubes-devel ML
- automatically schedule builds and tests - right now it is still
manual thing (ssh, up, enter)
- (maybe?) collect packages built in Travis-CI, create isolated
repository and run tests against it, so problems will be even detected
before package lands in testing repo; but there are more questions
than answers in this direction
The list could go on, but I think the above are the major points
happening right now.
[1] https://github.com/QubesOS/qubes-issues/issues/833
[2] https://github.com/QubesOS/qubes-issues/issues/2618
[3] https://github.com/QubesOS/qubes-issues/issues/833#issuecomment-277015652
[4] https://github.com/QubesOS/qubes-issues/issues/4186
[5] https://openqa.qubes-os.org/tests/overview?distri=qubesos&version=3.2&build=3.2-20180916-3&groupid=1
[6] https://openqa.qubes-os.org/tests/overview?distri=qubesos&version=4.0&build=4.0-20180916&groupid=1
[7] https://www.qubes-os.org/doc/templates/fedora/
[8] https://www.qubes-os.org/doc/templates/debian/
[9] https://www.qubes-os.org/news/2018/08/07/whonix-14-has-been-released/
[10] https://github.com/woju/qubes-core-qrexec/blob/master/Documentation/multifile-policy.markdown
[11] https://github.com/QubesOS/qubes-issues/issues?q=is%3Aopen+is%3Aissue+milestone%3A%22Release+4.0+updates%22+label%3Abug
[12] https://github.com/QubesOS/updates-status/issues/694
[13] https://markmail.org/thread/jlj4cxz5e33ile43
[14] https://github.com/QubesOS/qubes-core-admin/pull/209
[15] https://github.com/marmarek/openqa-tests-qubesos/blob/master/extra-files/convert_junit.py
[16] https://openqa.qubes-os.org/tests/overview?distri=qubesos&version=4.0&build=4.0-20180918&groupid=6
[17] https://github.com/marmarek/openqa-tests-qubesos/tree/master/extra-files/update
[18] https://github.com/QubesOS/qubes-issues/issues/2718
[19] https://github.com/QubesOS/updates-status/issues
[20] https://github.com/QubesOS/qubes-core-admin/pulls?utf8=%E2%9C%93&q=is%3Apr+author%3Arustybird+reflink
- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAluhzL4ACgkQ24/THMrX
1yx6AAf/a5YAfPOLsAt5Z2/n1WzGsNnRkspPfZlvwDHhqoqb4suQBOjCYnhWbk4z
luC82USvLUqKhpkoVHpKM9lElDLHuMBax+qRq24GSHILrZ+smePsdGSWgOdBQlDs
DDkZEGsqk0jOy2m+lZen0n940DmagV0+bgxSarDoTGXtHyEKsVNjH91TEl4pOAot
UXCTQu110oGtpVeLo9Yvkt6uCFxvR1c65bWbkUk1ruPPXAqn1h6aynargQjx8apD
UnhHlCnZMCFSCjtztRUjf3kgRRNm5HSOhir9nAUJ3zq2RxKuyz5INyGGcK16ysAZ
mfEKbpwlHmQNHXC7y85LqkS7jdY6Pg==
=t+K4
-----END PGP SIGNATURE-----
Reply all
Reply to author
Forward
0 new messages