Qubes + OpenXT
505 views
Skip to first unread message
Eric Shelton
Jan 7, 2017, 1:58:39 PM1/7/17
to qubes-devel
A couple of recent events prompted me to look more into the OpenXT project:
- The recent announcements about Qubes' funding and shift in development priorities. It is great to hear that Qubes will remain an open source effort. However, there has consistently only been a small handful of developers active in Qubes -- mainly those being paid by ITL (without Marek, I don't think there would be a Qubes today). With their priorities being set by commercial clients, it seems less certain what Qubes will focus on, and on what schedule they will get there. I suppose there just aren't all that many non-ITL open source contributors with the needed Linux internals and Xen internals expertise looking to chip in, and maybe this situation will not change anytime soon.
- It came to my attention that OpenXT has implemented Linux-based stubdomains using upstream QEMU - an issue long neglected by upstream Xen. Qubes is working on this, and maybe it will be a 4.0 feature.
A couple of quick summaries of the OpenXT effort:
OpenXT is also the open source core of a very Qubes-like OS: SecureView:
I believe I am reasonably familiar with the Qubes project, and to me there seems to be a significant amount of overlap in the missions and solutions of Qubes and OpenXT - especially where it counts: the "security research" and virtual desktop integration areas. This also translates into a lot of duplicated effort across these two projects. OpenXT has their own TPM/AEM/measured launch, PV-USB (implemented long ago, and with Windows support), stub domains (they have long used Linux-based stubdomains, and will be moving up to QEMU 2.6 soon), netvm, and a UIVM that has been separated from dom0. I'm not saying everything is a one-to-one match with the Qubes architecture, but I can say that Qubes has been, and continues to, spend its precious and limited resources implementing and updating features that are already implemented and being updated in OpenXT. Also, OpenXT is way ahead of Qubes on Windows guest support (which I suspect is important for the Qubes commercialization path). There are also many ideas implemented or being worked on in Qubes that strike me as positive improvements for OpenXT: usbvm, seamless desktop, template domains, etc.
Perhaps it is worth considering (or reconsidering?) bringing the efforts of these two projects together. OpenXT might offer a reasonable base architecture for Qubes, and allow the Qubes team to not spread itself so thin (Marek and company are tackling problems from the BIOS on up, and that presents a lots of software that is changing (changes in KDE broke the window decoration, Wayland in F25 will surely bring new headaches) and constantly is pulling the Qubes developers away from their core mission. In some ways, isn't this what the HAL effort (which unfortunately does not seen to have achieved much) was about - abstracting Qubes away from the base hypervisor architecture?
The only comment I have seen from the Qubes developers about OpenXT is criticism of their use of dbus in their interdomain communication architecture (https://twitter.com/rootkovska/status/505024924097196032). I imagine the Qubes team can find a reasonable way around that. I understand there is going to be a significant "not invented here" resistance to this idea, but keep in mind that there are at least as many smart and security conscious developers working on OpenXT, too.
Perhaps another issue is that the developers working on OpenXT, for the most part, work for US defense industry contractors. SecureView appears to have ties with the NSA and the US Air Force. On the other hand, I will point out that these same parties are also contributors to upstream Xen and Linux - although among many others (and hypothetically, many more "sets of eyes" - however well that concept actually works in practice). Plus, somehow people long ago rationalized Tor having arisen from US Naval Research Laboratory and receiving ongoing funding from the US government. For what it is worth, Qubes could be a viable competitor for SecureView (especially if built on OpenXT), if you want to pursue that avenue for commercial funding.
I love Qubes, and it would be great if it had loads of resources to continue to do everything from the bottom to the top and remain the somewhat independent effort it has been for the last couple of years. But that is not the case, is it? If building Qubes on top of OpenXT allows Qubes developers to focus on the unique aspects of Qubes instead of fixing up things being broken by other projects, that would allow Qubes developer hours to be focused on developing the aspects that are unique to Qubes, demonstrate the value of Qubes, and perhaps establish a more sustainable cycle for Qubes (user interest, commercial interest, developer interest, etc.) that better ensures the project can continue. Plus, I suspect both projects have a lot to gain from each other. Frankly, otherwise I fear that the Qubes project will, despite the team's best intentions and efforts, fizzle out over the next few years.
Best,
Eric
Joanna Rutkowska
Jan 9, 2017, 11:02:41 AM1/9/17
to Eric Shelton, qubes-devel
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
You've touched on an interesting topic: what's the unique value of the Qubes OS
Project? I like to think it is primarily the following aspects:
1. The combination of both offensive and devops expertise in the team,
2. Ability to find reasonable balance between "seeking perfection" on the one
hand ("let's rewrite the whole stack from scratch"), and "conformism" on the
other hand ("maybe this solution can be bypassed trivially, but it still
increases the bar, blah blah blah"),
3. Last but not least: unlike so many recent projects which want to use
virtualization for creation of multiple isolated "computers", Qubes philosophy
has always been to provide one integrated environment built on top of many
isolated VMs.
Now, having the above in mind, lets consider your proposition of potentially
merging with OpenXT, or to use OpenXT as a basis for Qubes OS:
First, I don't think the fact they decided to use the D-Bus protocol as a
universal solution for their inter-VM communication should be considered as
disqualifying reason for us reusing their code. Instead, we should treat it as a
red light for the security competence of the team that have created their
architecture. A Big Red Light with a Roaring Fire Siren, to be more precise.
Indeed, OpenXT, previously known as XenClient XT, has a history of what I see as
bad security architecture decisions. Besides the above-criticized choice of
D-Bus as a fundamental building block for any VM-spanning services, another
striking example is the adoption of GPU pass-through as their primary graphics
virtualization technology.
While I'm sure that the word "pass-through" brings very positive connotations to
most readers ("something that doesn't require much handling from the hypervisor
side"), and also looks great on powerpoint diagrams ("arrows indifferently
crossing the hypervisor box"), in reality GPU pass-through require lots of
complexity on the hypervisor side, which the slides often forget to mention.
E.g. what happens when the real GPU is taken away from the VM? Are we going to
give it some *emulation* for the time being to keep it happy? If so, what if
there is a bug in the emulation code (there surely is, 'cause GPU's are
complex)? What privileges would the VM obtain after exploiting the GPU emulation
code?
On more recent Intel GPUs the situation has presumably improved, but the fact XT
has decided to chose this insecure form of graphics virtualization over the much
more secure approach that we designed for Qubes OS, so many years ago, is
another fact that makes me very uneasy about the security expertise of the XT
team.
(Because of this architecture decision, BTW, what XT people call "UIVM", is not
the same as what we call "GUI domain", and which we plan to introduce some time
after 4.0 release. In XT architecture, UIVM is a natural consequence of the GPU
pass-through model, and is merely a label assigned to one-of-the-many VMs which
can get full pass-through access to the GPU. In Qubes, the GUI domain is planed
to be *the only one* VM granted pass-through access to the GPU, while we would
continue to be using our -- I believe -- very secure GUI virtualization protocol
to allow other VMs to access graphics.)
Last but not least, probably the most fundamental difference between XT and
Qubes (and partly also responsible for some architectural differences) is in the
philosophy. XT attempts to implement the classic
multiple-isolated-(virtual)-computers-inside-one-box model (aka Multi Level
Security, or MLS), which originates from the military world. Qubes, on the other
hand, attempts to provide one secure desktop platform, which uses multiple
isolated VMs to securely decompose various tasks.
I've never quite got convinced by this MLS approach, to be honest. After all,
what's the point of running a few monolithic Windows VMs, if each of them is
still... a buggy, monolithic VM which can get compromised just as easily? Of
course, the official ideology of MLS says that even if an attacker compromises
one of the VMs, then they would not be able to at least leak the data to other,
unclassified networks.[*] In any case, the MLS proponents believe that these
systems could be useful to e.g. stop an insider from stealing gigabytes of
confidential info and leaking them out somewhere. Looks like this might not have
quite worked out for some of the organizations who are well known to promote
this security model (and incidentally also XT and SELinux)... ;)
Contract this to the Qubes approach, where we would like to utilize multiple
domains in order to actually make the domains more difficult to attack. Whether
by making them "disposable", or by splitting the tasks between more than one VM
(and potentially having some of these other VMs as disposable, or easily
recoverable).[**]
All the above complains about XT focus on the architecture. And I'd argue that
for a project like Qubes the proper architecture is something that should always
come first. Or to stay it differently: even the best code cannot be "rescued" if
it implements a bad architecture.
But perhaps they could produce some good code that we could consume
nevertheless? Well, maybe yes. Or maybe not. Somebody would need to take a look.
Likely we could utilize some non-critical pieces. But surely the idea of using
OpenXT as a *basis* for Qubes OS is not a good idea.
I also feel obliged to protest slightly against singling out any single
individual as a central force which keeps the project going. First, this makes
disservice to many others (many of which enlisted on our always-lagging-behind
Team page, but some not recognized even there) who also have contributed to the
project. Sometimes under-recognized when looking at the git statistics.
Consider e.g. Rafał Wojtczuk, who implemented most of the GUI virtualization in
a way that it has not needed any modifications over the last 7 years... Yet,
many might be tempted to overlook this when counting commits. This is also
thanks to a reasonably good architecture of the GUI virtualization, I think,
another aspect that almost never shows itself in the log.
Also, we shall not forget about the mysterious forces in the universe who -- for
all these years -- have paid the salaries of the core team. It's quite likely
Qubes would have ceased to exist long ago, if not for these magical fairies ;)
In other words, Qubes should not be equated with any single person.
Speaking of good developers, I'd like to also share an observation that very
good and bright developers are often a source of troubles in security, and that
we should all, both developers and non-developers (e.g. researchers), be very
weary of that all the time.
This (surely controversial) paradox might be a result of the fact that being
developer -- in essence -- is about learning how to master complexity. Modern
languages and frameworks allows one, and in fact even encourage, to implement
ever more complex creations, almost always hiding all the complexity in the
shadow. (For example: to use D-Bus for communication between
differently-privileged domains[***])
But the complexity is still there. At the end of the day, whatever software we
write, it will finally get translated to the machine code and execute as such.
But enough bashing of developers :) Without developers (or devops) Qubes would
not have existed. Nor it would make sense without all the offensive security
expertise that have been invested into it, especially in the early days. And
this brings us back to where we started, i.e. the unique value of the project :)
Cheers,
joanna.
[*] For this reasons such system -- to be remotely meaningful -- must always
take great lengths to reduce cooperative covert channels. Something that is
very hard on x86, and also something that Xen doesn't make easy to achieve.
[**] And thanks to this alternative approach, we're not so much interested in
elimination of cooperative covert channels in Qubes, although we recognize that
in some situations it would be still nice, such as for Tor/Whonix-related
services that run on top of Qubes infrastructure.
[***] A curious person might try to search the oss-security for all bugs
affecting D-bus and conclude there are not many. I think this is not surprising
given that D-bus is 1) primarily used on Linux Desktops (and AFAICT the Year of
Linux on the Desktop is still ahead of us), and 2) even if an attacker is
interested in attacking a Linux desktop, then usually -- thanks to the
monolithic architecture of mainstream Linux distributions -- it is not
necessarily for the attacker to attempt to attack D-Bus after she successfully
manages to execute her code as one of the user applications.
-----BEGIN PGP SIGNATURE-----
iQIcBAEBCAAGBQJYc7QWAAoJEDOT2L8N3GcY1R4P/iz5+VY5u6bWPguAm+8ABNSn
uDJhCWC6IhbEysrGlCWoaZY0x9BZ+IB2zAVtu6wPn+dl2UOK72Ey6M4BX0GzYvsA
nXdfRGojQtPTLsh5czYilw/Kbgym8139Im424kCU73d6K1FkkXOZgOFgCD2OzydS
fpAjBz26WyEsZqZ0+iP9fy5A1O8TWiR9vhPLxtqMplBiTmR/iNnR1JIvGfRdL5LR
+LcTkDbzUmOYMACvB0VwRRTeO/q4z/bU/ztMRXoJAU/z2B65ABSDB5VkzysUPGUu
OYkCuMj6KnYeX2F6oA851ws30KXxXw/hGPSm/e7EHPjWJYA2mIzPyjxfTklU0imi
OdXlBx7sdTTn25khUKQ+ejSn4JeNC7yypOzNmquy+8Szz8k0YaPBkBy1b1/3IbaK
y1ioontNbKl/kxqkmcUiPmIDkkEeM1UhJDqdicP67xego48OuSb+ce65ggFAsE5q
i8VGE4OW5yIbOQ9w8yUJmjwlUKgjN0ZFajt6SeVNcKKhzhF9iRV775LocQuW6Mn2
x79/sDNrM1pdq4F6NooM3TnLYQh2fzKNH+PiDpQNJ/dBHnij4agl90eci/CU06j0
PGmK4qK5HzSfDJZ+LWSfusPq/apztcKRp5/AyMfTE0Y0JDZ3BkwfOowguvzh1ric
GrrFYVG4+LtndCFfb9Fa
=1zcp
-----END PGP SIGNATURE-----
Hash: SHA256
Project? I like to think it is primarily the following aspects:
1. The combination of both offensive and devops expertise in the team,
2. Ability to find reasonable balance between "seeking perfection" on the one
hand ("let's rewrite the whole stack from scratch"), and "conformism" on the
other hand ("maybe this solution can be bypassed trivially, but it still
increases the bar, blah blah blah"),
3. Last but not least: unlike so many recent projects which want to use
virtualization for creation of multiple isolated "computers", Qubes philosophy
has always been to provide one integrated environment built on top of many
isolated VMs.
Now, having the above in mind, lets consider your proposition of potentially
merging with OpenXT, or to use OpenXT as a basis for Qubes OS:
First, I don't think the fact they decided to use the D-Bus protocol as a
universal solution for their inter-VM communication should be considered as
disqualifying reason for us reusing their code. Instead, we should treat it as a
red light for the security competence of the team that have created their
architecture. A Big Red Light with a Roaring Fire Siren, to be more precise.
Indeed, OpenXT, previously known as XenClient XT, has a history of what I see as
bad security architecture decisions. Besides the above-criticized choice of
D-Bus as a fundamental building block for any VM-spanning services, another
striking example is the adoption of GPU pass-through as their primary graphics
virtualization technology.
While I'm sure that the word "pass-through" brings very positive connotations to
most readers ("something that doesn't require much handling from the hypervisor
side"), and also looks great on powerpoint diagrams ("arrows indifferently
crossing the hypervisor box"), in reality GPU pass-through require lots of
complexity on the hypervisor side, which the slides often forget to mention.
E.g. what happens when the real GPU is taken away from the VM? Are we going to
give it some *emulation* for the time being to keep it happy? If so, what if
there is a bug in the emulation code (there surely is, 'cause GPU's are
complex)? What privileges would the VM obtain after exploiting the GPU emulation
code?
On more recent Intel GPUs the situation has presumably improved, but the fact XT
has decided to chose this insecure form of graphics virtualization over the much
more secure approach that we designed for Qubes OS, so many years ago, is
another fact that makes me very uneasy about the security expertise of the XT
team.
(Because of this architecture decision, BTW, what XT people call "UIVM", is not
the same as what we call "GUI domain", and which we plan to introduce some time
after 4.0 release. In XT architecture, UIVM is a natural consequence of the GPU
pass-through model, and is merely a label assigned to one-of-the-many VMs which
can get full pass-through access to the GPU. In Qubes, the GUI domain is planed
to be *the only one* VM granted pass-through access to the GPU, while we would
continue to be using our -- I believe -- very secure GUI virtualization protocol
to allow other VMs to access graphics.)
Last but not least, probably the most fundamental difference between XT and
Qubes (and partly also responsible for some architectural differences) is in the
philosophy. XT attempts to implement the classic
multiple-isolated-(virtual)-computers-inside-one-box model (aka Multi Level
Security, or MLS), which originates from the military world. Qubes, on the other
hand, attempts to provide one secure desktop platform, which uses multiple
isolated VMs to securely decompose various tasks.
I've never quite got convinced by this MLS approach, to be honest. After all,
what's the point of running a few monolithic Windows VMs, if each of them is
still... a buggy, monolithic VM which can get compromised just as easily? Of
course, the official ideology of MLS says that even if an attacker compromises
one of the VMs, then they would not be able to at least leak the data to other,
unclassified networks.[*] In any case, the MLS proponents believe that these
systems could be useful to e.g. stop an insider from stealing gigabytes of
confidential info and leaking them out somewhere. Looks like this might not have
quite worked out for some of the organizations who are well known to promote
this security model (and incidentally also XT and SELinux)... ;)
Contract this to the Qubes approach, where we would like to utilize multiple
domains in order to actually make the domains more difficult to attack. Whether
by making them "disposable", or by splitting the tasks between more than one VM
(and potentially having some of these other VMs as disposable, or easily
recoverable).[**]
All the above complains about XT focus on the architecture. And I'd argue that
for a project like Qubes the proper architecture is something that should always
come first. Or to stay it differently: even the best code cannot be "rescued" if
it implements a bad architecture.
But perhaps they could produce some good code that we could consume
nevertheless? Well, maybe yes. Or maybe not. Somebody would need to take a look.
Likely we could utilize some non-critical pieces. But surely the idea of using
OpenXT as a *basis* for Qubes OS is not a good idea.
I also feel obliged to protest slightly against singling out any single
individual as a central force which keeps the project going. First, this makes
disservice to many others (many of which enlisted on our always-lagging-behind
Team page, but some not recognized even there) who also have contributed to the
project. Sometimes under-recognized when looking at the git statistics.
Consider e.g. Rafał Wojtczuk, who implemented most of the GUI virtualization in
a way that it has not needed any modifications over the last 7 years... Yet,
many might be tempted to overlook this when counting commits. This is also
thanks to a reasonably good architecture of the GUI virtualization, I think,
another aspect that almost never shows itself in the log.
Also, we shall not forget about the mysterious forces in the universe who -- for
all these years -- have paid the salaries of the core team. It's quite likely
Qubes would have ceased to exist long ago, if not for these magical fairies ;)
In other words, Qubes should not be equated with any single person.
Speaking of good developers, I'd like to also share an observation that very
good and bright developers are often a source of troubles in security, and that
we should all, both developers and non-developers (e.g. researchers), be very
weary of that all the time.
This (surely controversial) paradox might be a result of the fact that being
developer -- in essence -- is about learning how to master complexity. Modern
languages and frameworks allows one, and in fact even encourage, to implement
ever more complex creations, almost always hiding all the complexity in the
shadow. (For example: to use D-Bus for communication between
differently-privileged domains[***])
But the complexity is still there. At the end of the day, whatever software we
write, it will finally get translated to the machine code and execute as such.
But enough bashing of developers :) Without developers (or devops) Qubes would
not have existed. Nor it would make sense without all the offensive security
expertise that have been invested into it, especially in the early days. And
this brings us back to where we started, i.e. the unique value of the project :)
Cheers,
joanna.
[*] For this reasons such system -- to be remotely meaningful -- must always
take great lengths to reduce cooperative covert channels. Something that is
very hard on x86, and also something that Xen doesn't make easy to achieve.
[**] And thanks to this alternative approach, we're not so much interested in
elimination of cooperative covert channels in Qubes, although we recognize that
in some situations it would be still nice, such as for Tor/Whonix-related
services that run on top of Qubes infrastructure.
[***] A curious person might try to search the oss-security for all bugs
affecting D-bus and conclude there are not many. I think this is not surprising
given that D-bus is 1) primarily used on Linux Desktops (and AFAICT the Year of
Linux on the Desktop is still ahead of us), and 2) even if an attacker is
interested in attacking a Linux desktop, then usually -- thanks to the
monolithic architecture of mainstream Linux distributions -- it is not
necessarily for the attacker to attempt to attack D-Bus after she successfully
manages to execute her code as one of the user applications.
-----BEGIN PGP SIGNATURE-----
iQIcBAEBCAAGBQJYc7QWAAoJEDOT2L8N3GcY1R4P/iz5+VY5u6bWPguAm+8ABNSn
uDJhCWC6IhbEysrGlCWoaZY0x9BZ+IB2zAVtu6wPn+dl2UOK72Ey6M4BX0GzYvsA
nXdfRGojQtPTLsh5czYilw/Kbgym8139Im424kCU73d6K1FkkXOZgOFgCD2OzydS
fpAjBz26WyEsZqZ0+iP9fy5A1O8TWiR9vhPLxtqMplBiTmR/iNnR1JIvGfRdL5LR
+LcTkDbzUmOYMACvB0VwRRTeO/q4z/bU/ztMRXoJAU/z2B65ABSDB5VkzysUPGUu
OYkCuMj6KnYeX2F6oA851ws30KXxXw/hGPSm/e7EHPjWJYA2mIzPyjxfTklU0imi
OdXlBx7sdTTn25khUKQ+ejSn4JeNC7yypOzNmquy+8Szz8k0YaPBkBy1b1/3IbaK
y1ioontNbKl/kxqkmcUiPmIDkkEeM1UhJDqdicP67xego48OuSb+ce65ggFAsE5q
i8VGE4OW5yIbOQ9w8yUJmjwlUKgjN0ZFajt6SeVNcKKhzhF9iRV775LocQuW6Mn2
x79/sDNrM1pdq4F6NooM3TnLYQh2fzKNH+PiDpQNJ/dBHnij4agl90eci/CU06j0
PGmK4qK5HzSfDJZ+LWSfusPq/apztcKRp5/AyMfTE0Y0JDZ3BkwfOowguvzh1ric
GrrFYVG4+LtndCFfb9Fa
=1zcp
-----END PGP SIGNATURE-----
Matteo
Jan 9, 2017, 2:53:30 PM1/9/17
to qubes...@googlegroups.com
Il 09/01/2017 17.02, Joanna Rutkowska ha scritto:
> Contract this to the Qubes approach, where we would like to utilize multiple
> domains in order to actually make the domains more difficult to attack. Whether
> by making them "disposable", or by splitting the tasks between more than one VM
> (and potentially having some of these other VMs as disposable, or easily
> recoverable).
some time ago i wanted to make a video about this, i wanted to proof
> Contract this to the Qubes approach, where we would like to utilize multiple
> domains in order to actually make the domains more difficult to attack. Whether
> by making them "disposable", or by splitting the tasks between more than one VM
> (and potentially having some of these other VMs as disposable, or easily
> recoverable).
why/how Qubes OS is more secure than anything else i saw because normal
people don't see it. Also what Johanna said about GUI, i know nothing
about gui how does it work why it is secure...
my idea was opening a keylogger (because people know and understand
them) on windows and on qubes and show the differences. i proposed a
transcript but received no answer, so i try again by re-posting the
link; if noone answer again i don't know... maybe i will make it anyway
and present it.
https://groups.google.com/d/msg/qubes-devel/I4lTrYnxKHE/CXxGVIouBwAJ
About GPU virtualization/emulation bugs:
virtual box guest-to-host escape because of Open-GL bug, if i remember
right from an analisys i read it doesn't even crash the program so you
won't suspect anything
https://packetstormsecurity.com/files/127865/virtual_box_opengl_escape.rb.txt
Eric Shelton
Jan 19, 2017, 12:12:49 AM1/19/17
to qubes-devel, knock...@gmail.com, joa...@invisiblethingslab.com
On Monday, January 9, 2017 at 11:02:41 AM UTC-5, Joanna Rutkowska wrote:
...
Now, having the above in mind, lets consider your proposition of potentially
merging with OpenXT, or to use OpenXT as a basis for Qubes OS:
First, I don't think the fact they decided to use the D-Bus protocol as a
universal solution for their inter-VM communication should be considered as
disqualifying reason for us reusing their code. Instead, we should treat it as a
red light for the security competence of the team that have created their
architecture. A Big Red Light with a Roaring Fire Siren, to be more precise.
I suppose the extent to which such issues would be a liability for OpenXT (or certain parts of it) serving as a base for Qubes depends on what is being used, how it is incorporated into Qubes, and whether trust can be invested in the parts that are used. For example, like Qubes, OpenXT maintains a number of patches applied against upstream Xen. Perhaps Qubes might use the same hypervisor, if those patches were considered auditable. Additionally, as a downstream project, Qubes could pick and choose from the OpenXT patches and apply additional ones, if needed.
Indeed, OpenXT, previously known as XenClient XT, has a history of what I see as
bad security architecture decisions. Besides the above-criticized choice of
D-Bus as a fundamental building block for any VM-spanning services, another
striking example is the adoption of GPU pass-through as their primary graphics
virtualization technology.
While I'm sure that the word "pass-through" brings very positive connotations to
most readers ("something that doesn't require much handling from the hypervisor
side"), and also looks great on powerpoint diagrams ("arrows indifferently
crossing the hypervisor box"), in reality GPU pass-through require lots of
complexity on the hypervisor side, which the slides often forget to mention.
E.g. what happens when the real GPU is taken away from the VM? Are we going to
give it some *emulation* for the time being to keep it happy? If so, what if
there is a bug in the emulation code (there surely is, 'cause GPU's are
complex)? What privileges would the VM obtain after exploiting the GPU emulation
code?
On more recent Intel GPUs the situation has presumably improved, but the fact XT
has decided to chose this insecure form of graphics virtualization over the much
more secure approach that we designed for Qubes OS, so many years ago, is
another fact that makes me very uneasy about the security expertise of the XT
team.
Although the above discussion and criticisms may have been correct for OpenXT's predecessor, XenClient XT, my understanding is that approach was abandoned by OpenXT. Instead, they appear to run guests under QEMU, and make use of its emulated VGA adapter, much as done for an HVM appvm under Qubes (ignoring Windows Tools).
These slides discuss where the project is headed with display and input management:
As illustrated in slides 19-21 of that deck, a guest's use of a monitor is all or nothing, apparently requiring switching from one "virtual console" to another. However, although that sort of user interface experience is very different from Qubes, that is not important for the idea of incorporating OpenXT components as a base for Qubes. Instead, what is more relevant is whether their approach for input and display for HVM domains is something that can reasonably be plugged into the Qubes GUI framework for running an HVM domain. If so, their stubdomain approach might be borrowed with little modification. Qubes, even making use of OpenXT components, could still start up appvms, start applications within them, and provide input and display in pretty much the same way it does currently.
(Because of this architecture decision, BTW, what XT people call "UIVM", is not
the same as what we call "GUI domain", and which we plan to introduce some time
after 4.0 release. In XT architecture, UIVM is a natural consequence of the GPU
pass-through model, and is merely a label assigned to one-of-the-many VMs which
can get full pass-through access to the GPU. In Qubes, the GUI domain is planed
to be *the only one* VM granted pass-through access to the GPU, while we would
continue to be using our -- I believe -- very secure GUI virtualization protocol
to allow other VMs to access graphics.)
The UIVM is likely more than just a label. It suggests OpenXT has already done some degree of disaggregating dom0 from the domain that manages the display, and demonstrates an underlying use and division of domains that is compatible with the Qubes approach.
Last but not least, probably the most fundamental difference between XT and
Qubes (and partly also responsible for some architectural differences) is in the
philosophy. XT attempts to implement the classic
multiple-isolated-(virtual)-computers-inside-one-box model (aka Multi Level
Security, or MLS), which originates from the military world. Qubes, on the other
hand, attempts to provide one secure desktop platform, which uses multiple
isolated VMs to securely decompose various tasks.
I've never quite got convinced by this MLS approach, to be honest. After all,
what's the point of running a few monolithic Windows VMs, if each of them is
still... a buggy, monolithic VM which can get compromised just as easily? Of
course, the official ideology of MLS says that even if an attacker compromises
one of the VMs, then they would not be able to at least leak the data to other,
unclassified networks.[*] In any case, the MLS proponents believe that these
systems could be useful to e.g. stop an insider from stealing gigabytes of
confidential info and leaking them out somewhere. Looks like this might not have
quite worked out for some of the organizations who are well known to promote
this security model (and incidentally also XT and SELinux)... ;)
Contract this to the Qubes approach, where we would like to utilize multiple
domains in order to actually make the domains more difficult to attack. Whether
by making them "disposable", or by splitting the tasks between more than one VM
(and potentially having some of these other VMs as disposable, or easily
recoverable).[**]
I'm not convinced that a Qubes built on top of or using OpenXT components requires buying in to the MLS approach. You can use the same hypervisor and other components - just use them differently.
What could be beneficial for Qubes is not so much building it "on top of OpenXT" or adopting the OpenXT architecture as it is identifying software components might be packaged up in a way that they could be effectively shared and maintained across both projects. Stepping back and looking at the two projects from a 30,000 foot view, there are a fair number of things in common that might somehow be packaged up for common code across the projects. Things such as:
- measured boot (including extending measured boot to VMs)
- improvements/fixes to upstream Xen (OpenXT is exploring disabling unnecessary Xen features - https://openxt.atlassian.net/browse/OXT-839 - something that I think the Qubes dev team would also like to be doing with Xen to remove bells and whistles like nested HVM)
- patches to the Linux kernel (keeping up with new kernels)
- "modern" stubdomains (based on upstream QEMU, rather than QEMU 1.2 or so)
- PCI passthrough (their users want this feature too, including GPU passthrough)
- solid Windows guest support (a fair number of users need/want this. Qubes still has no solution for sound in Windows, but OpenXT does)
- a build framework (The Qubes build system is "unique." Will it be the desired long term approach?)
All the above complains about XT focus on the architecture. And I'd argue that
for a project like Qubes the proper architecture is something that should always
come first. Or to stay it differently: even the best code cannot be "rescued" if
it implements a bad architecture.
But perhaps they could produce some good code that we could consume
nevertheless? Well, maybe yes. Or maybe not. Somebody would need to take a look.
Likely we could utilize some non-critical pieces. But surely the idea of using
OpenXT as a *basis* for Qubes OS is not a good idea.
As I discussed above, I think the number and "criticality" of reasonably shared components could be much greater without compromising Qubes' security. Qubes and OpenXT have a number of similar features that might be implemented using common code.
I also feel obliged to protest slightly against singling out any single
individual as a central force which keeps the project going. First, this makes
disservice to many others (many of which enlisted on our always-lagging-behind
Team page, but some not recognized even there) who also have contributed to the
project. Sometimes under-recognized when looking at the git statistics.
Consider e.g. Rafał Wojtczuk, who implemented most of the GUI virtualization in
a way that it has not needed any modifications over the last 7 years... Yet,
many might be tempted to overlook this when counting commits. This is also
thanks to a reasonably good architecture of the GUI virtualization, I think,
another aspect that almost never shows itself in the log.
Also, we shall not forget about the mysterious forces in the universe who -- for
all these years -- have paid the salaries of the core team. It's quite likely
Qubes would have ceased to exist long ago, if not for these magical fairies ;)
In other words, Qubes should not be equated with any single person.
I 100% acknowledge that I am looking at which people have been active lately; they are the ones likely to make things happen in 2017, 2018, and beyond. I wonder where will the project go from here, and how? It was by no means intended to diminish past efforts and contributions such as Rafał's.
From that perspective, it has been mostly the efforts of a single developer, however that person's work was funded or inspired, that has been keeping Qubes going the last couple of years. Did the #2 Qubes contributor in 2016 do even 10% as much as Marek (by whatever metric you consider sensible: number of commits, lines of code, etc)? Somehow Marek keeps up with the relentless rollout of new versions of Fedora and other dependencies that inevitably break various aspects of Qubes, answers all manners of user questions, debugs or helps debug obscure and difficult issues arising from running a desktop OS on top of Xen, fixes things that "should" but don't quite work in Xen, makes sure Qubes keeps up with XSAs, and on top of that makes sure the project advances forward towards ver 4. So, although thanks to Rafał and others Qubes came into being years ago, I don't think it was inappropriate for me to say that "without Marek, I don't think there would be a Qubes today" - with "today" meaning just that: 2016-2017. Marek is the one putting fingers to keyboard and both realizing and maintaining Qubes in the present.
Speaking of good developers, I'd like to also share an observation that very
good and bright developers are often a source of troubles in security, and that
we should all, both developers and non-developers (e.g. researchers), be very
[wary] of that all the time.
This (surely controversial) paradox might be a result of the fact that being
developer -- in essence -- is about learning how to master complexity. Modern
languages and frameworks allows one, and in fact even encourage, to implement
ever more complex creations, almost always hiding all the complexity in the
shadow. (For example: to use D-Bus for communication between
differently-privileged domains[***])
But the complexity is still there. At the end of the day, whatever software we
write, it will finally get translated to the machine code and execute as such.
But enough bashing of developers :) Without developers (or devops) Qubes would
not have existed. Nor it would make sense without all the offensive security
expertise that have been invested into it, especially in the early days. And
this brings us back to where we started, i.e. the unique value of the project :)
I don't think any of us see Qubes as suffering from this. Also, Qubes gives us a great tool that allows us to contain the risk or damage resulting from imperfect software.
Back in 2009-2010 (the days of the Qubes architecture spec), it was reasonable to conclude that Qubes would have to do a lot of things "from scratch" - Qubes was doing things that no other open source project had done. However, now there are others that have been, and are trying to, solve similar problems, and implementing similar solutions. I think the Qubes team could competently identify parts of the OpenXT project that could become common software components across the two projects. I don't think Qubes has to adopt the OpenXT architecture (whatever that may be beyond the D-Bus inter-vm comm) to get the benefits of using OpenXT code. I think there is a reasonable prospect of reducing the amount of code unique to, and accordingly having to be maintained solely by, Qubes, and at the same time maybe picking up some features that the Qubes team does not have the resources to pursue on its own.
I'll also note that this is a very Qubes-centered pitch. OpenXT has things to gain as well from doing this. For one, disaggregation of USB out of dom0 into a USBVM is a problem that has been solved by Qubes.
Best,
Eric
Marek Marczykowski-Górecki
Jan 20, 2017, 7:18:36 AM1/20/17
to Eric Shelton, qubes-devel, joa...@invisiblethingslab.com
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Hash: SHA256
On Wed, Jan 18, 2017 at 09:12:49PM -0800, Eric Shelton wrote:
> On Monday, January 9, 2017 at 11:02:41 AM UTC-5, Joanna Rutkowska wrote:
> >
> > ...
> >
> > Now, having the above in mind, lets consider your proposition of
> > potentially
> > merging with OpenXT, or to use OpenXT as a basis for Qubes OS:
> >
> > First, I don't think the fact they decided to use the D-Bus protocol as a
> > universal solution for their inter-VM communication should be considered
> > as
> > disqualifying reason for us reusing their code. Instead, we should treat
> > it as a
> > red light for the security competence of the team that have created their
> > architecture. A Big Red Light with a Roaring Fire Siren, to be more
> > precise.
> >
>
> I suppose the extent to which such issues would be a liability for OpenXT
> (or certain parts of it) serving as a base for Qubes depends on what is
> being used, how it is incorporated into Qubes, and whether trust can be
> invested in the parts that are used. For example, like Qubes, OpenXT
> maintains a number of patches applied against upstream Xen. Perhaps Qubes
> might use the same hypervisor, if those patches were considered auditable.
> Additionally, as a downstream project, Qubes could pick and choose from
> the OpenXT patches and apply additional ones, if needed.
The reason pointed out by Joanna is exactly why we shouldn't blindly
> On Monday, January 9, 2017 at 11:02:41 AM UTC-5, Joanna Rutkowska wrote:
> >
> > ...
> >
> > Now, having the above in mind, lets consider your proposition of
> > potentially
> > merging with OpenXT, or to use OpenXT as a basis for Qubes OS:
> >
> > First, I don't think the fact they decided to use the D-Bus protocol as a
> > universal solution for their inter-VM communication should be considered
> > as
> > disqualifying reason for us reusing their code. Instead, we should treat
> > it as a
> > red light for the security competence of the team that have created their
> > architecture. A Big Red Light with a Roaring Fire Siren, to be more
> > precise.
> >
>
> I suppose the extent to which such issues would be a liability for OpenXT
> (or certain parts of it) serving as a base for Qubes depends on what is
> being used, how it is incorporated into Qubes, and whether trust can be
> invested in the parts that are used. For example, like Qubes, OpenXT
> maintains a number of patches applied against upstream Xen. Perhaps Qubes
> might use the same hypervisor, if those patches were considered auditable.
> Additionally, as a downstream project, Qubes could pick and choose from
> the OpenXT patches and apply additional ones, if needed.
import components from this project, even if they've solved a problem we
have. Instead if doing so, we should carefully analyse how they've
designed it and implemented and only then consider basing on it.
(...)
better implementations, for example AEM2 Turbo Edition [1] or Heads[2].
[1] http://mjg59.dreamwidth.org/35742.html
[2] https://github.com/osresearch/heads
> - improvements/fixes to upstream Xen (OpenXT is exploring disabling
> unnecessary Xen features - https://openxt.atlassian.net/browse/OXT-839 -
> something that I think the Qubes dev team would also like to be doing with
> Xen to remove bells and whistles like nested HVM)
> - patches to the Linux kernel (keeping up with new kernels)
both projects I try to send as much as possible to upstream. Especially
for new patches. There is a backlog of Xen patches which are currently
quite Qubes-specific (and purposely breaking non-Qubes use cases), so
those needs some work to be more generic. But we have some progress
here. I believe OpenXT have very similar approach, so in this area in
practice we do share work.
> - "modern" stubdomains (based on upstream QEMU, rather than QEMU 1.2 or so)
finished. Similar to above - there are some aspects of OpenXT we can
reuse and we do. But there are also significant differences (transport
channel for qmp, different gui-agent, and some more), which makes it
impossible (and undesirable) to reuse as is.
> - PCI passthrough (their users want this feature too, including GPU
> passthrough)
> - solid Windows guest support (a fair number of users need/want this.
> Qubes still has no solution for sound in Windows, but OpenXT does)
that OpenXT integration level is much much lower (very few inter-VM
services etc). So "solid support for Windows" really means "it's
possible to boot Windows". Which is also possible on Qubes OS.
As for sound support, it's mostly about having sound proxy in
stubdomain. Which indeed is something we could try to import. The Qemu
patch is very OpenXT-specific, but maybe some parts could be reused.
> - a build framework (The Qubes build system is "unique." Will it be the
> desired long term approach?)
actual package build is very similar to what particular upstream project
use (dpkg-buildpackage, rpmbuild, etc). It is possible to build Qubes
packages using just those tools - assuming you've setup build
environment (installed build dependencies etc). Each distribution have
also some higher level tool to take care of it (mock for Fedora, sbuild
for Debian etc). And actually we're in the process of using those tools
to install build dependencies, instead of our own scripts.
But the main purpose of qubes-builder still stands - download _and
verify_ the code. Something that most of similar tools are doing
partially (for example checking signature/hash only if some is
configured - and failing only when it's invalid, but not when it's
missing).
Actually, I have checked OpenEmbedded (used by OpenXT) in the past, and
concluded it would be tricky to make sure that the build framework
itself would not do careless "curl | bash" equivalent (disguised as
"wget; tar; cd; make"). And also OpenEmbedded is targeting, as the name
suggests, embedded platforms, where the artifact is a whole system
(firmware) image. Not individual packages. Something that could work for
stubdomain for example but probably none of other Qubes parts. And in
fact there are very few projects able to work with different package
types and at the same time handle multiple source components (with
reasonable security).
[3] https://www.qubes-os.org/news/2016/05/30/build-security/
(...)
- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
iQEcBAEBCAAGBQJYggAWAAoJENuP0xzK19cs41IH/j4JUNkFs0VWPL8SoEtapXL8
p8qGc1tPOArolPT9yV+yabker4Ul7SCepri6ExOeQvuip2xaEbf6aWhc2Qmz2ybF
5Mg/N5ToS41Tt3L2bXSbU0wW3Cf3DzkhNhRly/v5SJkufVXnUpNQkIcqiD+hN1tM
tQTxprc/C/e7QYol4AIBoSL1mw5FvFU9ebxb+DlDwsxkGiqc+g7Q8C0Ug6VUgr8J
67VSn4KI5rJcKIUwZxBe4/YofW6nZZ8zqJ6/oynAn9fXO8cCI9cpfTRKHjq+SYJH
YV5HjLlGDtKvVXcw+K6VFHkqMBhEHAU+0jzSmXQCuRcodMKiDLsUvPEstAz72jA=
=ND0v
-----END PGP SIGNATURE-----
rian...@gmail.com
Mar 9, 2017, 11:43:49 AM3/9/17
to qubes-devel
Actually... I don't think the Qubes goals and the OpenXT goals are all that dissimilar. As already stated, OpenXT used to be XenClient XT, and the open source community basically inherited a bunch of code from Citrix that really needed a LOT of work. The use of DBus for example is something everyone wants to get rid of (and has been brought up a lot). I would not "blindly" accept any code from anyone (as previously mentioned), but there are a lot of things that OpenXT does that could be done better in Xen, the QEMU stubdomain for example. Joanna's summary of the graphics layer is out-of-date as well (the link from Xen summit that Brendan presented is a good reference), and to address the above comment, OpenXT's main use case are Windows guest VMs, so there is a fair amount of work to support Windows really well.
Seems to me there is probably a things both teams to leverage / learn from.
- Rian
hpw...@gmail.com
Mar 9, 2017, 12:29:20 PM3/9/17
to qubes-devel
I just found this thread while googling for some stuff and figured I would chime in. I have been involved with OpenXT since it was open sourced by Citrix, but I am not a developer (I did forward this thread on to some of the OpenXT developers, so they may chime in as well).
I do not want to comment on what is better or worse in OpenXT or Qubes, that is for people much smarter than me to argue, but I would like to say that OpenXT and Qubes are BOTH client-side virtualization systems based on Xen. Yes, they have different use cases (MLS vs. a secure single user experience), but it sure seems like there should be a lot of commonality, experience and brainpower that could be leveraged between the two projects. I think it is safe to say that neither project has infinite resources for development, so doesn't it make sense to pool the finite resources we have?
I am not saying that Qubes should be based on OpenXT or anything like that, but if there is one feature, capability or module that can be shared why not?
Again the smarter people would need to work the details, but I am sure the OpenXT community would welcome constructive input, collaboration and contribution from the the Qubes community, and it seems visa versa as well.
Phil
I do not want to comment on what is better or worse in OpenXT or Qubes, that is for people much smarter than me to argue, but I would like to say that OpenXT and Qubes are BOTH client-side virtualization systems based on Xen. Yes, they have different use cases (MLS vs. a secure single user experience), but it sure seems like there should be a lot of commonality, experience and brainpower that could be leveraged between the two projects. I think it is safe to say that neither project has infinite resources for development, so doesn't it make sense to pool the finite resources we have?
I am not saying that Qubes should be based on OpenXT or anything like that, but if there is one feature, capability or module that can be shared why not?
Again the smarter people would need to work the details, but I am sure the OpenXT community would welcome constructive input, collaboration and contribution from the the Qubes community, and it seems visa versa as well.
Phil
Reply all
Reply to author
Forward
0 new messages