Difference between PVH and PVHVM

[Blacklight447]

In https://github.com/QubesOS/qubes-issues/issues/2185 it is mentioned that Qubes will be using PVHVM instead of PVH until the latter is completed. Anyway I can't seem to find what the difference in security is when searching the two of them up on the web. Could someone explain to me what the impact of chosing PVHVM for now is and what the difference in security is with PVH (if there is any), or point me in the right direction where I can find this information?

Friendly greetings,
Blacklight

[Jean-Philippe Ouellet]

Marmarek or HW42 could probably give you better answers, but the
following is my understanding:

The terminology is admittedly somewhat confusing, especially since Xen
people no longer talk about a discrete set of virt modes but it's now
thought of as more of a "spectrum".

Right now (R4-rc3) we are using a mode where memory management is
handled by hardware (SLAT), but QEMU is still involved in domain init
and provides device models for VMs which don't use PV drivers. The
goal in the future is to eliminate QEMU entirely, but this requires
kernel support which AFAIK deemed not mature enough the last time it
was evaluated for use in Qubes. Various names have been used for this
(and similar) virt mode at different points in time:
PVH/PVHv2/HVMlite/etc. You can find more info on the Xen wiki and in
various Xen developer summit presentation slides if you're so
inclined.

The benefits to removing QEMU entirely are:
1) reduced attack surface (both because you can't exploit qemu to
escalate privileges within the domain (relevant for VMs without
passwordless sudo), as well as eliminating the PV hypervisor interface
exposed to the *-dm domains)
2) decreased per-vm memory footprint (right now each running domain
requires an additional ~140mb mem for its corresponding *-dm domain)
3) lower CPU overhead (right now each *-dm domain takes ~10-15% CPU,
see #2849 [1], but even after fixing that there would still be some
overhead)

Regards,
Jean-Philippe

[1]: https://github.com/QubesOS/qubes-issues/issues/2849

[Vít Šesták]

Just few notes:

I believe that getting rid of QEMU is rather getting rid of PV domains than getting rid of QEMU itself.

* First, privilege elevation is not much a threat in Qubes. OTOH, VM escape is a fatal threat.
* I believe QEMU vulnerabilities typically require some lowlevel access to devices, so those vulnerabilities typically don't allow you to elevate privileges within VM.

The difference for most domains: In Q3.2, attacker needs a suitable vulnerability for PV domains implementation. In Q4.0, attacker either needs suitable vulnerabilities for both PV domains implementation and QEMU or a suitable vulnerability for hardware-virtualized domains implementation. In Q4.1, attacker will hopefully need a vulnerability in hardware-virtualized domains implementation. (For sake of simplicity, I consider shared parts as non-vulnerable, so I don't count potential vulnerabilities in GUI daemon, RPC services, hardware etc. I compare just the virtualization itself.)

This might not be clear what is more secure. The reason behind the change is that HVM implementation is much less complex (and thus much less error-prone) than PV implementation. As a result, QubesOS 4.1 (or whatever version that brings PVH) is expected to be more secure that Qubes 3.2 and Qubes 4.0. The situation in Qubes 4.0 is a bit controversial, but if hardware-virtualization-specific vulnerabilities are rare (and I believe they are), it should be also an improvement (compared to Q3.2).

Note that we cannot easily get rid of QEMU completely because of OSes like Windows, ReactOS or other fully-virtualized OSes.

This brings a question: Can we get rid of PVs for stubdomains, for example by using PVH for stubdomain? I believe this would be consistent with the attitude of removing PV. I am not sure how hard/easy it is, though.

Regards,
Vít Šesták 'v6ak'

[Holger Levsen]

is this a good enough write up to push this into qubes-doc.git so that
it doesnt get lost? :-)

+thanks for explaining, Jean-Philippe!


--
cheers,
Holger

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Sat, Dec 16, 2017 at 02:10:17AM -0800, Vít Šesták wrote:
> Just few notes:
>
> I believe that getting rid of QEMU is rather getting rid of PV domains than getting rid of QEMU itself.

Yes and no. From security POV this is correct. But at the same time,
having qemu (with appropriate isolation) use resources (RAM, CPU), which
already are scarce on Qubes. So, we'd like to not have qemu there, where
possible.

> * First, privilege elevation is not much a threat in Qubes. OTOH, VM escape is a fatal threat.
> * I believe QEMU vulnerabilities typically require some lowlevel access to devices, so those vulnerabilities typically don't allow you to elevate privileges within VM.
>
> The difference for most domains: In Q3.2, attacker needs a suitable vulnerability for PV domains implementation. In Q4.0, attacker either needs suitable vulnerabilities for both PV domains implementation and QEMU or a suitable vulnerability for hardware-virtualized domains implementation. In Q4.1, attacker will hopefully need a vulnerability in hardware-virtualized domains implementation. (For sake of simplicity, I consider shared parts as non-vulnerable, so I don't count potential vulnerabilities in GUI daemon, RPC services, hardware etc. I compare just the virtualization itself.)

Yes.

> This might not be clear what is more secure. The reason behind the change is that HVM implementation is much less complex (and thus much less error-prone) than PV implementation. As a result, QubesOS 4.1 (or whatever version that brings PVH) is expected to be more secure that Qubes 3.2 and Qubes 4.0. The situation in Qubes 4.0 is a bit controversial, but if hardware-virtualization-specific vulnerabilities are rare (and I believe they are), it should be also an improvement (compared to Q3.2).

As for PVHv2 - in theory it should be available in 4.0 already, if you
have VM kernel new enough (4.11+).

> Note that we cannot easily get rid of QEMU completely because of OSes like Windows, ReactOS or other fully-virtualized OSes.
>
> This brings a question: Can we get rid of PVs for stubdomains, for example by using PVH for stubdomain? I believe this would be consistent with the attitude of removing PV. I am not sure how hard/easy it is, though.

I'm not sure either, but this is definitely something that we'll look
into. As soon as we get stable PVHv2. Right now Xen do not support PVHv2
as stubdomain. Also, Xen do not support PVHv2 with PCI passthrough. At
least not yet.

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJaL3/PAAoJENuP0xzK19csomoH/0UFMApKJcB45/HDiiejZg4J
i6Ux0ErCWF83fWcuqZc+ZOm2Y/9YThnpvAPfX98JzQKcwKilxK/dP/DbntYPwtJz
+UDRq+uBP1PdGHijSkq1fxijq0sxzT5MDZOYJEVyqiFC1M6oS74gVU35hxt289PD
CcRvmv7dmWQs25sWaK4yWCAVibGJFSdUro4jAicRrA5/mRhQxbC7Lo9ObUEGo2vc
5M2HWB3Y8w8bU74OyY6dQ/QKY4A2Ol+aFoDwad55fRn9kQggLvjWkRRZ5TAMWVgV
rs0JsUUxIC/y0t4BnQxy0p4rYr/F239W1IH8CyMkG0ePyGgU9k686F7EDYM6h5w=
=52Bl
-----END PGP SIGNATURE-----

[Vít Šesták]

Hello,

> > I believe that getting rid of QEMU is rather getting rid of PV domains than getting rid of QEMU itself.
>
> Yes and no. From security POV this is correct. But at the same time,
> having qemu (with appropriate isolation) use resources (RAM, CPU), which
> already are scarce on Qubes. So, we'd like to not have qemu there, where
> possible.

You are right, I was focused on security, not on resource consumption.

> As for PVHv2 - in theory it should be available in 4.0 already, if you
> have VM kernel new enough (4.11+).

Good to know. I guess that when I have a suitable kernel, I also need to configure something to switch the mode.

Just an idea: It could be useful to provide bleeding-edge PVHv2 templates for those who want to test it in Q4.0.

> I'm not sure either, but this is definitely something that we'll look
> into. As soon as we get stable PVHv2. Right now Xen do not support PVHv2
> as stubdomain. Also, Xen do not support PVHv2 with PCI passthrough. At
> least not yet.

Hmm, hmm.

Regards,
Vít Šesták 'v6ak'

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Sat, Dec 16, 2017 at 04:58:20AM -0800, Vít Šesták wrote:
> > As for PVHv2 - in theory it should be available in 4.0 already, if you
> > have VM kernel new enough (4.11+).
>
> Good to know. I guess that when I have a suitable kernel, I also need to configure something to switch the mode.
>
> Just an idea: It could be useful to provide bleeding-edge PVHv2 templates for those who want to test it in Q4.0.

You don't need separate template, just switch 'virt_mode' property to
'pvh' (and set sufficiently new kernel).

> > I'm not sure either, but this is definitely something that we'll look
> > into. As soon as we get stable PVHv2. Right now Xen do not support PVHv2
> > as stubdomain. Also, Xen do not support PVHv2 with PCI passthrough. At
> > least not yet.
>
> Hmm, hmm.
>
> Regards,
> Vít Šesták 'v6ak'
>

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJaMFfxAAoJENuP0xzK19cs960H/0WKeXHwSifENsA1OShnEe88
67PvFdmGO1GzFTQtBLCvbfnVFlYymu++J09kM/OhRhMV+6HTdnE7CnbYSsTWG9FG
IEryGQcOwcNs2pPlwtfgdGADgDPxXZyFel0mRzo/Dc0rV1XiKnwuUe2Ln2MnmOkw
iOcgDCmAR5xIQmhuTxE6Jt3VLpE12rK3ieIv7XYJEaUwZFli7HAhp9R+0fN0NnDJ
aVZ+wSnPK3g6sIZrCM9e1+HV8sO90wNxHSnAPdUs7OIpbPp4HTXnxELvy3iRBYL7
evlFp10OWyP9aVQOzDA8T5P/I0Vr31nYyi2NoWHrzjy9aYvDz+GjPfx+VZEbFd8=
=OaaW
-----END PGP SIGNATURE-----

[Vít Šesták]

Cool, this can allow us to test PVH without switching to Q4.1.