Qubes 4.1 - pci passthrough problems for sys-net

[Thorsten Schierer]

Hi, I installed the latest test version of Qubes 4.1, which worked just fine.

Since there are some issues on my test machine, I always have to use pv for vm's in Qubes 4.0.

(vt-d is enabled but doesn't seem to work correctly, but since it's only my test/development machine it doesn't really matter)

Now in Qubes 4.1, sys-net fails to start with the message "libxl__domain_config_setdefault: passthrough not supported on this platform"

So my question at this point is: Is PV still working in Qubes 4.1 and if so, is this a known issue at the moment?

Anything I can do to fix this problem in 4.1?

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Yes, PV still works in Qubes 4.1 (we will phase it out some day, but not
yet). No, systems without working IOMMU are not supported anymore at all
(even with PV hack).
I know it may be inconvenient for test setups (in fact it also affects
some of our development setup too), but we won't support systems without
(or with broken) IOMMU in official builds anymore. This is mainly part
of ensuring that "I run Qubes OS" is a statement meaning some baseline
of system security, and having sys-net/sys-usb with unrestricted access
to the whole host physical memory falls outside of this baseline.

What you can probably do for the test setup, is to build Xen with this
check commented out. I haven't tried it, but it should be possible.

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAl7oHP4ACgkQ24/THMrX
1yy54Qf/aSsNJTfUN0j6qLV/DqvXDiaU2xzWBtFtppFyEr808imxt5eYeODQAvwj
p7Yj8YmshhtUjxx7DsavOlQSsD/5eVsDZQJvoEDp8/Kzfnrsiwdv5fB6a9Xq9BnO
4TXZdo4bNkiRsSnyVsOa1EWvi7G8o2kDxdCvGXflT0x7C9BZLS48ysDlXs1Gff/t
8sCbrmlSTac1iU+AiQQA5dXn2MUHcZON5fsYguaowD6e4PHywhmulAxpsxa0j3w5
yFzBJkxZJNUNAds0gORtqPPdmJYm8Rqe4H3Z0NKcBxNhMLbRGVGOyCXnWWaCYI8Y
0TxOpIi5xXCJfwnD7ewPSJQidm2pSw==
=CgZH
-----END PGP SIGNATURE-----

[Thorsten Schierer]

> No, systems without working IOMMU are not supported anymore at all
> (even with PV hack).
> I know it may be inconvenient for test setups (in fact it also affects
> some of our development setup too), but we won't support systems without
> (or with broken) IOMMU in official builds anymore. This is mainly part
> of ensuring that "I run Qubes OS" is a statement meaning some baseline
> of system security, and having sys-net/sys-usb with unrestricted access
> to the whole host physical memory falls outside of this baseline.

Thanks for clarification.
It's actually really a shame that it should no longer be supported at all, as it makes my development process a lot harder in the future.

As for official builds, I understand that it's a security issue, but even then (imo) it's still better to use Qubes OS with PV than using a regular OS, depending on the use case.
While you might be vulnerable to memory attacks in sys-net (which is also a problem in a normal OS), you'd still gain benefits from compartmentalization - e.g. protecting your separated data from spying software vendors or virus infections, webpage profiling/fingerprinting, etc.
Wouldn't it make more sense to keep supporting it as a less secure operation mode in Qubes OS and rather display a warning in the installer (as it already does) and if necessary also display a notification (or maybe even a permanent watermark) in the OS itself, which states clearly that it only runs in a less secure mode?
Users with full hardware support would get the full security level while incompatible PCs (like my test machine) would still run it in a less secure mode, which I really don't mind at all in this case and more importantly, which is still more secure than any other OS?

Since Qubes OS is open source and everyone can modify it to their needs, "I run Qubes OS" is not really an unambiguously statement.
I mean, you said it yourself, if someone modifies xen, the feature would be back... while still running Qubes OS.
And even without modifying the source code itself, e.g. if I modify my Qubes OS to allow direct networking in dom0, does it still meet that baseline of system security you were talking about?

I mean yes, Qubes OS indicates a certain level of security in general and that's absolutely correct. But since it's also highly customizable (which is great and it should stay that way!), one could easily void that security by modifying it that way.
So for me it's not a valid reason to no longer support broken IOMMU with PV mode and I think it should be up to the user to decide whether the security risk is acceptable or not.
What's the alternative for people who can't afford to get new hardware, use Windows or Linux? Are they more secure than Qubes OS with sys-net in pv mode?
In some years this might look different, as older hardware (with this problem) will get extinct, but currently this should still be a thing.

> What you can probably do for the test setup, is to build Xen with this
> check commented out. I haven't tried it, but it should be possible.

I might give this a go, but that sounds pretty annoying with so many new builds going on.

Instead of removing support at all, maybe we could add some boot option that allows xen to work with broken/missing IOMMU with PV?

It would be similar to compiling your own xen, as you have to actively enable this feature, but without the hastle of having to recompile and install it every time over and over again. At least for the time being, until PV will eventually be removed.

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Tue, Jun 16, 2020 at 02:49:32PM -0700, Thorsten Schierer wrote:
> > No, systems without working IOMMU are not supported anymore at all
> > (even with PV hack).
> > I know it may be inconvenient for test setups (in fact it also affects
> > some of our development setup too), but we won't support systems without
> > (or with broken) IOMMU in official builds anymore. This is mainly part
> > of ensuring that "I run Qubes OS" is a statement meaning some baseline
> > of system security, and having sys-net/sys-usb with unrestricted access
> > to the whole host physical memory falls outside of this baseline.
>
> Thanks for clarification.
> It's actually really a shame that it should no longer be supported at all,
> as it makes my development process a lot harder in the future.
>
> As for official builds, I understand that it's a security issue, but even
> then (imo) it's still better to use Qubes OS with PV than using a regular
> OS, depending on the use case.
> While you might be vulnerable to memory attacks in sys-net (which is also a
> problem in a normal OS), you'd still gain benefits from
> compartmentalization - e.g. protecting your separated data from spying
> software vendors or virus infections, webpage profiling/fingerprinting, etc.
> Wouldn't it make more sense to keep supporting it as a less secure
> operation mode in Qubes OS and rather display a warning in the installer
> (as it already does) and if necessary also display a notification (or maybe
> even a permanent watermark) in the OS itself, which states clearly that it
> only runs in a less secure mode?

But you do realize that "keep supporting" here actually means
introducing a patch that intentionally breaks a security feature of
an upstream Xen?

As for a clear message - currently installer does say pretty clearly,
that the system without IOMMU is not going to work. And yet, many people
choose to ignore this explicit warning and then complain it doesn't
work.

Anyway, a permanent watermark idea could actually work. Similar thing is
also implemented in Windows - you get a permanent marking on the
wallpaper in all kind of debug builds or debug configurations.
See more at the end of the message.

And BTW we do plan to remove PV support at some point too, to reduce
hypervisor attack surface. It is already possible to build upcoming Xen
4.14 without PV code included, but there are still some dependencies on
PV to be solved.

> Users with full hardware support would get the full security level while
> incompatible PCs (like my test machine) would still run it in a less secure
> mode, which I really don't mind at all in this case and more importantly,
> which is still more secure than any other OS?
>
> Since Qubes OS is open source and everyone can modify it to their needs, "I
> run Qubes OS" is not really an unambiguously statement.
> I mean, you said it yourself, if someone modifies xen, the feature would be
> back... while still running Qubes OS.
> And even without modifying the source code itself, e.g. if I modify my
> Qubes OS to allow direct networking in dom0, does it still meet that
> baseline of system security you were talking about?

It's funny you mention this as an example: we did have
qvm-dom0-network-via-netvm tool some years ago, and removed it exactly
to make such modifications harder.

> I mean yes, Qubes OS indicates a certain level of security in general and
> that's absolutely correct. But since it's also highly customizable (which
> is great and it should stay that way!), one could easily void that security
> by modifying it that way.

This is actually a thing that is also slowly changing. Greatly
customizable system is good for power users who indeed do understand
consequences of various configuration changes. But as a general
direction, as the system matures, we try to reduce required knowledge to
use Qubes, while still maintain strong security properties. This
basically means making harder to shoot oneself in the foot.

The work on GUI domain which isolates user interface from all-powerful
dom0 also is a step in this direction.

Another thing is, supporting many different configurations is difficult.
Our priority is trustworthiness of the system, flexibility comes only
later. Phasing out PV, while primarily done for security reasons, also
reduces complexity of supported configurations.

> So for me it's not a valid reason to no longer support broken IOMMU with PV
> mode and I think it should be up to the user to decide whether the security
> risk is acceptable or not.

The problem with this approach is such decisions in many cases have
quite severe consequences that may not be obvious at first sight. In
case of IOMMU-less sys-net, this for example means you are really
exposed when connecting to various wifi networks in random coffee shops.
And also you are no longer protected from some random USB stick you've
found on a parking lot.

> What's the alternative for people who can't afford to get new hardware, use
> Windows or Linux? Are they more secure than Qubes OS with sys-net in pv
> mode?
> In some years this might look different, as older hardware (with this
> problem) will get extinct, but currently this should still be a thing.

This is less and less about "older hardware" already. IOMMU is available
on the market since over 15 years. This is more about broken
implementations - and those probably will always be there.
Microsoft is increasing its requirements related to security features to
allow hardware to be named "Windows compatible", which generally does
have positive effect on availability of those features on consumer
hardware. Maybe one day they will also require IOMMU there. Or maybe one
day Qubes OS will have enough market share to dictate such requirements
;)

> > What you can probably do for the test setup, is to build Xen with this
> > check commented out. I haven't tried it, but it should be possible.
>
> I might give this a go, but that sounds pretty annoying with so many new
> builds going on.
> Instead of removing support at all, maybe we could add some boot option
> that allows xen to work with broken/missing IOMMU with PV?
> It would be similar to compiling your own xen, as you have to actively
> enable this feature, but without the hastle of having to recompile and
> install it every time over and over again. At least for the time being,
> until PV will eventually be removed.

I think a workable solution would require:

1. Very explicit user decision. Something that user needs to find
separately, possibly also being forced to read some warning.
Specifically this exclude any kind of semi-automatic enabling like a
"missing IOMMU, do you want to use PV?" prompt.

2. When enabled, a permanent and clearly visible warning that the system
configuration is less secure. Preferably including also an easy way to
restore "optimal" configuration.

For the first point, some boot option could work.
For the second one, some watermark on the desktop maybe? Forced a
specific wallpaper?

My proposal is this: if enough people are interested in adding/restoring
ability to run on machines with broken IOMMU (at least until PV is
completely removed), please provide patches implementing the second
point above. Then, I'll add the first one.

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAl7pcpkACgkQ24/THMrX
1yxgDQf+LemMz2ykgUyDHH3Gn/8KTtxYansfwh3ncKEUQoUoPln06CNON5vYaHvv
FVVpSZc+q6Kc3lhB4fY0Y38u2/6+yEMVwD+uo5q44JbLsiuKttphqRenJT9HcJZc
yHw/nedb2OIaarsfQpUsoGH8f8RpajOsKJIvkqSs8HQOhzL+zvnRJbfnamJrPIqV
kh0ZTBgLzIY0Kdq/W+o6s9yzP83wtZ3kgSaHHxnpJekXuey2XwHfd7gYXG75BcxZ
bM7P+rFmnymC0nCyE85CLeBBdT94h4gNwatYCzSCgrheu/dpOFEb1n4kLMz2GPIJ
1ldK2uypQ1CfqYWoailGw2Xck98eWQ==
=uAGB
-----END PGP SIGNATURE-----

[Thorsten Schierer]

> But you do realize that "keep supporting" here actually means
> introducing a patch that intentionally breaks a security feature of
> an upstream Xen?

I wasn't aware of that, as the xen wiki states "On systems without an IOMMU, devices can be passed through to trusted PV guests, but doing so removes the security or stability advantages (though not the performance advantages)".
Therefor I thought it was a standard feature of xen? Perhaps the wiki is just outdated?

> As for a clear message - currently installer does say pretty clearly,
> that the system without IOMMU is not going to work. And yet, many people
> choose to ignore this explicit warning and then complain it doesn't
> work.

> The problem with this approach is such decisions in many cases have
> quite severe consequences that may not be obvious at first sight. In
> case of IOMMU-less sys-net, this for example means you are really
> exposed when connecting to various wifi networks in random coffee shops.
> And also you are no longer protected from some random USB stick you've
> found on a parking lot.

It couldn't hurt to add these examples explicitly into the warning in the installer, so the impact here is a little bit more transparent?
Also it could be made more clear that there is no support whatsoever in this mode, aka the user is completely on his own.

I mean, why even allow to install Qubes without iommu in the first place, when there is no (easy) way for advanced users to fully use it?

If you have no idea what you are doing and you simply ignore the warning, it still doesn't work out of the box without iommu, you still have to switch to pv and edit boot parameters, the installed apps are not listed, etc.
There is even a warning in the GUI when switching to PV, stating that it's insecure.
And thb, if you have been warned, you accepted the risk, you manually enabled the functionality (either by switching to pv or even by editing boot parameters, etc.), it should be enough patronizing?

So alltogether I'd say, the risk that a "newbie" installs it on unsupported hardware and uses it as daily driver, not knowing that there are security issues, is pretty much zero.

> Anyway, a permanent watermark idea could actually work. Similar thing is
> also implemented in Windows - you get a permanent marking on the
> wallpaper in all kind of debug builds or debug configurations.
> See more at the end of the message.

I've been thinking about the watermark idea and don't think it's the best idea. Besides the fact that a user (probably) would want to remove that "ugly" waterwark in daily use, it seems to be too much work and had no really benefit.

> > I mean yes, Qubes OS indicates a certain level of security in general and
> > that's absolutely correct. But since it's also highly customizable (which
> > is great and it should stay that way!), one could easily void that security
> > by modifying it that way.
>
> This is actually a thing that is also slowly changing. Greatly
> customizable system is good for power users who indeed do understand
> consequences of various configuration changes. But as a general
> direction, as the system matures, we try to reduce required knowledge to
> use Qubes, while still maintain strong security properties. This
> basically means making harder to shoot oneself in the foot.
>
> The work on GUI domain which isolates user interface from all-powerful
> dom0 also is a step in this direction.
>
> Another thing is, supporting many different configurations is difficult.
> Our priority is trustworthiness of the system, flexibility comes only
> later. Phasing out PV, while primarily done for security reasons, also
> reduces complexity of supported configurations.

I'm not a fan of locked down systems, therefor I hope these changes are more related to optical/gui features?
I mean, If I wanted to be patronized by my own OS, I'd buy an iPhone ;)
Just for clarification, if the GUI domain is fully established, it's still possible to get into dom0 as root, right?

> Or maybe one day Qubes OS will have enough market share to dictate such
> requirements
> ;)

That would be nice ;)

> 2. When enabled, a permanent and clearly visible warning that the system
> configuration is less secure. Preferably including also an easy way to
> restore "optimal" configuration.

As already stated above, as a user I wouldn't want to see a permanent message on my system, but rather one which can be disabled "if you really understand what you are doing".
Thb, I'd even rather recompile xen or the package that is responsible for the watermark, only to not have it displayed permanently :)

> For the first point, some boot option could work.
> For the second one, some watermark on the desktop maybe? Forced a
> specific wallpaper?
>
> My proposal is this: if enough people are interested in adding/restoring
> ability to run on machines with broken IOMMU (at least until PV is
> completely removed), please provide patches implementing the second
> point above. Then, I'll add the first one.

A boot parameter would be a good choice I think. Nobody would add a boot parameter and switch vms to pv mode "by accident" and there could be a big red warning on the wiki page (if there even should be one, since as we already said, it's unsupported)?
And later on just kill it alltogether with PV at the given time?

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Wed, Jun 17, 2020 at 12:48:48PM -0700, Thorsten Schierer wrote:
> > But you do realize that "keep supporting" here actually means
> > introducing a patch that intentionally breaks a security feature of
> > an upstream Xen?
>
> I wasn't aware of that, as the xen wiki states "On systems without an
> IOMMU, devices can be passed through to trusted PV guests, but doing so
> removes the security or stability advantages (though not the performance
> advantages)".
> Therefor I thought it was a standard feature of xen? Perhaps the wiki is
> just outdated?

I guess the wiki wasn't update yet. This is a new thing in Xen 4.13.

> > As for a clear message - currently installer does say pretty clearly,
> > that the system without IOMMU is not going to work. And yet, many people
> > choose to ignore this explicit warning and then complain it doesn't
> > work.
>
> > The problem with this approach is such decisions in many cases have
> > quite severe consequences that may not be obvious at first sight. In
> > case of IOMMU-less sys-net, this for example means you are really
> > exposed when connecting to various wifi networks in random coffee shops.
> > And also you are no longer protected from some random USB stick you've
> > found on a parking lot.
>
> It couldn't hurt to add these examples explicitly into the warning in the
> installer, so the impact here is a little bit more transparent?
> Also it could be made more clear that there is no support whatsoever in
> this mode, aka the user is completely on his own.
>
> I mean, why even allow to install Qubes without iommu in the first place,
> when there is no (easy) way for advanced users to fully use it?

I agree the existence of "Continue" button on that message is a bug. The
override of this check should be somewhere else (possibly together with
an option actually allowing the system to run using PV).

> If you have no idea what you are doing and you simply ignore the warning,
> it still doesn't work out of the box without iommu, you still have to
> switch to pv and edit boot parameters, the installed apps are not listed,
> etc.
> There is even a warning in the GUI when switching to PV, stating that it's
> insecure.
> And thb, if you have been warned, you accepted the risk, you manually
> enabled the functionality (either by switching to pv or even by editing
> boot parameters, etc.), it should be enough patronizing?

I still think there should be some obvious indicator all the time about
the unsupported/insecure configuration. See below.

> So alltogether I'd say, the risk that a "newbie" installs it on unsupported
> hardware and uses it as daily driver, not knowing that there are security
> issues, is pretty much zero.

And yet I've seen it happening (someone suggested it as a "fix" on IRC
without providing much context)...

> > Anyway, a permanent watermark idea could actually work. Similar thing is
> > also implemented in Windows - you get a permanent marking on the
> > wallpaper in all kind of debug builds or debug configurations.
> > See more at the end of the message.
>
> I've been thinking about the watermark idea and don't think it's the best
> idea. Besides the fact that a user (probably) would want to remove that
> "ugly" waterwark in daily use, it seems to be too much work and had no
> really benefit.

So, maybe not a watermark, but some clear message on the wallpaper (but
not covering open windows)?

> > > I mean yes, Qubes OS indicates a certain level of security in general
> and
> > > that's absolutely correct. But since it's also highly customizable
> (which
> > > is great and it should stay that way!), one could easily void that
> security
> > > by modifying it that way.
> >
> > This is actually a thing that is also slowly changing. Greatly
> > customizable system is good for power users who indeed do understand
> > consequences of various configuration changes. But as a general
> > direction, as the system matures, we try to reduce required knowledge to
> > use Qubes, while still maintain strong security properties. This
> > basically means making harder to shoot oneself in the foot.
> >
> > The work on GUI domain which isolates user interface from all-powerful
> > dom0 also is a step in this direction.
> >
> > Another thing is, supporting many different configurations is difficult.
> > Our priority is trustworthiness of the system, flexibility comes only
> > later. Phasing out PV, while primarily done for security reasons, also
> > reduces complexity of supported configurations.
>
> I'm not a fan of locked down systems, therefor I hope these changes are
> more related to optical/gui features?
> I mean, If I wanted to be patronized by my own OS, I'd buy an iPhone ;)
> Just for clarification, if the GUI domain is fully established, it's still
> possible to get into dom0 as root, right?

Yes and no. There are going to be less and less things in dom0.
Ideally, dom0 should only be about setting up the environment and then
enforcing policies(*), but everything else should be disaggregated.
Specifically, we do plan at some point for dom0 system to be a small,
read-only system image. At that point, running any command in dom0
wouldn't make much sense (perhaps besides debugging).
What would make sense is to run a command in relevant service VM and
just make sure dom0 will allow it (if it involves inter-vm
communication).

(*) Actually, maybe even those two tasks could be separated.

> > Or maybe one day Qubes OS will have enough market share to dictate such
> > requirements
> > ;)
>
> That would be nice ;)
>
> > 2. When enabled, a permanent and clearly visible warning that the system
> > configuration is less secure. Preferably including also an easy way to
> > restore "optimal" configuration.
>
> As already stated above, as a user I wouldn't want to see a permanent
> message on my system, but rather one which can be disabled "if you really
> understand what you are doing".

Switching to "suboptimal" configuration should be very explicit action,
but it should be also clearly visible when such configuration is active.
What I want to avoid is setting the system up as an experiment but later
slowly starting using it for more serious things - forgetting you've
enabled a bunch of dangerous options before.

> Thb, I'd even rather recompile xen or the package that is responsible for
> the watermark, only to not have it displayed permanently :)

I think we need to find the right balance here, for the indicator to be
not too-annoying. But I still think it should be clearly visible all the
time, not only during the switch action.

> > For the first point, some boot option could work.
> > For the second one, some watermark on the desktop maybe? Forced a
> > specific wallpaper?
> >
> > My proposal is this: if enough people are interested in adding/restoring
> > ability to run on machines with broken IOMMU (at least until PV is
> > completely removed), please provide patches implementing the second
> > point above. Then, I'll add the first one.
>
> A boot parameter would be a good choice I think. Nobody would add a boot
> parameter and switch vms to pv mode "by accident" and there could be a big
> red warning on the wiki page (if there even should be one, since as we
> already said, it's unsupported)?
> And later on just kill it alltogether with PV at the given time?

Yes, it will go away at some point together with the whole PV.

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAl7qvSMACgkQ24/THMrX
1yyLPQgAmqsa8QgeYuqKuN3piH/MwAWvsRMrSmH5ajYtrrddd4RUt+xOqSJzuIBv
Pjiiz/QGIkaQIk/WjJQTfy0sAM1JIPBUZrHEirYLSBF9Mbg7CMErwSCj9p9q52jb
N9ZhVZp4PS+pemniXJVXM53EnAboyLIaOtZa08dbgytam4OxA+pcffRzE83Gc+xI
xc8LXl0NsOUZ8jT8seVYbSDzi6+7vrEa/zo9OcRkMMWmypSvmyX2jduJ2nlUuSfB
QXC4lCJZnw5Si6ciPzBPVI6H+3I0U08gBK1B8QhGKTgxbwmXGDLdp4WTqnLH18i4
GGiXODwuuLLU6IS6MNEKh52iEE/INw==
=Kixm
-----END PGP SIGNATURE-----

[Thorsten Schierer]

> I agree the existence of "Continue" button on that message is a bug. The
> override of this check should be somewhere else (possibly together with
> an option actually allowing the system to run using PV).

It would also be ok if you had to switch to console in Qubes OS Installer and run some command there to enable iommu-less installation.
Also... Does it really make sense to implement new features that get obsolete when pv is removed? Do you have an estimated time when it will be removed completely?

> > So alltogether I'd say, the risk that a "newbie" installs it on unsupported
> > hardware and uses it as daily driver, not knowing that there are security
> > issues, is pretty much zero.
>
> And yet I've seen it happening (someone suggested it as a "fix" on IRC
> without providing much context)...

From years of experience I can tell you, there will always be people using commands, tools, etc. without knowing what they are doing. You just can not prevent it.
Currently, if someone wants to run Qubes without iommu (and it doesn't work out of the box), they will search for instructions and possibly find people telling them to switch to pv.
Whereas if you remove the feature in source code, they might possibly find people giving them self-compiled "patched" packages and instructions to install them in dom0.
Which variant could possibly cause more harm?

If people really want to do something, they will try and try until it works. Even if that means to compromise their own machine unknowingly.
A good example is "cracked" android apps - people want to use them for free (aka piracy), so they download some random apk from the web and install it.
It happens every day and there are a lot out there coming with malware, so...

Same goes for the nag-screen / permanent message. There will be people trying to get rid of it, because it's "ugly" (and completely useless if you willingly decided to use it for daily work), I am one of them ;)
Of course I'd likely patch the appropriate package myself, instead of downloading them from untrusted source, but less experienced users will do.

> I still think there should be some obvious indicator all the time about
> the unsupported/insecure configuration. See below.
 

> So, maybe not a watermark, but some clear message on the wallpaper (but
> not covering open windows)?

> Switching to "suboptimal" configuration should be very explicit action,
> but it should be also clearly visible when such configuration is active.
> What I want to avoid is setting the system up as an experiment but later
> slowly starting using it for more serious things - forgetting you've
> enabled a bunch of dangerous options before.

> I think we need to find the right balance here, for the indicator to be
> not too-annoying. But I still think it should be clearly visible all the
> time, not only during the switch action.

If really needed, maybe a little script could run on startup, perform a check for iommu and display a warning message?

> Yes and no. There are going to be less and less things in dom0.
> Ideally, dom0 should only be about setting up the environment and then
> enforcing policies(*), but everything else should be disaggregated.
> Specifically, we do plan at some point for dom0 system to be a small,
> read-only system image. At that point, running any command in dom0
> wouldn't make much sense (perhaps besides debugging).
> What would make sense is to run a command in relevant service VM and
> just make sure dom0 will allow it (if it involves inter-vm
> communication).
>
> (*) Actually, maybe even those two tasks could be separated.

As long as your can get full access to every part as root, it's fine.

[Chris Halliger]

I stumbled upon qubes when it was in version 3.2 or something and decided to test it out. I liked the idea that it separates user data from each other and so I kept playing around with it from time to time, mainly to learn how things work. The lack of gpu acceleration was a big problem for me back then, which made me hesitate to use it as my main os. Some time later I finally installed qubes version 4 on my main system and fully transfered everything from windows 10 to my new qubes 4 installation, which is quite some work in order to get everything set up as best as possible. It's been a great experience so far, even though it is a bit more complex than windows 10 and I still have to look up some linux stuff from time to time in oder to change things to my liking. The problem is that even though in bios vt-d is enabled, qubes doesn't seem to be able to use it. So I had to use the pv option in my qubes in oder to use it properly. While you consider it a security risk, I'd say windows 10 was no better regarding the possible memory attack and with data separation, split gpg, split bitcoin and so on that qubes throws in the pot, I consider it waaaaaay more secure than windows 10. So long things short: Please don't disable support for pv and broken vt-d, as it would leave me (and probably others) totally hanging. Either I'd have to keep on using an outdated Qubes version 4 *yikes* or go the extensive way to move all data back to Windows 10 *double yikes*

[c...]

> I still think there should be some obvious indicator all the time about
> the unsupported/insecure configuration. See below.

Maybe as an alternative is to have a 'qubes with yellow warning sign in the corner' icon at system tray indicator, showing by click an extra 'insecure configuration warning' line at the top of running appvm's?

[Teqleez Motley]

Ageed. Colored tray icon with accompanying info text is a better solution than "watermark", IMO.

(I also very much second this effort of having the flexibility of running a less secure environment wherever we see fit to do so. There are several valid use cases, and who among Qubers-interested people are running everything on their main ("only"??) computer these days? We probably all have several spares in addition to the main one for many practical reasons.)

Great to see that Marek is taking a pragmatic/open-minded approach to this, and taking the time to provide excellent insights for the relevant considerations/discussions/dilemmas/etc.
Kudos.

--
Regards,
Teqleez



> You received this message because you are subscribed to the Google
> Groups "qubes-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to qubes-devel...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/qubes-devel/719ec53e-48fa-4586-b529-3aa10026c69eo%40googlegroups.com.
>

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Thu, Jun 18, 2020 at 06:18:01AM -0700, Thorsten Schierer wrote:
> > I agree the existence of "Continue" button on that message is a bug. The
> > override of this check should be somewhere else (possibly together with
> > an option actually allowing the system to run using PV).
>
> It would also be ok if you had to switch to console in Qubes OS Installer
> and run some command there to enable iommu-less installation.
> Also... Does it really make sense to implement new features that get
> obsolete when pv is removed? Do you have an estimated time when it will be
> removed completely?

While this is also my initial reaction for allowing back PCI passthrough
without IOMMU, I am kind of convinced that in some cases it is still a
useful thing to have. While I would like to remove PV sooner than later,
the "sooner" is not really "soon" - PV remains an option in Qubes 4.1,
which will be supported at least half a year after releasing the next
version (Qubes 4.2 or 5.0). And I'm not yet sure if we manage to get rid
of all the PV usage in 4.2 either.

> > > So alltogether I'd say, the risk that a "newbie" installs it on
> unsupported
> > > hardware and uses it as daily driver, not knowing that there are
> security
> > > issues, is pretty much zero.
> >
> > And yet I've seen it happening (someone suggested it as a "fix" on IRC
> > without providing much context)...
>
> From years of experience I can tell you, there will always be people using
> commands, tools, etc. without knowing what they are doing. You just can not
> prevent it.

Yes, I know. But I'd like to make it hard to do _without clear message
about consequences_. Which includes also "protection" against quickly
dismissing a warning confirmation dialog and forgetting it ever popped
up.

> Currently, if someone wants to run Qubes without iommu (and it doesn't work
> out of the box), they will search for instructions and possibly find people
> telling them to switch to pv.
> Whereas if you remove the feature in source code, they might possibly find
> people giving them self-compiled "patched" packages and instructions to
> install them in dom0.
> Which variant could possibly cause more harm?
>
> If people really want to do something, they will try and try until it
> works. Even if that means to compromise their own machine unknowingly.
> A good example is "cracked" android apps - people want to use them for free
> (aka piracy), so they download some random apk from the web and install it.
> It happens every day and there are a lot out there coming with malware,
> so...

And yet android tries to sandbox them to limit the impact. It is far
from perfect, it is easy enough to grant way too broad permissions to an
app, but at least it tries to do something...

> Same goes for the nag-screen / permanent message. There will be people
> trying to get rid of it, because it's "ugly" (and completely useless if you
> willingly decided to use it for daily work), I am one of them ;)

As noted in my previous message - this is indeed a tricky topic. It
should be noticeable enough to be noticed, but also non-intrusive enough
to not motivate enough its removal.
Maybe a Teqleez idea about tray icon is not a bad one? Or something like
an additional item with the warning in already existing domains widget?
I'd still combine it with a text on a wallpaper.

> Of course I'd likely patch the appropriate package myself, instead of
> downloading them from untrusted source, but less experienced users will do.
>
> > I still think there should be some obvious indicator all the time about
> > the unsupported/insecure configuration. See below.
>
> > So, maybe not a watermark, but some clear message on the wallpaper (but
> > not covering open windows)?
>
> > Switching to "suboptimal" configuration should be very explicit action,
> > but it should be also clearly visible when such configuration is active.
> > What I want to avoid is setting the system up as an experiment but later
> > slowly starting using it for more serious things - forgetting you've
> > enabled a bunch of dangerous options before.
>
> > I think we need to find the right balance here, for the indicator to be
> > not too-annoying. But I still think it should be clearly visible all the
> > time, not only during the switch action.
>
> If really needed, maybe a little script could run on startup, perform a
> check for iommu and display a warning message?

Yes, that would also work. But I think something more than just a
message at startup is needed, especially on systems that are not so
frequently restarted.

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAl7wBIsACgkQ24/THMrX
1yx8ogf9EEBl81r3jJOnA06wAOlQag4YTRO3gclejjOowV03bQcKM+oECg9KZZLf
Qm6Ek6MH7EqBk01imJA5ymqWm34svnmoxXj8bJkQuxxXONBe8G93QarX+e7xNRxX
Gw8YFtBMuYB0BoZ50XaxXqlPGtCOXT4BA8LjdJvkOIl+9A/SQ61yNiRc6u7vb6ny
fy7fDqorACwVCWBiZHudGDzswqmbv1cm57GetuZ1UYOsjo+A+7urmyKB99XkSwNY
O56uv6RMgPkbVrFLDJmU6agIumLBxQV+oBVSZh4nvcjhtdT34bC+t4E9hfNGjaDm
KRNp+RJolFL1iImyHZEb4uG9XltFfQ==
=x6+w
-----END PGP SIGNATURE-----

[Thorsten Schierer]

My first thought was also to include the warning into the Qubes logo in tray, but since it's the Qubes Domain widget and not a general Qubes OS widget, it somehow felt like the wrong place.
Also, you have to be careful here, since displaying a permanent warning icon could desensitize the user over time. If you later add some other warnings or hints into this widget, people might ignore them, thinking it's the IOMMU warning.
So an official way to "hide" the warning is highly appreciated.

Real-life story:
Some years ago a friend installed some anti-virus software on Windows. When he disabled their cloud-scanning option, its tray icon turned yellow and inside the application there was a permanent warning that the cloud-scanning is disabled. No option to disable it. With time he got used to the yellow icon. Some time later he set up a new firewall system and the software was unable to download updates. Now there was a second warning inside the app, but still the same yellow tray icon. Since he was used to that tray icon, he didn't know about the second warning and only opened the software several weeks later by coincidence. Needless to say that the software was completely outdated by that time ;)
(Meanwhile in newer versions, the vendor added fine-grained options to disable any kind of warnings, which restores the green tray icon if no more warnings are present after hiding them)

Anyways, I think a small (dedicated) application that checks for IOMMU and displays a tray icon with a notification warning and a dialog window (when you click it) explaining the issue more detailed, would be a better approach?
It could either run (and also be disabled) via "Session and Startup" (just keep things simple) or it could be launched by some other part of Qubes OS directly and be disabled by some command line or configuration file.

Since it's a dedicated application, it can simply be removed together with PV mode, without having to fiddle with code in the Qubes Domain widget again. I mean, we already know it will only exist for a limited time.