GPU passthrough: 2000 USD bounty

[Stickstoff]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Hello everyone,

I would like to be able to do a little gaming on my regular computer
from time to time, for sanity reasons. I use Qubes OS on a dual GPU
notebook. I don't want to compromise security with unsafe code in DOM0
nor dual booting. My budget towards this is up to 2000 USD.

Options I can think of (ordered by preference):

- - put 2000 USD to a bounty for programming of general (secondary) GPU
passthrough to an app-VM (including consumer nvidia GPUs)

- - replace my computer with an nvidia quadro equipped computer, put
whats left of the 2000 USD towards a bounty to get ATI and nvidia
quadro GPUs (apparently both easier to do than consumer nvidia GPUs)

- - buy an additional computer and stream the gaming via VNC or the like
to a Qubes app-VM

- -buy and use an additional computer


Gaming on Qubes is a niche and unrelated to its real goal. Still, it
would open new possibilities with running different OS' in VMs with
hardware acceleration, from gaming to grafics rendering to video
editing to scientifical calculations. It would be a big step towards
one-system-fits-all for the security conscious.
If some universally useable code came from this, it would make
migration from windows to "regular" linux distros much easier for a
lot of people who still need some gpu-dependent windows function.

I understand that 2000 USD is probably too little for a project of
such magnitude. Maybe it's a start of a bounty that becomes big enough
for this.

What do you people think? Or am I overlooking other options?
Kernel 4.10 adds "virtual GPU support" [1], will that make things
easier?

Cheers,

Stickstoff


[1]
http://news.softpedia.com/news/linux-kernel-4-10-officially-released-wit
h-virtual-gpu-support-many-features-513077.shtml
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBCAAGBQJY+mOSAAoJEPyQPtcO3Q1iyc4P/3u79Lx+8vXJ1/wyfcoKljI6
LYVEIC5ZaUNNl1k14rOL69V3Ndf3AFTPdRLUV9j5pvqpBCRHzrokKAJJ32vfQg6R
8uiJaDaYgje8RYUDx8K4U3oq69ETWx1aLYANnp5gV71IoMES2mK+XOW71+EhfjhF
GE7XQob/dgYXLWRHExarTGy1Rr+Nr3rScdGc3mAWAPqlreN58OZmkS0T/K7HCCcR
NPDpne7Pljb6MM8rBb9cZcG4Vz6nHOdJyuKKqEnquYLU8hoKsFEO90k7xK1GEFP1
iyBwK7yV0vauLmaHkf4HXN3PMRo4Hhuz2RfrHkW+AP0j5wIaqk4Wq2FZFvxz4C3n
ErQrYgqHi7eFrcBm+rwSedbi6BfgYqK15lRRqXwLsYbMUKdaN1eYnYpLKV/sl6UK
FGv9Y08G44ZPhNS5JAGbxBdvsKe+Nde0V/H/u8MzRXCLmkk8XKRbKyf+lQ5ZTmtd
r+XLmWiQ5DwOKUi24h8pMltngWc/nqhSDMy7mbf4JBBhjWV1T3o0o4MDg4YatR4d
x8vDs64U4A1lqTMbw+U4mZU2crka4xSFJ+OZk3h76heIrVF/jOwGzGpKGFL0+cHH
yDWFQj8r+PZ/BHChkJluthD0mj1bkDebilA33K1tMXOvbA3/Xd+1WDg1Q9YvskNv
ExN45lREneOMcWeLiHUV
=Up+F
-----END PGP SIGNATURE-----

[pixel fairy]

On Friday, April 21, 2017 at 12:55:07 PM UTC-7, Stickstoff wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> Hello everyone,
>
> I would like to be able to do a little gaming on my regular computer
> from time to time, for sanity reasons. I use Qubes OS on a dual GPU
> notebook. I don't want to compromise security with unsafe code in DOM0
> nor dual booting. My budget towards this is up to 2000 USD.
>

havent tried this yet, but you can stream from ps4 to windows (or mac), the requirements are pretty light and imply no need for accelerated graphics on the client end. please mention me if you try this and post back to the group. i have a ps4, but still working on getting qubes running.

if your willing to get a separate system for games, the nintendo switch looks pretty nice on paper. havent seen one in person yet.

[Grzesiek Chodzicki]

IIRC secondary GPU passthrough won't work on laptops because the discrete GPU is a render device and not a full display device (it's not directly connected to the laptop display or the video outs, instead it's framebuffer is copied to the framebuffer of the integrated GPU that handles the actualy physical displays) Copying the framebuffer between VMs with minimal latency sounds challenging. Although I remember one guy on this group that successfully passed through an AMD GPU to a Windows VM and was able to play games in the VM. This was a desktop PC though.

[Mathew Evans]

Biggest issue that limits you with nvidia is the fact that drivers detect that it is running in a VM / HVM etc.. It is possible to get nvidia drivers installed into Qubes (dont recommend it) and then you can pass a prime device through to a app VM with little issue. Ive done this for doing cuba-cat for password cracking and it works but you wont get any output on the screen at all. (goto dump to file)

Would love to play games on Qubes though. implenetation of OpenGL for the Qubes GFX driver would go a very very long way.

[Stickstoff]

On 04/21/2017 10:13 PM, pixel fairy wrote:
> havent tried this yet, but you can stream from ps4 to windows (or
> mac), the requirements are pretty light and imply no need for
> accelerated graphics on the client end. please mention me if you try
> this and post back to the group. i have a ps4, but still working on
> getting qubes running.
>
> if your willing to get a separate system for games, the nintendo
> switch looks pretty nice on paper. havent seen one in person yet.

There are several streaming solutions. Steam does it natively [1],
Nvidia has something [2], and there's an OSS implementation of that
Nvidia thing [3].
I, personally, would rather get some additional computerhardware instead
of a gaming console. More versatile and reusable later on.

On 04/22/2017 03:02 AM, Grzesiek Chodzicki wrote:
> IIRC secondary GPU passthrough won't work on laptops because the
> discrete GPU is a render device and not a full display device (it's
> not directly connected to the laptop display or the video outs, instead
> it's framebuffer is copied to the framebuffer of the integrated GPU that
> handles the actualy physical displays) Copying the framebuffer between
> VMs with minimal latency sounds challenging.

That's bad news. Thanks for the hint.
I suppose copying the framebuffer still is more reasonable than
streaming the game over ethernet, which, surprisingly, works quite good
with [1] to [3].

> Although I remember one guy
> on this group that successfully passed through an AMD GPU to a Windows
> VM and was able to play games in the VM. This was a desktop PC though.

Yes, I remember that too. He installed quemu in DOM0 for this, which is
too much of a security compromise for me.

On 04/22/2017 12:57 PM, Mathew Evans wrote:
> Biggest issue that limits you with nvidia is the fact that drivers
> detect that it is running in a VM / HVM etc.. It is possible to get
> nvidia drivers installed into Qubes (dont recommend it) and then you
> can pass a prime device through to a app VM with little issue. Ive
> done this for doing cuba-cat for password cracking and it works but
> you wont get any output on the screen at all. (goto dump to file)

You mean you installed the nvidia drivers in DOM0? Yes, I'd avoid that
on my regular system.

> Would love to play games on Qubes though. implenetation of OpenGL for
> the Qubes GFX driver would go a very very long way.

I tried to force software rendering in the VM, no luck even with that.

There are projects which split the opengl driver into a local proxy-like
part and a remote actually-rendering part, I found at least three [4] of
them. Maybe that would be usable? The rendering part would have to run
in DOM0 though, and I'm not sure if we could trim it down enough to
trust it though.
Heck, even just connecting to the rendering part via network would be a
good start I guess.


So, I'm not even sure what is technically possible, with 3D acceleration
in a VM (from GPU passthrough or rendering-proxy) without compromising
security of the system? There's thunderbolt for external pci too. Is any
of those ideas even possible?


Ente


[1] http://store.steampowered.com/streaming/
[2] https://www.nvidia.com/en-us/shield/games/gamestream/
[3]
https://github.com/moonlight-stream/moonlight-docs/wiki/Moonlight-Overview

[4]
https://www.mesa3d.org/osmesa.html
https://arrayfire.com/remote-off-screen-rendering-with-opengl/
http://www.virtualgl.org/About/Introduction

[Grzesiek Chodzicki]

With latest version of Xen it is possible to pass through a PCI device to a HVM (without running qemu in dom0) so we could at least try. I have one gpu in my system but somebody here ought to have two.

[Jean-Philippe Ouellet]

I don't know anything about your specific hardware, but it is true
that secondary GPUs are often not connected to the display itself, but
rather the rendering takes place there and then the rendered frames
are passed back to the host and to the integrated gpu to be put on
your display. From a Qubes perspective I believe this is actually a
very good thing since it means we could keep the integrated GPU
statically assigned to dom0, and keep the qubes gui protocol largely
unchanged. The question would be one of getting the passed through GPU
to render its output to some buffer which we pass back to dom0.

There are still firmware-security issues associated with passing the
discrete GPU between VMs of different trust levels, because someone
who has full control of the GPU may be able to re-flash its firmware
with something that would later perform a DMA attack against the 2nd
VM it's attached to. However, if you only ever wish to pass it through
to a single "gaming" windows HVM or such, this is not a problem.

The reason integrated GPUs are interesting in this regard is that they
do not have firmware which is persistently stored on the device,
rather it is loaded externally on each power-on and subject to normal
boot-security measures. The thinking is that by rebooting between
assigning your integrated GPU to different VMs, you prevent one from
compromising another via the GPU by making GPU compromise ephemeral.

As for previous successes requiring upstream-QEMU in dom0, the problem
here is that Xen only supports a very old forked QEMU in stubdomains,
but this is something that will change. Progress in this area has
stalled because there was an effort to run QEMU in a very minimal
unikernel-style environment, but this effort has been abandoned and
work is now underway towards making it run on top of linux (still in a
separate stubdomain), which should take less work to bring to a usable
state than the previous minimal-stubdom effort.

[cooloutac]

You have a ps4 and you want to game on the pc? why? Pc gaming died a decade ago cause piraters, cheaters, and ddos.

League of Legends is the only pc game on windows I would consider "popular" tks to asian countries who take e-sports as serious as football. But On Linux the only popular games are cs:go and Dota2 and unless you're a gaming pro or someone who doesn't mind trolls, that would be sadistic...lol

I would stick to single player games for consoles until they start jailing kids like in Japan and Korea. Man do I miss ea-sports on the pc. 95-2005 was a great decade.

Hardware industry has been steady tankin since, and I don't blame tablets or smartphones. I built a computer for the first time in years only for Qubes, but no way I'd waste money on a gaming rig for me and my hardware to get abused.

[cooloutac]

Actually I called dota2 and cs:go popular, but only by linux standards. millions at a time playing LoL compared to maybe 50,000 playing dota2, 20,000 playing cs:go and I'm sure those numbers are fabricated. And thats world wide.

And I find it such a shame that only moba games are popular. But they are the hardest games for anarchists to undermine I guess...

[Tai...@gmx.com]

On 04/25/2017 11:29 AM, cooloutac wrote:

> You have a ps4 and you want to game on the pc? why? Pc gaming died a decade ago cause piraters, cheaters, and ddos.

What? there are still many decent new games being released. I play BF4
and only encounter obvious cheaters once in a blue moon and they always
get banned by stat based anti-cheat like fairfight (server side
anti-cheat is the only way to go, no bullshit kernel drivers required
either)

Consoles suck, even the new versions of the PS4/Xbone can't play at
native resolutions with at least 60FPS and once the OEM shuts down the
servers your games are useless - people are still playing BF1942 because
they were able to easily reverse engineer a master server and anyone can
DL the server files but that wouldn't be possible on a console.
Not to mention the DRM and always-online requirements for singleplayer
games (yeah PC is DRM'ed too, but there are still great AAA games that
get released without it such as The Witcher 3 and the Metro series)

Piracy doesn't result in bad game sales, only bad games do and denuvo
proves that - the witcher 3 released without DRM sold many more copies
in the first week than Mass Effect 3.

[Grzesiek Chodzicki]

That escalated quickly...

Guys come on, this was supposed to be gpu passthrough thread not pc fanboy vs console fanboy thread.