nVidia binary in dom0 (ThinkPad P1 gen4)

16 views
Skip to first unread message

Michał "rysiek" Woźniak

unread,
Apr 17, 2022, 10:57:46 PM4/17/22
to qubes...@googlegroups.com
Hey all,

I am trying to get the nVidia binary driver to work in dom0. Using the latest
version of it (510.60.02) always ends up with this line in `Xorg.0.log`,
followed by X crashing:
> Failed to allocate push buffer

Running `nvidia-smi` shows the card is available, kernel modules loaded; I can
get temperature readouts, for example.

Tried changing UEFI settings ("discrete" vs "hybrid" mode), tried fiddling with
kernel parameters (modesetting, iommu), to no avail.

Not sure what I could try next. Any ideas welcome!

--
Best,
rysiek


Demi Marie Obenour

unread,
Apr 18, 2022, 9:37:30 AM4/18/22
to Michał "rysiek" Woźniak, qubes...@googlegroups.com
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Have you considered using sys-gui-gpu instead? Binary drivers in dom0
are a bad idea, both from a security and reliability perspective. In
particular, dom0 has an old version of both X11 and Mesa, which may well
be incompatible with the blob driver.

Alternatively, if your computer has an integrated Intel GPU, you could
use that. Nouveau might also be an option, if it supports your card.

- --
Sincerely,
Demi Marie Obenour (she/her/hers)
Invisible Things Lab
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEEdodNnxM2uiJZBxxxsoi1X/+cIsEFAmJdaZYACgkQsoi1X/+c
IsEXhw/+LB9voMNQ+1WEr8/ipDJ1ZSggH+EzoCONhmhlM/X+oeyZ6OHzqJ4FKbgv
U+zI/CH05rZoRd4Fda6CQRiH4xMIoCQ+rIHp2+15vkof8FAIOQpykDWWm/pJ4G/F
kfK6oh/OStZcHQWh8d3ShiDi5bbeVIFmW1ix4CvJ+F2wpPMX2vOYUbgI+j2SxasU
n0s4EVnjdRfFBuxpGG4pK+6YJ/Tepkf25izQZyEQuZ1PgrJL7sgKpieBRWNyCw7q
+wJDAV77vx3/rBp90zCEWJ0HR3CJGXWLduwCs0HN2Zg/jJ7zs55cViOK79D3bAlS
DrH7TWOwHqDMj8C0zPuozEMb5a+W7dPZZOlQy9KLWAmy8swnC1ED0SKf+u5v+qvn
4BbFVgxYAYisDMAwDAY/ZwU+AgfxMHHgVucauM20MVJPgxV0NObMgddw//iWHWGx
fMG6F4KI28R7L0asa/Tpc982GaisRZQ/MC5WPJamE7ZIIAA95lDEXqnJXOKTG4GQ
NNtmTYN65JC+lAkGKOV6ZwagxgWCRUAHWQk6WRWNY9uyD5cKHCpZz+Qi9JPwcEXn
iWQ52795aEUTT1ehZZgXU9hjVRSoGzXccvS/ZKw4X1Au/Cgux/hupagIMeKVk/Lt
8KkS5QNOIgDD3plK7qGs57ZzzC5bG3Qj7A5fqfbLB5fyxhNOtFg=
=pAEG
-----END PGP SIGNATURE-----

Michał "rysiek" Woźniak

unread,
Apr 18, 2022, 11:40:47 AM4/18/22
to qubes...@googlegroups.com
Hey,

On Monday, 18 April 2022 13:37:25 GMT Demi Marie Obenour wrote:
> On Mon, Apr 18, 2022 at 02:57:32AM +0000, Michał "rysiek" Woźniak wrote:
> > Hey all,
> >
> > I am trying to get the nVidia binary driver to work in dom0. Using the
> > latest version of it (510.60.02) always ends up with this line in
> > `Xorg.0.log`,>
> > followed by X crashing:
> > > Failed to allocate push buffer
> >
> > Running `nvidia-smi` shows the card is available, kernel modules loaded; I
> > can get temperature readouts, for example.
> >
> > Tried changing UEFI settings ("discrete" vs "hybrid" mode), tried fiddling
> > with kernel parameters (modesetting, iommu), to no avail.
> >
> > Not sure what I could try next. Any ideas welcome!
>
> Have you considered using sys-gui-gpu instead?

I have not, for a very specific reason: in the laptop in question, all external
video outputs are hard-wired to the nvidia card. That is, I *cannot* use an
external display *unless* I get the nvidia card to work, somehow.

I had *some* small progress: if I go the nouveau route (with
nouveau.modeset=1), I get an external display recognized and configured in X
now. I can move my mouse to it. I cannot display any actual windows on it.

> Binary drivers in dom0 are a bad idea, both from a security and reliability
> perspective.

I am well aware of that. But as far as security is concerned, GPU passthrough
has its own problems. It's turtles all the way down!

> In particular, dom0 has an old version of both X11 and Mesa, which may well
> be incompatible with the blob driver.

That's true. I am going to try some older ones. Already tried 495.46, got a
different error:
> Failed to allocate shared surface

I guess I should try to figure out which binary driver was the newest when the
dom0 versions of xorg and Mesa were released.

> Alternatively, if your computer has an integrated Intel GPU, you could
> use that.

That's what I've been using, but that leaves me without the ability to use
external displays.

> Nouveau might also be an option, if it supports your card.

It doesn't seem to (see above).

--
Best,
rysiek


Michał "rysiek" Woźniak

unread,
Apr 18, 2022, 2:05:12 PM4/18/22
to qubes...@googlegroups.com
On Monday, 18 April 2022 15:40:32 GMT Michał "rysiek" Woźniak wrote:
> > Have you considered using sys-gui-gpu instead?
>
> I have not, for a very specific reason: in the laptop in question, all
> external video outputs are hard-wired to the nvidia card. That is, I
> *cannot* use an external display *unless* I get the nvidia card to work,
> somehow.

Wait, that was almost certainly a brainfart. With sys-gui-gpu, the nVidia GPU
would get passed-through to that VM, and so any driver installed there (and
presumably more likely to work than in dom0) would also have access to all the
external displays hard-wired to that GPU. Right?

That's something I will try next then, and report back. Thank you for the
suggestion!

--
Best,
rysiek


Demi Marie Obenour

unread,
Apr 18, 2022, 5:43:02 PM4/18/22
to Michał "rysiek" Woźniak, qubes...@googlegroups.com
On Mon, Apr 18, 2022 at 06:04:59PM +0000, Michał "rysiek" Woźniak wrote:
> On Monday, 18 April 2022 15:40:32 GMT Michał "rysiek" Woźniak wrote:
> > > Have you considered using sys-gui-gpu instead?
> >
> > I have not, for a very specific reason: in the laptop in question, all
> > external video outputs are hard-wired to the nvidia card. That is, I
> > *cannot* use an external display *unless* I get the nvidia card to work,
> > somehow.
>
> Wait, that was almost certainly a brainfart. With sys-gui-gpu, the nVidia GPU
> would get passed-through to that VM, and so any driver installed there (and
> presumably more likely to work than in dom0) would also have access to all the
> external displays hard-wired to that GPU. Right?

Yup! You can even assign the nvidia card to the VM while leaving the
iGPU in dom0. If you do that, you will want to set your qrexec policies
to ensure that sys-gui-gpu can’t harm VMs it isn’t the GUIVM of. That
basically means removing lots of Admin API access from it that it
doesn’t need.

> That's something I will try next then, and report back. Thank you for the
> suggestion!

You’re welcome!
signature.asc
Reply all
Reply to author
Forward
0 new messages