VGL for 3D acc. apps in docker

[romangrothausmann]

Hi DRC

Following

https://groups.google.com/d/msg/virtualgl-users/BHF8rHMeeh4/QgmTskSaAAAJ

and

https://github.com/plumbee/nvidia-virtualgl/blob/6ee0057490f192f2a2e0f9692b7103aeddc830b7/Dockerfile

I created a dockerfile for VGL accelerated ITKSnap docker image (https://gitlab.com/romangrothausmann/dfitksnap/blob/itksnap-3.4_Qt4_VGL/)

which seem to work fine on a host with nvidia GPUs, drivers and nvidia-docker2.

Would you say the VGL configuration is appropriate? E.g. you recommended +t while plumbee uses -t:

https://gitlab.com/romangrothausmann/dfitksnap/blob/49c4b7f4b90ade9eeace7296d2b20b1997431589/Dockerfile#L75

In addition to what is recommended for X11 apps from within docker (http://wiki.ros.org/docker/Tutorials/GUI#The_isolated_way) I had to add both the 3D acc. X-display as well as the VNC display used for displaying:

https://gitlab.com/romangrothausmann/dfitksnap/blob/49c4b7f4b90ade9eeace7296d2b20b1997431589/run.sh#L9

Is that the correct way for X11 with VGL over VNC?

With vglconnect (avoiding VNC) I get an error similar to:

ITK-SNAP: cannot connect to X server localhost:10.0

Do I need to determine the display for the xauth setting differently then?

VGL is great! Many thanks for continuous development, maintaining and supporting.

Best,

Roman

[DRC]

It appears from your run.sh script that both the 3D X server and the VNC
server are running on the host instead of in the Docker container. In
that case, I don't really understand why all of the xauth stuff is
necessary. Can't you just share your existing ~/.Xauthority file with
the Docker container with read/write access? In fact, you wouldn't even
necessarily need to open up the 3D X server to all users. You could
still use the vglusers group if you wanted. You would just need to run
'xauth merge /etc/opt/VirtualGL/vgl_xauth_key' to add the 3D X server
cookie to ~/.Xauthority prior to starting the Docker instance.
Otherwise, your basic approach seems reasonable to me.

If I've misunderstood and you are actually running one of the X servers
(3D X server or VNC server) inside of the Docker container, then my
answer would be different. The answer I gave in the linked thread below
assumed that the 3D X server was running inside of the Docker container.
That appears to be the case with the plumbee Dockerfile you posted. In
your case, however, if there is no X server running in the Docker
container, then you don't need to run vglserver_config in the Docker
container at all. It appears that your run.sh sets things up so that
the Docker container makes X connections out to the host, so that
simplifies things considerably. In that case, your Docker container
only needs the X11 and OpenGL client libraries, plus the VirtualGL faker
libraries. It doesn't need any of the Xorg server infrastructure.

[romangrothausmann]

Many thanks DRC for your valuable comments. I tried to realize some of your suggestions but did not succeed in all:

Am Freitag, 17. August 2018 20:11:57 UTC+2 schrieb DRC:

It appears from your run.sh script that both the 3D X server and the VNC
server are running on the host

Yes, both are on the host.

 

In that case, I don't really understand why all of the xauth stuff is
necessary.  Can't you just share your existing ~/.Xauthority file with
the Docker container with read/write access?  In fact, you wouldn't even
necessarily need to open up the 3D X server to all users.  You could
still use the vglusers group if you wanted.  You would just need to run
'xauth merge /etc/opt/VirtualGL/vgl_xauth_key' to add the 3D X server
cookie to ~/.Xauthority prior to starting the Docker instance.

 

Removing the vglserver config (https://gitlab.com/romangrothausmann/dfitksnap/commit/4df27e3dfeba40dfabbd8989af0c80a8ecb8bf0b) works,

and using vgl_xauth_key instead of VGL_DISPLAY=:0 (https://gitlab.com/romangrothausmann/dfitksnap/commit/14207f23c29ea7ace9eb086b5471d02cc232532d) works as well.

However, if I share ~/.Xauthority (rw) with a prior 'xauth merge /etc/opt/VirtualGL/vgl_xauth_key'

both VNC and VGL-X are not accessible (https://gitlab.com/romangrothausmann/dfitksnap/commit/5dfa12f1e5669376137ff2c6b57b5a64170eb29f).

To my understanding that originates from the missing "sed -e 's/^..../ffff/'",

whose documentation I could not find but which to my understanding removes

the binding to the host name (which is different in the docker container).

This could also be the problem I am facing when running run.sh from a vglconnect terminal,

where $DISPLAY is something like localhost:N.n which does not make sense within the

docker environment. However, just stripping localhost from DISPLAY did not make it work either.

If I've misunderstood and you are actually running one of the X servers
(3D X server or VNC server) inside of the Docker container, then my
answer would be different.  The answer I gave in the linked thread below
assumed that the 3D X server was running inside of the Docker container.
 That appears to be the case with the plumbee Dockerfile you posted.

Hm, is that so or is their config line just unnecessary as well?

If I understand the post https://medium.com/@pigiuz/hw-accelerated-gui-apps-on-docker-7fd424fe813e

correctly, it is run on a remote (AWS G2 instance) that is running a VGL X, as setup by

https://github.com/plumbee/nvidia-hw-accelerated-box/blob/master/root_setup.sh#L140

which (installs docker on the host for other use but) is not running in a docker container.

Or in another way round, https://github.com/plumbee/nvidia-virtualgl would be lacking the

command that starts the VGL X-server within the container prior to running any program with vglrun.

In your case, however, if there is no X server running in the Docker
container, then you don't need to run vglserver_config in the Docker
container at all.  It appears that your run.sh sets things up so that
the Docker container makes X connections out to the host, so that
simplifies things considerably.  In that case, your Docker container
only needs the X11 and OpenGL client libraries, plus the VirtualGL faker
libraries.  It doesn't need any of the Xorg server infrastructure.

This made me wonder if it is possible to create a docker image that would

allow a contained app to use 3D acc. from the host even for non-NVIDIA

graphics-cards such as amd-radeon avoiding nvidia-docker.

However, if I remove "--runtime=nvidia" from my docker run command the

GL context seems to be gone:

QGLTempContext: No GL capable X visuals available.

QGLContext::makeCurrent(): Cannot make invalid context current.

Any idea of how to realize this scenario?

Do I need any changes in the dockerfile to make this work?

[DRC]

On 8/20/18 6:09 AM, romangrothausmann wrote:
> Removing the vglserver config
> (https://gitlab.com/romangrothausmann/dfitksnap/commit/4df27e3dfeba40dfabbd8989af0c80a8ecb8bf0b)
> works,
> and using vgl_xauth_key instead of VGL_DISPLAY=:0
> (https://gitlab.com/romangrothausmann/dfitksnap/commit/14207f23c29ea7ace9eb086b5471d02cc232532d)
> works as well.
> However, if I share ~/.Xauthority (rw) with a prior 'xauth merge
> /etc/opt/VirtualGL/vgl_xauth_key'
> both VNC and VGL-X are not accessible
> (https://gitlab.com/romangrothausmann/dfitksnap/commit/5dfa12f1e5669376137ff2c6b57b5a64170eb29f).
>
> To my understanding that originates from the missing "sed -e
> 's/^..../ffff/'",
> whose documentation I could not find but which to my understanding removes
> the binding to the host name (which is different in the docker container).
>
> This could also be the problem I am facing when running run.sh from a
> vglconnect terminal,
> where $DISPLAY is something like localhost:N.n which does not make sense
> within the
> docker environment. However, just stripping localhost from DISPLAY did
> not make it work either.

In a remote SSH environment with X11 forwarding enabled, localhost:N.n
is the server-side endpoint of the forwarded X11 connection.
localhost:N.n is a TCP listener, which is why simply stripping
"localhost" from DISPLAY didn't work (removing the hostname causes Xlib
to attempt to make the connection with a Unix domain socket rather than
TCP.) By default, SSH will not allow connections to the endpoint except
through the loopback (127.0.0.1) interface, so Docker wouldn't be able
to connect to the endpoint by default. You could set "X11UseLocalhost
no" in sshd_config, thus allowing connections to the endpoint from
anywhere, but there are security ramifications to this. It sounds like
what you really need is the Unix domain socket forwarding feature in
OpenSSH 6.7 and later, although I have no idea how to set that up to
accommodate X11 forwarding.

> Hm, is that so or is their config line just unnecessary as well?
> If I understand the post
> https://medium.com/@pigiuz/hw-accelerated-gui-apps-on-docker-7fd424fe813e
> correctly, it is run on a remote (AWS G2 instance) that is running a VGL
> X, as setup by
> https://github.com/plumbee/nvidia-hw-accelerated-box/blob/master/root_setup.sh#L140
> which (installs docker on the host for other use but) is not running in
> a docker container.
> Or in another way round, https://github.com/plumbee/nvidia-virtualgl
> would be lacking the
> command that starts the VGL X-server within the container prior to
> running any program with vglrun.

That was just my first impression. I wouldn't be able to say for sure
without attempting to bring up their container.

> This made me wonder if it is possible to create a docker image that would
> allow a contained app to use 3D acc. from the host even for non-NVIDIA
> graphics-cards such as amd-radeon avoiding nvidia-docker.
> However, if I remove "--runtime=nvidia" from my docker run command the
> GL context seems to be gone:
> QGLTempContext: No GL capable X visuals available.
> QGLContext::makeCurrent(): Cannot make invalid context current.
> Any idea of how to realize this scenario?
> Do I need any changes in the dockerfile to make this work?

I don't think you could make this work without specific support from the
GPU vendor. My understanding is that the nvidia-docker stuff sets up an
OpenGL stack within the Docker container that allows OpenGL commands to
be passed from guest to host. Doing that in a performant way requires
hooking into the stack at the driver level, which isn't something that
can be done generically.