Run level 3 app server

[eresrch]

I sort of have the 2D rendering working between two linux systems using VGL.  The application runs on a large server which is at run level 3, so there is no active X server running.  Using vglconnect to the server and then vglrun -d localhost:10.0 <app> I get all the 2D images just fine, but no 3D renderings.  This makes some sense because the server isn't running an X server at all.  But it does have V100 GPU's, so it could do 3D rendering.

I have a second, smaller server which does run at level 5 and also has good graphics cards.  I don't have local access to it.  So I'm wondering if it makes sense at all (probably not, but I'm asking) if it's possible to use the X server on a different system than the application and send the rendered results to a 3rd system.  This is similar to the figure under section 9.2 of the user's guide, except that the GPU driver and 3D X server would be on the X proxy host.

Alternatively, is there a way to set up a virtual X server which uses the local GPU, but does not connect to anything else except VGL requests? 

How I got here:  I attempted to put the large server up to run level 5, and it hosed the system so no one could log into it.  Power cycle was the only recourse.  It's designed for our network to live at run level 3, so that's where it stays.

I really don't understand all the details of X, VGL or ssh tunneling so my assumption is that this is not simple.  It would be great if there is a way to take advantage of the hardware for graphics as well as for the compute power.

Thanks.

[DRC]

On 3/25/20 4:59 PM, 'eresrch' via VirtualGL User Discussion/Support wrote:
> I sort of have the 2D rendering working between two linux systems
> using VGL.  The application runs on a large server which is at run
> level 3, so there is no active X server running.  Using vglconnect to
> the server and then vglrun -d localhost:10.0 <app> I get all the 2D
> images just fine, but no 3D renderings.  This makes some sense because
> the server isn't running an X server at all.  But it does have V100
> GPU's, so it could do 3D rendering.
>
> I have a second, smaller server which does run at level 5 and also has
> good graphics cards.  I don't have local access to it.  So I'm
> wondering if it makes sense at all (probably not, but I'm asking) if
> it's possible to use the X server on a different system than the
> application and send the rendered results to a 3rd system.  This is
> similar to the figure under section 9.2 of the user's guide, except
> that the GPU driver and 3D X server would be on the X proxy host.

No, that is not possible with VirtualGL.  The application must be
running on the same machine as the 3D X server and GPU.  What you're
proposing would involve some form of indirect OpenGL rendering, needed
in order to send the OpenGL commands from the application server to the
server containing the GPU.  Conceivably, the OpenGL commands could be
forwarded using remote X11 via an SSH tunnel, but since indirect OpenGL
via remote X11 doesn't support OpenGL > 1.4, it wouldn't work with most
modern OpenGL applications.  The only other option would be for the
VirtualGL Faker to intercept every OpenGL command from the application
(instead of just every GLX command) and forward it to the GPU server
using a custom OpenGL wire protocol.  That would be a huge undertaking
to develop, not to mention a compatibility and performance minefield. 
Also, even if we created our own OpenGL wire protocol, it still wouldn't
be possible to support all modern OpenGL features using indirect
rendering.  From a project management point of view, such a solution
would also be much more complex than I'm capable of developing and
maintaining on my own (bearing in mind that VGL is one of three open
source products I maintain for a living.)  The only solution I know of
that does what you propose is NICE DCV (now owned by Amazon), but it is
proprietary, and there are some limitations (performance being a big
one) to using a GPU on a different machine than the application.

> Alternatively, is there a way to set up a virtual X server which uses
> the local GPU, but does not connect to anything else except VGL
> requests? 

A "virtual" X server, by definition, doesn't use a GPU at all (Xvfb, for
instance.)  You can set up a "headless" X server (without a physical
display) that uses the GPU.  It should be possible to start such a
headless X server in Run Level 3.  To mimic the supported behavior of
vglserver_config, you would create an XAuth cookie under
/etc/opt/VirtualGL/vgl_xauth_key, instruct the headless X server to use
that cookie, and set the permissions of the cookie so that only certain
users (members of the vglusers group, for instance) can access it.  But
there isn't any way to limit access to an X server on an
application-by-application basis.  Access can only be limited on a
per-user basis.

[Mike Rosing]

Absolutely awesome answer! I guess "hard to do" is an understatement.
I will give the second idea a try. I only need to limit by user, not by
app, so that's OK. I'm sure I'll have more questions, and I hope I will
ask before I screw it up so bad only reboot saves the system.
Thank you!

On Wed, 25 Mar 2020, DRC wrote:

> Date: Wed, 25 Mar 2020 19:02:13 -0500
> From: DRC <d...@virtualgl.org>
> Reply-To: virtual...@googlegroups.com
> To: virtual...@googlegroups.com
> Subject: Re: [VirtualGL-Users] Run level 3 app server

> --
> You received this message because you are subscribed to the Google Groups "VirtualGL User Discussion/Support" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to virtualgl-use...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/virtualgl-users/42500406-7281-c782-e896-a7aaeb94192e%40virtualgl.org.
>

[Jason Edgecombe]

Hello,

At work, we run several Linux servers in mult-user mode (runlevel 3) with a headless X service in the background. Two of them have nVidia Tesla V100 3D cards the others run nViidia Quadros. The remote graphical logins are handled by StarNet FastX. All of the servers have integrated graphics cards with an additional Nvidia or AMD card for server-side 3D rendering.

Here is the bash code to add the headless X service on DISPLAY  :1 (run code as root)

==========================cut===================

umask 022 #not sure if this is needed

cat  > /etc/systemd/system/headlessx@.service << 'EOF'
[Unit]
Description=VirtualGL Headless X server on display %I
After=multi-user.target

[Service]
Type=simple
ExecStart=/bin/bash -c "Xorg :%i  -listen tcp -config /etc/X11/xorg.%i.conf"
Restart=on-failure

[Install]
WantedBy=multi-user.target

EOF

systemctl daemon-reload

systemctl enable headlessx@1

# set VGL_DISPLAY = :1 to use the correct graphics card

cat > /etc/profile.d/virtualgl.sh  << 'EOF'
# Configure the default DISPLAY for VirtualGL vglrun
export VGL_DISPLAY=':1'

EOF

==========================cut===================

The above config does NOT include the Nvidia or VirtualGL setup and is probably missing some other supporting configs.. Let me know if you have more questions about my setup.

Sincerely,

Jason

---------------------------------------------------------------------------
Jason Edgecombe | Linux Administrator
UNC Charlotte | The William States Lee College of Engineering
9201 University City Blvd. | Charlotte, NC 28223-0001
Phone: 704-687-1943
jwed...@uncc.edu | http://engr.uncc.edu |  Facebook
---------------------------------------------------------------------------
If you are not the intended recipient of this transmission or a person responsible for delivering it to the intended recipient, any disclosure, copying, distribution, or other use of any of the information in this transmission is strictly prohibited. If you have received this transmission in error, please notify me immediately by reply e-mail or by telephone at
704-687-1943.  Thank you.

To view this discussion on the web visit https://groups.google.com/d/msgid/virtualgl-users/alpine.LRH.2.21.2003251823550.31125%40centos7.eskimo.com.

[Mike Rosing]

The difference would be we don't have graphics cards for server-side 3D
rendering, I will have to steal GPU cycles from the V100's. That should
not matter, other than I'm losing compute power for visual capability.

I guess I'm learning way more than I wanted to about X servers. I should
be able to understand and modify your script for our system. It should be
pretty damn close to the same thing, but even 1 bit wrong can cause a lot
of problems.

Very much appreciated, thank you!
Mike

On Thu, 26 Mar 2020, Jason Edgecombe wrote:

> Date: Thu, 26 Mar 2020 10:44:04 -0400
> From: Jason Edgecombe <jwed...@uncc.edu>

> To view this discussion on the web visithttps://groups.google.com/d/msgid/virtualgl-users/alpine.LRH.2.21.2003251823550.31125%40ce
> ntos7.eskimo.com.

>
> --
> You received this message because you are subscribed to the Google Groups "VirtualGL User
> Discussion/Support" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to
> virtualgl-use...@googlegroups.com.

> To view this discussion on the web visithttps://groups.google.com/d/msgid/virtualgl-users/CAAR6MGAMMnd_mxx1TKES8-PH8Wwb3ws0ZBp2rVW
> 4h-m8nxcaQQ%40mail.gmail.com.
>
>