Help with configuring Leela (2 Geforce RTX 2080 Ti graphics cards)

[Metaphysician]

Hello. I am wondering if I could get some advice from the experts here.

I have just installed 2 Geforce RTX 2080 Ti graphics cards on my workstation,, which has two Xeon processors and a total of 24 cores.  As far as I can tell, the graphics cards, which were installed by a friend who is an expert at these things, are operating properly; Windows task manager says they are.

After flailing around a bit, I seem to have succeed in downloading and installing Leela.

My problem is that Leela seems to be running much more slowly than it should.  I am running it in the Chessbase interface as a UCI engine.  After about a minute of letting Leela analyze the starting position, I am getting a depth of 18 and am at 37 kN/s  (Now, a few minutes later, I am at 26 kN/s.  .Lower in the engine information panel I saw 6306kN after a short while; now, some minutes later, it says 17448kN, at a depth of 19.

When I open Task Manager, I see that one of my two graphics cards is working at about 5% or less, and the other at 0.

What am I doing wrong?  Leela has so many parameters to play with (backend, backendOptions . . .), and I have no idea what values to input.  Naively, I thought that the program would fly on my machine with my two new graphics cards, but obviously I don't know what I am doing.

Can anyone please offer help?

Thank you.

[Hoang Hiep Vu]

Hi,

I don't know what is wrong, but there are following "standard" steps:

- Download lc0 cuda version: https://github.com/LeelaChessZero/lc0/releases/download/v0.21.0/lc0-v0.21.0-windows-cuda.zip

- With RTX GPU, if you want to use lc0 with the first GPU, just use: "--backend=cudnn-fp16 --backend-opts=gpu=0" (change gpu=1 to use the second GPU). You can use both GPU in the same time, but it is different parameter value. Just test with one GPU to see if it can attain 100%. Test with T40, it should attain near 100%

Moreover, use GPUZ to see GPU workload (https://www.techpowerup.com/gpuz/). If you use Task Manager, by default, it will display "3D" workload that has nothing to do with lc0, you need to change to "compute_0". 

I guess there is nothing wrong with your gpu, 37 kn/s is expected, just the Task Manager displays wrong information.

[Metaphysician]

Thank you so much for your helpful reply!

First, I do have the CUDA version, as I want Leela to make use of my graphics cards.  I will take your suggestion and download GPU-Z.

A couple of questions:

I DO want to use both graphics cards.  So in the "Backend" parameter, which of the various options do I choose?

If I wish to use both graphics cards, what exactly do I type in the "BackendOptions" parameter?  Could you please write it down for me without abbreviations?  (I am hopeless with software / computers, but I can follow directions if they are idiot-proof.)

What do you recommend I set "Threads" to?  It if matters, I have two Xeon processors, with a total of 24 cores.

The sad fact is that I do not know what "T40" is.  What is it, where do I find it, and how do I test with it?

Finally, about your sentence ending "you need to change to "compute_0":  What exactly do I need to change to "compute_0"?

Many thanks once more for all your kind help.

Best wishes,

Meta

[gvergh...@gmail.com]

To check gpu usage, use GPU-Z instead of the task manager :)

For T40 nets, download the 41xxx networks from here -- https://lczero.org/networks/

And for your twin gpu's, I would suggest using fp16 with roundrobin -- it alternates iterations between both gpus.

Test between 2 to 3 threads for faster nps.

The command-line would be something like

   --backend=roundrobin --backend-opts=(backend=cudnn-fp16,gpu=0),(backend=cudnn-fp16,gpu=1)

It would be easier to use the lc0.cfg. The engine would load all the settings from the cfg file.

Create a lc0.cfg in the engine dir, open with notepad and add these settings.

One command line option per line and the "--" is optional.

For example, an lc0.cfg could look like this...

--threads=3

--backend=roundrobin

--backend-opts=(backend=cudnn-fp16,gpu=0),(backend=cudnn-fp16,gpu=1)

--cpuct=2.5000

--syzygy-paths=X:\sygygy

Good luck :)

[Hoang Hiep Vu]

If you want to use Task Manager instead of GPU-Z, select "compute_0" as the attached file.

[Metaphysician]

Thank you so much for your help.

I've used your recommended parameters.  

I am using the engine in ChessBase.  I am getting about 21kN/s and, after about 12 minutes analyzing a middlegame position, a depth of about 25.  And at the bottom of the engine output screen, to the right of the elapsed time, I see 16220kN.

Do you think I am getting the results I should from 2 Nvidia Geforce RTX 2080 Ti graphics cards?  It not, do you have anyideas about how to do better?  (I have 2 Xeon processors and a total of 24 cores, if this matters.)

And if not, can you please recommend settings if I want to use just a single graphics card?

Thank you again for all your kind help!

Meta   .

[gvergh...@gmail.com]

I don't own those gpus - visit the Leela discord channel and ask about the benchmarks.

https://discordapp.com/channels/425419482568196106/425419483016855563

Start an infinite analysis session and open gpu-z.

Check the usage of both gpus.

If both are running in the range of 75% - 100%, then everything is set.