Running LCZero with multiple GPUs

[Cary Knoop]

How do I setup LCZero to run with multiple GPUs?

I have both the Fritz and Nibbler UI.

[Anil Dharan]

+1. I'd also like to know. Also, can I combine my existing 1080 Ti with 2080 Ti ?

[Brian Richardson]

run lc0.exe -h to see the command line options, or type uci after running lc0.exe to see the UCI options.

For more info see:

https://blog.lczero.org/search?q=backend

run one client.exe for each gpu.

You can use the 1080ti with a 2080ti for self-play games, but it would not be very effective for playing games using both gpus.

The 1080ti does not support fp16, so running without that will slow down the 2080ti by a lot--just use the 2080ti by itself for playing. 

[mjlitola]

I was in the understanding that multi-GPU -systems do not scale well, but it seems that I was wrong. They do seem to scale almost 100%, but it seems to do this up to only two GPUs. I have a server board that supports up to 19 GPUs, but can’t utilize it with LCZero. Anymore than two GPUs and you start getting only 30-40% increase of what the card’s original performance in nps was alone. There are no tests on multiple GPUs from GTX 1000 and RTX 2000 series mixed, but the speed increase found in fp16 would seem to limit the usage for using several GTX 1000’s or several RTX 2000’s and not mixing the different generations.

https://docs.google.com/spreadsheets/d/1lGFf6PLGmBUSMan-YP7Vul4DpRNfn6K8oeCjBILe6uA/edit#gid=1508569046

[Cary Knoop]

It is not clear how to use the command line options in the UI.

For instance, in Fritz there is a backend field under the engine options, but what EXACTLY do I put there, say for two GPUs?

[John Upper]

In Fritz 15 (as well as Chessbase 15, both of which seem to use the same UCI user interface) I have these for my RTX 2080 + GTX 1080ti:

backend: multiplexing

backendOptions: cudnn(gpu=1),cudnn-fp16(gpu=0)

NOTE: in my case, the RTX card is gpu 0, and the GTX 1080 is gpu 1. When I reverse the order of gpu 1 and gpu 0, Lc0 does not run. It seems it has to have the fp16 gpu second in the backendoptions.
 

[Cary Knoop]

Thank you John for providing this info.

It does not work for me, I have two 1080TI's so I used: 

backend: multiplexing

backendOptions: cudnn(gpu=1),cudnn(gpu=0) 

The engine does nothing after this.  Is there a log file generated somewhere I am able to look at?

[John Upper]

I don't know about log files.

Before multiplexing, test if you can get leela to run on each card separately (one at a time). If not, the problem may be that it can't find the network, or something that has nothing to do with your gpus.

[Cary Knoop]

Thanks, folks, I got it to work.

For two GPUs (2 x 1080 Ti) I used:

Backend: multiplexing

Backend Options: (backend=cudnn,gpu=0),(backend=cudnn,gpu=1)

[Anil Dharan]

Hi

    Is it possible to combine a 780 Ti and a 1080 Ti for better performance ?

[Anil Dharan]

Agreed that the 1080 Ti doesn't support cudnn fp 16 while 2080 Ti does.

BUT, if I choose cudnn for BOTH the 1080 Ti and 2080 Ti, will they work together ? I'm assuming of course, that the 2080 Ti also supports plain cudnn.

[John Upper]

They should work together, but you'd get MUCH better performance using only the 2080 with fp16 than using both at fp32 since fp16 is about 4x faster than fp32.

[Anil Dharan]

THAT'S what I wanted to know.

Thank You.

[Alexander Lyashuk]

It's possible to use 2080 in cudnn-fp16 mode and 1050 in cudnn mode at the same time.

I'm not sure what could be the benefits of using both GPUs in cudnn mode.

There is a blog post about backend configuration:

http://blog.lczero.org/2019/04/backend-configuration.html

(it can give you a hint why `cudnn(gpu=1),cudnn-fp16(gpu=0)` works, while `cudnn(gpu=1),cudnn(gpu=0)` doesn't, you need `backend=cudnn,(gpu=1),(gpu=0)` instead :)

--
You received this message because you are subscribed to the Google Groups "LCZero" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lczero+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lczero/e6bf82e2-f893-465d-bdae-928510bd45a3%40googlegroups.com.

[Anil Dharan]

Hi

   Actually I didn't know this earlier. I thought it was compulsory that both GPUs should use the same Mode for lc0 to take advantage.

So, I suppose this means that I need not discard my old 1080 Ti when my new 2080 Ti arrives and use both together for additional boost on lc0 ?

 Also, can you tell me the EXACT Commands to use in the Backend Options if a System has a 1080Ti and 2080 Ti ( and also for two 2080 Tis in round robin).

Thank You

On Thursday, October 10, 2019 at 4:57:13 PM UTC+5:30, Alexander Lyashuk wrote:

It's possible to use 2080 in cudnn-fp16 mode and 1050 in cudnn mode at the same time.

I'm not sure what could be the benefits of using both GPUs in cudnn mode.

There is a blog post about backend configuration:

http://blog.lczero.org/2019/04/backend-configuration.html

(it can give you a hint why `cudnn(gpu=1),cudnn-fp16(gpu=0)` works, while `cudnn(gpu=1),cudnn(gpu=0)` doesn't, you need `backend=cudnn,(gpu=1),(gpu=0)` instead :)

On Wed, Oct 9, 2019 at 4:39 PM Anil Dharan <anild...@gmail.com> wrote:

THAT'S what I wanted to know.

Thank You.

On Wednesday, October 9, 2019 at 7:38:20 PM UTC+5:30, John Upper wrote:

They should work together, but you'd get MUCH better performance using only the 2080 with fp16 than using both at fp32 since fp16 is about 4x faster than fp32.

On Wednesday, 9 October 2019 01:19:02 UTC-4, Anil Dharan wrote:

Agreed that the 1080 Ti doesn't support cudnn fp 16 while 2080 Ti does.

BUT, if I choose cudnn for BOTH the 1080 Ti and 2080 Ti, will they work together ? I'm assuming of course, that the 2080 Ti also supports plain cudnn.

--
You received this message because you are subscribed to the Google Groups "LCZero" group.

To unsubscribe from this group and stop receiving emails from it, send an email to lcz...@googlegroups.com.

[Alexander Lyashuk]

Indeed it's possible to mix different backends (even CUDA + CPU, or CUDA + OPENCL, for example)

How to configure it:

backend: either roundrobin or multiplexing, it's hard to guess which one will give higher nps. There is also demux which you can try but probably it's not good idea to use it in asymmetric configurations, it will be slower. But it doesn't hurt to try. Also for some of configurations (mainly demux, although it may help with others too), it may make sense to increase minibatch-size from 256 to, say, 512.

Also, optimal number of theads may be 2, 3 or 4, depending on hardware configuration and backend (roundrobin or multiplexing or demux).

As for backend-opts, here it is, the same for all cases. If GPU0 is 1080ti and GPU1 is 2080ti

(gpu=0,backend=cudnn),(gpu=1,backend=cudnn-fp16)

or if your GPU0 is 2080ti and GPU1 is 1080ti:

(gpu=1,backend=cudnn),(gpu=0,backend=cudnn-fp16)

For two 2080ti's it will be

(gpu=0,backend=cudnn-fp16),(gpu=1,backend=cudnn-fp16)

P.S. There are many ways to write the same configuration, that's why there are many variants mentioned on the forum. In the end they are all identical all work the same way.

E.g. instead of 

(gpu=0,backend=cudnn),(gpu=1,backend=cudnn-fp16)

it's possible to write e.g.

cudnn(gpu=0),cudnn-fp16(gpu=1)

and instead of

(gpu=0,backend=cudnn-fp16),(gpu=1,backend=cudnn-fp16)

it's possible to write

backend=cudnn-fp16,(gpu=0),(gpu=1)

To unsubscribe from this group and stop receiving emails from it, send an email to lczero+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lczero/b372c35f-a313-443e-a0f4-2358b2acf34a%40googlegroups.com.

[Anil Dharan]

Hi Alexander

                  Fantastic Post !

Excellent detailed explanation in simple language. That Blog Post you linked to earlier was almost impossible to understand.

Thank You so much !

Regards

To view this discussion on the web visit https://groups.google.com/d/msgid/lczero/b372c35f-a313-443e-a0f4-2358b2acf34a%40googlegroups.com.