I ran an overnight test - Leela beat Stockfish 9 64-bit!

[Margus Riimaa]

Processor Intel i7-8700 CPU @ 3.20GHz, 6 Core(s), 12 Logical 

Memory 16GB

Graphics         1080Ti

Storage 230GB SSD

Hash Engine Default Settings: Stockfish-32MB, Lc0-128M

Ponder ON

Syzygy TB 5-man

Time Controls 2-min match + 2 sec per move

Match Adjust. Draw after 60 moves or win if eval > 500 centipawns - better ideas welcome!!! Sadly there is no option to continue depending on the eval of engines.(Personally find it quite stupid)

GUI         ChessOK Aquarium 2011

Opening Book Max 4 moves from a Rybka 4 book

Engines:         Lc0 v0.17.0-rc2 Net-ID Don´t know where to look - 8.27.2018 - 11175???

        Stockfish 9 64-bit

Match arrang. Each opening played twice with switched sides

Games played 60

Score: Lc0 31

        Sf9 29

Games are available for download as an attachment.

[Paul Berger]

This is great but I suspect that such quick games are actually favoring Leela (SF has better scaling), anyway, thanks for the games :-)

[Margus Riimaa]

You are welcome!

This is quite possible, that SF scales better, however I would like to point out an observation I made during the games regarding the tablebases - SF searches massively quicker (roughly 120 times) than Lc0 and achieves much much greater depths under such short games than Leela does (sometimes Leela has depth 7 when SF has 27, or over 30, in some cases if I recall it correctly)  and reaches tablebases way earlier. SF "touches" the TB in the middlegame, while Lc0 reaches them much later in the endgame. So it seemed to me at least, that TB benefit SF more in this case.

[Jonathan Rosenthal]

If SF is only getting 120 times the nps, then you have a Leela ratio around 7.3 on your setup. Most probably you are running SF on only one of your cores, which you should specify and be aware that your setup is very GPU heavy.

You should give SF more than 32mb of hash, UCI engines are supposed to have a low default value for hash and 32mb is very low. I am less familiar with Leela's hash needs, but I am assuming you can go up more than the default 128mb as well. You have 16gb ram, use some of it.

I personally am not a fan of ponder on matches in general, but if you are intending to handicap SF to a single core than you can do it. More issues start arising when you want comparable results between GPU engines or engines where you give access to more of your hardware. I understand that some people feel this is more like in human games, but humans don't share brains and other organs ;) If you are running 2 completely separate systems, ponder makes more sense.

You should be aware that 1 node in Leela means something different than 1 node in SF and depth in Leela means something different than depth in SF, so avoid comparing them as they are absolutely apples to oranges. In general these numbers are not even well defined or comparable amongst AB engines, let alone with Leela.

Probably your biggest mistake is adjudicating to draws after 60 moves. That is a very small number and you will be adjudicating many games which may have ended in decisive results in either direction.

[Margus Riimaa]

Thank You, Jonathan!

I was completely unaware that I ran Sf on just only 1 core. Now I checked - indeed, threads 1. My bad.

And thanks for the other info aswell!

Now I learned something.