the truth about leela and stockfish

[eahit...@gmail.com]

as i already said many times before - the only way for games between sf and leela to be fair is to turn the fish into a gpu using program, then make it play against leela on the same hardware (gpu). otherwise, games between sf and leela can NEVER be fair

the latest version of leela - ID454 - is 6013.66 after 19, 194, 035 games. this means that leela now has an elo of 3096.496 

according to TCEC, stockfish 180614 has an elo of 3519. so leela is still far behind the fish. games between sf and leela, at this stage, can NEVER be fair, even on the same gpu hardware (assuming sf uses gpus) 

for leela to be as strong as sf, she must be around 6718

[Edward Panek]

SF can never be as strong as Carlsen until we can somehow put Magnus' brain into a CPU. I wish there were some way to determine which is stronger ; Magnus or SF?

[Salah Abbas]

For NN GPU is very efficient to run. But on chess engines GPUs aren't better. It just comes down to the type of calculations. Chess Engines have a lot of algorithms that tell it what would be the best move in a certain situation. That's the same with Leela, but it is a NN, not the same as a traditional chess engine. GPUs are great for machine learning and they aren't great for recursion and floating. If you put Stockfish on a GPU, and run it, it would way slower in terms of kn/s and would have a lower elo.

[Joe Thornber]

On Saturday, 30 June 2018 15:46:38 UTC+1, eahit...@gmail.com wrote:

as i already said many times before - the only way for games between sf and leela to be fair is to turn the fish into a gpu using program, then make it play against leela on the same hardware (gpu). otherwise, games between sf and leela can NEVER be fair

You are correct; 

- any comparison between Leela and Stockfish needs to specify the hardware that it was run on otherwise it is meaningless.  

- Leela and stockfish are never going to run on the same hardware. 

- Stockfish is currently a lot stronger than Leela (any hardware).

Everybody is aware of this.  Why does it upset you so?

[telsyak]

In my opinion, the goal is get the strongest chess program to run in a top desktop/laptop pc (a intel i9 cpu + nvidia titan v gpu, for example), running in the same computer is fair enough for me. Video games programs use both cpu and gpu, if a chess program  cannot use all hardware avaliable is due to algorithm limitations but all hardware is ready for be used. Some programs not use all resources and still compared them in rating list, for example, single processor version, not tablebases support, hash size limitation, etc.

[ovi...@gmail.com]

If at some moment Leela becomes stronger than traditional engines (using whatever is necessary) I guess grandmasters will use it despite unfairness... Is a GPU so expensive?

[fkra...@gmail.com]

Build two computers and limit them to 1000 Watts.

One customized for SF and the other for Leela.

[graci...@gmail.com]

The only fair match and most people would really care is to use the price of the hardware as a leveling field. So if stockfish is running on a 1000 dollar cpu then to be fair leela should run on a thousand dollar gpu too.

Let's face it, most people only care how strong the engine runs on their personal rig and what hardware/engine configuration gives the best elo for the money. No one really gives a damn about sf running on supercomputers and/or leela/a0 on some top secret TPU, it's just all a matter of speculation. 

[Edward Panek]

This is correct. For the average person only $$ is a factor. Deep Blue played incredible chess back in its day but noone had millions of dollars. Today SF plays very well and its free. Imagine a version of SF came out that ran on GPU and was 10X as strong and could be run on a 1050GTX for $200. It would be big big news.

[Lothar Jung]

Recenty on my 1.000 Watt-PC (Ryzen 1800X, 3.800 MHz, 2x GTX 1080, OC 150/500) Lc0 454 won against SF9 on 4 Cores with 3:2:5 in a Blitz-Match with 5min./sec. TC.

[Nabil Danial]

A $600 system optimized for Stockfish will beat a $600 system optimized for Leela any day of the week but a $60000 system optimized for Leela might beat a $60000 system optimized for Stockfish. 

[Edward Panek]

GPU performance is kicking CPU performance tho.

[Francesco Tommaso]

I think this is the whole point. There has been some questioning about Leela's potential and if it will ever beat Stockfish. I think this chart is more than enough to dissolve such doubts.