CCRL Rating

[Sam Jukes]

I felt the ratings on this website didn't say much and I thought it would be cool to get a CCRL rating out of this engine. 

The fact LCZ can play under time constraints means it should be allowed on the CCRL lists so I imagine it will show up soon. 

Until then I have decided to play LCZ_ID_112 against JanWillem 1.13 which has a CCRL rating of 1945. They are playing many blitz games with increments right now and will stop after 100 games. From that I will calculate a CCRL performance rating for LCZ. 

I think once you have one rating you can just find the formula to switch between the two rating systems and so can calculate the CCRL rating for all LCZ releases.

So far the score is 6-5 to LCZ_ID_112 which suggest it probably has a CCRL rating of about 2000. Still too early to tell.

[luis....@gmail.com]

This is with CPU only, right?

[Sam Jukes]

I am using my GPU

[Joules Kin]

Is leela weaker without the gpu?

[jkiliani]

Usually yes, but it really depends only on how many nodes per second your setup can evaluate. People have already managed to get excellent calculation speeds with multi-CPU multi-thread setups.

[mkh]

it is not very easy to get a comparable CCRL elo. I do myself some tests and let Lczero play against Adam 3.3 and Obender 3.2.x both rated 40/4 around 2220 elo. Before i did that i benchmarked my computer with  crafty 19.17 bh to get comparable results. But exactly  there arises a problem to get a proper CCRL rating theer is no such benchmark for a program like Lczero which uses for best strength GPU intead of CPU.
I for example did the tests on depktop replacement notebook with an gtx 980m. This is certainly not the same speed/strength that would be possible  with e.g. a GTX1080ti.

By the way the results on my computer shows Lczero id 113 plays nearly same strength as Adam and Obender. It depends a little bit what kind of blitz chess i choose. If I play the time exaclty calibrated like CCRL procedure descriped I play on my machine 40/2  and lczero is nearly exaclty same strength. If i play 40/48s+0.2s like in the excel sheet you can find in faq's it is a little bit weaker than adam and obrender. So CCRL rating with a gtx 980m  should IMHO be around 2200 . If you play faster blitz like 40/48s+0.2s it would be somewhat weaker.
 

[mkh]

40/48+0.2 is not correct I mean 48 second for the hole game +0.2s per move.

[Joules Kin]

Yeah I'm about to install the gpu version to see how it goes, but before I do, can you tell me if the "Intel HD Graphics 620" counts as a gpu?

[jkiliani]

Should certainly count as a GPU, yes. Whether you'll actually get more nodes per second with an Intel HD than with the CPU, I'm personally very dubious about that. But regardless, you can use the GPU to fully utilise your system for training game generation, if you run it in parallel to CPU clients.

[Sam Jukes]

I couldn't get the CPU version to work. I have 4CPU and a decent computer so it should be good. I have a good graphics card as well so I doubt LCZ will be weak with my GPU. But yeh I understand it is not 100% fair as I am not using exact time controls etc... I'm just doing this to get a rough idea as to the CCRL rating. 

[Sam Jukes]

I'm at work and have let it run at home while I am out. I'll post final results when I get back.

[Юрий Павлович]

Go version of LZ uses fixed 3200 visits for Elo evaluation of CGOS. This ensures same playing strength independent of hardware and very quick moves so engine wont lose on time. Yes, this will handicap already weak engine but if we want to see progress with less statistical noise it's better to use small amount of fixed visits.

вторник, 10 апреля 2018 г., 14:54:53 UTC+5 пользователь mkh написал:

[Sam Jukes]

I got home and found the result to be:

LCZ_112     59 - 41     JanWillem_1.13

JanWillem_1.13 is 1945 so the performance of LCZ_112 is 2017

[mkh]

may I ask you what hardware you used  and what kind of time control was set for the games ?