CCRL Benchmark Net

[Dietrich Kappe]

I can never lay my hands on a CCRL data trained net when I want one. I've made a 128x10 classical value head and convolutional policy head net available. https://github.com/dkappe/leela-chess-weights/wiki/CCRL-Net

It looks to be about 200 elo weaker than Bad Gyal 7, which is surprising to me.

[Alejandro de Diego]

Thank you for your great contributions!!!😀

[Brian Richardson]

Thank you for sharing more great work.

I had trained a CCRL net using the blog yaml except the value weight was set to 1.0.

If I can find it again (sigh), I'll run a match vs yours (value 0.25)

Separately, I'm wondering about RL v SL.  I tend to think of SL as training from pgn input.

Of course, nets trained without policy info (beyond only the one move played) are many hundreds of Elo weaker.

Another way to think of SL is not using the more rapid training cycle with many nets and some regression test like what Lc0 does.

If a net is trained with policy data (either from pgn with it added, or from Lc0 self-play games with it already included) in a longer training run (many more steps and samples), does that count as SL?

Perhaps a name for something in-between would be helpful?

[Alexey Eromenko]

CCRL seems to use GTX 1050 which is wholly inadequate for running Leela. One needs an RTX GPU, due to it being 4x times faster for Leela ! (Due to tensor cores)

[Dietrich Kappe]

So, this is a net trained on 2.5 million CCRL pgn games (ab engines going head to head in 40/4 and 40/40). It has nothing to do with CCRL and it’s gtx 1050.

[Brian Richardson]

On Wednesday, August 28, 2019 at 10:56:57 PM UTC-4, Alexey Eromenko wrote:

CCRL seems to use GTX 1050 which is wholly inadequate for running Leela. One needs an RTX GPU, due to it being 4x times faster for Leela ! (Due to tensor cores)

CCRL scales time controls in an effort to be consistent:

"Time control: Equivalent to 40 moves in 40 minutes on Athlon 64 X2 4600+ (2.4 GHz), about 15 minutes on a modern Intel CPU."

While this is reasonable for A/B engines, of course it does not apply to testing NN engines.

Nonetheless, testing NN engines at all is helpful, and CCRL does provide some reference points.

The "CCRL Standard Dataset" is a collection of games used to provide a baseline for net training per this blog post:

http://blog.lczero.org/2018/09/a-standard-dataset.html