Cutechess-cli one-liner to avoid configuration

[MindMeNot]

Here's some batch .bat scripts to launch a tournament on cutechess-cli that configures everything on the fly instead of relying on configuration files and whatnot.

You can add "-repeat" before "-recover" to repeat openings twice if you need.

Opening book is http://rebel13.nl/download/books.html

More info at https://github.com/cutechess/cutechess/blob/master/docs/cutechess-cli.6

Modify paths to reflect your setup:

=================

@echo off

SET outfile="out.txt"

echo Cutechess started at %date% %time%. Output is redirected to %outfile%

echo Cutechess started at %date% %time% > %outfile%

cutechess-cli.exe -tournament round-robin -rounds 40 -games 1 -concurrency 1 -pgnout out.pgn -recover ^

-resign movecount=2 score=500 -draw movenumber=100 movecount=3 score=50 ^

-engine name=SFDev nodes=1000000 option.Threads=2 option.Hash=1024 cmd=engines\SFDev\stockfish_18090410_x64_bmi2.exe ^

-engine name=Lc0.11262 nodes=3000 cmd=engines\Lc0\Lc0.exe arg="--weights=C:\Users\[redacted]\Desktop\Cutechess\engines\lc0\11262.pb" ^

-engine name=Lc0.11089 nodes=3000 cmd=engines\Lc0\Lc0.exe arg="--weights=C:\Users\[redacted]\Desktop\Cutechess\engines\lc0\11089.pb" ^

-engine name=Lc0.11198 nodes=3000 cmd=engines\Lc0\Lc0.exe arg="--weights=C:\Users\[redacted]\Desktop\Cutechess\engines\lc0\11198.pb" ^

-engine name=Lc0.11250 nodes=3000 cmd=engines\Lc0\Lc0.exe arg="--weights=C:\Users\[redacted]\Desktop\Cutechess\engines\lc0\11250.pb" ^

-each proto=uci timemargin=10 tc=inf book=book\bookfish.bin bookdepth=2 dir=C:\Users\[redacted]\Desktop\Cutechess >> %outfile%

echo Tournament ended at %date% %time%.

echo Tournament ended at %date% %time%. >> %outfile%

pause

=================

Btw in case you are curious this was the result table of that particular tournament:

Rank Name                         Elo     +/-   Games   Score     Draws

   1 Lc0.11198                       7      35     160       50.9%     58.1%

   2 SFDev                            2      39     160       50.3%     48.1%

   3 Lc0.11250                       0      34     160       50.0%     60.0%

   4 Lc0.11262                       0      34     160       50.0%     60.0%

   5 Lc0.11089                      -9      31     160       48.8%     66.3%

I also tested the CCRL dataset 128x10 network published in the Lc0 blog (blog.lczero.org/2018/09/a-standard-dataset.html). Here's the batch file:

=================

@echo off

SET outfile="out.txt"

echo Cutechess started at %date% %time%. Output is redirected to %outfile%

echo Cutechess started at %date% %time% > %outfile%

cutechess-cli.exe -tournament round-robin -rounds 100 -games 1 -concurrency 1 -pgnout out.pgn -recover ^

-resign movecount=2 score=500 -draw movenumber=100 movecount=3 score=50 ^

-engine name=Lc0.cclr nodes=6700 cmd=engines\Lc0\Lc0.exe arg="--weights=C:\Users\[redacted]\Desktop\Cutechess\engines\lc0\128x10-base-200000.pb" ^

-engine name=Lc0.11198 nodes=1100 cmd=engines\Lc0\Lc0.exe arg="--weights=C:\Users\[redacted]\Desktop\Cutechess\engines\lc0\11198.pb" ^

-each proto=uci timemargin=10 tc=inf book=book\bookfish.bin bookdepth=2 dir=C:\Users\[redacted]\Desktop\Cutechess >> %outfile%

echo Tournament ended at %date% %time%.

echo Tournament ended at %date% %time%. >> %outfile%

pause

=================

and it's... very bad:

Score of Lc0.cclr vs Lc0.11198: 1 - 98 - 1  [0.015] 100

Elo difference: -726.94 +/- nan

One question, if i use the following script that has time control (tc=30/30+1) instead of nodes some games result in 11198 losing on time. anybody knows why?

=================

@echo off

SET outfile="out.txt"

echo Cutechess started at %date% %time%. Output is redirected to %outfile%

echo Cutechess started at %date% %time% > %outfile%

cutechess-cli.exe -tournament round-robin -rounds 10 -games 1 -concurrency 1 -pgnout out.pgn -recover ^

-resign movecount=2 score=500 -draw movenumber=100 movecount=3 score=50 ^

-engine name=Lc0.cclr cmd=engines\Lc0\Lc0.exe arg="--weights=C:\Users\[redacted]\Desktop\Cutechess\engines\lc0\128x10-base-200000.pb" ^

-engine name=Lc0.11198 cmd=engines\Lc0\Lc0.exe arg="--weights=C:\Users\[redacted]\Desktop\Cutechess\engines\lc0\11198.pb" ^

-each proto=uci timemargin=10 tc=30/30+1 book=book\bookfish.bin bookdepth=2 dir=C:\Users\[redacted]\Desktop\Cutechess >> %outfile%

echo Tournament ended at %date% %time%.

echo Tournament ended at %date% %time%. >> %outfile%

pause

=================

Started game 1 of 10 (Lc0.cclr vs Lc0.11198)

Finished game 1 (Lc0.cclr vs Lc0.11198): 1-0 {Black loses on time}

[MindMeNot]

Surprise surprise, 20* nets are not very good yet:

Rank Name          Elo      +/-   Games   Score     Draws

   1 Lc0.11198       255      68     120      81.3%    19.2%
   2 SFDev            228      67     120      78.8%    17.5%
   3 Lc0.20595      -232      72     120      20.8%    10.0%
   4 Lc0.20712      -250      73     120     19.2%     11.7%

Alternative ratings courtesy of bayeselo.exe:

Rank Name         Elo       +      -    games   score   oppo.   draws
   1 Lc0.11198      234     54    51     120       81%     -78      19%
   2 SFDev           222     54    51     120       79%     -74      18%
   3 Lc0.20595     -221     53    57     120      21%      74       10%
   4 Lc0.20712     -235     54    58     120      19%      78       12%

The script used to run the tournament:

==============
@echo off
SET outfile="out.txt"
echo Cutechess started at %date% %time%. Output is redirected to %outfile%
echo Cutechess started at %date% %time% > %outfile%

cutechess-cli.exe -tournament round-robin -rounds 40 -games 1 -repeat -concurrency 1 -pgnout out.pgn -recover ^

-resign movecount=2 score=500 -draw movenumber=100 movecount=3 score=50 ^

-engine name=SFDev nodes=300000 option.Threads=1 option.Hash=1024 cmd=engines\SFDev\stockfish_18090410_x64_bmi2.exe ^
-engine name=Lc0.11198 nodes=1000 cmd=engines\Lc0\Lc0.exe arg="--weights=C:\Users\[redacted]\Desktop\Cutechess\engines\lc0\11198.pb" ^
-engine name=Lc0.20595 nodes=1000 cmd=engines\Lc0\Lc0.exe arg="--weights=C:\Users\[redacted]\Desktop\Cutechess\engines\lc0\20595.pb" ^
-engine name=Lc0.20712 nodes=1000 cmd=engines\Lc0\Lc0.exe arg="--weights=C:\Users\[redacted]\Desktop\Cutechess\engines\lc0\20712.pb" ^