LcZero on Picochess

[Henri Spijkerman]

Much to my liking I found out that the LcZero engine can be installed on the Pi in Picochess. It plays an interesting game of chess, not strong as it is mainly a CUDA-driven engine, but highly original and more humanlike than others.

I've tried to use stronger and more recent weight-files, like 11258 and the dkappe distilled networks, but they don't work.

Is it possible to compile a newer version that can do that trick? I would _love_ that! :)

[Scally]

Hi Henri,

Yes I’ve been following this Project since the earlier Lczero, it’s now called Lc0.
The latest official release is Lc0 v0.21.0 and v0.22.0 is in dev.

As you say the best networks to use on the Raspberry Pi are the distilled networks as these are smaller and faster on the RPi. I have all versions working, but different versions need different weights file versions. Attached is a chart I knocked up for someone else who asked me the same thing.

I estimate it’s about 2400 elo on the Raspberry Pi 3, but it plays a woeful endgame even when referencing the picochess 3-4 men endgame tablebases.

Over the next few days I can add the necessary files so that anyone with terminal access can add them.

Lc0 is now the 2nd strongest engine behind Stockfish, but will overtake it soon. This is only true however when running it on a high end GPU, it’s much weaker on CPU based systems and weaker still on the RPi.


Cheers,

Al.

[Henri Spijkerman]

Op maandag 1 april 2019 13:12:07 UTC+2 schreef Scally:

Hi Henri,

Over the next few days I can add the necessary files so that anyone with terminal access can add them.

That would be wonderful! :)

dkappe himself hinted that his 24x3-se network might be the best for pi:

http://talkchess.com/forum3/viewtopic.php?t=69645&start=20

Greetings,

Henri

[Scally]

Hi Henri,

I use the 112x9 which topped his little tournament.
Someone also kindly converted the 9149 weights file for me which works well on the latest versions of Lc0 too.

Al.

[Marco Baiocco]

Sounds interesting. Expecially the fact that it plays "more like a human".

Is this easily configurable in terms of playing strength, once installed? Or do you have to mess a lot with nn weights?

If it is, it would be nice if someone could package it for the Raspberry, like having the code and the installation instruction (which I assume would be similar to those explained in another thread to install any engine on Picochess)...

Marco

--
You received this message because you are subscribed to the Google Groups "PicoChess" group.
To unsubscribe from this group and stop receiving emails from it, send an email to picochess+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Scally]

Hi Marco,

I do a static compile with Lc0 so you won’t need to install any other required programs. I will supply the binary and uci file together with an example engine.ini file and instructions. All you will need to do is download a weights file of your choice, again instructions and an example will be given.

My equipment is still packed away after a holiday and my immersion heater has failed meaning no hot water. So I’m not in the position to do this immediately, but hopefully by the end of the week.

Al.

[Marco Baiocco]

Hi Al,

Sure, there is no hurry at all... I would do myself if only I had the time (and will, I confess) to install the compiler and all the stuff. :P

I can wait, I won't die if I don't play Lc0 in a week. :)

I am interested expecially in low level human play, which is something quite difficult to realize, in my opinion.

Marco Baiocco

[Scally]

Hi all,

Here is a link to download a folder containing all you will need to add Lc0 to your Picochess Engines: http://www.filedropper.com/addlc0
I tried attaching it here, but the file is too large. You will need to be able to login to your Raspberry Pi and enter 4 commands using a terminal. I use my iPad and an App called Termius to enter my commands and an App called Documents to upload and download files to and from my Raspberry Pi.

Once you have copied the attached file to your Raspberry Pi you will need to type these commands:

unzip addlc0

cd addlc0 (Change your current directory to the newly added folder)

sudo chmod a+x addfiles (Give execution permissions to run the file)

./addfiles (This will copy all required files to /opt/picochess/engines/armv7l, preserve your current engines.ini to engines.ini.save, create a new engines.ini to include lc0v0210 & reboot your system)

I have included a weights file (it’s learning file) which I use, but if you wish to try your own, these distilled weights have been purposely built for slower CPU based systems like the Raspberry Pi:
https://github.com/dkappe/leela-chess-weights/wiki/Distilled-Networks
I prefer the ‘ID11258-112x9-se’ file, this will need to be copied to /opt/picochess/engines/armv7l and my provided ‘weights9149.converted.txt’ file will need to be moved elsewhere.
You can also try the newest weights files here:
https://lczero.org/networks/
These are really best on GPU based systems, they will take a while to load on a RPi and will probably play weaker because of the slow nodes per second due to the size of these weights files. I find the converted file and distilled networks mentioned above much better, but please try yourselves.

If your Raspberry Pi or DGT Pi does not have a cooling fan I would not recommend choosing 4 cores as your level with Lc0.


Have fun,

Al.

[Henri Spijkerman]

Hi Al,

My expectations were high (I have to confess) but you managed to beat them flat down. Thank YOU for a wonderful, adjustable en fascinating Lc0 engine-package. It works like a charm (of course...) and gives me lot of chances to experiment! Great stuff!

Greetings,

Henri

[Marco Baiocco]

Hi Al,

Thanks for the package and the instructions!

I installed the engine on my raspberry, the script really does it all for you, thanks in name of all the lazy people of the world!

I have a question and an issue, though.

Question: how do I set the strength of the system? I guess the weights file changes the way the engine plays... and how do you set "how weak it can be"? Except for the number of cores?

Issue: The engine actually appears in the list of selectable engines and I can select it. If I do, it asks me how many cores to use. However when I confirm the number of cores, picochess appears to freeze: Any button I press won't do anything, and also moving chess pieces seems to make any difference. Picochess doesn't answer, and if I try to move a piece on the board, nothing happens. What am I doing wrong?

Perhaps I should uncomment any of the ; parameters you set in the engines.ini?

Thanks

Marco Baiocco

[Scally]

Hi Henri,
I’m glad you’re happy, enjoy experimenting.

Hi Marco,

Unfortunately it’s one of those engines without levels, the strength is governed by the weights file, so if you use a weaker weights file the engine will be weaker.
I added the 4 levels to control wether it uses 1, 2, 3 or 4 cores so a slight increase in strength.

I did test my files before I uploaded them.

If yours isn’t working it’s 1 of 2 things.

1) The weights file is missing or in the wrong format (the weights file you download must not be unzipped but can be renamed to anything you like and must be placed in the /opt/picochess/engines/armv7l folder)

2) The engines.ini file is not correct.

You can check option 1 by typing these commands:

cd /opt/picochess/engines/armv7l
./lc0v0210
uci
go infinite

This will load the weights file and start to list the moves, if the weights file is missing it will tell you.

Hit control and c to cancel the engine.

For option 2 type:

cd /opt/picochess/engines/armv7l
cat engines.ini

There should be a space between your last engine and the lc0v0210 entries, similar to:

name = ......
small = ......
medium = ........
large = ........
elo = xxxx

[lc0v0210]
;Threads = 2
;VerboseMoveStats = False
Etc..

Just add this space via:

sudo nano engines.ini
Scroll to the the above lines, add a space then crtl-x and y to save.

Al.

[Marco Baiocco]

Hi Al,

Got it, is there a way to know which network weight is weaker than the other? I don't need the elo but a rough relative idea...

About the issue, except for the fact that since I normally work on Unix systems where nano is not available and I therefore prefer vi:

- As you guessed there was no space between [lc0v0210] and the previous parameters. I checked this first because I actually noticed when checking engines.ini on my own. I would have said that if parsing the engines.ini was the problem, picochess wouldn't even have identified [lc0v0210] as a separate engine, but... Okay, I added the empty space.

- I knew that the weight file was ok, your script just copied it in the right directory, and it was already there. However I tried to launch the engine and found the issue.

I get this error message:

root@raspberrypi:/opt/picochess/engines/armv7l# ./lc0v0210

./lc0v0210: error while loading shared libraries: libopenblas.so.0: cannot open shared object file: No such file or directory

Why am I missing a shared library that all of you have in the "pre-imaged" raspberry??

Thanks

Marco

Marco Baiocco

[Marco Baiocco]

Okay,

I installed the Openblas library, and now Lc0 works on my Raspy too!

Still wondering why I was missing that library while you were not but, hey, who knows!

Cheers

Marco Baiocco

[Scally]

Hi Marco,

From what I understood, a static compile means you don’t need to add the required libraries.

However in the root directory just type:
sudo apt-get install libopenblas-dev

And everything should work.

As for strengths of the weights files, there is an rough elo shown against those in https://lczero.org/networks/ (which is down for 24 hours maintenance)

For the distilled networks in https://github.com/dkappe/leela-chess-weights/wiki/Distilled-Networks, there is a tournament result if you scroll down, if nothing else, it shows what weights are stronger than others.

I estimate the included converted weights file ‘weights9149.converted.txt’ as around 2400 ELO on the Raspberry Pi, but it may be stronger.


Al.

[Marco Baiocco]

Hi Al,

Yes, you probably missed my mail from 10 minutes ago. :) I already did that (thankfully I know my way around Linux a bit) and it actually works.

Thanks, I will check the resources you linked.

How strong is the network you included with the engine, if you have a rough idea?

Thank you so much for your help!

Marco Baiocco

[Scally]

Hi Marco,

Yes I was typing my reply when you responded.

As just reported my included file is around 2400 ELO on the Raspberry Pi, maybe a little stronger.

Don’t forget Lc0 plays a laughable endgame so don’t expect too much. I have linked it to the picochess endgame tablebases but these are only for 3 and 4 men.

Al.

[Henri Spijkerman]

Don’t forget Lc0 plays a laughable endgame so don’t expect too much. I have linked it to the picochess endgame tablebases but these are only for 3 and 4 men.

Off topic, I know, but about tablebases, I've added 5 pieces tablebases to the picochess tablebases folder, still room enough for more... Doesn't that work?

Greetings,

Henri

 

[Michael Byrne]

No, don’t add more than 5 , 5 will work, 6 men syzygy will not work on 32 bit OS. 

Sent from my iPhone

--

[Scally]

Hi Henri,

Yes they work as I’ve tried them.

The 5 men syzygy tablebases are 7 gb & 6 men are 150 gb but only work on 64 bit machines as verified by Michael Byrne of McCain fame.

Al.

[Scally]

Haha, another post replied to as I was typing, thanks for the verification Michael.

Al.

[Henri Spijkerman]

The 3+4+5 men tablebases are only 938 mb in total. You can download them here: https://chess.massimilianogoi.com/download/tablebases/

Greetings,

Henri

[Scally]

Hi Guys,

I forgot to say, for best results you should select ‘no book’ and let Lc0 play it’s best ‘learnt’ openings unless you want to see how it plays against some openings of course.

Also, I’m interested in your findings, like preferred weights file, estimated strength, playing style or any other comments ...


Cheers,

Al.

[Ruskie]

I only tried the "prepackaged" weights, though I have downloaded a couple more.

I am a mediocre chess player, and I normally played reduced strength engines, so that might influence my impressions. Anyway, so far, it looks very aggressive to me. I feel pressured from move 4 on, both with White and Black. Very hard to play against.

I wish they could produce a weight file for a 1300, 1500, 1700 ELO game, but I understand that is not the purpose of the project.

Cheers

Marco

[Scally]

Hi Marco,

If you look here you will find the 51*** range weights files:

https://lczero.org/networks/

Scroll down a bit and you should see some around your grade. The downside of these weights files are that they are quite big.
You will will find that they take 10-15 seconds to load on Picochess before you can play. Also they have slow access on the RPi, but this might suit lower grade players.
Try them and let me know.
Make sure you delete or move away any other weights files from /opt/picochess/engines/armv7l

Al.

[Marco Baiocco]

Hi Al,

They are not "distilled" network, right?

Will they work anyway?

What's the differnce in using non-distilled weights? Is it only that they are not optimized for normal CPU, or am I just mixing up things?

Rgds

Marco Baiocco

[Scally]

Hi Marco,

Yes they still work but they are huge therefor slower on a CPU.

The beauty of the distilled networks are they are a lot smaller and have faster access.
Yesterday somebody suggested I should use the even smaller distilled networks for greater speed.

In your case you are looking for a lower Elo network, the 5**** networks have just begun training so are a lot weaker, suiting your needs.

Al.

[Marco Baiocco]

Okay, I would say huge means slower, and slower means weaker (in a way), so this would be an added benefit, actually.

[Marco Baiocco]

Unfortunately, those weight files are in "the old text format no longer supported by Lc0". They need to be converted via command-line tools:

" error Text format weights files are no longer supported. Use a command line tool to convert it to the new format.  " 

I found a thread from you in the LCZero group where you ask how to do that, and they point you to the training repository on Github. I went there but I have not been able to identify the tool to use.

Cheers

Marco Baiocco

Mail priva di virus. www.avast.com

[Al]

Hi Marco,

That can’t be right? The 41xxx and 51xxx are being built as we speak. I was talking about the 51xxx networks at the top, if you scroll down a little there’s some in the 1400-1500 Elo range.

I haven’t tried them but they are new and should work, I’ll try this morning and get back to you .....


Al.

[Marco Baiocco]

Oh, ok ok. I scrolled all the way down!!

[Al]

No problem,

Yes the very old ones don’t work as you said.

I hope you find one at your level or just above. They will take a time to load, say 10-15 seconds until picochess loads the engine after selecting the engine and level.

Enjoy,

Al.

[Scally]

Hi Marco,

I downloaded 51051 graded just under 1600. I was surprised at the size, they are 10 times smaller than the 41*** networks and the same size as my converted 9149 network.
It takes less than 5 seconds to load on my Raspberry Pi 3b+ and plays well on Picochess.

Al.

[Marco Baiocco]

Hi Al,

Curiously enough, without reading your mail I did the very same. I downloaded the very weak ones (rate 500 elo or so) and then all the way up until 1600 elo (skippng only a few). They all worrk, but I haven't had the chance to play against them yet.

Thanks

Marco 

[Henri Spijkerman]

I am thoroughly enjoying the new Lc0 package! I am playing a lot with Kappe's new 32930-112x9.se distilled network, which I find very strong and fast.

It's difficult to compare weights, it's one of the most favorite subjects among Lc0-followers, and of course it all depends on the kind of hardware you are using. But I am really (pleasantly) surprised to see the slow BLAS version of Lc0 work that well on the even slower pi!

Looking at temperature, on my 3b+ 3 cores is on the edge but safe, as long as I don't use 'Ponder on'.

Lots more to explore!

Henri

[Henri Spijkerman]

To increase the fun even more I acquired a Pi Zero W and have Picochess installed. Everything works wonderfully (and that for a mere $10!), except the latest Lc0 package. I've nano'd the 'addfiles' to get the engine in the armv6l folder and adjusted the 'lc0v0210.uci' to 1 core and less hash (64), but it does not work yet. Any ideas Al?

To get back to the normal 3+ install, the latest 5XXXX weights are really good and fast!

Henri

[Scally]

Hi Henri,

Not everything that’s compiled on a armv7 processor works on an armv6 processor.
I have commitments on 3 different forums plus my normal picochess weekly compiles, but will try to compile Lc0 on my Raspberry pi zero w that is laying here in readiness to swap places with the Raspberry Pi zero in my eagerly awaited DGT Centaur that’s on order.

I’ll get back to you within a week,


Cheers,

Al.

[Henri Spijkerman]

Hi Al,

Take your time! You are a busy man and I owe a lot to you when it comes to Picochess and the Pi... :)

Henri

[Scally]

Hi Henri,

The compile for Lc0 takes 3 hrs on a 1 core Raspberry Pi Zero W.
First I compiled with my gcc-8.1.0 compiler, which seemed to work, it started and allocated the weights file, but it failed with ‘Illegal instruction’
Then I compiled with the native RPi gcc-6.3.0 compiler, with the same result.
Finally I tried my Clang 6.0.0 compiler but got a segmentation fault at the start of the compile.

I’ve attached both files here as they DO run on armv7l proving there’s nothing wrong with the compiled binaries. There’s obviously something with armv6 that doesn’t like Lc0, looking up similar error messages seems to point to a memory problem.

If you use these binaries, don’t forget to add permissions via:
sudo chmod a+x engine-name


Cheers,

Al.

[Henri Spijkerman]

Hi Al,

I feel sorry for all the effort you put into a Lc0 Picochess Pi Zero (w) version. And I guess wanting to play with such an engine on such a slow machine is a bit awkward from the start. I tried the versions you made and of course you are true again, Picochess acknowledges the engines ('engine ok'), but they do not work on my Zero W too.

They run on my Pi 3+ as you predicted, so I am grateful for your latest compiled versions of Lc0!

Thanks again,

Henri

[Henri Spijkerman]

Until lately, the distilled networks of Kappe were the strongest weights for Lc0 on Picochess. Now there is a new king in town: Little Demon 2. Check it out!

https://lc0.org/ld2

Henri

[Scally]

Hi Henri,

Yes I’ve seen that, however have you tried Dkappe’s good, bad & evil Gyal weights files?

https://github.com/dkappe/leela-chess-weights/releases


Cheers,

Al.

[Henri Spijkerman]

Hi Al,

Keeping an eye out for Gyal, but strictly speaking they're not LC0 weights:

"How were they trained? By taking lichess games and producing Q and policy data from low depth SF10 searches"

https://github.com/dkappe/leela-chess-weights/wiki/Bad-Gyal

So I look at Good and Eval Gyal as 'new' engines....

Greetings,

Henri