Self-Learning stockfish upgraded

7919 views
Skip to first unread message

Kelly Kinyama

unread,
May 6, 2019, 10:58:45 AM5/6/19
to FishCooking
Self-learning stockfish has been upgraded and has beaten orignal stockfish on my computer by +21 Elo points.

Lets build on this idea to withstand pressure for NN engines like leela Zero. The future belongs to learning engines!

jgonza...@gmail.com

unread,
May 6, 2019, 11:15:27 AM5/6/19
to FishCooking
+21 Elo?!? That sounds too good to be true. What was changed? And can you validate the supposed Elo gains by scheduling a bench on Fishtest?

Kelly Kinyama

unread,
May 6, 2019, 11:53:14 AM5/6/19
to FishCooking
Thanks for the reply. I do not know how to use Fishtest. I need a guide.

Kelly Kinyama

unread,
May 6, 2019, 11:58:34 AM5/6/19
to FishCooking
It learns the evaluation of states from the games it plays.

lucasme...@gmail.com

unread,
May 6, 2019, 12:50:32 PM5/6/19
to FishCooking
The future is self-learning. Thanks Kelly for the amazing work you have been done.

mts...@gmail.com

unread,
May 6, 2019, 5:03:26 PM5/6/19
to FishCooking
What to do about the countless bin files created every game? Also, all we have to do is use the .exe, right? Nothing else?

Kelly Kinyama

unread,
May 7, 2019, 1:30:18 AM5/7/19
to FishCooking
Those are openings used by the .exe. Its just that computers understand numbers. So one bin file might be equivalent to the french defense, another to sicilian etc. That is where the games it plays are saved

The only thing else is to restart the engine for every new game. Otherwise, the learning wont work

Alayan

unread,
May 7, 2019, 9:30:42 AM5/7/19
to FishCooking
I tried looking up your code changes, but your github repo is messy :(

The concept of avoiding to play again moves with a bad history in previous games is rather nice.

Do you think the learning files can be considered a book ?

Kelly Kinyama

unread,
May 7, 2019, 10:06:07 AM5/7/19
to FishCooking

That is debatable. But when you look at how leela zero learns, they do the same thing. It is just that they hide their evaluations in a network. Is using a trained network a book? I do not think so.
My same technique can be used to buid endgame databases. Are databases books?

Kelly Kinyama

unread,
May 7, 2019, 10:07:29 AM5/7/19
to FishCooking
On Tuesday, 7 May 2019 15:30:42 UTC+2, Alayan wrote:

The code is messy. I just need to comment the code

manza...@gmail.com

unread,
May 8, 2019, 10:01:16 AM5/8/19
to FishCooking
Effectively, for example, in search.cpp, the function searchMCTS is never called.
Moreover, I'll try to render this code compatible with MinGW to use with Linux.

manza...@gmail.com

unread,
May 8, 2019, 4:33:32 PM5/8/19
to FishCooking
Il giorno martedì 7 maggio 2019 16:07:29 UTC+2, Kelly Kinyama ha scritto:
I simplified, updated to the last patch and rendereed the code compatible with MinGW (so, also for Linux):

https://github.com/amchess/Stockfish/tree/STockfishMCTS

Kelly Kinyama

unread,
May 10, 2019, 2:10:10 AM5/10/19
to FishCooking
Thanks Mr Manza for the great work! You have saved some of my time.

manza...@gmail.com

unread,
May 10, 2019, 5:02:02 AM5/10/19
to FishCooking
It's the goal of a community to support us each other. I told I appreciated your work and my goal was to valorize it.

ahmetbu...@gmail.com

unread,
May 12, 2019, 1:12:03 AM5/12/19
to FishCooking
https://github.com/amchess/ShashChess A.Manzo and Kelly Kinyama did a great job with NN Persisted Self Learning.

peter.zs...@gmx.at

unread,
May 12, 2019, 8:34:31 AM5/12/19
to FishCooking
Can you rebase this branch to latest official / master?
That would give us the possibilty to see the changes ...
Maybe update first your local master, it seems to be the one from stockfish 9 (really old)

Thanks in advance

manza...@gmail.com

unread,
May 12, 2019, 9:03:29 AM5/12/19
to FishCooking
My branch is updated at last Stockfish patch.

Peter Zsifkovits

unread,
May 12, 2019, 11:01:49 AM5/12/19
to manza...@gmail.com, FishCooking
Hmm, ok fine, but we can't really see the difference to latest master or
can we?

I mean make it look like this:

https://github.com/CoffeeOne/Stockfish/tree/MCTS


manza...@gmail.com

unread,
May 12, 2019, 11:06:30 AM5/12/19
to FishCooking
Yes, I didn't update master.
I don't have a lot of time because I'm very busy with the development of ShashChess engine.
We have a lot of improvement ideas and we need some help....

Alayan

unread,
May 12, 2019, 2:34:25 PM5/12/19
to FishCooking
The base branch being very old means the diff is unreadable as there is way too much unrelated changes cluttering it.

That, and the absence of proper commits in Kelly's branch, was what I meant when I wrote that the "github repo is messy". The code is there but it's very unpractical to analyze.

Rebasing it and with proper commits would be very helpful.

peter.zs...@gmx.at

unread,
May 12, 2019, 2:36:23 PM5/12/19
to FishCooking

I did it, see my last posting.

manza...@gmail.com

unread,
May 12, 2019, 6:05:50 PM5/12/19
to FishCooking

I updated the master.
Now the branch STockfishMCTS is aligned.

Kelly Kinyama

unread,
May 12, 2019, 11:06:18 PM5/12/19
to FishCooking
On Sunday, 12 May 2019 07:12:03 UTC+2, ahmetbu...@gmail.com wrote:
> https://github.com/amchess/ShashChess A.Manzo and Kelly Kinyama did a great job with NN Persisted Self Learning.

Sorry. The zobrist keys have to be constants and hard-coded. Copy them from my code in position.cpp

akir...@gmail.com

unread,
May 14, 2019, 12:14:53 PM5/14/19
to FishCooking
Fascinating idea. I'm running a 100 game match between this and Stockfish 19050918 bmi2. Will be interested to see the results.

Kelly Kinyama

unread,
May 14, 2019, 3:12:59 PM5/14/19
to FishCooking
I will happy to see the results as well
Message has been deleted

denis....@gmail.com

unread,
May 15, 2019, 4:20:00 AM5/15/19
to FishCooking
A lot of warnings and errors via gcc version 8.3.0 (Rev2, Built by MSYS2 project):

[…]
position.cpp:940:26: warning: integer constant is so large that it is unsigned
Zobrist::castling[14] = 14147118971151220805;
^~~~~~~~~~~~~~~~~~~~
position.cpp:942:18: warning: integer constant is so large that it is unsigned
Zobrist::side = 11598212307497933772;
^~~~~~~~~~~~~~~~~~~~
position.cpp:943:21: warning: integer constant is so large that it is unsigned
Zobrist::noPawns = 11773087935368354602;
^~~~~~~~~~~~~~~~~~~~
position.cpp: In member function 'Position& Position::set(const string&, Color, StateInfo*)':
position.cpp:1224:77: error: no matching function for call to 'transform(std::__cxx11::basic_string<char>::iterator, std::__cxx11::basic_string<char>::iterator, std::__cxx11::basic_string<char>::iterator, <unresolved overloaded function type>)'
std::transform(sides[c].begin(), sides[c].end(), sides[c].begin(), tolower);
^
In file included from C:/msys64/mingw64/include/c++/8.3.0/algorithm:62,
from position.cpp:21:
C:/msys64/mingw64/include/c++/8.3.0/bits/stl_algo.h:4293:5: note: candidate: 'template<class _IIter, class _OIter, class _UnaryOperation> _OIter std::transform(_IIter, _IIter, _OIter, _UnaryOperation)'
transform(_InputIterator __first, _InputIterator __last,
^~~~~~~~~
C:/msys64/mingw64/include/c++/8.3.0/bits/stl_algo.h:4293:5: note: template argument deduction/substitution failed:
position.cpp:1224:77: note: couldn't deduce template parameter '_UnaryOperation'
std::transform(sides[c].begin(), sides[c].end(), sides[c].begin(), tolower);
^
In file included from C:/msys64/mingw64/include/c++/8.3.0/algorithm:62,
from position.cpp:21:
C:/msys64/mingw64/include/c++/8.3.0/bits/stl_algo.h:4330:5: note: candidate: 'template<class _IIter1, class _IIter2, class _OIter, class _BinaryOperation> _OIter std::transform(_IIter1, _IIter1, _IIter2, _OIter, _BinaryOperation)'
transform(_InputIterator1 __first1, _InputIterator1 __last1,
^~~~~~~~~
C:/msys64/mingw64/include/c++/8.3.0/bits/stl_algo.h:4330:5: note: template argument deduction/substitution failed:
position.cpp:1224:77: note: candidate expects 5 arguments, 4 provided
std::transform(sides[c].begin(), sides[c].end(), sides[c].begin(), tolower);
^
make[2]: *** [<builtin>: position.o] Error 1
make[2]: Leaving directory '/home/dew/self_tmp/Stockfish/src'
make[1]: *** [Makefile:512: gcc-profile-make] Error 2
make[1]: Leaving directory '/home/dew/self_tmp/Stockfish/src'
make: *** [Makefile:418: profile-build] Error 2

manza...@gmail.com

unread,
May 15, 2019, 4:43:48 AM5/15/19
to FishCooking
I put the constants.
Now the code is updated at 9/5/2019.
You can check if it's all good.
Please tell me if it's ok.
https://github.com/amchess/Stockfish/tree/STockfishMCTS
Anyway, it works also without explicit constants.
We made a test on ShashChess 40 position suite:
https://github.com/amchess/ShashChess/wiki/Match
The result was the following:
First try: (without learning)
+7/=66/-7 games download:
https://www.mediafire.com/file/fzlgpyn4tqf9l73/FirstMatch.pgn/file
After:
+12/=62/-6 for StockLearn (more than 100 points ELO of increment !) :argh: games download:
https://www.mediafire.com/file/onow87acmv2s3kw/AfterLearningMatch.pgn/file

akir...@gmail.com

unread,
May 15, 2019, 8:40:56 AM5/15/19
to FishCooking
I ran a 100 game match but Stockfish dev prevailed.

1: Stockfish dev 52.0/100
2: StockLearn 090519 64 BMI2 48.0/100

I'm running another one now.

akir...@gmail.com

unread,
May 15, 2019, 8:42:03 AM5/15/19
to FishCooking
Can someone compile either the regular or popcnt version of this? I can't run bmi2 engines on my home computer, they crash for some reason.

denis....@gmail.com

unread,
May 15, 2019, 9:53:16 AM5/15/19
to FishCooking
On Wednesday, May 15, 2019 at 3:42:03 PM UTC+3, akir...@gmail.com wrote:
> Can someone compile either the regular or popcnt version of this? I can't run bmi2 engines on my home computer, they crash for some reason.

You could to compile yourself via Visual Studio - just add NDEBUG;IS_64BIT;USE_PREFETCH;USE_POPCNT;
into a project and do some optimizations like PGO and etc..

Kelly Kinyama

unread,
May 15, 2019, 10:07:38 AM5/15/19
to FishCooking
Delete line 47 in search.cpp
Change line 1020 in search.cpp from "SE = false" to "bool SE = false".
SE is a local variable.

I do not know the effect of code from line 775 to 790. I probes the MCTS table even at PvNodes; It has to be tested seperately. So I removed it from my code.
I do not have enough hardware to test the code.

manza...@gmail.com

unread,
May 15, 2019, 10:29:49 AM5/15/19
to FishCooking
I corrected the local variable: you're right even if it worked.
The lines from 775 to 790 are from the old function searcMCTS: I refactored the code.
A curiosity: why the Zobrist constant are to be constants even if the value are always randomly generated?

denis....@gmail.com

unread,
May 15, 2019, 10:34:13 AM5/15/19
to FishCooking
@Kelly You don't need other hardware, just install MSYS2 or Cygwin for gcc testing
Each fishtester used gcc or clang ant that it is...

manza...@gmail.com

unread,
May 15, 2019, 10:45:53 AM5/15/19
to FishCooking
Il giorno mercoledì 15 maggio 2019 16:34:13 UTC+2, denis....@gmail.com ha scritto:
> @Kelly You don't need other hardware, just install MSYS2 or Cygwin for gcc testing
> Each fishtester used gcc or clang ant that it is...

Yes, so we can also have the builds for other platforms and a compatibility with linux.
Mingw on Windows and gcc or clang on Linux.
I tested my code and the engine learns with great results, but, because it was not trivial at all the refactoring/simplification and elimination of errors/warnings also from benchmark, I need a confirmation from the author.
More simply, I work with
msys, mingw compiler and Eclipse CDT, also a lot lighter than Visual Studio and totally free.

Kelly Kinyama

unread,
May 15, 2019, 11:08:59 AM5/15/19
to FishCooking
So that we can be sharing the bin files. So that the engine uses the same numbers at different runtimes or on different computers. If you use random numbers, you have to be retaining the engine everytime you run it. Thats one of the basis of the learning algorithm

manza...@gmail.com

unread,
May 15, 2019, 11:21:40 AM5/15/19
to FishCooking
Ok. Now, I understand.
Thanks

manza...@gmail.com

unread,
May 15, 2019, 1:30:19 PM5/15/19
to FishCooking
I updated the code with the today patch:
https://github.com/amchess/Stockfish/tree/STockfishMCTS

I also commented your own code to facilitate the eventual debug.

Alexandre Meirelles

unread,
May 15, 2019, 2:08:16 PM5/15/19
to FishCooking
Can you pls give the executable? 

Ty very much in advance!

aden...@gmail.com

unread,
May 15, 2019, 2:13:00 PM5/15/19
to FishCooking
This is fascinating stuff! How long do we have to train before it takes over SFdev? And is there a way to somehow combine these trained bin files into a single exe?

manza...@gmail.com

unread,
May 15, 2019, 2:23:37 PM5/15/19
to FishCooking
Il giorno mercoledì 15 maggio 2019 20:13:00 UTC+2, aden...@gmail.com ha scritto:
> This is fascinating stuff! How long do we have to train before it takes over SFdev? And is there a way to somehow combine these trained bin files into a single exe?

I wait for a validation of my code by the original author.
In my opinion, it's essential to have the code compatible with MinGW
to be aligned with the community.

Kelly Kinyama

unread,
May 16, 2019, 2:10:25 AM5/16/19
to FishCooking
Thank you for completing a torne of work Mr Manza. I use Visual Studio because I could not compile in mingW. Maybe I am missing something. I will need help with mingW.

manza...@gmail.com

unread,
May 16, 2019, 3:37:45 AM5/16/19
to FishCooking
Me too I used Visual Studio, but MinGW and, for example, Eclipse CDT, it's a lot lighter. Moreover, your great work can be more appreciated, also by the community.
As demonstrated in TCEC15, the future is this learning approach.
You can contact me in private if you need help...
Andrea Manzo

Ente

unread,
May 16, 2019, 7:21:08 AM5/16/19
to FishCooking
As far as I am concerned, it also works with MSYS2 which is also a way described on github so we could refer to that.

The thing I thought about is how to combine our CPU power to get the best results. E.g. I have two computers but I have no way of combining the experience gathered. I would be most appreciative if you could write a tool to combine the .bin files.

JOSE LOPEZ

unread,
May 16, 2019, 10:40:11 AM5/16/19
to FishCooking


El jueves, 16 de mayo de 2019, 7:21:08 (UTC-4), Ente escribió:


The thing I thought about is how to combine our CPU power to get the best results. E.g. I have two computers but I have no way of combining the experience gathered. I would be most appreciative if you could write a tool to combine the .bin files.



I  think this is a key factor. Should be awesome to see the results. 

akir...@gmail.com

unread,
May 16, 2019, 10:49:12 AM5/16/19
to FishCooking
Yeah there definitely needs to be some way to pools these results in a central repository.

manza...@gmail.com

unread,
May 16, 2019, 11:49:47 AM5/16/19
to FishCooking
In my opinion, the merge is not trivial.
The engine suppose the bin files he reads are the best at the moment, in particular, better than its evaluation without learning, but they can came from another less powerful pc.
Moreover, for example, if you have pawns1.bin and pawns2.bin and you want a final pawns12.bin (a merge), if a position (its hashValue)
- is only in one file you can put it in the final one and it's trivial
- is in both files, it's associated, in particular, to its score. So, what score to take? The answer obviously is the one of the file by generating which the position's analysis quality and time spent are better, but how we can etablish this?

Ente

unread,
May 16, 2019, 12:10:52 PM5/16/19
to FishCooking
Maybe it needs additional computation time (set by the user) to confirm the quality of analysis if a hash collision occurs.

If the learning is written down in the standard polyglot format we could also use the learn byte so it can keep track of how often it was played and how often it lost/won/drew.

Ente

unread,
May 16, 2019, 12:16:48 PM5/16/19
to FishCooking
Here is a link. There you can find some additional information about polyglot. I think the user should get the choice what happens when a hash collision occurs. I would take depth over score. Maybe one could also do x versions of the merge (depending on how many collisions occur) and then test for elo gain/loss.

manza...@gmail.com

unread,
May 16, 2019, 12:23:53 PM5/16/19
to FishCooking
The file format is .bin like polyglot, but the information stored is not the same:
hashkey, depth, score and move.
So, we can't use the probabilities like in polybook.

Ente

unread,
May 16, 2019, 12:28:42 PM5/16/19
to FishCooking
Then we are left with an arbitrary choice or must recalculate. Unless we could also store number of nodes searched. Then we could do a rough comparison.

Kelly Kinyama

unread,
May 16, 2019, 12:29:00 PM5/16/19