The AntiFish NN Project Has Started (Stockfish 2.3.1 is AntiFish's Primary Goal)

4,786 views
Skip to first unread message

Cscuile

unread,
Dec 7, 2018, 11:41:52 PM12/7/18
to LCZero
Brief Summary: AntiFish's primary purpose is to test whether or not a Neural Network can train to target holes within the AB engine Stockfish. If AntiFish can surpass Leela ID 9155 (3074 Elo or around SF2.3.1 Strength) then it shows that AntiFish is specifically training to target SF's evaluation holes. This can one day be used to improve Stockfish Dev, if this concept works. 


AntiFish Details:
-6x64 Block
-20,000 Games Generated per network
-2,000 Steps (Looking to perhaps increase) 
-AntiFish 400 Nodes trains against Stockfish 2.3.1 with increasing nodes. 
-1024 Batch-Size
-0.2 LR Value
-For more details please feel free to ask me. 



Challenges and Difficulties: 
-Resources!
-As of right now, in order to get to 100k steps, it would take around 70 days worth of game generation, even with a 6 block. This is due to my CPU bottlenecking my GPU in game generation.
-At around 50,000 Nodes for Stockfish, training time will double from 1 day to 2 days to 4 days...etc as nodes double. (20,000 games)


Even with a 6 block, AntiFish requires quite a substantial amount of resources to train. Perhaps a solution is to train off of FishTest data then, switch back to AntiFish-Stockfish 10 Game generation. 


Thoughts & Conclusion:

I was wondering if any of you have any thoughts, suggestions, advice, or anything of that such. 

The main "Hub" of AntiFish is on the Stockfish Discord, a link will be provided below. 

A contribution client will likely be needed in the future if AntiFish is to become serious at targeting Stockfish 10's evaluation holes at higher nodes (100-500+ Million). 

Resources is the biggest issue with this project. My CPU is causing the main bottleneck in game generation. 

Some questions I have are, will AntiFish be able to Scale against Stockfish with higher nodes? Will Stockfish lose its evaluation weaknesses with increasing nodes? Will the most recent AlphaZero Cpuct client allow AntiFish to scale 50+ or more Elo? 


If you have any questions or advice please share it with me below. 


Thank you, 
Cscuile

Stockfish/AntiFish Discord: https://discord.gg/kfyhvX3
AntiFish ID Progress Rating List: https://docs.google.com/spreadsheets/d/1ZAIuHR6n-5JTxKQc0XUSx1jyUrgVEcj8DNLKA7-urBw/edit#gid=442598453

Margus Riimaa

unread,
Dec 8, 2018, 2:14:48 PM12/8/18
to LCZero
I like your project very much.

Cscuile

unread,
Dec 8, 2018, 2:24:18 PM12/8/18
to LCZero
Thank you!

Cscuile

unread,
Dec 8, 2018, 2:27:14 PM12/8/18
to LCZero
Just an update, my CPU is the main bottle neck in game generation. I generate 20k games per 2k steps. Since I have to increase nodes every time I train, the training will take twice as long (At 50k Nodes plus)

Because of this, I have to use Fishtest data now to train.

brian

unread,
Dec 8, 2018, 3:23:53 PM12/8/18
to LCZero
Just curious, how were you generating the games between sf and nn?

Cscuile

unread,
Dec 8, 2018, 4:31:07 PM12/8/18
to LCZero
The way I generate games is by playing AntiFish against Stockfish 2.3.1 at increasing nodes. This method is unfortunately far too time consuming since my CPU bottlenecks game generation.

Right now I am trying to find a way to use FishTest games, then bootstrap regular generated games to the network. 

smashu2

unread,
Dec 8, 2018, 6:19:47 PM12/8/18
to LCZero
Your strategy should be start with net 11248 and train it vs SF to see if you can squeeze more juice out of it than when you Have squeeze all you can than net 30 might be ready I would wait for a net to reach is ceiling and try to squeeze more out of it by training vs SF when everybody start a new net like 40 don't start from scratch.

Cscuile

unread,
Dec 8, 2018, 6:33:04 PM12/8/18
to LCZero
Hmm, good idea. I'll have a ton of data I'll have to download for ID 11248/11258 though. 

Daniel Rocha

unread,
Dec 8, 2018, 8:39:15 PM12/8/18
to csc...@gmail.com, lcz...@googlegroups.com
If you are doing with 11248/58, you might as well to start with SF10

--
You received this message because you are subscribed to the Google Groups "LCZero" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lczero+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lczero/074d7f98-9c50-4b8a-9964-ec873b25c5bc%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


--
Daniel Rocha - RJ

Cscuile

unread,
Dec 8, 2018, 10:05:42 PM12/8/18
to LCZero
I'm not certain how long it will take to generate games for a 20b, but why not right?

Daniel Rocha

unread,
Dec 8, 2018, 10:51:37 PM12/8/18
to csc...@gmail.com, lcz...@googlegroups.com
Well, I might help you. I have a 1070 here.


For more options, visit https://groups.google.com/d/optout.

Cscuile

unread,
Dec 8, 2018, 10:55:44 PM12/8/18
to LCZero
Wow awesome! I really do appreciate the help with a project like this.

I'll need some way to gather games and to update the cutechess-cli game commands, but for now I think the Stockfish Discord should do just fine hopefully. 

Daniel Rocha

unread,
Dec 8, 2018, 11:01:56 PM12/8/18
to csc...@gmail.com, lcz...@googlegroups.com
Just send me a message on LC0 discord, I am Daniel, you know. I'm there frequently


For more options, visit https://groups.google.com/d/optout.
Message has been deleted

smashu2

unread,
Dec 8, 2018, 11:53:52 PM12/8/18
to LCZero
So it was previously thought that net 11248 as reach is limit your project will either confirm it or show it's still possible to improve 11248, also by improving vs SF I presume the NN will also improve in general vs all other AB engine it will be interesting to test if it's true or not.


On Friday, December 7, 2018 at 11:41:52 PM UTC-5, Cscuile wrote:

smashu2

unread,
Dec 8, 2018, 11:57:20 PM12/8/18
to LCZero
yes start training vs SF10 the purpose is to see if 11248 can develop opening strategy and pawn structures that sf 10 will struggle with.


On Saturday, December 8, 2018 at 8:39:15 PM UTC-5, Daniel Rocha wrote:

fauzi...@revton.com

unread,
Dec 9, 2018, 4:18:27 AM12/9/18
to LCZero
nice interesting project, please keep us updated

Cscuile

unread,
Dec 9, 2018, 8:11:30 AM12/9/18
to LCZero
Sure thing! I'll update you guys here as well as on the stockfish discord. 

Though, I will need help with the 20b and game generation. 

Pawel Powet

unread,
Dec 9, 2018, 10:10:43 AM12/9/18
to LCZero
So why don't you buy a new PC or new CPU at least, having such a big project in hand? Upgrading PC while running such a great project should be a priority for you! 

Edward Panek

unread,
Dec 9, 2018, 11:15:09 AM12/9/18
to LCZero
Thats being presumptuous.

No person has the same situation as another. 

Cscuile

unread,
Dec 9, 2018, 11:44:52 AM12/9/18
to LCZero
I wish I could! A Threadripper would make this project go much much faster! But I just can't afford to get a new CPU right now. 


On Sunday, December 9, 2018 at 10:10:43 AM UTC-5, Pawel Powet wrote:

Pawel Powet

unread,
Dec 9, 2018, 11:54:27 AM12/9/18
to LCZero
I don't agree.  His idea is to run a project that demand computation power. If this is what he want to do, so it is up to him to acquire needed components that let him to accomplish it and His situation has nothing to do with it. !

Edward Panek

unread,
Dec 9, 2018, 11:57:23 AM12/9/18
to LCZero
Or,,,a radical idea..engage people on a forum to help him?

Cscuile

unread,
Dec 9, 2018, 12:14:01 PM12/9/18
to LCZero
+1

Cscuile

unread,
Dec 9, 2018, 12:16:49 PM12/9/18
to LCZero
In the future once I increase SF to nodes above 50,000, I will definitely need help generating games or else it would take me decades to finish alone. (Training time doubles per doubling nodes)

If this concept works, it may have vast implications for chess engines so it is definitely worth experimenting! 

Daniel Rocha

unread,
Dec 9, 2018, 12:20:06 PM12/9/18
to Edward Panek, lcz...@googlegroups.com
Yes, this is why I am offering him my computer. In fact, I guess this is why he is posting on a forum!

--
You received this message because you are subscribed to the Google Groups "LCZero" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lczero+un...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

pradip ingale

unread,
Dec 9, 2018, 12:22:16 PM12/9/18
to LCZero
Leela 11248 already strong why not you continue this network against sf10 only three to four week you can achieve you're goal

Cscuile

unread,
Dec 9, 2018, 12:28:15 PM12/9/18
to LCZero
I currently have Test10's latest data downloading and I'm experimenting with a few training parameter configurations =D

Edward Panek

unread,
Dec 9, 2018, 12:52:46 PM12/9/18
to LCZero
+1!!!!

Jupiter

unread,
Dec 9, 2018, 1:35:13 PM12/9/18
to LCZero
Brief Summary: AntiFish's primary purpose is to test whether or not a Neural Network can train to target holes within the AB engine Stockfish.
 
Could you elaborate what these evaluation holes are all about. Or you just run match games between the project net and SF2.3.1 (Stockfish 2.3.1) and see if project net would defeat SF2.3.1?

Did you try to match between id 9155 and Sf2.3.1? What is the result and on what conditions. One of the issues of determining which is best between Lc0 and Sf is the  hardware conditions?

1. Test it at same hardware cost?

a. Lc0
High end gpu + low end cpu to minimize cost

b. Stockfish
High end cpu + low end gpu to match the cost of Lc0

c. Limit the total cost to n usd, since Sf2.3.1 does not scale well with number of cpu, it is an old program.

d. Limit project net to use GTX 1060 or something lower, test it against Sf2.3.1 on what cpu? on a cpu such that they can play on equal TC and Sf2.3.1 must have a better result in the beginning or on certain stage of training. This gap in strength will be your target to achieve your goal to defeat St2.3.1. You need a clear metric to determine if you already achieve your goal or not.

Cscuile

unread,
Dec 9, 2018, 5:58:21 PM12/9/18
to LCZero
Stockfish sometimes misevaluates positions as winning, when Lc0 finds it to either be losing or a draw. AntiFish will attempt to specifically get into these positions against SF.

Daniel Rocha

unread,
Dec 9, 2018, 7:09:58 PM12/9/18
to csc...@gmail.com, lcz...@googlegroups.com
I thought you'd go for 11248/58, instead of going all the way from the bottom.

--
You received this message because you are subscribed to the Google Groups "LCZero" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lczero+un...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Cscuile

unread,
Dec 9, 2018, 7:18:32 PM12/9/18
to LCZero
Sorry, I should have said that this will be done after the initial bootstrap. 

Daniel Rocha

unread,
Dec 9, 2018, 7:26:21 PM12/9/18
to csc...@gmail.com, lcz...@googlegroups.com
What is the importance of the initial bootstrap?


For more options, visit https://groups.google.com/d/optout.

Cscuile

unread,
Dec 9, 2018, 7:30:59 PM12/9/18
to LCZero
The main issue with getting a network going is getting enough games to train off off. A initial bootstrap will allow for the NN to get relatively strong. After this is done you can start generating games via AntiFish vs Stockfish 10 play. 

Right now I am trying out a 15x192 block network. For Leela this is around 3300 Elo. From what I've heard, a 20b is too much for my hardware and too time consuming. 

If this doesn't work, I'll have to lower it one more step. 

Daniel Rocha

unread,
Dec 9, 2018, 7:34:18 PM12/9/18
to csc...@gmail.com, lcz...@googlegroups.com
Why not literally programing an FPGA with neurons? I mean, not you, but would there be an advantage? Every time you need more, you just increase the space.


For more options, visit https://groups.google.com/d/optout.

Cscuile

unread,
Dec 9, 2018, 7:50:11 PM12/9/18
to LCZero
Hmm, I'm not sure. 

Cscuile

unread,
Dec 9, 2018, 9:21:57 PM12/9/18
to LCZero
AntiFish 15x192 Bootstrap just finished. At .1+.1 TC it is +400 Elo to SF9. In comparison ID 11248 is +600 Elo. Now the question is, do I go for a 20b? 

Cscuile

unread,
Dec 9, 2018, 9:30:05 PM12/9/18
to LCZero
.01+.01 TC**

Cscuile

unread,
Dec 9, 2018, 9:33:32 PM12/9/18