stockfish-NNUE as grist for SF development?

[Warren D Smith]

"stockfish NNUE" is a clone that has a neural-net based evaluator replacing the usual eval.

It has been trained by some people based on the usual SF evaluation plus mucho lookahead.

It is claimed to be about 50 elo weaker than the latest SF, which they regard as promising.

A pretty obvious thing to try is to create dataset of chess positions for which the NNUE and

usual SF evals differ by a lot.  From them see how to improve the usual SF eval.  Then repeat.

[guyv...@gmail.com]

I don't get this, if you're creating an NN based on the stockfish evaluation function, why would you interpret differences between the two as evaluation function dificiencies instead of as an NN that isn't producing the required results? If the NN was trained with another dataset (and was strong enough) this would make more sense in my opinion.

Op zondag 21 juni 2020 om 03:57:28 UTC+2 schreef warre...@gmail.com:

[Warren D Smith]

On 6/23/20, guyv...@gmail.com <guyv...@gmail.com> wrote:
> I don't get this, if you're creating an NN based on the stockfish
> evaluation function,

--it is based on SF eval LOOKE AHEAD by a lot of ply. Not the plain SF eval.

> why would you interpret differences between the two as
> evaluation function dificiencies instead of as an NN that isn't producing
> the required results? If the NN was trained with another dataset (and was
> strong enough) this would make more sense in my opinion.

--I think I have answered ...

--
Warren D. Smith
http://RangeVoting.org <-- add your endorsement (by clicking
"endorse" as 1st step)

[guyv...@gmail.com]

No, you didn't answer my question, you just repeated your previous statement.

Why do we need an NN for this, why can't we just compare the results to a deeper stockfish evaluation itself instead of to an NN trained to behave like a deeper stockfish evaluation?

Op dinsdag 23 juni 2020 om 13:27:29 UTC+2 schreef warre...@gmail.com:

[Warren D Smith]

On 6/23/20, guyv...@gmail.com <guyv...@gmail.com> wrote:
> No, you didn't answer my question, you just repeated your previous
> statement.
> Why do we need an NN for this, why can't we just compare the results to a
> deeper stockfish evaluation itself instead of to an NN trained to behave
> like a deeper stockfish evaluation?

--oh. Well the point is, the SF eval is not capable of predicting the
looked-ahead SF eval,
in general, or anyhow already is doing that about as well as it can
thanks to tuning.
But the NN finds things about the looked-ahead SF eval, which it really
*is* capable of predicting algorithmically fairly effectively.
Maybe some predictions wrong, but overall the successes
outweigh the failures and pay off. But those things, if an NN can do them,
maybe a non-NN can also do them (by adding new eval terms).

So the approach I am suggesting sort of automatically finds useful ideas
for SF-eval-designers.

[fengya...@gmail.com]

--oh.  Well the point is, the SF eval is not capable of predicting the
looked-ahead SF eval,
in general, or anyhow already is doing that about as well as it can
thanks to tuning.
But the NN finds things about the looked-ahead SF eval, which it really
*is* capable of predicting algorithmically fairly effectively.
Maybe some predictions wrong, but overall the successes
outweigh the failures and pay off.  But those things, if an NN can do them,
maybe a non-NN can also do them (by adding new eval terms).

So the approach I am suggesting sort of automatically finds useful ideas
for SF-eval-designers.

 

I agree the looked-ahead eval is difficult to statically predict, but it's not super clear that the NN will be easier, even if the NN identifies features of the looked-ahead eval that are able to be statically approximated. NNs are pretty opaque and there's not really a way to "decompile" the NN.

[Warren D Smith]

> I agree the looked-ahead eval is difficult to statically predict, but it's
> not super clear that the NN will be easier, even if the NN identifies
> features of the looked-ahead eval that are able to be statically
> approximated. NNs are pretty opaque and there's not really a way to
> "decompile" the NN.

--I agree with all that, but you might be able to dodge any need to
"decompile" the NN by simply finding a lot of positions where
1. the NN was a lot closer than SF's static eval to the looked-ahead eval
2. the NN & and the static eval differed by a lot
and then simply staring at those positions.

Then you observe some common patterns in that set of positions, and "aha" some
ideas occur to you for a new term to add to SF's static eval which will be both
cheap and effective.

[Kelly Kinyama]

Can I be tutored on how stockfish NN works?

[cameron....@gmail.com]

Hi Kelly,

Please check the link and use google translate for Japanese part Stockfish NNUE – The Complete Guide

it is better to contact the author for better explanation here is his twitter link nodchip

[Kelly Kinyama]

Thank you Mr Cameron for reply