Slight improvement of the SPSA method

[Ivan Ivec]

Dear friends,

I managed to find a slight improvement of the SPSA method for tuning chess engines parameters. You can find my paper at
https://arxiv.org/abs/2205.15602

The paper will be published in Acta Mathematica Spalatensia:

https://amas.pmfst.unist.hr/ams/index.php

You can find the link to the repository with the tuner under references in the paper.

And finally I need a help: I want to tune NNUE parameters while also searching for even better tuner as explained in the conclusion of the paper. However, I don't know how to prepare that tuning in the Stockfish code. There were some tuning attempts here in the framework few years ago, but the method doesn't work with the newest net architecture.

Thank you in advance for any help.

[Ivan Ivec (Mathematics explained)]

search.cpp: In member function 'virtual void Stockfish::MainThread::search()':
search.cpp:278:44: error: 'struct Stockfish::Eval::NNUE::Network' has no member named 'biases'
  278 |         Stockfish::Eval::NNUE::network[j]->biases[0] = nbnw[j*33];
      |                                            ^~~~~~
search.cpp:282:44: error: 'struct Stockfish::Eval::NNUE::Network' has no member named 'weights'
  282 |         Stockfish::Eval::NNUE::network[j]->weights[i] = nbnw[(i+1)+(j*33)];

Any ideas? Am I simply missing some #include somewhere?

[Aram Tumanian]

Hello, Ivan!

Thank you for your work (although I haven't read it yet).

You get this error because 'biases' and 'weights' are not members of the Stockfish::Eval::NNUE::Network class. They are members of the Stockfish::Eval::NNUE::FeatureTransformer class instead. The latter is defined in "nnue/nnue_feature_transformer.h". So you probably need:
278 |         Stockfish::Eval::NNUE::featureTransformer->biases[0] = nbnw[j*33];

неділя, 18 вересня 2022 р. о 23:10:15 UTC+3 ivan...@gmail.com пише:

[Ivan Ivec (Mathematics explained)]

Thanks Aram! I'll try at some point.

[Jörg Oster]

Ivan, you might also have a look at nnue-small which implements a small net from the engine Smallbrain.

The size of this net is 768 x 256 x 1, which gives you a lot of parameters for tuning purposes.

Have a look at neuralnet.cpp and neuralnet.h.

If you have questions you can contact me by email, please.

[Ivan Ivec (Mathematics explained)]

Thanks, it sounds interesting for tuning.

However, my first goal is to find even better tuner first.

Namely, SPSA and variants work best if parameters are symmetric around the optimum.

Handling asymmetric case successfully is in my opinion the most promising thing for any tuning method.