Slight improvement of the SPSA method

Skip to first unread message

Ivan Ivec

Aug 5, 2022, 2:34:35 PM8/5/22
to FishCooking
Dear friends,

I managed to find a slight improvement of the SPSA method for tuning chess engines parameters. You can find my paper at

The paper will be published in Acta Mathematica Spalatensia:

You can find the link to the repository with the tuner under references in the paper.

And finally I need a help: I want to tune NNUE parameters while also searching for even better tuner as explained in the conclusion of the paper. However, I don't know how to prepare that tuning in the Stockfish code. There were some tuning attempts here in the framework few years ago, but the method doesn't work with the newest net architecture.

Thank you in advance for any help.

Ivan Ivec (Mathematics explained)

Sep 18, 2022, 4:10:15 PM9/18/22
to FishCooking
search.cpp: In member function 'virtual void Stockfish::MainThread::search()':
search.cpp:278:44: error: 'struct Stockfish::Eval::NNUE::Network' has no member named 'biases'
  278 |         Stockfish::Eval::NNUE::network[j]->biases[0] = nbnw[j*33];
      |                                            ^~~~~~
search.cpp:282:44: error: 'struct Stockfish::Eval::NNUE::Network' has no member named 'weights'
  282 |         Stockfish::Eval::NNUE::network[j]->weights[i] = nbnw[(i+1)+(j*33)];

Any ideas? Am I simply missing some #include somewhere?

Aram Tumanian

Sep 19, 2022, 8:51:39 PM9/19/22
to FishCooking
Hello, Ivan!
Thank you for your work (although I haven't read it yet).
You get this error because 'biases' and 'weights' are not members of the Stockfish::Eval::NNUE::Network class. They are members of the Stockfish::Eval::NNUE::FeatureTransformer class instead. The latter is defined in "nnue/nnue_feature_transformer.h". So you probably need:
278 |         Stockfish::Eval::NNUE::featureTransformer->biases[0] = nbnw[j*33];

неділя, 18 вересня 2022 р. о 23:10:15 UTC+3 пише:

Ivan Ivec (Mathematics explained)

Sep 23, 2022, 3:21:09 PM9/23/22
to FishCooking
Thanks Aram! I'll try at some point.

Jörg Oster

Sep 24, 2022, 8:50:10 AM9/24/22
to FishCooking
Ivan, you might also have a look at nnue-small which implements a small net from the engine Smallbrain.
The size of this net is 768 x 256 x 1, which gives you a lot of parameters for tuning purposes.

Have a look at neuralnet.cpp and neuralnet.h.
If you have questions you can contact me by email, please.

Ivan Ivec (Mathematics explained)

Sep 29, 2022, 8:11:24 AM9/29/22
to FishCooking
Thanks, it sounds interesting for tuning.
However, my first goal is to find even better tuner first.

Namely, SPSA and variants work best if parameters are symmetric around the optimum.
Handling asymmetric case successfully is in my opinion the most promising thing for any tuning method.
Reply all
Reply to author
0 new messages