Goodbye CLOP, hello SPSA

1049 views
Skip to first unread message

Gary Linscott

unread,
May 17, 2014, 2:31:09 PM5/17/14
to fishc...@googlegroups.com
CLOP never worked properly because it was hard to integrate the clop C++ code with fishtest.  So, it's been removed.

On the bright side, Joona published spsa.pl, which was a very elegant, and small framework for doing SPSA based tuning.  It was exceptionally easy to integrate his SPSA logic into the framework, and so I've just enabled SPSA tests.

The SPSA parameters are well described by Joona's documentation (note, I swapped the order of the min and max columns from spsa.pl, if you are familiar with that).

Column 0: Variable name (alphanumeric string)
Column 1: Variable initial value (float)
Column 2: Variable minimum value (float)
Column 3: Variable maximum value (float)
Column 4: Perturbation "ck" for the last iteration (float)
Column 5: Relative apply factor "Rk" for the last iteration (float)

To make it easy for the worker code, and avoid threading messes like we had with CLOP, I've had each worker play concurrency * 2 games as one SPSA iteration.  I then collapse the final WLD score into the range -2..2.  I think this is theoretically sound, but Id love to have someone with more math confirm :).

The code for processing SPSA is a bit spread around, but here are the main locations:

Gary Linscott

unread,
May 17, 2014, 2:53:23 PM5/17/14
to fishc...@googlegroups.com
And a bug found almost immediately :).  The variables need to be changed randomly per run, otherwise it will bias towards always trying higher or lower values for each variable.

X

unread,
May 17, 2014, 3:38:02 PM5/17/14
to fishc...@googlegroups.com
> I think this is theoretically sound, but Id love to have someone with more math confirm :).
No math here, but I think spsa.pl does not collapse anything, and I guess you shouldn't either. If you do, precision of results of high-core machines will be drowned in noise from low-core ones, and thus be partially wasted.

Gary Linscott

unread,
May 17, 2014, 4:03:10 PM5/17/14
to fishc...@googlegroups.com
spsa.pl, only plays two games in each iteration though (although it can do so in parallel), so it doesn't need to collapse the results into -2..2, they are already there! :)  However, with the worker code, it's not feasible to start games at the same time with different parameters currently.  Given those restrictions, I'm sure there is a better way to use the fact that the bigger machines are playing more games, but I took the simple way out.

Eelco de Groot

unread,
May 17, 2014, 4:06:42 PM5/17/14
to fishc...@googlegroups.com
Op zaterdag 17 mei 2014 20:31:09 UTC+2 schreef Gary Linscott:
I just wanted to say that I think it is great to have a working parallel SPSA, and it seems also a way to keep the framework occupied :) Thanks Gary!

zamar

unread,
May 17, 2014, 5:39:45 PM5/17/14
to fishc...@googlegroups.com
1 game = 1 point.

If you play 16 games with same parameters (which is of course not ideal), the result should be in -16,...,16, there is no need to scale anything.

Gary Linscott

unread,
May 17, 2014, 5:44:20 PM5/17/14
to fishc...@googlegroups.com
Ok, thanks!  For the iteration parameter, I'm only incrementing it by one, regardless of how many games the worker actually plays.  Should this be changed as well?

mattsull...@gmail.com

unread,
May 17, 2014, 9:25:02 PM5/17/14
to fishc...@googlegroups.com
This sounds great. Would it be easy to disable the stats panel on the side and hide the users' residuals? These numbers aren't meaningful for SPSA tests, and all the colored residuals might give people the wrong idea.

Gary Linscott

unread,
May 17, 2014, 11:48:38 PM5/17/14
to fishc...@googlegroups.com, mattsull...@gmail.com
Yes, good idea, done.

zamar

unread,
May 18, 2014, 3:20:40 AM5/18/14
to fishc...@googlegroups.com
I think so.

Arjun Temurnikar

unread,
May 21, 2014, 3:25:02 AM5/21/14
to fishc...@googlegroups.com
What value of ck do you use?

I see that the default in the "New test" section is set to "10", and Joona in his spsa readme recommends "8". I use "6" sometimes when the values of parameters are small and optimum is somewhere very close by and I want some extra stability. I have found in some of my local tests ck = 6 slightly more reliable.

The test I have submitted is using ck = 6.

Also, I would mention, @Gary, the ck and rk values are not shown on the test page. http://tests.stockfishchess.org/tests/view/537c19880ebc595b33677281

Perhaps you should add that in the spsa params div.

Thanks :)

Gary Linscott

unread,
May 21, 2014, 12:23:37 PM5/21/14
to fishc...@googlegroups.com
Ah, interesting, thanks.  I was just using the default values from the aggressiveness sample in Joona's SPSA package.  6 or 8 probably makes more sense.

lucas....@gmail.com

unread,
May 22, 2014, 8:07:56 PM5/22/14
to fishc...@googlegroups.com
Not sure 8 or 6 are better than 10. In practice 10 worked well amd in Arjun's tests the param almost never change. I also followed Arjun's recommendation in my tune_lmr and parameters almost do not move. I will wait and see how it goes a bit, but if continues I will have to stop and restart: what a waste!

Please leave ck=10 by default umtil we see any concrete evidence that another value is better.

Arjun Temurnikar

unread,
May 22, 2014, 8:15:30 PM5/22/14
to fishc...@googlegroups.com, lucas....@gmail.com
Be patient Lucas. They will change, just slower than what it would with 10.

Wait till at least about 20-30K games, which when you will really start to see the progress. :)

Arjun Temurnikar

unread,
May 22, 2014, 8:34:13 PM5/22/14
to fishc...@googlegroups.com, lucas....@gmail.com
But don't mistake that for slower convergence. It will just fluctuate less rapidly... If the optimum values are close by, I have found lower ck values are slightly more beneficial otherwise large fluctuations create noise, but if the optimum is far from your initial, you would want higher values so as to speed up the convergence.

I may be completely wrong here (I am just speaking from experience)


:)

On Thursday, 22 May 2014 17:07:56 UTC-7, lucas....@gmail.com wrote:

Arjun Temurnikar

unread,
May 22, 2014, 8:34:54 PM5/22/14
to fishc...@googlegroups.com, lucas....@gmail.com
Just to add, if you are not sure about the initial, I think CLOP is a better way to go.


On Thursday, 22 May 2014 17:07:56 UTC-7, lucas....@gmail.com wrote:

Arjun Temurnikar

unread,
May 22, 2014, 8:47:46 PM5/22/14
to fishc...@googlegroups.com, lucas....@gmail.com
I have also in the past experimented with SPSA in 2 stages -- first with high ck so that the parameters "roughly converge" fast, and then with a low ck value so as to tune with precision the "roughly converged" parameters.

For eg.
First with ck = 10 (20K games)
Then, using the obtained parameter values, new SPSA session with ck = 6 (30K games)

That might work better if the optimum and initial are much further apart.

I can't say this is the most ideal way of doing things though... it was all just for fun. :)


On Thursday, 22 May 2014 17:07:56 UTC-7, lucas....@gmail.com wrote:

lucas....@gmail.com

unread,
May 23, 2014, 12:52:41 AM5/23/14
to fishc...@googlegroups.com
You make a good point about the two step approach. Would be interesting to hear Joona's views on that.

Ideally the Spsa implementation should embed it, for example having a time decay in ck or whatever makes sense.

zamar

unread,
May 23, 2014, 2:58:19 AM5/23/14
to fishc...@googlegroups.com, lucas....@gmail.com
SPSA has a time decay for ck. And this has already been implemented... When scheduling a test, you set the final value for ck and decay factor.

zamar

unread,
May 23, 2014, 3:15:06 AM5/23/14
to fishc...@googlegroups.com, lucas....@gmail.com
http://www.jhuapl.edu/spsa/PDF-SPSA/Spall_Implementation_of_the_Simultaneous.PDF

Spall recommends using Gamma = 0.101 which is the lowest possible values which theoretically guarantees convergence. We follow this recommendation.
Reply all
Reply to author
Forward
0 new messages