shgo and minimizer_kwargs

[Olivier B.]

Hi everyone,

We are trying to find the global optimum of a minimisation problem. We have tried the basin hopping and shgo (simplicial homology global optimization) algorithm from scipy via the lmfit 'interface'. Using basin hopping, we can succesfully converge and tune the local minimizer to our liking. When we use shgo however, we cannot pass 'minimizer_kwargs' to the local minimizer. Illustration of the problem:

Basinhopping:

minner=Minimizer(func,params_instance,fcn_args=arg),disp=True,niter=200,minimizer_kwargs='method':'Nelder-Mead','options':'disp':'True','maxiter':10000}}) 

result = minner.minimize(method='basinhopping' ) 

print(result.call_kws) 

Results in:

{'niter': 200, 'T': 1.0, 'stepsize': 0.5, 'minimizer_kwargs': {'method': 'Nelder-Mead', 'options':{'disp': 'True', 'maxiter': 10000}}, 'take_step': None, 'accept_test': None, 'callback': None, 'interval': 50, 'disp': True, 'niter_success': None, 'seed': None}

This is as expected, since the options for Nelder-mead are correctly passed on as {'disp': 'True', 'maxiter': 10000}.

Now for SHGO:

minner = Minimizer(func, params, fcn_args=arg),sampling_method='sobol',n=600,minimizer_kwargs={'method':'Nelder-Mead','options':{'disp':'True','maxiter':10000}})

result = minner.minimize(method='shgo')

 print(result.call_kws) 

Results in:

{'constraints': None, 'n': 600, 'iters': 1, 'callback': None, 'minimizer_kwargs': {'method': 'Nelder-Mead', 'options': {}}, 'options': None, 'sampling_method': 'sobol'}

As you can see, suddenly the 'options' for Nelder-Mead is an empty dictionary. Any ideas on how to solve this issue?

Thanks!

Olivier

[Renee Otten]

Hi Olivier, 

I agree with you that it seems like this should work… not sure yet why it doesn’t. Let me take a look at this in the next few days. 

Are you using the latest released versions of lmfit and its dependencies?

Best, 

Renee

On Jun 30, 2021, at 3:44 PM, Olivier B. <garageb...@gmail.com> wrote:

Hi everyone,

--
You received this message because you are subscribed to the Google Groups "lmfit-py" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lmfit-py+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lmfit-py/b36afca4-9301-4ab0-b459-18177b427ce2n%40googlegroups.com.

[Renee Otten]

Okay, adding some print statements to debug this in “minimizer.py”:

  2164         result.call_kws = shgo_kws

+ 2165     ┆   print(f"\n\nshgo_kws (final, before fit) = {shgo_kws}")

  2166     ┆   try:

  2167     ┆   ┆   ret = scipy_shgo(self.penalty, bounds, **shgo_kws)

  2168     ┆   except AbortFitException:

  2169     ┆   ┆   pass

  2170

+ 2171     ┆   print(f"\n\nresult.call_kws (after fit) = {result.call_kws}”)

and running an example with your options shows:

shgo_kws (final, before fit) = {'constraints': None, 'n': 600, 'iters': 1, 'callback': None, 'minimizer_kwargs': {'method': 'Nelder-Mead', 'options': {'disp': True, 'maxiter': 10000}}, 'options': None, 'sampling_method': 'simplicial'}

result.call_kws (after fit) = {'constraints': None, 'n': 600, 'iters': 1, 'callback': None, 'minimizer_kwargs': {'method': 'Nelder-Mead', 'options': {}}, 'options': None, 'sampling_method': 'simplicial'}

So the call to the minimizer [ ret = scipy_shgo(self.penalty, bounds, **shgo_kws) ] seems to be causing this. I *think* this is because “options” happens both within the ‘minimizer_kwargs’ and in the “main” dictionary. I think indeed that the call to the minimizer is already not correct, so we likely should separate the “minimizer_kwargs” form the rest and then change the call to:

ret = scipy_shgo(self.penalty, bounds, **shgo_kws, minimizer_kwargs)

I haven’t had time to actually try that or perhaps others have another/better suggestion - if not I’ll experiment a bit over the weekend. 

Best,

Renee

[Matt Newville]

Hi Olivier, Renee,

Yeah, I agree that this should work and it looks like a bug.

To view this discussion on the web visit https://groups.google.com/d/msgid/lmfit-py/75A92CEF-3B7A-4F02-9107-64B7E4D05113%40gmail.com.

--

--Matt Newville <newville at cars.uchicago.edu> 630-327-7411

[Olivier B.]

Hi Matt and Renee,

Thanks for your answers. 

I would like to help, but I don't feel very comfortable trying to debug this myself in the lmfit code, since I am not a very experienced programmer. Would you like me to open an issue on GitHub for this?

Kind regards,

Olivier

Op vrijdag 2 juli 2021 om 13:56:26 UTC+2 schreef Matt Newville:

[Renee Otten]

Hi Olivier, 

It turns out this is not something we do “wrong” in lmfit, it’s caused by the SHGO code in SciPy. I just went through that code (https://github.com/scipy/scipy/blob/v1.7.0/scipy/optimize/_shgo.py), added some print statements in there and you see that the “options” are filtered out here (https://github.com/scipy/scipy/blob/44e31b1eb202ef91caad059952ba268d2610a60b/scipy/optimize/_shgo.py#L584). Since “disp” and ‘maxiter” are not in the specified in the “solver_args” for any of the solvers including “nelder-mead” (https://github.com/scipy/scipy/blob/44e31b1eb202ef91caad059952ba268d2610a60b/scipy/optimize/_shgo.py#L559), these are filtered out. 

In short, what happens in lmfit is the “correct” behavior from our point of view. This also explains why printing the “result.call_kws” are different before and after the call to SHGO… Whether this is a “bug” in the SHGO algorithm in SciPy or works as intended that I don’t know. You could check the GitHub page of the SciPy library to see if there are reports about this and/or open an issue there.

Sorry, I am afraid we cannot be more helpful at this point. 

Best, 

Renee

[Olivier B.]

Hi Renee,

Thanks for all the help and the clear explanation!

Kind regards,

Olivier

Op zaterdag 3 juli 2021 om 15:48:05 UTC+2 schreef Renee Otten: