Use basinhopping algorithm with lmfit

[Yann Fichou]

Hi everyone,

I'm using the lmfit tools to fit multidimensional data (mostly following this pattern https://github.com/lmfit/lmfit-py/blob/master/examples/fit_multi_datasets.py).

My global fit works well, but I find myself often stuck in local minima. My experience is that the basinhopping algorithm is good to avoid this kind of issue.

Is there a way to implement scipy.optimize.basinhopping with the lmfit library?

It is my understanding that scipy.optimize.minimize, which is called by scipy.optimize.basinhopping, is slightly different from lmfit.minimize. In particular, I would not know how to pass the Parameters into the scipy.optimize.basinhopping.

To be clear, here is my current fitting line :

>res = lmfit.minimize(residGlob, fit_params, args=(xdata, dataToFit1, dataToFit2))

Can we do something like :

basinhopping(residGlob, fit_params, args=(xdata, dataToFit1, dataToFit2))?

Alternatively, any suggestion on how to find global minima with the lmfit tools?

Thanks a lot,

Yann

[Andrew Nelson]

You can try differential_evolution, it's also a good global minimum finder.

--
You received this message because you are subscribed to the Google Groups "lmfit-py" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lmfit-py+unsubscribe@googlegroups.com.
To post to this group, send email to lmfi...@googlegroups.com.
Visit this group at https://groups.google.com/group/lmfit-py.
To view this discussion on the web visit https://groups.google.com/d/msgid/lmfit-py/622fbec3-299b-411f-b44d-9778a9980196%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

_____________________________________
Dr. Andrew Nelson


_____________________________________

[Yann Fichou]

Thanks for your answer Andrew. It throws me this error :

ufunc 'isfinite' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe'. 

Do you know where it comes from?

Yann

On Tuesday, October 31, 2017 at 4:39:34 PM UTC-7, Andrew Nelson wrote:

You can try differential_evolution, it's also a good global minimum finder.

On 1 November 2017 at 10:36, Yann Fichou <y.fi...@gmail.com> wrote:

Hi everyone,

I'm using the lmfit tools to fit multidimensional data (mostly following this pattern https://github.com/lmfit/lmfit-py/blob/master/examples/fit_multi_datasets.py).

My global fit works well, but I find myself often stuck in local minima. My experience is that the basinhopping algorithm is good to avoid this kind of issue.

Is there a way to implement scipy.optimize.basinhopping with the lmfit library?

It is my understanding that scipy.optimize.minimize, which is called by scipy.optimize.basinhopping, is slightly different from lmfit.minimize. In particular, I would not know how to pass the Parameters into the scipy.optimize.basinhopping.

To be clear, here is my current fitting line :

>res = lmfit.minimize(residGlob, fit_params, args=(xdata, dataToFit1, dataToFit2))

Can we do something like :

basinhopping(residGlob, fit_params, args=(xdata, dataToFit1, dataToFit2))?

Alternatively, any suggestion on how to find global minima with the lmfit tools?

Thanks a lot,

Yann

--
You received this message because you are subscribed to the Google Groups "lmfit-py" group.

To unsubscribe from this group and stop receiving emails from it, send an email to lmfit-py+u...@googlegroups.com.

To post to this group, send email to lmfi...@googlegroups.com.
Visit this group at https://groups.google.com/group/lmfit-py.
To view this discussion on the web visit https://groups.google.com/d/msgid/lmfit-py/622fbec3-299b-411f-b44d-9778a9980196%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Andrew Nelson]

Have you set finite bounds for each of your parameters?

To unsubscribe from this group and stop receiving emails from it, send an email to lmfit-py+unsubscribe@googlegroups.com.

To post to this group, send email to lmfi...@googlegroups.com.
Visit this group at https://groups.google.com/group/lmfit-py.

To view this discussion on the web visit https://groups.google.com/d/msgid/lmfit-py/c3ac840d-9b12-4197-903b-97e78f696de0%40googlegroups.com.

[Yann Fichou]

I did, but 2 param were set "vary=False", and the differential_evolution method doesn't seem to like that. It works now. However there is no errors given on each parameter.

Any help to try the basin-hopping algorithm with lmfit?

Thanks.

Yann 

[Matt Newville]

Hi Yann,

On Tue, Oct 31, 2017 at 6:36 PM, Yann Fichou <y.fi...@gmail.com> wrote:

Hi everyone,

I'm using the lmfit tools to fit multidimensional data (mostly following this pattern https://github.com/lmfit/lmfit-py/blob/master/examples/fit_multi_datasets.py).

My global fit works well, but I find myself often stuck in local minima. My experience is that the basinhopping algorithm is good to avoid this kind of issue.

Is there a way to implement scipy.optimize.basinhopping with the lmfit library?

At the moment, the only "global optimizer" option for lmfit is `differential_evolution`.  I think that options for other "global solvers, such as basin-hopping or AMPGO (see http://infinity77.net/global_optimization/) would be very valuable additions.

I am pretty sure that making either compatible with lmfit would be a small amount of work (not zero, but not too hard either) in order to accommodate Parameters and to use lmfit's minimize instead of scipy's.  And, I see that Andrew has the code for AMPGO on github, which would make that much easier.

Unfortunately, I'm not able to drop everything else and do this within a couple days.   The effort could really use a volunteer.   While dreaming of volunteers to add capabilities, it would also be very useful to wrap ODR for lmfit.  If anyone is interested, please let us know.

Cheers,

--Matt

[Renee Otten]

Admittedly, not as sophisticated as differential evolution or bassin-hopping…. but, if you don’t have too many parameters, it might be an option to use the “brute” method. It’s pretty much implemented as in spicy, and just performs a grid search - you can start from the best grid point (which is hopefully close to your real minimum) and do a leastsq minimization from there. 

To view this discussion on the web visit https://groups.google.com/d/msgid/lmfit-py/a7974075-1dda-4d04-8444-e7298c54f41d%40googlegroups.com.

[Yann Fichou]

Thanks for you answers.

Matt, I agree with you that it basically "only" asks volunteer work. I actually looked into the basinhopping module, naively hoping to simply switch a optimize.minimize function to a lmfit.minimize, but it does not seem to be so simple :). It would requires a bit of formal programming training, which I completely lack. It makes me even more grateful to you guys for developing such nice tools and for making it public.

I do hope someone will be able to implement basinhopping or .AMPGO (didn't know it)  to accomodate Parameters, I feel like it would add some value to the lmfit project.

Renee, that's a good idea and that's what I'm doing, running global search and then do local minimization.

Yann 

[Matt Newville]

Hi Yann, All,

On Wed, Nov 1, 2017 at 1:13 PM, Yann Fichou <y.fi...@gmail.com> wrote:

Thanks for you answers.

Matt, I agree with you that it basically "only" asks volunteer work. I actually looked into the basinhopping module, naively hoping to simply switch a optimize.minimize function to a lmfit.minimize, but it does not seem to be so simple :). It would requires a bit of formal programming training, which I completely lack. It makes me even more grateful to you guys for developing such nice tools and for making it public.

I do hope someone will be able to implement basinhopping or .AMPGO (didn't know it)  to accomodate Parameters, I feel like it would add some value to the lmfit project.

Renee, that's a good idea and that's what I'm doing, running global search and then do local minimization.

Yann 

Just to agree with Renee:  Yes, I should have mentioned brute-force. For medium scale problems or even for a selected subset of parameters in a larger problem, a brute force grid approach is often very effective.  If nothing else, it can be used to pick the best handful of starting parameter values.

I added "lmfit needs basinhopping or AMPGO" as a Github issue -- I agree that these would be very useful.

--Matt