How can I find good initial guess (best fit)

[Leona]

Hi all, I created the LM algorithm based on lmfit (Minimizer) and all works well. However, sometimes the model does not find the best fit. I have to manually change for example param d to 20 (initial value) and then it works again well.   

I would like to ask you if is here is some way how the created LM algorithm can find it themselves.

For example, is it possible to add more initial values or increase somehow the number of iteration (use all values in parameters bound)?

Please let me know.

I really appreciate your help.

Sample of my code:

# set parameters incluing bounds

params = Parameters()

params.add('a', value=38, min=10.0, max=100.0)

params.add('b', value=36.6, min=0.0, max=100.0)

params.add('c', value=0, min=-100.0, max=100.0)

params.add('d', value=0.00, min=0.00, max=0.02)

params.add('f', value=1, vary=False)

params.add('g', value=2, vary=False)

# do fit, here with leastsq model

minner = Minimizer(fcn2min, params, fcn_args=(x, data), iter_cb=per_iteration)

result = minner.minimize()

[Mark Dean]

Hi Leona,

If you know limits for all parameters and aren't prioritizing speed. I would suggest using the brute-force fitting method. 

minner = Minimizer(fcn2min, params, fcn_args=(x, data), iter_cb=per_iteration, method='brute')

You should consider setting the step parameter for the parameters. 

You can use those results as the starting point for an LM minimization if you like. 

Mark

[Matt Newville]

Hi Leona, 

I agree with Mark's suggestion of using brute to do the sort of thing you are asking for.

Basically, LM cannot do this sort of search by itself -- the different solvers all take a different approach, and LM is definitely about efficiently finding the local minimum from the starting values you provide it.  

Whether "global minimum" and "local minimum" are the same, and how good initial values need to be, both depend on the details of the problem to be solved - your 'fcn2min'.  You didn't provide this, so we don't know.

That said, I also want to caution you about a couple of things.  

First, you said that if you start with "d=20", that "it works again".  But your example code fragment shows

        params.add('d', value=0.00, min=0.00, max=0.02)

Now, whether d=0 is "close enough" to d=20 depends on the problem.  But here you are not allowing d to get anywhere near 20 -- you are explicitly saying that this value is invalid.

Second, you are putting bounds on every parameter, and what appear to be fairly tight bounds on 'd'.   Bounds should be used to prevent values that make the model invalid or are absurd.   For a peak-like model, saying the width cannot be negative makes sense.  In many cases, you know the sign of an amplitude parameter, so that again, setting min=0 or max=0 might be sensible.  

Setting min and max because you think "well, the value ought to be in this range" is not a good idea.  Let the fit do its work. 

Finally, setting an initial value to a boundary value is an exceptionally bad idea -- the fit is quite likely to get stuck there.  

This is especially true when that value is 0.  Actually, an initial value of 0 is never a great idea because "how zero" is unclear.  Is that 0 like 0.0001 or 0.0000001, or 1.e-32.  In casual conversation, those are all zero-ish.  For the math of minimization, they are very different.  So, if you mean a value in the 0.0001 range, guess that instead of zero.

--Matt

[Leona]

Hi Mark and Matt,

thank you so much!! You are the legends!

These are my current parameters and it works well now! I attached also the graph, so you can see how the function looks like.

params = Parameters()

params.add('a', value=50, min=5.0, max=200.0)

params.add('b', value=40, min=10.0, max=100.0)

params.add('c', value=1, min=-200.0, max=200.0)

params.add('d', value=0.001, min=0.00, max=0.02)

params.add('f', value=1, vary=False)

params.add('g', value=2, vary=False)

I changed the value d from 0 to 0.001 and now it performs well! In the previous example, I thought that if I change value c to 20 and not d. I am sorry for my mistake there!

I probably don't need the method='brute'. I tried it, but I got an error mesage:

  File "C:\Users\TEST\Anaconda3\lib\site-packages\lmfit\minimizer.py", line 1568, in leastsq

    lsout = scipy_leastsq(self.__residual, variables, **lskws)

TypeError: leastsq() got an unexpected keyword argument 'method'

However, thank you guys it helped me a lot!!!

[Deniz Bozyigit]

Hey Leona,
I've been following this thread and wanted to chime in. I actually built a library called ZeroGuess specifically to solve the kind of initial parameter problem you're describing. I created it after years of frustration with exactly these kinds of fitting issues.
The idea is simple - use machine learning to figure out good starting parameters automatically. Here's how you'd use it with your lmfit setup:

!pip install zeroguess

from zeroguess.integration import ZeroGuessModel

# It integrates with lmfit through this wrapper

model = ZeroGuessModel(

your_function,

independent_vars_sampling={"x": x_data},

estimator_settings={

"snapshot_path": "your_model.pth", # This saves the model for reuse

}

)

# You still use your same parameter bounds

model.set_param_hint('a', min=5.0, max=200.0)

model.set_param_hint('b', min=10.0, max=100.0)

model.set_param_hint('c', min=-200.0, max=200.0)

model.set_param_hint('d', min=0.0, max=0.02)

# Then let ZeroGuess handle the initial guessing

params = model.guess(data, x=x)

result = model.fit(data, x=x, params=params)

The first run will train on synthetic data from your function (taking a few minutes), but then it saves to your_model.pth so future fits are super quick. It's basically teaching itself what good parameters look like for your specific function.
For your case, it would likely catch that d=0 is a problematic starting point and give you something more reasonable automatically.
I'd be happy to help if you have any questions about using it. The repo is at https://github.com/deniz195/zeroguess if you want to check it out.
Good luck with your fits!
Cheers,

Deniz