"Tolerance seems to be too small.. Could not estimate error-bars" and Large Errorbars

[Sterling Smith]

Thanks for lmfit.  It is very useful.

In my curve fitting, I am getting the message "Tolerance seems to be too small.. Could not estimate error-bars" often.  Which of the ftol, gtol, xtol should I change to start with (if a general statement can be made)?

On the other hand, I sometimes get very large errorbars.  Is there any parameter I can change to decrease the size of the errorbars?

[Matt Newville]

Hi Sterling,

On Fri, Jan 22, 2016 at 11:05 AM, Sterling Smith <smi...@fusion.gat.com> wrote:

Thanks for lmfit.  It is very useful.

In my curve fitting, I am getting the message "Tolerance seems to be too small.. Could not estimate error-bars" often.  Which of the ftol, gtol, xtol should I change to start with (if a general statement can be made)?

That message seems fairly unusual to me.   But, all of ftol, gtol, and xtol are probably too small for most real data analysis problems, and the defaults should probably 1.e-5 instead of 1.e-7.    It's hard to predict which of these most important for any particular problem, so increasing all of them is probably reasonable.      Are parameters changing values from their initial values?

FWIW, if you have parameters with wildly varying scales (say, some values that are 1.e-6 and some that are 1.e8) these problems with tolerances being too tight can be exacerbated.  Generally speaking, MINPACK (leastsq) does a very good at handling large difference in scales, but the scale can't be infinite.  Also, setting bounds on parameters can make the automatic scaling (and the values for the tolerance parameters) harder.

On the other hand, I sometimes get very large errorbars.  Is there any parameter I can change to decrease the size of the errorbars?

Well, if the fit is "bad" the errorbars will tend to be large.  That would generally imply that the model is not perfect..  But "bad" is also relative to the estimated uncertainty in the data (epsilon in the docs). 

Hope that helps.   Of course, giving more details about what you're doing would give us much more to go on.

--Matt

[Sterling Smith]

Matt,

On Friday, January 22, 2016 at 8:51:35 PM UTC-8, Matt Newville wrote:

Hi Sterling,

On Fri, Jan 22, 2016 at 11:05 AM, Sterling Smith <smi...@fusion.gat.com> wrote:

Thanks for lmfit.  It is very useful.

In my curve fitting, I am getting the message "Tolerance seems to be too small.. Could not estimate error-bars" often.  Which of the ftol, gtol, xtol should I change to start with (if a general statement can be made)?

That message seems fairly unusual to me.   But, all of ftol, gtol, and xtol are probably too small for most real data analysis problems, and the defaults should probably 1.e-5 instead of 1.e-7.    It's hard to predict which of these most important for any particular problem, so increasing all of them is probably reasonable.      Are parameters changing values from their initial values?

Yes, the parameters are changing from their initial values.  I will try the larger 1e-5 for all of them.

FWIW, if you have parameters with wildly varying scales (say, some values that are 1.e-6 and some that are 1.e8) these problems with tolerances being too tight can be exacerbated.  Generally speaking, MINPACK (leastsq) does a very good at handling large difference in scales, but the scale can't be infinite.  Also, setting bounds on parameters can make the automatic scaling (and the values for the tolerance parameters) harder.

Yes, I the parameters can be largely different.  I noticed the `diag` keyword for leastsq, but I wasn't sure how that array should correspond to the parameters of my lmfit.Parameters object.  I am also using constraints.

On the other hand, I sometimes get very large errorbars.  Is there any parameter I can change to decrease the size of the errorbars?

Well, if the fit is "bad" the errorbars will tend to be large.  That would generally imply that the model is not perfect..  But "bad" is also relative to the estimated uncertainty in the data (epsilon in the docs). 

Actually, the nominal values of the fit seem reasonable. 

Hope that helps.   Of course, giving more details about what you're doing would give us much more to go on.

I appreciate your time and attention.  Since you ask, here are some more details:

In a tokamak, 1d profiles from core to edge are traditionally sufficient to characterize the plasma.  For a typical physical quantity (electron density or temperature, for example) 10-30 measurements are made across the profile.  Codes which predict the radial turbulent transport of energy out of the plasma need the local radial gradient scale lengths of the physical quantity.  By fitting the data with the integral of scale lengths at a few locations, we are hoping to provide a robust characterization of the profile in such a way that is similar to what the turbulence codes do.  So the free parameters in our fit are a series of interior knot locations (with constraints that these should be sufficiently separated and ordered), the scale length at those locations and the endpoints, and a constant of integration (this is the parameters that could be very large for some physical parameters, but I suppose we could divide out the order of magnitude).  If you have further interest, I could post the code I am using for the fitting.

 

--Matt

-Sterling 

[A V]

I am having this problem with the initial values of a few parameters not changing, is there a usual cause for this?

Best wishes,

Adam

[Matt Newville]

Adam,

On Mon, Oct 10, 2016 at 3:16 PM, 'A V' via lmfit-py <lmfi...@googlegroups.com> wrote:

I am having this problem with the initial values of a few parameters not changing, is there a usual cause for this?

You're having the same problem that Sterling had and getting a "Tolerance seems to be too small" message?  Did you follow advice given? 

It sounds like you might be having a different problem.   There *are* indeed some common reasons for parameters not changing from their initial values.  This can happen if changing the value for a parameter  by the small amount used to calculate the derivative is not making a noticeable difference in the residual array.  There are many potential reasons for that, many of them discussed previously in this mailing list.

If you have an actual question, ask in a way that make us understand what problem you are actually having.  Include a small example showing the problem.

--Matt