Minimum Number of Iterations Using ftol

[Gabriel Myers]

Hello all,

I am writing with a question regarding the optional argument to minimize, ftol. When working with the below example (from https://lmfit.github.io/lmfit-py/examples/example_fit_with_bounds.html), I found that when the ftol was set to massive values (say ftol = 10000000),  so large that one would expect immediate termination of the algorithm, the code still took 4 function evaluations to produce a fit.

p_true = Parameters()

p_true.add('amp', value=14.0)

p_true.add('period', value=5.4321)

p_true.add('shift', value=0.12345)

p_true.add('decay', value=0.01000) 

def residual(pars, x, data=None):

    argu = (x * pars['decay'])**2 

    shift = pars['shift']

    if abs(shift) > pi/2:

        shift = shift - sign(shift)*pi 

        model = pars['amp'] * sin(shift + x/pars['period']) * exp(-argu)

    if data is None:

        return model

    return model - data random.seed(0)

x = linspace(0, 250, 1500)

noise = random.normal(scale=2.80, size=x.size)

data = residual(p_true, x) + noise

fit_params = Parameters()

fit_params.add('amp', value=13.0, max=20, min=0.0)

fit_params.add('period', value=2, max=10)

fit_params.add('shift', value=0.0, max=pi/2., min=-pi/2.)

fit_params.add('decay', value=0.02, max=0.10, min=0.00) 

out = minimize(residual, fit_params, args=(x,), kws={'data': data}, ftol = 10000000)

fit = residual(out.params, x)

report_fit(out, show_correl=True, modelpars=p_true)

What is the reason for these preliminary iterations? I would think that since ftol is set so high the convergence criteria is immediately met, therefore there would not be the need for so many iterations.

Thanks,

Gabriel Myers

[Matt Newville]

On Fri, Apr 2, 2021 at 4:39 PM Gabriel Myers <myersga...@gmail.com> wrote:

Hello all,

I am writing with a question regarding the optional argument to minimize, ftol. When working with the below example (from https://lmfit.github.io/lmfit-py/examples/example_fit_with_bounds.html), I found that when the ftol was set to massive values (say ftol = 10000000),  so large that one would expect immediate termination of the algorithm, the code still took 4 function evaluations to produce a fit.

Yes, ftol of 1e+7 is absurd.  The algorithm did terminate immediately.  I cannot imagine what you were expecting setting ftol to 1e+7 to do.   That's 14 orders of magnitude larger than the default value, and sort of breaks the concept of what that value means.

Well, it has to compute the Jacobian by finite differences.  It is "minimal" to make 4 function evaluations for 4 variables.  If 4 function calls seem excessive, you're looking in the wrong place.  If you were trying to understand the algorithm in detail, the source code (Fortran MINPACK-1, lmdif.f) is readily available and pretty readable, especially considering its age.   

[Gabriel Myers]

Hello Matt,

Thank you for taking the time to read and respond to my question. I suppose I was only setting ftol to such a large value to play with the deeper function of the algorithm at an absurd value. I appreciate you mentioning the source code, I will look deeper into that fortran and see what I can find.

Gabriel Myers