differential evolution with workers>1

[Frederic Noel]

Hi together,

I am currently trying to use the method='differential_evolution' with the keyword workers, which enables parallelisation using multiple cpu-cores. 

With workers=1 everything runs fine.

When i set workers>1 on my windows maschine the prozess just runs forever/freezes, while i can see the number of selected cores on my pc to start to work for a few seconds, but then stop again. I can only end this my manually restarting the kernel.

I also tried the same thing in a colab notebook (as it probably uses some linux system). There i get after some time the error message "AbortFitException: fit aborted: too many function evaluations 12000".

When i call the differential_evolution method of scipy directly this does not happen and the fit works out fine, also with workers>1.

I set up an example code (based on the differential evolution example from scipy), comparing the two cases:

# scipy directly

import numpy as np
from scipy.optimize import rosen, differential_evolution
bounds = [(0,2), (0, 2), (0, 2), (0, 2), (0, 2)]
if __name__ == "__main__":
    result = differential_evolution(rosen, bounds, updating='deferred',workers=2)
result.x, result.fun

-> (array([1., 1., 1., 1., 1.]), 0.0)

# lmfit

import lmfit
def resid(params):
    x1 = params['x1'].value
    x2 = params['x2'].value
    x3 = params['x3'].value
    x4 = params['x4'].value
    x5 = params['x5'].value
    x=np.array([x1,x2,x3,x4,x5])
    return rosen(x)
params = lmfit.Parameters()
params.add('x1', 0.5, min=0, max=2.0)
params.add('x2', 1.0, min=0, max=2.0)
params.add('x3', 1.0, min=0, max=2.0)
params.add('x4', 1.5, min=0, max=2.0)
params.add('x5', 1.0, min=0, max=2.0)
if __name__ == "__main__":
    o2 = lmfit.minimize(resid, params, args=None, method='differential_evolution', updating='deferred', workers=2)
print("\n\n# Fit using differential_evolution:")
lmfit.report_fit(o2)

-> freezes/runs forever

# expected result/result with workers=1

-> # Fit using differential_evolution: [[Fit Statistics]] # fitting method = differential_evolution # function evals = 12001 # data points = 1 # variables = 5 chi-square = 3.5540e-09 reduced chi-square = 3.5540e-09 Akaike info crit = -9.45518933 Bayesian info crit = -19.4551893 ## Warning: uncertainties could not be estimated: [[Variables]] x1: 0.99987098 (init = 0.5) x2: 0.99981640 (init = 1) x3: 0.99996904 (init = 1) x4: 0.99925413 (init = 1.5) x5: 0.99844780 (init = 1)

Thus this code gives sensible results for workers=1, but runs forever/gives the abovementioned error for workers>2.

Any ideas what is generating this behaviour? Do i have to use/pass the workers keyword in a different way within lmfit?

Thanks a lot for the help.

Cheers,

Frederic

[Matt Newville]

Hi Frederic, 

Sorry for the trouble.  I have to admit that I never use differential evolution and have no idea what could be going wrong.   I would guess that there may have been changes upstream (scipy's differential_evolution) that we have not properly adapted to.   If we don't have any tests that use more than 1 worker, we probably should!

If you or anyone else has suggestions for fixing or improving this or adding/adapting tests, please let us know with a github issue or pull request.

--
You received this message because you are subscribed to the Google Groups "lmfit-py" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lmfit-py+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lmfit-py/41962cde-3ea2-4161-a196-3ec96cd1fe47n%40googlegroups.com.

--

--Matt Newville <newville at cars.uchicago.edu> 630-327-7411

[Andres Zelcer]

Hi all

This issue arises because you are indeed exceeding the default number of function evaluations that lmfit sets by default. Just add `max_nfev=55000` as a parameter to `lmfit.minimize` and it should work.

Nevetheless, lmfit wrapper around scipy's  differential_evolution does not forward this limit to the optimizer. I will try to make a PR for this soon.

It seems to be frozen (or running forever), because lmfit wraps your objective function in a little python snippet, and GIL contention happens when more than one thread wants to run python code. It is slower using many workers than using a single thread (I *did* test it, it happens also with bare scipy's). In my laptop, running the provided example takes 5.7 seconds for scipy's differential_evolution and 40s for lmfit's with `max_nfev=55000. With only one worker the figures are 5.3 s and 9.5 s.

   Cheers,

     Andrés

[Renee Otten]

Hi all, 

Andrés is correct that it works when increasing the number of allowed function evaluations (`max_nfev`, is the argument for this used in lmfit); the fit finishes properly with acceptable fitting parameters. The value that you specify as ‘max_nfev’ is forwarded to the DE method in SciPy as “maxiter”; from the documentation that perhaps isn’t exactly what one would expect to happen. The exact equivalent in the DE solver would be “maxfun”, whereas when setting “maxiter” the maximum number of function evaluations (with no polishing) is: ``(maxiter + 1) * popsize * N``. The SciPy authors suggest that setting “maxiter” likely makes more sense so that’s what we do and I am not convinced that should change… we anyway calculate the number of function evaluations based on how often our “residual” function is called.

If I set "max_nfev=100000” both with workers=1 and workers=2, then lmfit reports “functions evals” as 49431 versus 52356, so that seems all reasonable to me. Doing the same directly with SciPy as you showed in your example as well gives:

 (nit: 631, nfev: 47406) and (nit: 626, nfev: 47031), respectively. To me that seems all very close and thus should be a workable situation. 

Again, doing multiprocessing adds quite a bit of overhead and might only be worthwhile for certain situations; clearly the example shown here it isn’t the case…. The implementation of multiprocessing has an OS dependent component as far as I know; it works for me on Linux and on macOS (by switching to “fork” instead of “spawn”: import multiprocessing ; multiprocessing.set_start_method('fork’)). I haven’t tested it on Windows so it could be that there is an issue on that platform. In any case I doubt that would be a specific lmfit problem as we just pass stuff directly to scipy.differential_evolution.

Best, 

Renee

To view this discussion on the web visit https://groups.google.com/d/msgid/lmfit-py/ae322b20-eb5a-4ead-848b-cf12285ea356n%40googlegroups.com.