missing attributes in MinimizerResult

[Marc W. Pound]

Hi all,

Is there a reliable way to check the validity of a MinimizerResult based on its attributes for all (or even most) available fitting methods?  Specifically, I’m referring to the MinimizerResult attributes such as status, success, errorbars, message.   The documentation says these are “possible” attributes -- one learns after the fact how important that single word is in the docs as indeed they are only sometimes present.  Having the attributes always initialized to a known default if not set by the method would be kinder to the developer (I note that Minimizer does initialize these, while MinimizerResult does not).   Then something like this would not raise AttributeError depending on the method chosen:

fitresult = minimize(...,method=kwargs[‘method’])

if fitresult.success and fitresult.errorbars:

# do something

else:

# do something else

I think the workaround is to use Minimizer.minimize()  instead of minimize() and check the attributes on the Minimizer instance. 

Version info: lmfit 1.0.3, python 3.7.10

thanks,

Marc

[Matthew Newville]

Hi Marc,

On Tue, Nov 2, 2021 at 2:17 PM Marc W. Pound <mpo...@umd.edu> wrote:

Hi all,

Is there a reliable way to check the validity of a MinimizerResult based on its attributes for all (or even most) available fitting methods? 

I guess it sort of depends on what you mean by "validity".

Specifically, I’m referring to the MinimizerResult attributes such as status, success, errorbars, message.   The documentation says these are “possible” attributes -- one learns after the fact how important that single word is in the docs as indeed they are only sometimes present.  Having the attributes always initialized to a known default if not set by the method would be kinder to the developer (I note that Minimizer does initialize these, while MinimizerResult does not).   Then something like this would not raise AttributeError depending on the method chosen: 

fitresult = minimize(...,method=kwargs[‘method’])

if fitresult.success and fitresult.errorbars:

# do something

else:

# do something else

Yeah, I don't disagree with that, at least not in principle.  Certainly, *some* attributes could have default values indicating "not a valid value", perhaps initialized as `None`.  That is, if `status` is `None`, the fit probably never really even happened.....

I think the workaround is to use Minimizer.minimize()  instead of minimize() and check the attributes on the Minimizer instance. 

Yeah, I do agree that MinimizerResult could be better.  A PR to give initial values of `None` (or some other invalid value) would be welcome.

Version info: lmfit 1.0.3, python 3.7.10

thanks,

Marc

--
You received this message because you are subscribed to the Google Groups "lmfit-py" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lmfit-py+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lmfit-py/3ae05403-5753-4050-ac3e-2ed78655fcefn%40googlegroups.com.

--Matt 

[Marc W. Pound]

Hi Matt,

On Thursday, November 4, 2021 at 12:19:41 AM UTC-4 matt.n...@gmail.com wrote:

Hi Marc,

On Tue, Nov 2, 2021 at 2:17 PM Marc W. Pound <mpo...@umd.edu> wrote:

Hi all,

Is there a reliable way to check the validity of a MinimizerResult based on its attributes for all (or even most) available fitting methods? 

I guess it sort of depends on what you mean by "validity".

Yes, I struggled with that noun when writing the original message!  I didn't want to use 'status' since that is the name of one of the attributes in question.     But I think you understand the basic idea is 'sanity check.'

Specifically, I’m referring to the MinimizerResult attributes such as status, success, errorbars, message.   The documentation says these are “possible” attributes -- one learns after the fact how important that single word is in the docs as indeed they are only sometimes present.  Having the attributes always initialized to a known default if not set by the method would be kinder to the developer (I note that Minimizer does initialize these, while MinimizerResult does not).   Then something like this would not raise AttributeError depending on the method chosen: 

fitresult = minimize(...,method=kwargs[‘method’])

if fitresult.success and fitresult.errorbars:

# do something

else:

# do something else

Yeah, I don't disagree with that, at least not in principle.  Certainly, *some* attributes could have default values indicating "not a valid value", perhaps initialized as `None`.  That is, if `status` is `None`, the fit probably never really even happened....

Some of the fitting methods seem to expect (or assume) the attributes were set because they only set them if something bad happened. For instance, emcee() sets result.success = False for an aborted fit, but doesn't  otherwise set results.success = True.   

 

I think the workaround is to use Minimizer.minimize()  instead of minimize() and check the attributes on the Minimizer instance. 

Yeah, I do agree that MinimizerResult could be better.  A PR to give initial values of `None` (or some other invalid value) would be welcome.

Since Minimizer has already made a choice on the default attribute values, I propose  changing line 695 in Minimizer.prepare_fit  to pass those defaults when its MinimizerResult is instantiated.   i.e.  from

 self.result = MinimizerResult()

to 

 self.result = MinimizerResult(covar=self.covar, success = self.success, errorbars = self.errorbars, etc)

(but neatly wrapping them in a kwargs dict instead of a long line of parameters).   

I'm happy to do a PR on this.

thanks,

Marc

Version info: lmfit 1.0.3, python 3.7.10

thanks,

Marc

--Matt 

[Renee Otten]

Hi Marc, Matt, 

Sure for some attributes this is possible, but for others we add things depending on what the solver in SciPy returns and I think that’s fine. 

 

I think the workaround is to use Minimizer.minimize()  instead of minimize() and check the attributes on the Minimizer instance. 

Yeah, I do agree that MinimizerResult could be better.  A PR to give initial values of `None` (or some other invalid value) would be welcome.

Since Minimizer has already made a choice on the default attribute values, I propose  changing line 695 in Minimizer.prepare_fit  to pass those defaults when its MinimizerResult is instantiated.   i.e.  from

 self.result = MinimizerResult()

to 

 self.result = MinimizerResult(covar=self.covar, success = self.success, errorbars = self.errorbars, etc)

(but neatly wrapping them in a kwargs dict instead of a long line of parameters).   

I'm happy to do a PR on this.

It seems to me that for at least a subset of them (i.e, "aborted" and "errorbars") we already set the defaults on lines 717-718. “success” is set to False when the fit is aborted, and its a attribute of the OptimizerResult from all SciPy algorithms so it should be set to True when a fit actually converged. “covar” is set when it succeeds in calculating the covariance matrix, either directly from the SciPy algorithm or trough the use of numdifftools; if not, it will indeed not be present. 

So I guess I am not really clear about which attributes other than “covar” you are exactly missing, but admittedly haven’t spend much time looking at it either. But sure, for a *limited* number of attributes we can set a default value either by adding them around lines 717-718 or perhaps as you suggested in the call of MinimizerResult. I’m not sure yet what would be preferable.

Renee

[Matthew Newville]

Hi Marc, Renee, 

Would it be sufficient to change `MinimizerResult.__init__()` to make sure to always initialize values for `success`, `errorbars`, `covar`, and maybe a few others to `None`?  That way if the fit aborts or something unexpected happens, those attributes will be there with values that can at least be interpreted as "something went wrong".

--

You received this message because you are subscribed to the Google Groups "lmfit-py" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lmfit-py+u...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/lmfit-py/1947ED98-8A14-4CB7-AF43-1922DC32E399%40gmail.com.

[Marc W. Pound]

Hi Matt, Renee,

Thank you for your comments, as it helps me to understand the whys along with the whats.   I originally ran into this because I happened to try emcee after using leastsq.  emcee is an outlier in that it does not use scipy.optimize so many of the scipy.optimize attributes don't apply, and naturally aren't initialized.    But there are a few that should apply.  Matt's minimum list of success, errorbars, covar all initialized to None is a good one for all possible methods.   Then a typical test if result.success and  result.errorbars  does what is expected. 

Marc