Numpy masked_arrays are not treated as I would except

[Axel Buß]

I think lmfit is not treating np.masked_arrays as well as it could. 

For instance, if I want to perform a fit on a simple pair of x-y arrays,

and have y changed to an np.masked_array, then the mask is ignored. 

It seems that lmfit is doing it correctly when applying the mask on x as well, though.

However, this can become confusing and there should be at least a warning I guess.

In particular, when then doing the result.plot() method, it can look like the fit just went completely wrong, because then, it does ignore the masked values.

I suggest that lmfit helps to prevent such easily made mistakes.

[Matt Newville]

Hi Alex, 

Please provide a short example that shows the problem, or some examples. 

To be clear, the underlying fitting algorithms will not deal with Nans or Infs or any non-finite double-precision numeric values at all.   We could probably do a better job at more gracefully doing that for the user, but I'm sure there are complications and corner cases that might make that a bit challenging.

--Matt

--
You received this message because you are subscribed to the Google Groups "lmfit-py" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lmfit-py+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lmfit-py/9533e1bf-4b5f-41ca-b8cf-4b24aabf77a7n%40googlegroups.com.

--Matt

[Axel Buß]

No Problem, Matt. 

https://gist.github.com/xolotl90/76182e7e5356aeec7385e7ba9cfdaf2b

I uploaded some code to this gist.

Cheers,

Axel

[Vital Fernández]

I wonder if you found the solution to this issue.

In this stackoverflow post, Matt Newville sugest using the good_points argument but I think that it  is not longer availble in the ".fit" function.

Thank you for any advice

Vital

[Matt Newville]

Hi Vital,

On Fri, Apr 22, 2022 at 5:41 PM Vital Fernández <lati...@gmail.com> wrote:

I wonder if you found the solution to this issue.

 

The fitting algorithms do not ever handle NaN (or Inf) values, no matter whether they are meant to signal "bad floating-point value" or "missing".  I do not mean "they don't handle them as well as they could", I mean "they will always and reliably fail" when sent any values that are NaN or Inf.  

What they require is exactly and only ndarrays of type Float64.  Anything else has to be coerced to finite and valid Float64 before sending it to the actual fitting functions.  Lmfit tries to handle some of the more common cases of values of "arrays that are not valid Float64".  Maybe it could look for more cases and try to guess what the user meant when it passed in values of NaN, but by far the best approach is to remove these yourself.  

In this stackoverflow post, Matt Newville sugest using the good_points argument but I think that it  is not longer availble in the ".fit" function.

Why do you think that no longer works?  It seems like it should work to me.

Thank you for any advice

The advice continues to be: remove NaNs.

--Matt