Specific array failing to be modeled, code claims NaN inputs

Dakota Rose

unread,

Aug 27, 2019, 5:55:14 PM8/27/19

to lmfit-py

So I have been using lmfit to model the scattered light Bidirectional Scatter Distribution Function (BSDF). It has been working wonderfully, until I tried a new set of data. Below is sufficient snippets of my code, including the arrays I am using.

def BSDF(theta_s,b0,l,s):

theta_0_radx = 0*pi/180 #change to call in incident angle dynamically

theta_s_radx = theta_s*pi/180

beta_0x = np.sin(theta_0_radx) # this is the sin of the incident angle

betax = np.sin(theta_s_radx) # this is the sin of the measured angle

BSDF = b0*(1+((abs(betax - beta_0x))/l)**2)**(-s/2)

return np.log(BSDF)

I am returning the log of the BSDF to ease the fitting, and this has worked for all other data sets. The two arrays I am giving lmfit are:

angle = [-75.022, -74.022, -73.022, -72.022, -71.022, -70.022, -69.022, -68.022, -67.022,

-66.022, -65.022, -64.022, -63.022, -62.022, -61.022, -60.022, -59.022, -58.022,

-57.022, -56.022, -55.022, -54.022, -53.022, -52.022, -51.022, -50.022, -49.022,

-48.022, -47.022, -46.022, -45.022, -44.022, -43.022, -42.022, -41.022, -40.022,

-39.022, -38.022, -37.022, -36.022, -35.022, -34.022, -33.022, -32.022, -31.022,

-30.022, -29.022, -28.022, -27.022, -26.022, -25.022, -24.022, -23.022, -22.022,

-21.022, -20.022, -19.022, -18.022, -17.022, -16.022, -15.022, -14.022, -13.022,

-12.022, -11.022, -10.022, -9.022, -8.022, -7.022, -6.022, -5.022, -4.022,

-3.022, -2.022, -1.522, -1.022, 0.588, 0.738, 0.888, 1.038, 1.538,

2.038, 2.538, 3.038, 4.038, 5.038, 6.038, 7.038, 8.038, 9.038,

10.038, 11.038, 12.038, 13.038, 14.038, 15.038, 16.038, 17.038, 18.038,

19.038, 20.038, 21.038, 22.038, 23.038, 24.038, 25.038, 26.038, 27.038,

28.038, 29.038, 30.038, 31.038, 32.038, 33.038, 34.038, 35.038, 36.038,

37.038, 38.038, 39.038, 40.038, 41.038, 42.038, 43.038, 44.038, 45.038,

46.038, 47.038, 48.038, 49.038, 50.038, 51.038, 52.038, 53.038, 54.038,

55.038, 56.038, 57.038, 58.038, 59.038, 60.038, 61.038, 62.038, 63.038,

64.038, 65.038, 66.038, 67.038, 68.038, 69.038, 70.038, 71.038, 72.038,

73.038, 74.038]

BSDF = [-13.566, -14.176, -11.592, -9.306, -7.071, -7.184, -7.169, -7.161, -7.147,

-7.136, -7.131, -7.126, -7.112, -7.099, -7.078, -7.066, -7.058, -7.047,

-7.027, -7.003, -6.979, -6.959, -6.944, -6.916, -6.907, -6.912, -6.893,

-6.863, -6.836, -6.805, -6.778, -6.755, -6.722, -6.688, -6.665, -6.629,

-6.603, -6.578, -6.518, -6.479, -6.427, -6.379, -6.339, -6.323, -6.296,

-6.275, -6.243, -6.198, -6.146, -6.101, -6.029, -5.964, -5.925, -5.867,

-5.796, -5.715, -5.61, -5.509, -5.388, -5.27, -5.139, -5.056, -4.903,

-4.653, -4.424, -4.184, -3.959, -3.718, -3.446, -3.178, -2.926, -2.67,

-2.315, -1.855, -1.5, -0.826, 2.6, 2.099, 0.545, -0.46, -1.702,

-2.033, -2.187, -2.39, -2.72, -3.022, -3.346, -3.644, -3.939, -4.159,

-4.368, -4.579, -4.795, -5.02, -5.192, -5.313, -5.437, -5.563, -5.645,

-5.728, -5.828, -5.885, -5.958, -6.055, -6.125, -6.17, -6.198, -6.247,

-6.324, -6.375, -6.416, -6.44, -6.462, -6.457, -6.475, -6.496, -6.545,

-6.6, -6.661, -6.696, -6.736, -6.773, -6.802, -6.82, -6.844, -6.85,

-6.878, -6.917, -6.95, -6.975, -6.999, -7.017, -7.035, -7.054, -7.073,

-7.076, -7.101, -7.112, -7.126, -7.138, -7.15, -7.155, -7.173, -7.178,

-7.186, -7.193, -7.2, -7.208, -7.202, -7.213, -7.221, -7.233, -7.238,

-7.246, -7.251]

These are filtered arrays imported from a .csv file, however in my attempts to troubleshoot I copy/pasted them hardcoded. As can be seen, they are simple arrays and there are no NaN values in either array. Also note, the BSDF array has already had the log applied to it, so the format matches the output of the BSDF function already. Below is the very simple fitting code, which has always worked in the past.

gmodeltest = Model(BSDF)

seedparameterstest = gmodeltest.make_params(b0=1,l=0.1,s=1)

resulttest = gmodeltest.fit(BSDF,seedparameterstest,theta_s=angle)

There should be no issues, as this could not be simpler. But it fails, with the error being returned as "ValueError: The input contains nan values". I will post the full error text at the end of this post.

A little more context:

I have actually been working with this data set for a while, without issues. It was originally larger, however I was asked to filter down the data points by limiting (sin(theta_s) - sin(theta_0)) ≥ 0.01. In this specific case, theta_0 = 0, so this is simple. All that filter did was remove small angles, between around -1 and 0.5 in the original array. All the other data points were left untouched (as I filtered using Pandas).

But, after that filter was applied I continually get this error, and I cannot understand why. The data has not been altered, only shortened. Any light that could be shed on this problem would be greatly appreciated. Thank you!

Full error text:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-480-df058a11608d> in <module>
     73 #print(wholething_BSDF)
     74 
---> 75 resulttest = gmodeltest.fit(BSDF,seedparameterstest,theta_s=angle)

~/anaconda3/lib/python3.7/site-packages/lmfit/model.py in fit(self, data, params, weights, method, iter_cb, scale_covar, verbose, fit_kws, nan_policy, calc_covar, **kwargs)
   1005                              nan_policy=self.nan_policy, calc_covar=calc_covar,
   1006                              **fit_kws)
-> 1007         output.fit(data=data, weights=weights)
   1008         output.components = self.components
   1009         return output

~/anaconda3/lib/python3.7/site-packages/lmfit/model.py in fit(self, data, params, weights, method, nan_policy, **kwargs)
   1353         self.userkws.update(kwargs)
   1354         self.init_fit = self.model.eval(params=self.params, **self.userkws)
-> 1355         _ret = self.minimize(method=self.method)
   1356 
   1357         for attr in dir(_ret):

~/anaconda3/lib/python3.7/site-packages/lmfit/minimizer.py in minimize(self, method, params, **kws)
   1947                         val.lower().startswith(user_method)):
   1948                     kwargs['method'] = val
-> 1949         return function(**kwargs)
   1950 
   1951 

~/anaconda3/lib/python3.7/site-packages/lmfit/minimizer.py in leastsq(self, params, **kws)
   1490 
   1491         try:
-> 1492             lsout = scipy_leastsq(self.__residual, variables, **lskws)
   1493         except AbortFitException:
   1494             pass

~/anaconda3/lib/python3.7/site-packages/scipy/optimize/minpack.py in leastsq(func, x0, args, Dfun, full_output, col_deriv, ftol, xtol, gtol, maxfev, epsfcn, factor, diag)
    394             maxfev = 200*(n + 1)
    395         retval = _minpack._lmdif(func, x0, args, full_output, ftol, xtol,
--> 396                                  gtol, maxfev, epsfcn, factor, diag)
    397     else:
    398         if col_deriv:

~/anaconda3/lib/python3.7/site-packages/lmfit/minimizer.py in __residual(self, fvars, apply_bounds_transformation)
    536         else:
    537             return _nan_policy(np.asarray(out).ravel(),
--> 538                                nan_policy=self.nan_policy)
    539 
    540     def __jacobian(self, fvars):

~/anaconda3/lib/python3.7/site-packages/lmfit/minimizer.py in _nan_policy(arr, nan_policy, handle_inf)
   2164 
   2165         if contains_nan:
-> 2166             raise ValueError("The input contains nan values")
   2167     return arr
   2168 

ValueError: The input contains nan values

Matt Newville

unread,

Aug 27, 2019, 8:32:46 PM8/27/19

to lmfit-py

On Tue, Aug 27, 2019 at 4:55 PM Dakota Rose <dakota....@gmail.com> wrote:

So I have been using lmfit to model the scattered light Bidirectional Scatter Distribution Function (BSDF). It has been working wonderfully, until I tried a new set of data. Below is sufficient snippets of my code, including the arrays I am using.

def BSDF(theta_s,b0,l,s):
theta_0_radx = 0*pi/180 #change to call in incident angle dynamically
theta_s_radx = theta_s*pi/180
beta_0x = np.sin(theta_0_radx) # this is the sin of the incident angle
betax = np.sin(theta_s_radx) # this is the sin of the measured angle

BSDF = b0*(1+((abs(betax - beta_0x))/l)**2)**(-s/2)

return np.log(BSDF)

log(x) is NaN for x < 0. See https://lmfit.github.io/lmfit-py/faq.html#i-get-errors-from-nan-in-my-fit-what-can-i-do

Dakota Rose

unread,

Aug 28, 2019, 12:42:05 PM8/28/19

to lmfit-py

Yes, that is true. However there is no explicit logarithms of zero being taken in my code. the BSDF array has already had the log taken, and that needn't happen again. And for the angle, it is fed into abs(np.sin(theta_s)-np.sin(0)), and the absolute value eliminates negatives before log(BSDF) is returned.

Once again, this code has worked repeatedly with other data sets of the same format. I am trying to understand why this exact set is not working, when the same arrays including some data that was filtered out work. If you see a log issue, please point it out more specifically, as I am fairly sure I have checked that already.

What I mean by filtered data is this: below are the same two arrays but with more values included in the center of the array, and these arrays work in that code without issue. I am asking why.

Worth noting that I did do some rounding of that data that I sent you before as part of my attempts to get it to work, so the below arrays simply include more significant figures.

angle2 = [-7.5022e+01, -7.4022e+01, -7.3022e+01, -7.2022e+01, -7.1022e+01, -7.0022e+01,

-6.9022e+01, -6.8022e+01, -6.7022e+01, -6.6022e+01, -6.5022e+01, -6.4022e+01,

-6.3022e+01, -6.2022e+01, -6.1022e+01, -6.0022e+01, -5.9022e+01, -5.8022e+01,

-5.7022e+01, -5.6022e+01, -5.5022e+01, -5.4022e+01, -5.3022e+01, -5.2022e+01,

-5.1022e+01, -5.0022e+01, -4.9022e+01, -4.8022e+01, -4.7022e+01, -4.6022e+01,

-4.5022e+01, -4.4022e+01, -4.3022e+01, -4.2022e+01, -4.1022e+01, -4.0022e+01,

-3.9022e+01, -3.8022e+01, -3.7022e+01, -3.6022e+01, -3.5022e+01, -3.4022e+01,

-3.3022e+01, -3.2022e+01, -3.1022e+01, -3.0022e+01, -2.9022e+01, -2.8022e+01,

-2.7022e+01, -2.6022e+01, -2.5022e+01, -2.4022e+01, -2.3022e+01, -2.2022e+01,

-2.1022e+01, -2.0022e+01, -1.9022e+01, -1.8022e+01, -1.7022e+01, -1.6022e+01,

-1.5022e+01, -1.4022e+01, -1.3022e+01, -1.2022e+01, -1.1022e+01, -1.0022e+01,

-9.0220e+00, -8.0220e+00, -7.0220e+00, -6.0220e+00, -5.0220e+00, -4.0220e+00,

-3.0220e+00, -2.0220e+00, -1.5220e+00, -1.0220e+00, -5.2200e-01, -3.7200e-01,

-2.2200e-01, -2.0700e-01, -1.9200e-01, -1.7700e-01, -1.6200e-01, -1.4700e-01,

-1.3200e-01, -1.1700e-01, -1.0200e-01, -8.7000e-02, -7.2000e-02, -5.7000e-02,

-4.2000e-02, -2.7000e-02, -1.2000e-02, 3.0000e-03, 1.8000e-02, 3.3000e-02,

4.8000e-02, 6.3000e-02, 7.8000e-02, 9.3000e-02, 1.0800e-01, 1.2300e-01,

1.3800e-01, 1.5300e-01, 1.6800e-01, 1.8300e-01, 1.9800e-01, 2.1300e-01,

2.2800e-01, 2.4300e-01, 2.5800e-01, 2.7300e-01, 2.8800e-01, 4.3800e-01,

5.8800e-01, 7.3800e-01, 8.8800e-01, 1.0380e+00, 1.5380e+00, 2.0380e+00,

2.5380e+00, 3.0380e+00, 4.0380e+00, 5.0380e+00, 6.0380e+00, 7.0380e+00,

8.0380e+00, 9.0380e+00, 1.0038e+01, 1.1038e+01, 1.2038e+01, 1.3038e+01,

1.4038e+01, 1.5038e+01, 1.6038e+01, 1.7038e+01, 1.8038e+01, 1.9038e+01,

2.0038e+01, 2.1038e+01, 2.2038e+01, 2.3038e+01, 2.4038e+01, 2.5038e+01,

2.6038e+01, 2.7038e+01, 2.8038e+01, 2.9038e+01, 3.0038e+01, 3.1038e+01,

3.2038e+01, 3.3038e+01, 3.4038e+01, 3.5038e+01, 3.6038e+01, 3.7038e+01,

3.8038e+01, 3.9038e+01, 4.0038e+01, 4.1038e+01, 4.2038e+01, 4.3038e+01,

4.4038e+01, 4.5038e+01, 4.6038e+01, 4.7038e+01, 4.8038e+01, 4.9038e+01,

5.0038e+01, 5.1038e+01, 5.2038e+01, 5.3038e+01, 5.4038e+01, 5.5038e+01,

5.6038e+01, 5.7038e+01, 5.8038e+01, 5.9038e+01, 6.0038e+01, 6.1038e+01,

6.2038e+01, 6.3038e+01, 6.4038e+01, 6.5038e+01, 6.6038e+01, 6.7038e+01,

6.8038e+01, 6.9038e+01, 7.0038e+01, 7.1038e+01, 7.2038e+01, 7.3038e+01,

7.4038e+01]

BSDF2 = [-13.56630947, -14.17576332, -11.59218515, -9.30553056, -7.07133359,

-7.18443147, -7.16924992, -7.16097151, -7.14690119, -7.13566015,

-7.13114886, -7.12628449, -7.11207708, -7.09903686, -7.07795066,

-7.06581358, -7.05822939, -7.04690241, -7.02744012, -7.00316546,

-6.9793587, -6.95873283, -6.94390073, -6.91639247, -6.90675578,

-6.91236589, -6.89286667, -6.86278191, -6.83636528, -6.80519869,

-6.77848294, -6.75503419, -6.72210593, -6.68781686, -6.6648091,

-6.62860954, -6.60321609, -6.57845153, -6.51841955, -6.47857364,

-6.4271827, -6.37889329, -6.33903818, -6.32330752, -6.2963606,

-6.27488903, -6.24300757, -6.19824274, -6.14648227, -6.10127941,

-6.02937412, -5.96423856, -5.92492802, -5.86712527, -5.79622677,

-5.71534911, -5.61001894, -5.50854469, -5.38824207, -5.26953557,

-5.13909386, -5.05552806, -4.90317199, -4.65268117, -4.42368231,

-4.18383173, -3.95859064, -3.71769054, -3.44640397, -3.17797383,

-2.92620621, -2.67017655, -2.31506261, -1.85469927, -1.50013518,

-0.82645003, 0.18398684, 1.0321163, 3.3127304, 3.48859757,

3.26231838, 3.88362353, 4.83310225, 5.27453684, 5.34901048,

5.7547924, 6.39107924, 7.01121399, 7.74283596, 8.63657495,

10.37847872, 12.53681421, 13.94478289, 14.13686916, 13.23265173,

11.12741003, 9.24956109, 7.66387726, 6.31842743, 5.43720937,

4.90749454, 4.59096984, 4.06748711, 3.55305973, 3.28914788,

2.8953593, 2.86903462, 2.99071973, 2.98517676, 2.68648602,

2.3580198, 2.17361471, 1.98622916, 1.71900955, 2.59972232,

2.09863124, 0.54522705, -0.46044942, -1.7015532, -2.03255796,

-2.18658142, -2.39010569, -2.71992037, -3.02228161, -3.34642515,

-3.64352375, -3.93888046, -4.15856313, -4.36772933, -4.57852826,

-4.79499986, -5.01962559, -5.19161775, -5.313025, -5.43666032,

-5.56332512, -5.64532491, -5.72817763, -5.82764646, -5.88530435,

-5.95802977, -6.05461736, -6.12522454, -6.17011286, -6.19775098,

-6.24713129, -6.32442296, -6.37477685, -6.41550104, -6.44025478,

-6.46178823, -6.45731681, -6.47532372, -6.49564563, -6.54450202,

-6.59953556, -6.6608952, -6.69587492, -6.73632616, -6.77322439,

-6.80249477, -6.8197444, -6.84384195, -6.85043021, -6.87819648,

-6.91740166, -6.95045403, -6.97539193, -6.99910332, -7.01745854,

-7.0352478, -7.05370633, -7.07262992, -7.07641065, -7.10146139,

-7.11219975, -7.12566256, -7.1384271, -7.14995424, -7.1553196,

-7.17276303, -7.17799044, -7.18575108, -7.19304023, -7.19984706,

-7.20818492, -7.20212634, -7.21278699, -7.22123391, -7.23293156,

-7.23820568, -7.2461694, -7.25066821]

Matt Newville

unread,

Aug 28, 2019, 1:58:55 PM8/28/19

to lmfit-py

On Wed, Aug 28, 2019 at 11:42 AM Dakota Rose <dakota....@gmail.com> wrote:

Yes, that is true. However there is no explicit logarithms of zero being taken in my code. the BSDF array has already had the log taken, and that needn't happen again. And for the angle, it is fed into abs(np.sin(theta_s)-np.sin(0)), and the absolute value eliminates negatives before log(BSDF) is returned.

The code you posted has:

BSDF = b0*(1+((abs(betax - beta_0x))/l)**2)**(-s/2)

return np.log(BSDF)

where 'b0', 'l', and 's' are passed in as function argument and made into Parameters. They can potentially have any floating point value. So BSDF can definitely be negative (any negative value for b0 will ensure that), and the log of that will definitely give NaN.

--Matt

Dakota Rose

unread,

Aug 28, 2019, 4:20:52 PM8/28/19

to lmfit-py

Ok, I totally see what is going on now. Thank you! Previously, with the unfiltered data, b0 was always positive since it is the height of the peak, which was very large. However, I filtered out that peak, and thus b0 was chosen to be negative instead, absolutely giving this error.

Again, thank you!

Dakota

On Wednesday, August 28, 2019 at 10:58:55 AM UTC-7, Matt Newville wrote:

Reply all

Reply to author

Forward