Loading Unbound min and max in JSON string

[Matthew Giammar]

Hi there,

I work in a chemistry lab and we've written an analysis library which uses LMFIT for spectral fitting. Right now we're designing a web-app to act as a simple interface to the library.

The logic in the app is mostly written in Python using Plotly Dash and there are no problems on the Python side. However, due to some Dash limitations storing and updating the LMFIT Parameters object is done in JavaScript.

Here is an overview of my issue:

1.  a JSON string is sent to the JavaScript code using Parameters.dumps()
   
2.  The symbols "NaN", "Infinity", and "-Infinity" are replaced with "null" (the JavaScript null value, not a string). This is because JavaScript does not recognize these symbols.
   
3. This string is parsed to create a JavaScript object.
   
4. As the user interacts, the JavaScript object is updated. Unbounded min/max are represented by null.
5. At a certain point the JavaScript recreates the JSON string so it can be sent back to the Python side. null values for min and max are replaced with "-Infinity" and "Infinity" respectively. This JSON string looks the same as the original string sent to the JavaScript functions (minus any modifications made).
6. Parameters().loads(json_str) is called in Python to recreate the parameters object and run a fitting routine. Here is where the issue lies: The new JSON cannot be parsed and I'm getting an error "TypeError: '>' not supported between instances of 'int' and 'NoneType'"

I've tried other values other than "Infinity" including "Infinity" (a string), Number.POSITIVE_INFINITY (JavaScript infinity), "inf" (a string), and keeping the value null. Each time I still get a TypeError from a comparison.

Is there a certain value/string I should set min and max to in the JSON string for it to parse correctly? Please let me know if any additional info would be helpful or of any potential fixes. Thanks!

Best,

Matthew

[Renee Otten]

Hi Matthew, 

Your workflow on paper sounds reasonable and should work I think. The error message in you show is pretty clear: you cannot find out if an integer is bigger than a NoneType. 

A minimal, reproducing example is always useful - so in this case I would say the bare minimum to provide is the two JSON strings that you pass to and return from the JavaScript, which throws you this error.

Renee

--
You received this message because you are subscribed to the Google Groups "lmfit-py" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lmfit-py+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lmfit-py/6dc7272b-e814-4a76-bf22-f679caa8b26en%40googlegroups.com.

[Matt Newville]

Hi Matthew, Renee,

I have seen similar problems with JSON redefining Inf and NaN, though not with lmfit -- I guess I have not tried that.  Anyway, I agree it is a problem.  We should try to better accept JSON from other sources for `Parameters.loads()`.  I don't know that we'll ever get this to 100% correct, but we can try. 

I have an example, and I think a simple fix... so I'll bring that over to github.

To view this discussion on the web visit https://groups.google.com/d/msgid/lmfit-py/DFF34C17-77CA-4200-9A4A-3627D568A8AC%40gmail.com.

--

--Matt Newville <newville at cars.uchicago.edu> 630-327-7411

[Matthew Giammar]

Hi Renee and Matt,

Thank you for the quick responses! Here are the the JSON strings being passed and modified. I've bolded the values of interest.

JSON sent to JavaScript: 

{"unique_symbols": {"True": 1, "sys_0_site_0_isotropic_chemical_shift": 0.0, "None": null, "False": 0, "NAN": NaN, "erfc": {"__class__": "Callable", "__name__": "erfc", "pyversion": "3.7", "value": null, "importer": "scipy.special"}, "erf": {"__class__": "Callable", "__name__": "erf", "pyversion": "3.7", "value": null, "importer": "lmfit.confidence"}, "wofz": {"__class__": "Callable", "__name__": "wofz", "pyversion": "3.7", "value": null, "importer": "scipy.special"}, "pi": 3.141592653589793, "gamfcn": {"__class__": "Callable", "__name__": "gamma", "pyversion": "3.7", "value": null, "importer": "scipy.special"}, "inf": Infinity, "newaxis": null, "mth_0_rotor_frequency": 0.0, "Inf": Infinity, "sys_0_abundance": 100, "e": 2.718281828459045, "little_endian": 1, "nan": NaN, "infty": Infinity}, "params": [["sys_0_site_0_isotropic_chemical_shift", 0.0, true, null, -Infinity, Infinity, null, null, null, 0.0, null], ["sys_0_abundance", 100, false, "100", 0, 100, null, null, null, 100.0, null], ["mth_0_rotor_frequency", 0.0, true, null, -100.0, 100.0, null, null, null, 0.0, null]]}

JSON cleaned by JavaScript:

{"unique_symbols": {"True": 1, "sys_0_site_0_isotropic_chemical_shift": 0.0, "None": null, "False": 0, "NAN": null, "erfc": {"__class__": "Callable", "__name__": "erfc", "pyversion": "3.7", "value": null, "importer": "scipy.special"}, "erf": {"__class__": "Callable", "__name__": "erf", "pyversion": "3.7", "value": null, "importer": "lmfit.confidence"}, "wofz": {"__class__": "Callable", "__name__": "wofz", "pyversion": "3.7", "value": null, "importer": "scipy.special"}, "pi": 3.141592653589793, "gamfcn": {"__class__": "Callable", "__name__": "gamma", "pyversion": "3.7", "value": null, "importer": "scipy.special"}, "inf": null, "newaxis": null, "mth_0_rotor_frequency": 0.0, "Inf": null, "sys_0_abundance": 100, "e": 2.718281828459045, "little_endian": 1, "nan": null, "infty": null}, "params": [["sys_0_site_0_isotropic_chemical_shift", 0.0, true, null, null, null, null, null, null, 0.0, null], ["sys_0_abundance", 100, false, "100", 0, 100, null, null, null, 100.0, null], ["mth_0_rotor_frequency", 0.0, true, null, -100.0, 100.0, null, null, null, 0.0, null]]}

JSON recreated by JavaScript:

{"unique_symbols": {"True": 1, "sys_0_site_0_isotropic_chemical_shift": 0.0, "None": null, "False": 0, "NAN": NaN, "erfc": {"__class__": "Callable", "__name__": "erfc", "pyversion": "3.7", "value": null, "importer": "scipy.special"}, "erf": {"__class__": "Callable", "__name__": "erf", "pyversion": "3.7", "value": null, "importer": "lmfit.confidence"}, "wofz": {"__class__": "Callable", "__name__": "wofz", "pyversion": "3.7", "value": null, "importer": "scipy.special"}, "pi": 3.141592653589793, "gamfcn": {"__class__": "Callable", "__name__": "gamma", "pyversion": "3.7", "value": null, "importer": "scipy.special"}, "inf": Infinity, "newaxis": null, "mth_0_rotor_frequency": 0.0, "Inf": Infinity, "sys_0_abundance": 100, "e": 2.718281828459045, "little_endian": 1, "nan": NaN, "infty": Infinity}, "params":[["sys_0_site_0_isotropic_chemical_shift",0,true,null,null,null,null,null,null,0,null],["sys_0_abundance",100,false,"100",0,100,null,null,null,100,null],["mth_0_rotor_frequency",0,true,null,-100,100,null,null,null,0,null]]}

Also here are the lines which replace null min/max with infinity. It seems to be showing up as null in the JSON:

// Replace null min/max with +-Infinity

if (tmp[4] == null) {

     tmp[4] = -Infinity;

}

if (tmp[5] == null) {

     tmp[5] = Infinity;

}

Best,

Matthew