Interest in interactive front-end

[Jonathan Okasinski]

Hello everyone!

I was wondering if there was interest in having an interactive front-end for some of lmfit's capabilities.  I'm new to lmfit, but need some specific functionality for my current project, and don't know of any good solutions already in the Python ecosystem.  

I understand that this would be a very large project in terms of maintenance & (optional) dependencies, so I completely understand (and rather expect) that this is out of scope. However, if there is interest, I'm happy to develop this either as part of lmfit or some kind of extension rather than as an unrelated module.

A little about me - I'm interested in these capabilities for my current work in x-ray detector development.  Matt, I don't know if you recall specific detectors on the APS beamlines, but I work for the group behind the MM-PAD detectors (Cornell, Sol Gruner).  They're high dynamic range, high frame rate, low pixel count units with Si (and soon CdTe!) sensors.  

The functionality I'm looking to write is basically:

1) Interactive selection of initial parameters.  Rather than laboriously reading every coordinate off a graph, just drag around and resize the model components

2) Interactive plot of both the fit and the residuals

3) Interactive way to modify the composition of the model

I've got a matplotlib prototype of all of this by building off of HyperSpy's fitting capabilities, and this has been immensely improved my work "quality of life".  However, HyperSpy is extremely heavy for what I need and GPLv3.  Additionally, I'm trying to migrate off matplotlib and towards the Bokeh/pyviz ecosystem.

A little more on my use case: I usually know the form of what I'm trying to fit and have good initial guesses, but have a lot of free parameters.  For example, I might have 5 Gaussian peaks and a linear background, which is 17 free parameters.  This sort of tool makes it much, much easier for me to get accurate results quickly.  Plus, it's always grinded my gears a bit that (to my knowledge) no one in the Python community has built an open-source & improved version of Matlab's curve fitting toolbox, so there's some personal motivation in there for me as well.

Thanks a lot for your time & consideration!  Looking forward to hearing back from you guys either way.  Great work so far, been loving it!

All the best,

Jonathan

[Matt Newville]

Hi Jonathon,

On Fri, Nov 6, 2020 at 11:40 PM Jonathan Okasinski <jo...@cornell.edu> wrote:

Hello everyone!

I was wondering if there was interest in having an interactive front-end for some of lmfit's capabilities. 

Sure, maybe depending on what that all means to you.  I would say that an application something like fityk, based on Python and lmfit could be very useful to many people.  I have a couple of GUI apps that have dedicated fitting for specific types of data based on lmfit.   These are decidedly not a "general-purpose fitting app",  but one of them is actually not too far off, but only for 1D data.  I have not gone down the route of  "general-purpose fitting app", and am rather over-extended in projects I am working on.

I'm new to lmfit, but need some specific functionality for my current project, and don't know of any good solutions already in the Python ecosystem.  

Well, what specific functionality?   I can't really tell what your project is about except to add meta-data to images.

I understand that this would be a very large project in terms of maintenance & (optional) dependencies, so I completely understand (and rather expect) that this is out of scope. However, if there is interest, I'm happy to develop this either as part of lmfit or some kind of extension rather than as an unrelated module.

Without knowing what it is, it's hard to comment in detail,  but, sure!

A little about me - I'm interested in these capabilities for my current work in x-ray detector development.  Matt, I don't know if you recall specific detectors on the APS beamlines, but I work for the group behind the MM-PAD detectors (Cornell, Sol Gruner).  They're high dynamic range, high frame rate, low pixel count units with Si (and soon CdTe!) sensors.  

Great!  We use tend to use a variety of detectors, including PAD detectors.

The functionality I'm looking to write is basically:

1) Interactive selection of initial parameters.  Rather than laboriously reading every coordinate off a graph, just drag around and resize the model components

2) Interactive plot of both the fit and the residuals

3) Interactive way to modify the composition of the model

It's hard to disagree with any of these, but it sort of quickly leads to "what are you fitting"?  If the idea is to fit peaks in the images, then that's all probably doable.  Then again, many people use the sort of detectors you're talking about for X-ray Diffraction where the image is mapped in a very non-linear way to another "space" (basically, scattering angles), and then a model for intensity vs scattering angle is used.   That's somewhat harder, both because pre-processing (the sort of thing you might need meta-data for) is critical and because the conversion of model parameters to image data is actually complex and complicated.  I know that things quickly get complicated for many types of image processing and data analysis.

For one of the fitting GUIs I have (for X-ray fluorescence), there are a few critical but physical parameters (roughly, "calibration and noise"), a few "fudgy parameters" to control the known-to-be-not-quite-Gaussian-ness of the peaks, and then a selection of (literally!) atomic components that each imply a single weighting parameter (yeah, the problem is really "almost linear") for a fairly complex response function. 

For another fitting GUI (for X-ray absorption spectra) it really is more "select a background function" and "select a series of peak shapes", and the user can click on a graph to help set initial parameter values.  Bounds and constraint expressions can be entered into a form.  I also have a few other GUIs for XAS for background subtraction and other data fitting, but these fits tend to be embedded into a single function call with a few parameters.   Then there are a few comman-line /script-level interfaces to fitting some other types of data.... maybe akin to that HyperSpy.

I've got a matplotlib prototype of all of this by building off of HyperSpy's fitting capabilities, and this has been immensely improved my work "quality of life".  However, HyperSpy is extremely heavy for what I need and GPLv3.  Additionally, I'm trying to migrate off matplotlib and towards the Bokeh/pyviz ecosystem.

OK, I don't know much about HyperSpy.  It looks to be command-line based, but I'm not certain of that.

For desktop applications, I'm pretty happy with matplotlib, but have and use a user-configurable wrapping of it for 1- and 2-D data (wxmplot).  It is often criticized for not being very fast, but it can be made reasonably responsive with some care.

I have been liking plotly quite a bit for web applications, but I would not consider writing a fitting application with a browser interface.  I am generally skeptical of browser-based applications for complex data analysis.  I also find Jupyter/IPython to be unsuitable for anything except simple demos and training.   I realize that this puts me firmly in the minority.  My experience writing and supporting scientists using GUI and non-GUI apps also puts me firmly in the minority. 

A little more on my use case: I usually know the form of what I'm trying to fit and have good initial guesses, but have a lot of free parameters.  For example, I might have 5 Gaussian peaks and a linear background, which is 17 free parameters.  This sort of tool makes it much, much easier for me to get accurate results quickly.  Plus, it's always grinded my gears a bit that (to my knowledge) no one in the Python community has built an open-source & improved version of Matlab's curve fitting toolbox, so there's some personal motivation in there for me as well.

I'm not familiar Matlab at all, but would guess that to be something like fityk or Origin?  I think that is completely doable...

Thanks a lot for your time & consideration!  Looking forward to hearing back from you guys either way.  Great work so far, been loving it!

OK. Cheers,

--Matt

[Jonathan Okasinski]

Hi Matt,

Great to hear from you.  A quick note, the data I'm interested in fitting is just 1D, so I'm looking for a general purpose 1D fitting app.  

As for my project, that's just it: image stacks with metadata added transparently.  While this may seem mundane or trivial it's only been possible recently due to i.e. pymeasure and has become more important due to the increased complexity of next-gen PADs with adaptive gain, 10kHz framing, and so on.  I also think that the knock-on effects will be large - speaking for myself at least, it's transformed my workflow from data collection-centric to analysis-centric.

PyX also enables automation, letting me acquire meaningful 5- or 6- dimensional datasets very quickly, but then I somehow have to interpret that data.  Enter my interest in interactive regression!  I want to extract some regressed values at every point (10s to 1000s) of my dataset.  But then I must have some way to quickly ensure that all regressions are sufficiently well-conditioned, and fix those that are not, very quickly.  I think this is where some degree of graphical interactivity is called for.  If lmfit had an interface that allowed me to quickly and interactively show the fit, change the regression parameters, and re-run the fit, I could use this interface to manually fix 10-100 of the worst fits without too much pain.

Interactivity is also something I think may be able to help crack the eventual challenge of visualizing the entire dataset.  This is what I call "sparse, medium-D data" - say from 4-8 data points per dimension and 4-8 dimensions - and I want to use interactive regression tools as a small-scale test of these ideas.

You may be surprised to know that I am also not a fan of Jupyter & broadly skeptical of web interfaces as well.  After all I grew up with matplotlib and never really transitioned away.  However, I don't want to write a web app: I just want the browser as my front-end, controlled interactively by a terminal.  One reason for this is portability: the browser makes cross-platform remote work (highly relevant right now!) far more practical.  

But a bigger driver for me is that I'm looking for a lot of interactivity, and despite my preference for matplotlib I think that within the past 6~12 months the Bokeh ecosystem has begun to outstrip matplotlib in this regard.  It's not that one couldn't do it in matplotlib, but 1) there'd probably be backend-specific code and 2) I think the high-level API offered by Panel and holoviews is unmatched, and upcoming projects like hvPlot and Lumen will only further this.  In principle, and I think in practice too, no custom JS will be needed; Bokeh manages both the JS and the client/server connection all on its own.

I rather like plotly as well, but it seems to me that I can put plotly inside Bokeh (via Panel) a lot easier than I can do the reverse.  But for this particular application, Bokeh already has built-in support for draggable points & shapes, whereas plotly doesn't and apparently won't (and this demo JS hack no longer works for me on Chrome/Firefox).

I understand if this isn't a direction you want to pursue for lmfit, but, if you're curious about the recent progress on Bokeh & co, such an add-on might serve as an interesting risk-free demo/way to see what it is or isn't capable of!

~ Jonathan

--
You received this message because you are subscribed to a topic in the Google Groups "lmfit-py" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/lmfit-py/oFv_6Y96qXc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to lmfit-py+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lmfit-py/CA%2B7ESbp8UdSjWLqzEO1cCin5MedCyR3tKyMK%2BjMOvcqZb%3D2DJw%40mail.gmail.com.

[William Holden]

Hi Jonathan,

Sounds like I neat project, I'll be interested to see how it progresses!

I explored a jupyter-based frontend a couple years ago (https://github.com/wholden/jupyter-lmfit) as an experiment, but that was before I knew about fityk, which ended up satisfying my needs. 

At the time I was interested in trying to do what you're describing by making some kind of clickable interface in plotly, but ran into the same general roadblock you described about lack of a high-level api for handling those kind of events. I haven't used Bokeh myself and didn't know they exposed that functionality, I'll have to check it out.

Will

[Evan Groopman]

I've been pretty pleased with PyQt (PySide) and pyqtgraph for building my own data analysis GUIs. pyqtgraph adds extra functionality for mouse/keyboard interaction with plots, and adds a host of useful widgets. Cross-platform, plenty of support around the web, and you can use Qt Designer to lay out the interface. I use Veusz a lot from publication figures, which is also based on PyQt.

I'm also fairly oversubscribed and it's a bit difficult for me to release software developed at work, but I could perhaps tinker with and share a basic implementation of a GUI for these purposes. 

- Evan

You received this message because you are subscribed to the Google Groups "lmfit-py" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lmfit-py+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lmfit-py/78c5f9cb-8214-48ed-b217-80b82f5de0bbn%40googlegroups.com.

[Ray Osborn]

Hi Jonathan,

I'm a bit slow to respond to this, and it may not be what you are looking for, but there is an interactive front-end to lmfit built into NeXpy (https://nexpy.github.io/nexpy/), which is a PyQt GUI for analyzing NeXus files containing x-ray or neutron scattering data. Although this seems very narrow, it will read most generic HDF5 files, and has import modules for multi-column text files and SPEC files. You can see a screenshot at https://nexpy.github.io/nexpy/pythongui.html#fitting-nexus-data. Every time 1D data are plotted, a 'Fit' button is added to the plot tabs, which will open a dialog that allows lmfit models to be added in arbitrary order with default parameters. Tooltips will display the model doctoring as you hover over the model selection menu.

It's open source, so you are welcome to see if any of the code is of interest to your needs. I suppose it could be generalized as a separate package with a bit of work. If someone wanted to work on adding support for linked parameters, for example, I would be very interested.

By the way, Jacob Ruff uses NeXpy at CHESS, so you might be able to ask him about it.

Ray

[Matt Newville]

Hi Jonathon, 

On Mon, Nov 9, 2020 at 12:57 PM Jonathan Okasinski <jo...@cornell.edu> wrote:

Hi Matt,

Great to hear from you.  A quick note, the data I'm interested in fitting is just 1D, so I'm looking for a general purpose 1D fitting app.  

As for my project, that's just it: image stacks with metadata added transparently. 

While this may seem mundane or trivial it's only been possible recently due to i.e. pymeasure and has become more important due to the increased complexity of next-gen PADs with adaptive gain, 10kHz framing, and so on.  I also think that the knock-on effects will be large - speaking for myself at least, it's transformed my workflow from data collection-centric to analysis-centric.

Sorry for the delay - busy times!  I think this all sounds interesting.    A general process for data reduction from "raw+meta" to "1D dataset" is not clear to me.  Sometimes an image *is* just an image and sometimes each pixel needs to be remapped to a dataspace (for example, scattering angle).  

PyX also enables automation, letting me acquire meaningful 5- or 6- dimensional datasets very quickly, but then I somehow have to interpret that data.  Enter my interest in interactive regression!  I want to extract some regressed values at every point (10s to 1000s) of my dataset.  But then I must have some way to quickly ensure that all regressions are sufficiently well-conditioned, and fix those that are not, very quickly.  I think this is where some degree of graphical interactivity is called for.  If lmfit had an interface that allowed me to quickly and interactively show the fit, change the regression parameters, and re-run the fit, I could use this interface to manually fix 10-100 of the worst fits without too much pain.

I don't disagree with any of this, but the phrase "General Purpose" you used above still sort of scares me.    

That is, here on the "lmfit side", we do have the basic tools to automate the fitting of the 1D datasets: "define a model as a set of peaks and lineshapes, save a model, load model, apply it to 10,000 1D datasets".   A GUI that helped with that graphically fitting of 1D datasets seems doable and interesting if actually some real, non-trivial work.  I think the specialized examples that exist, and very nice tools like NexPy show that this can be done.

A lmfit GUI that could set up and test models and then save the model definition and even **write the python script-let" to use that would be very good.  Then you could embed that 1d-model fitting into your own data reduction and processing workflow.  So that could be something like fityk in that it is general-purpose for "1-D" and maybe "simple 2-D" -- just Intensity(x, y) with data reduction at all.  

Interactivity is also something I think may be able to help crack the eventual challenge of visualizing the entire dataset.  This is what I call "sparse, medium-D data" - say from 4-8 data points per dimension and 4-8 dimensions - and I want to use interactive regression tools as a small-scale test of these ideas.

OK... I don't really understand you here, but maybe that's OK.   I sort of think that anything beyond 1-D or "simple 2-D"  cannot really be "general purpose". 

You may be surprised to know that I am also not a fan of Jupyter & broadly skeptical of web interfaces as well.  After all I grew up with matplotlib and never really transitioned away.  However, I don't want to write a web app: I just want the browser as my front-end, controlled interactively by a terminal.  One reason for this is portability: the browser makes cross-platform remote work (highly relevant right now!) far more practical.  

But a bigger driver for me is that I'm looking for a lot of interactivity, and despite my preference for matplotlib I think that within the past 6~12 months the Bokeh ecosystem has begun to outstrip matplotlib in this regard.  It's not that one couldn't do it in matplotlib, but 1) there'd probably be backend-specific code and 2) I think the high-level API offered by Panel and holoviews is unmatched, and upcoming projects like hvPlot and Lumen will only further this.  In principle, and I think in practice too, no custom JS will be needed; Bokeh manages both the JS and the client/server connection all on its own.

I have no real objection to any of that.  Well, except that yes, of course, one would have to pick a toolkit or backend at some point, including if you use holoviews, no?  Anyway, maybe a browser would make a good interface to anything as complicated as building models and doing and displaying fits to loads of user data,  but I would not be sure of that.  

I rather like plotly as well, but it seems to me that I can put plotly inside Bokeh (via Panel) a lot easier than I can do the reverse.  But for this particular application, Bokeh already has built-in support for draggable points & shapes, whereas plotly doesn't and apparently won't (and this demo JS hack no longer works for me on Chrome/Firefox).

Are draggable data points desirable?

I understand if this isn't a direction you want to pursue for lmfit, but, if you're curious about the recent progress on Bokeh & co, such an add-on might serve as an interesting risk-free demo/way to see what it is or isn't capable of!

I don't have a strong preference.  Again, I have been happy with plotly for timeseries or x/y plots, but I would not start with any browser-based app that read in and used data from users.

Anyway, it's all interesting to speculate about.  Personally, I would probably suggest starting with 1-D or simple 2-D fitting as a standalone unit ("lmfit GUI - fityk replacement") and think about how to have that as part of the analysis workflow along with domain-specific data reduction.

Cheers,

--Matt

[Shane Caldwell]

(Adding myself to the latecomers to this discussion.)

I've got two things to add here, but before I say them I'll just introduce how I use lmfit. I work for a private company and maintain software that depends on lmfit, ever since I happily found it on my introduction to Python about 5 years ago. We've used it to perform about 1e7 fits in production over that time.

My comments are:

1) I think the features being proposed here would be amazing to have in our use cases, and thanks Jonathan for proposing this. Since most of our fitting succeeds automatically, we would want to activate the interactivity on demand, less than 1% of the time. For example, when a fit fails it's often because our automated parameter initialization was bad, so we would interactively fix them.

2) We value the small dependency footprint that lmfit currently has. I'm not yet clear on how packaging would work with this proposal, and the notion of optional dependencies is new to me (reading about it now....). I'd be happy to have it tightly integrate with lmfit, but I think consumers should be free to install lmfit "headless." Sounds like that is the intent.

Best and thanks for a great library,

Shane

--

You received this message because you are subscribed to the Google Groups "lmfit-py" group.

To unsubscribe from this group and stop receiving emails from it, send an email to lmfit-py+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lmfit-py/CA%2B7ESbqkfOELTaLS7%3DgFaVBkg-p9g%2BebGhwzD84tMu5KORpuhw%40mail.gmail.com.

[Matt Newville]

Hi Shane, all,

Thanks for the discussion.  I think I am still unsure about what might be needed or what might be the preferred way to start working on this (or inspire folks to work on it ;)).   I'm aware of William Holden's ipython widgets and Ray Osborn's nexpy.  

My own GUI code with lmfit-fitting is sort of domain-specific and embedded in a larger application.  I put some screenshots of this at:   https://github.com/lmfit/lmfit-py/wiki/GUI-ideas. 

(BTW: if anyone would like to add to this or edit these pages, let me know -- I'm sort of new to github wikis, but it seemed like the right thing for this example and I'd be happy to give others access to that).

For sure, this is domain-specific at the moment for how to get data in and for any "pre-processing steps". It also makes a particular choice for toolkits (GUI and plotting). But it might be possible to pull out many of the components to make into a stand-alone app.  For example, the code to generate the page of parameters from a model component is mostly encapsulated in a class that could be used in a more general-purpose app. 

Is this the sort of thing you would want?  Use such a GUI app for interactive and/or problematic cases but be able to save the Model this interactive fit generates to then be able to "apply to 10,000 similar datasets"?

To view this discussion on the web visit https://groups.google.com/d/msgid/lmfit-py/CAOUO9Y0r0SFwatsGLnDsrtuLcS3FRNEW6AX%3D1LoPdsE%3D1WZN%3Dw%40mail.gmail.com.

--

--Matt Newville <newville at cars.uchicago.edu> 630-327-7411

[Jonathan Okasinski]

Hello everyone!

Apologies I got briefly pulled off to another project.  However, I much appreciate everybody's thoughts and I'm very glad there's interest.  I've got a relatively small Bokeh/Panel prototype put together.  You can only fit one set of data, but I think it's illustrative (details below).

William - Here are Bokeh's graph-editing tools, if that's of interest.

Evan - I also like pyqtgraph, though I've not tried Veusz.  I think for a desktop app this would be a great choice, but I've had some other issues with Qt in the past - remote access, low-level interface, and integrating other technologies with it.  

Ray - I did not know about NexPy's fitting interface!  That's very close to the base of what I was imagining, and I've replicated part of its interface.  I'll touch base with Jacob and see if he had some more thoughts.

Shane - Very glad to hear that someone who's already had to work around this problem thinks this is a viable solution!  1e7 fits is certainly a big number.  And I agree that lmfit should preserve the option to ship headless.

Matt - Apologies for any confusion with earlier remarks.  I hope my description below will clear things up & thanks for putting together the wiki.  Also:

> Is this the sort of thing you would want?  Use such a GUI app for interactive and/or problematic cases but be able to save the Model this interactive fit generates to then be able to "apply to 10,000 similar datasets"?

Yes, that's pretty much exactly what I'm looking for.

> Are draggable data points desirable?

See below, not dragging the data points, but dragging "handle points" for model components.  Though now you mention it, maybe that's a good strategy for exploring the impact of outliers on a regression!

> Personally, I would probably suggest starting with 1-D or simple 2-D fitting as a standalone unit ("lmfit GUI - fityk replacement") and think about how to have that as part of the analysis workflow along with domain-specific data reduction.

I think we're on the same page.  As for the analysis workflow, my approach here has been to make regression interactive & easy: then you simply re-run the analysis and regression top-to-bottom.  However, it would be nice to have changes in pre-processing reflected live in your regression window.  Doing this manually "`igs.data = new_data`" is certainly possible, but you've got me thinking...doing this in a truly live/transparent way may be possible in an ipython context.  

--- About my prototype ---

My code's not well documented & I'd like to put together an illustrative demo before I show everyone.  However, I think my description below will help add some specifics to discuss.

The prototype I wrote extends Model by adding an `interactive_guess` function.  It works like `guess`, but returns an `InteractiveGuessSession` object instead of a `ModelResult`.  `InteractiveGuessSession` is conceptually similar to `ModelResult`, but it's also attached to a front-end.  For example, it has a `params` attribute to get the current parameters displayed by the front-end.

Here's a picture of the front-end in my browser (aesthetics very much in-progress):

This is linked to a command-line session.  You can edit parameters in one of three ways:

1) Programmatically via `igs.params['g0_amplitude'].value = 1`

2) Interactively via the table on the right

3) Interactively by the dragging red points on the graph, or using the scroll wheel

This works with CompositeModels and all of this is synced together with the front-end.  FYI this is about ~300 lines of code.

To make a model component interactive, it must implement two functions:

1) a map from its Parameters to a polygon shape

2) a map from a polygon shape to its Parameters

It may optionally implement:

3) a way of updating its parameters when the scroll wheel is scrolled

If initial parameters are not specified to `interactive_guess`, it must also support `guess`.

Current holes include:

- Changing models is somewhat awkward.  It's patched in via a `set_model` method to `InteractiveGuessSession`, but you essentially have to rebuild everything.  You also can't change the data.  However, if you want to change that, is having a method on Model even a logical entry point for this function?

1) The interactivity relies on callbacks added to the Parameter objects.  This means if you replace the Parameter objects, for example using Parameters.update, the interactivity breaks; you have to modify the attributes of the Parameter objects without replacing them.

2) Because JSON doesn't support infinities, the min/max bounds are clipped to 1e308, which just looks ugly

3a) Right now you have to select the model component you want to drag via a dropdown.  It'd be nice if you could just double click, or if all of the model handles could be shown together.

3b) A bug in Bokeh means you have to click 4 times after each drag to drag again.  

As for dependency management, I think the easiest option is to attempt to import the requisite libraries:

- If it fails, issue a warning (or a log message)

- If it works, dynamically replace the class definitions with the interactive versions.  Or hack in some mixins by modifying __bases__.  

An alternative that I personally don't prefer is:

- Attempt to dynamically load the requisite libraries when any interactive functions are called (as I also modify Parameters, Parameter, pre-built models).  However, this requires distributing the dependencies around in the source code, and users will see functions that they expect to work but then might fail anyway.

Please let me know if anyone has thoughts on my approach here.  I'll try to send out a nice usable demo within the next few days.

Thanks again everyone for the informative discussion!

Jonathan

You received this message because you are subscribed to a topic in the Google Groups "lmfit-py" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/lmfit-py/oFv_6Y96qXc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to lmfit-py+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lmfit-py/CA%2B7ESbqhcn34vSot1DC3mUYyGfUZHV0W9PDq6FjiPeeW2%3DC7jg%40mail.gmail.com.

[Matt Newville]

Hi Jonathon,

On Wed, Nov 18, 2020 at 12:45 PM Jonathan Okasinski <jo...@cornell.edu> wrote:

Hello everyone!

Apologies I got briefly pulled off to another project.  However, I much appreciate everybody's thoughts and I'm very glad there's interest.  I've got a relatively small Bokeh/Panel prototype put together.  You can only fit one set of data, but I think it's illustrative (details below).

William - Here are Bokeh's graph-editing tools, if that's of interest.

Evan - I also like pyqtgraph, though I've not tried Veusz.  I think for a desktop app this would be a great choice, but I've had some other issues with Qt in the past - remote access, low-level interface, and integrating other technologies with it.  

Ray - I did not know about NexPy's fitting interface!  That's very close to the base of what I was imagining, and I've replicated part of its interface.  I'll touch base with Jacob and see if he had some more thoughts.

Shane - Very glad to hear that someone who's already had to work around this problem thinks this is a viable solution!  1e7 fits is certainly a big number.  And I agree that lmfit should preserve the option to ship headless.

Matt - Apologies for any confusion with earlier remarks.  I hope my description below will clear things up & thanks for putting together the wiki.  Also:

> Is this the sort of thing you would want?  Use such a GUI app for interactive and/or problematic cases but be able to save the Model this interactive fit generates to then be able to "apply to 10,000 similar datasets"?

Yes, that's pretty much exactly what I'm looking for.

OK.  I think I cannot tell if you think the screenshots shown on the wiki pages interesting or worth trying to use or emulate. 

For sure, this sort of conversation can turn into "well, I do this" with a reply of "well, this other thing works for me".   And, I think that could be OK: we don't currently have a "general-purpose lmfit app", ad maybe the desire to have one is simply is not that strong.   OTOH, if that is the actual goal, I think it would be helpful to discuss this as if the intention was to consider working together on such a project.  I'll continue making that assumption for a few more rounds of this discussion.  But I should also be realistic: I don't know that we'll agree on how to do this, even if the time and resources were available.

> Are draggable data points desirable?

See below, not dragging the data points, but dragging "handle points" for model components.  Though now you mention it, maybe that's a good strategy for exploring the impact of outliers on a regression!

I'm afraid I may not fully understand you here.  I think you might mean that you want to click and drag on a plot of data and have a model function "follow the mouse" - redrawing.   I'm not certain of that.  I'm also not certain how you would like that to happen, at least for a truly general case.

> Personally, I would probably suggest starting with 1-D or simple 2-D fitting as a standalone unit ("lmfit GUI - fityk replacement") and think about how to have that as part of the analysis workflow along with domain-specific data reduction.

I think we're on the same page.  As for the analysis workflow, my approach here has been to make regression interactive & easy: then you simply re-run the analysis and regression top-to-bottom. 

When you say "regression" do you mean "fit with lmfit" or do you really mean regression, which would not need an iterative approach?

However, it would be nice to have changes in pre-processing reflected live in your regression window.  Doing this manually "`igs.data = new_data`" is certainly possible, but you've got me thinking...doing this in a truly live/transparent way may be possible in an ipython context.  

Not sure I follow you there, but maybe that is specific to the toolkit choices you're making.  Should I understand what 'igs' is?

--- About my prototype ---

My code's not well documented & I'd like to put together an illustrative demo before I show everyone.  However, I think my description below will help add some specifics to discuss.

The prototype I wrote extends Model by adding an `interactive_guess` function.  It works like `guess`, but returns an `InteractiveGuessSession` object instead of a `ModelResult`.  `InteractiveGuessSession` is conceptually similar to `ModelResult`, but it's also attached to a front-end.  For example, it has a `params` attribute to get the current parameters displayed by the front-end.

Here's a picture of the front-end in my browser (aesthetics very much in-progress):

This is linked to a command-line session. 

Is this IPython or Jupyter?  Or is it in some browser?  I don't recognize it. 

You can edit parameters in one of three ways:

1) Programmatically via `igs.params['g0_amplitude'].value = 1`

2) Interactively via the table on the right

3) Interactively by the dragging red points on the graph, or using the scroll wheel

This works with CompositeModels and all of this is synced together with the front-end.  FYI this is about ~300 lines of code.

How do you know what parameters to change on mouse or scroll-wheel events?

To make a model component interactive, it must implement two functions:

1) a map from its Parameters to a polygon shape

2) a map from a polygon shape to its Parameters

 

What is polygon shape? 

It may optionally implement:

3) a way of updating its parameters when the scroll wheel is scrolled

If initial parameters are not specified to `interactive_guess`, it must also support `guess`.

Current holes include:

- Changing models is somewhat awkward.  It's patched in via a `set_model` method to `InteractiveGuessSession`, but you essentially have to rebuild everything.  You also can't change the data.  However, if you want to change that, is having a method on Model even a logical entry point for this function?

I may not be getting the expected processing steps well enough to be able to comment on that...

1) The interactivity relies on callbacks added to the Parameter objects.  This means if you replace the Parameter objects, for example using Parameters.update, the interactivity breaks; you have to modify the attributes of the Parameter objects without replacing them.

Hm, that seems kind of fragile.  Would subclassing Parameters be helpful for this? 

2) Because JSON doesn't support infinities, the min/max bounds are clipped to 1e308, which just looks ugly

Wait, JSON doesn't support infinities?  I think it does:

    >>> import json, numpy

    >>> json.loads(json.dumps(numpy.inf)) == numpy.inf

    True

3a) Right now you have to select the model component you want to drag via a dropdown.  It'd be nice if you could just double click, or if all of the model handles could be shown together.

3b) A bug in Bokeh means you have to click 4 times after each drag to drag again.  

As for dependency management, I think the easiest option is to attempt to import the requisite libraries:

- If it fails, issue a warning (or a log message)

- If it works, dynamically replace the class definitions with the interactive versions.  Or hack in some mixins by modifying __bases__.  

An alternative that I personally don't prefer is:

- Attempt to dynamically load the requisite libraries when any interactive functions are called (as I also modify Parameters, Parameter, pre-built models).  However, this requires distributing the dependencies around in the source code, and users will see functions that they expect to work but then might fail anyway.

Please let me know if anyone has thoughts on my approach here.  I'll try to send out a nice usable demo within the next few days.

What is your target user and what do you expect them to have to do to be able to do a fit?  It might also be good to start with a use-case.

I hope that helps,

--Matt

[Jonathan Okasinski]

Hello everyone!

As promised I've cleaned up the code for my demo; it's here on GitHub (gist).  There's a quick guide included.  Let me know if you have any issues playing with it or other thoughts!  

Matt - 

As my response is admittedly quite long, I've divided it into sections.  

Meta: about this discussion itself 

Goals: identifying users, use-cases, justifying this approach

Implementation: responding to your comments on my prototype so far

 --- meta ---

> For sure, this sort of conversation can turn into "well, I do this" with a reply of "well, this other thing works for me".   And, I think that could be OK: we don't currently have a "general-purpose lmfit app", ad maybe the desire to have one is simply is not that strong.   OTOH, if that is the actual goal, I think it would be helpful to discuss this as if the intention was to consider working together on such a project.  I'll continue making that assumption for a few more rounds of this discussion. 

Apologies, I'm trying to avoid the "well, I do this" sort of conversation.  My intention is also to work together, but I know many people are already fairly oversubscribed.  Thus, I was planning to do most or all of the coding work, and wanted feedback & ideas for usability/ecosystem/interface/code style/strategy sort of thing.  I figured I'd come out with a more complete/polished/widely applicable solution that hopefully someone else could find useful.  If this model is not desirable to you, I'd be happy to discuss distributing the work in a different manner.     

>  But I should also be realistic: I don't know that we'll agree on how to do this, even if the time and resources were available.

I could be wrong, but I really suspect we agree on almost all the major points.  With that being said, I'm having a hard time describing what's in my head, and I think that accounts for much of the apparent difference in opinion.  I hope that looking at the demo will help with this.  If this continues to be a problem I'm happy to set up a time to talk; if you're amenable to this, I've found it's a lot easier and faster to communicate clearly in real-time.  

 --- goals ---

> What is your target user and what do you expect them to have to do to be able to do a fit?  It might also be good to start with a use-case.

What is your target user?

Anyone who wants to fit complex models built out of simple components, and/or a lot of data to apply the same model to. Experimental physicists come to mind.

What do you have to do for a fit?

Call `model.interactive_guess(data)`, then edit your model to your satisfaction.  No more no less.

I'll talk about target users first.  Who's got...

1) Complex models from simple components?  While I don't know enough about experimental physics to argue it permeates the field, it seems packages that think primarily in this manner are pretty widespread.  Already 5 have been mentioned:

  - NexPy, as Ray mentioned  

  - XASViewer, as you mentioned on the wiki

  - HyperSpy, and its nearly 30 pre-built 1D model components

  - I can speak somewhat for the PAD group here at Cornell - this kind of modeling is the basis for many of our bread-and-butter measurements

  - Will's prototype GUI

I think there are probably others if we went hunting.  So now:

a) if you believe that there are users who want to regress complex models that are simple components summed together

b) and that the simple components have a natural interactive interface, such as the line I showed on the Gaussian peak that you can drag to resize it

Then I think this is a good use-case!

2) A lot of data to apply the same model to?  Part of what I'm selling with PyX is that even if you don't think you have a lot of data, you actually want & can have a lot of data, making this capability useful.  So I'd tell you that this is everyone in the hard sciences - but I know that's more conjectural, so I'll stick to:

  - Shane described this as exactly his use-case (in quantum computing)

  - Again speaking for my group, we'd love to be able to, say, automatically optimize detector performance, as measured by metrics derived from a fit

  - Materials scientists & co are also starting to think hard about high-throughput characterization & automated phase space exploration.  See for example SARA (recently brought in $15M) or this review in Joule.  One may consider those proposals relatively aggressive, but even in a conservative sense, deriving performance metrics from a fit seems perfectly reasonable to do with tons of phase-space data. 

I think Shane nailed the reasons why an interactive interface would be good for this use case:

> Since most of our fitting succeeds automatically, we would want to activate the interactivity on demand, less than 1% of the time. For example, when a fit fails it's often because our automated parameter initialization was bad, so we would interactively fix them.

Now, on to the actual fitting:

`model.interactive_guess(data)` is really a replacement for the `guess`/`fit`/`plot` workflow.  You gather your data, and do all the preprocessing on the command line.  Then, rather than trying to guess reasonable initial parameters ala `guess`/`fit`/`plot`, you call `interactive_guess` to launch a non-blocking GUI.  You select your first guess, and can then fit, interactively in a live setting.  If the fit isn't good enough, you can change your initial guesses/fitting parameters as-needed until it is, either via the GUI or the command line.  If you want to change your preprocessing and thus the data you're fitting, you may do that via the command line (again, GUI's still open, no need to explicitly "save" model/parameters/etc, it doesn't block the command line).  Once you're happy with your data & fit you can close out the GUI.  You're left with a command line object that's a complete record of your data, model, parameters, and fit results.  This can be saved/processed however you like.

I like this workflow for small-scale fitting (1-10 datasets).  When you have multiple data sets and well-defined models with many parameters (e.g. sum of 5 Gaussians), this is a significant quality of life & speed bump in my opinion.  But I think this actually enables new possibilities in the 10-10000 dataset range (more on the importance of this below).  Shane's figure on doing many regressions was a <1% failure rate.  At 10000 datasets that's <100 that need fixing.  Fixing 100 regressions by tweaking parameters, repeatedly doing `fit`, tweaking `params`, and looking at `plot`, sounds pretty time-consuming to me.  I think it'd take me at least 4-5 minutes each.  500 minutes is 8 hours - an entire work day.  However, with `interactive_guess`, I could:

1) Easily come up with a good first guess to apply to 10k datasets, via the already described workflow

2) Write a script to apply it to all 10000 datasets, then identify failed regressions (~100)

3) Call `interactive_guess` sequentially on all 100 datasets.  As soon as the user is satisfied with the fit/closes the GUI, launch the next GUI with the next dataset.  

I think I could do that at about 1 minute per regression.  That's 1.6 hours total - half my morning.  While that's not great, it's a lot better than a full day!  While you can disagree with my specific numbers, my core argument is that I expect this workflow to be 4-5x faster per regression, and for a sufficiently large number of regressions, this becomes an enabling difference in what is possible.

And finally, a use-case:

My prototypical use case is in extracting gain & noise figures from x-ray detectors.  This essentially involves taking 3D image stacks, doing some preprocessing down to 1D, and then fitting to a sum of 5 or so Gaussians. In theory this is only one fit - but these are early stage detectors.  Given the current empirical understanding of variations in gain & noise, I actually need to do 6 separate, but similar, fits, to properly characterize one detector at one operating point.  This is a little painful to do without the interactive interface.

But now, I want to optimize this detector, let's say to decrease its performance variation.  I'll do this by changing operating parameters.  But for every single set of operating parameters I try, I've got to do another 6 regressions.  Two things here:

1) There are literally hundreds of operating parameters with maybe 6-10 steps each.  A complete exploration would require 200**7 = 1e16 sets of parameters.  With some intuition, maybe I only care about 2-5 parameters and 3-6 steps at a time.  That's between 2**3 = 8 and 5**6 = 15000 sets of parameters.  At every point I need 6 regressions.  So that's between 54 - 90000 regressions.   

2) The "6 regressions" figure is empirically determined.  What if the nature of the performance variation has been misunderstood?  A complete exploration would require 2*128*128 (every pixel) = 16000 fits, per set of operating parameters.  Realistically, with maybe 100 regressions I can be fairly confident my set of 6 is sufficient.

What's my point?  I've got a space that theoretically requires 1e20 regressions to explore, but I've only got the tools to do 1-10 regressions.  I've got to guess, via my best intuition, the behavior of my detector in this huge space from a limited set of data points.  If I could do more regressions, on the 100-10000 scale, relatively easily, that would be tremendously beneficial, and massively limit the guesswork required.

Traditionally people close this gap with intuition & experience.  That's hard.  Moreover, even experienced people can't be 100% certain, because they've had to make assumptions; they simply don't have the data.  And with regression capability in the 100-10000 range, we could have much, if not all, of the data required.

Don't forget that my task here is to come up with just one number, supposedly requiring just one regression.  Imagine what a more complex task might require!  

I've described a narrow use-case: radiation imaging detector development.  But I think this general case of having a large operating parameter space and metrics to be extracted from regression is common, perhaps much more common than is broadly admitted.  For example, of the past 3 major projects I've worked on, I would claim that at least 2 were majorly held back for lack of practical bulk (100-1000 fits) regression ability.  I had enough raw data, but not the tools to make sense of it.

To support my claim I'll speak briefly of another project.  I was essentially doing I-V characterization of some novel perovskites on the ~100fA scale.  I discovered that these perovskites had all kinds of nasty hysteretic behaviors to voltage (literally degrading during measurement), light, air, humidity, temperature, you name it.  I wanted to understand how the apparent trap density & carrier mobility changed (again - just two numbers), which required a fit to a piecewise linear & quadratic model.   I stared at those numbers for months, one dataset at a time, but there were just too many variables that I couldn't directly control.  I couldn't understand any of the trends.  One day I buckled down and wrote a highly kludgy auto-regression framework that barely worked - and literally had insights overnight (which I manually confirmed, of course!)  

If I could have had easy access to the ability to perform several hundred regressions and verify the output earlier, it would have absolutely been a game changer for me.

 --- Implementation ---

> OK.  I think I cannot tell if you think the screenshots shown on the wiki pages interesting or worth trying to use or emulate. 

I do think it's interesting and worth emulating.  I've already taken the parameter table, and plan to implement the same kind of interface for `expr` & fit results.  Perhaps more uniquely, I really like the sidebar where it appears one can scroll through different files - I hadn't considered that yet, and I think that could be quite a valuable option to have especially in my & Shane's use cases.

> I'm afraid I may not fully understand you here.  I think you might mean that you want to click and drag on a plot of data and have a model function "follow the mouse" - redrawing.   I'm not certain of that.  I'm also not certain how you would like that to happen, at least for a truly general case.

Yes, I want the model to "follow the mouse".  I agree there's no true general solution, but recall that my use-case here is specifically models with a number of simple components.  I think this use is far more straightforward:

1) Select the component model you want to edit

2) Each component decides how it should "follow the mouse" - interpreting drags or scrolls

3) Only one component at a time is following the mouse; the model as a whole updates when any component changes

I've found this greatly speeds up the regression process, which as I discussed in "goals" is what I'm looking for.

This also relates to:

> How do you know what parameters to change on mouse or scroll-wheel events?

The component being actively edited decides for itself how to respond

Thus, each component in the model has to implement this interface.  However, many models are reusable, it takes very little code (~20 easy lines) to do so, and the interface is pretty reusable between shapes.  For example, the interface for a Gaussian distribution would be nearly identical to that for the Voigt or Lorentzian distributions, just with renamed parameters for center/spread/height.

Related:

> What is a polygon shape?

Each component should represent itself as a connected series of line segments (a "polygon shape").  When those line segments move, the component updates itself to match.

> Hm, that seems kind of fragile.  Would subclassing Parameters be helpful for this?

I agree that this is fragile and want to find a better way.  I've already had to subclass Parameter to hack in the callbacks.  I can think of a few possible workarounds, but none are great:

1) Put watchers on Parameters so whenever a Parameter is replaced, we transfer the callbacks.  However, Parameters subclasses OrderedDict.  Off the top of my head, one could replace a Parameter with bracket-indexing, via `.set`, or via `.update`.  I feel like there are probably others - I just don't know readily where to put a callback to catch all instances of a Parameter being replaced.  

2) When a user wants to replace a Parameter object, instead, just copy all the attributes over.  In addition to sharing the problem with 1), this sounds like it could have a lot of unintended side effects.

3) Watch for the Parameter attributes themselves changing.  However, I personally don't know of any way to just "watch" an arbitrary Python variable change.

> Not sure I follow you there, but maybe that is specific to the toolkit choices you're making.  Should I understand what 'igs' is?

Apologies, I had an editing error.  `igs` is the InteractiveGuessSession instance - I'm saying, you could send new data to the front-end display via a method call from the Python terminal.

> Is this IPython or Jupyter?  Or is it in some browser?  I don't recognize it. 

This is not IPython or Jupyter and does not depend on them in any way.  This is Panel, which provides HTML/JS front-end tools for Python, and is primarily designed to run standalone with a web browser.   It is very high-level and built on top of Bokeh.  

> Wait, JSON doesn't support infinities?

That's what I said too!   As it turns out, Python's default implementation (`allow_nan=True`) is not officially JSON-compliant, as the JSON specification doesn't support NaN/Inf.  For this reason, Bokeh has had to disable Python's NaN/Inf serialization.  And forcibly enabling Bokeh's NaN/Inf serialization indeed causes errors JS-side for me.

I hope this was informative and worth reading.  Thank you for working with me on this!

~ Jonathan

--

You received this message because you are subscribed to a topic in the Google Groups "lmfit-py" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/lmfit-py/oFv_6Y96qXc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to lmfit-py+u...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/lmfit-py/CA%2B7ESbouksnaEYGCuBUedrquUM9S%2BecoA9NDXsFu32YP7yVazA%40mail.gmail.com.

[Shane Caldwell]

I've taken a look at the demo, and I think it looks great. It delivers on the features that were advertised, and I definitely buy the workflow arguments for a tool like this based both on efficiency and making new things possible. If this were available and polished I would probably use it.

Thoughts to share. (I know you may have addressed some of this along the way, apologies if I've missed where you did so.)

* In addition to the CLI, I'd also like to integrate this with a UI for a larger application. For that I'd want a factory method on InteractiveGuessSession taking a ModelResult, in the case of a bad fit.

* I noticed that access to the last ModelResult is available, so that's great. That's the other big thing I'd need.

* The dragging feature is a killer idea that would be amazing with the Bokeh bug fixed and some polish. The red dots were pretty confusing to me at first.

* Would it be feasible to generalize InteractiveModel so that it doesn't explicitly depend on a subtype of Model? It seems awkward to have to get in there and tell it where to put the red dots. I would think we'd want to render a fairly smooth version of the line with Model.eval and just splat the params into there in a generic way.

* How to make it generic? Ex: if I drag a Gaussian sideways, do I want to vary sigma or center? I'd tentatively suggest that when dragging we usually only mean to vary one parameter at a time. Would it be workable to have a parameter selection within the EditTool that performs the drag? In the case of oscillating signals, particularly with more than one frequency component, this might be quite a bit easier on the user than defining/using a polygon.

* Similar vein as the ^^ question, if the InteractiveModel were generic could the InteractiveModel then be instantiated from a Model generically? If so, it would seem like a cleaner separation of concerns to also make a factory method that takes a Model. That way, the various ways to create an InteractiveGuessSession are contained there.

* (Okay, I'll admit it: The InteractiveModelMixin class gives me the heebie jeebies... I'm wondering if it can be boiled down to methods on the InteractiveGuessSession, and object composition be used rather than inheritance where we need to operate on Lmfit objects. In general it doesn't feel right to subclass Lmfit objects, and then have to implement per-subclass behavior on each.)

Details:

* Printing out the fit report info once the fit is done would be useful.

* It would be slick if the scrolling to adjust float values were sensitive to the position of the cursor in the FloatWidget. (I haven't used Panel before, don't know if cursor readout is available.)

* I didn't confirm behavior with a non-composite model, but that would be my main usage. I assume it's fine.

Thanks again for the prompting on all of this, Jonathan. Looks promising to me.

Shane

You received this message because you are subscribed to the Google Groups "lmfit-py" group.

To unsubscribe from this group and stop receiving emails from it, send an email to lmfit-py+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lmfit-py/CAOu--pEQ-9n6tzejsxQRT5Lh%3DkGyg5YYCUPaoiPFKG5ez4T2OA%40mail.gmail.com.

[Jonathan Okasinski]

Shane,

Thanks so much for the quick & detailed feedback!  I'm very happy to hear that it fits your expectations & use-cases, and that my broader argument has also resonated.

> In addition to the CLI, I'd also like to integrate this with a UI for a larger application. For that I'd want a factory method on InteractiveGuessSession taking a ModelResult, in the case of a bad fit.

That's a good point and easy to implement.  On a related note I think that InteractiveGuessSession also shouldn't open a browser from the constructor.  I think perhaps we should spin off the "run" functionality - removing `stop` and `is_alive` and letting the user manage the thread's lifecycle.  Instead maybe a `igs.show()` method which would return a thread.  The downside of this is that we push the "thinking about threads" problem to the user.  Any thoughts on this? 

Sidebar, if you choose to write your UI with Panel, you can simply include `igs.panel` in a layout like any other object, which I think is kind of slick.

> I noticed that access to the last ModelResult is available, so that's great. That's the other big thing I'd need.

> The dragging feature is a killer idea that would be amazing with the Bokeh bug fixed and some polish. The red dots were pretty confusing to me at first.

Glad you liked it!  Just to clarify - were the red dots confusing because of the Bokeh bug, or it just wasn't obvious that they were "special" handle points of the model at all?

As for the bug - I've come up with a patch that's been tentatively accepted to Bokeh.  I'm not sure when it'll be released.  While this project probably can't be officially released with lmfit until then, I could always create a patched fork of Bokeh 2.2.3 as a workaround.  

Hmm...looking at the rest of your comments together, I'm assuming you don't like the general concept of dragging "handle points" on the models, but you want the same dragging capability.  I'm not terribly attached to "handle points", and I agree that solving the problem for the general case would be better, but I'm not exactly sure how you're proposing to do that conceptually. 

 

Spitballing a bit, perhaps we could draw a bounding box that you could squish around? That's relatively straightforward to do for a general peaked model, but seems to me it doesn't work great with lines or polynomials.  

I had also considered that if your data is very noisy, perhaps you just want to draw a line through it and fit to those points for an initial guess.  That's certainly possible, but I don't know if it's useful vs say just running a moving average.  For a sum of well-defined peaks that feature could also help, but it doesn't seem particularly nice to use.  

I also wanted to unpack the problems you had with the handle points.  For example, the Gaussians work like this:

- The right point is always tied to the top of the distribution.  Wherever you drag the right point, the center and height of the model stay fixed to that point.  The FWHM stays constant.

- The left point is always tied to the -1 sigma point of the distribution.  Wherever you drag the left point, its horizontal coordinate will determine the -1 sigma point.  The center & height of the model stay fixed.

I like how the right point works, I don't really like how the left point works.  But separate from those details, I think there are two broad potential issues:

1) It's not clear from the red dots that that's the behavior you should expect

2) You fully expect that behavior, but just think it's unintuitive

For you is the problem more 1), 2), or both?

> Would it be feasible to generalize InteractiveModel so that it doesn't explicitly depend on a subtype of Model? It seems awkward to have to get in there and tell it where to put the red dots. I would think we'd want to render a fairly smooth version of the line with Model.eval and just splat the params into there in a generic way.

I suppose it is awkward, but I'm hung up on the fact that models can have so many different shapes.  How would you "splat the params" for a generic model...I mean there's no telling how a given param affects the model's shape.  I originally intended the polygons to resemble the Model's shape, which is why I wanted the Models to decide for themselves how to sort of "self-describe". 

If your concern is the burden implementing this for every model, I'll comment that non-interactive model components work just fine.  They just won't show up in the list of components you can select to drag, but you can still use the Parameters table and everything else.

> How to make it generic? Ex: if I drag a Gaussian sideways, do I want to vary sigma or center? 

Not exactly sure exactly what you mean by "sideways".  Right now the Gaussians have two points - left one varies width, right one varies center (& amplitude).  Do you mean both points?  

> I'd tentatively suggest that when dragging we usually only mean to vary one parameter at a time. 

In principle I agree...but I really think that idealizes what a "parameter" is too much.  For example, I think dragging the top point of the Gaussian to move the peak location is intuitive.  However, this actually requires varying all three parameters:

1) center, to match the x-coordinate of the top point

2) amplitude, to match the y-coordinate of the top point

3) sigma, because your amplitude changed, so your FWHM is going to change.  If you leave sigma fixed, it looks like your distribution shrinks/expands

Of course, you could re-normalize your Gaussian such that your FWHM is constant.  Then you'd only be changing two parameters - center and amplitude.  But notice how fungible the parameters are, for ostensibly the same Gaussian distribution?  Then you need to consider other distributions...then entirely non-distributions. 

Also, in either case, you're varying two "fundamental" parameters at a time: center and amplitude.  Perhaps I'll tentatively counter-propose this broad statement:

> When dragging we want to vary two "fundamental" parameters at a time.  

Where "fundamental" parameters are "direct observables on a graph".  Sigma, for example, isn't "directly observable" spread; FWHM is.  

Why two parameters?  Well, you've got an x-coord and a y-coord and ideally both mean something.  With my current Gaussian model, the left point's y-coord is thrown out, and I think that helps make it awkward.  

Broadly I'm arguing there's a lot of valid ways to map parameters to models and draggable points.  Thus I don't see how to write "model handles" in a truly general sense.  However, perhaps there's something clever I'm missing, and this certainly doesn't exclude a solution outside of the "model handles" conceptual idea.

Here's another example in the same vein: consider a sine wave fixed at (0,0).  If I dragged a point, I'd probably put it at a peak.  Then I'd want it to reflect the period (x-coord) and the amplitude (y-coord).  Perhaps instead of fixing it at (0,0), I'd put another point there.  That point would represent phase shift (x-coord) and offset (y-coord).  I guess the question is...if I drag the "(0,0)" point upwards, do I want my amplitude to be constant, thus raising the other point, or do I want to keep the other point fixed?  Similarly, if I drag the same point to the right, do I want to keep phase shift or period constant?  IMO either is reasonable, if application-specific.

(There's also the question of, if you believe that one point is two parameters, what do you do for models with odd numbers of parameters?  That's why I introduced the scroll wheel.  In my original prototype, Gaussians were just one point - at the top - and you changed width with the scroll wheel.  Personally I love it, but not everyone has or likes scroll wheels, so I really feel like there should be an alternative.)

> Would it be workable to have a parameter selection within the EditTool that performs the drag? In the case of oscillating signals, particularly with more than one frequency component, this might be quite a bit easier on the user than defining/using a polygon.

In general Bokeh will send key presses/etc back Python side...though it doesn't suppress those browser events.  i.e. you may have noticed using the scroll wheel on the Gaussians will scroll the page too, which isn't nice.  But in short yes: I had also been thinking of mapping the arrow keys to change components, so you don't have to go over and click on the dropdown.

When you say "multiple frequency components", do you mean a single monolithic model with multiple frequencies that aren't a sum of sines?  (Fourier notwithstanding)

Assuming that is what you mean, I guess the question is "what do you want the point to mean?"  For example, where do you draw it - on the model? if so which point?  I guess I can infer you only want one parameter to change...do you want one point for each parameter?  How would you tell them apart, if they weren't referenced to "special" points on the model like the Gaussian peak?

One option would be to just not draw the line between the points, so it's not a "polygon" just a "collection of points".  I think it would really help me if you could send a screenshot maybe with some Paint edits so I can see what kind of model you're talking about and what interface you want.

It could help intuition to have the model update as you drag.  However, I think that will require serializing models and sending them JS-side, which doesn't sound worth it.  Alternatively Bokeh can forward all mouse events to Python, but this reportedly jams up due to bandwidth.  

> Similar vein as the ^^ question, if the InteractiveModel were generic could the InteractiveModel then be instantiated from a Model generically? If so, it would seem like a cleaner separation of concerns to also make a factory method that takes a Model. That way, the various ways to create an InteractiveGuessSession are contained there.

Yes, if there was a conceptual way to have a generic drag interface, I definitely agree this is the way to go and will lead to a better separation of concerns.

> (Okay, I'll admit it: The InteractiveModelMixin class gives me the heebie jeebies... I'm wondering if it can be boiled down to methods on the InteractiveGuessSession, and object composition be used rather than inheritance where we need to operate on Lmfit objects. In general it doesn't feel right to subclass Lmfit objects, and then have to implement per-subclass behavior on each.)

(In conclusion for the section below, I think the right Python solution really depends on the generality of the drag interfaces, and I think there are viable Python solutions for whatever level of generality we can come up with.)

I'll be the first to admit that some spots are sketchy prototype techniques but the InteractiveModelMixin class didn't ring my alarm bells haha!  I agree that it's wrong, if there's a generic drag interface.  However, without such an interface...I feel like you have to allow each of the models to say something.

There's a middle ground that might be better though.  Perhaps each Model, rather than implementing code, just says something like "I'm a `distribution` with my center parameter named `center` and my spread parameter `sigma` etc etc".  Then, the InteractiveGuessSession or some InteractiveModel factory can interpret that.  So you have a per-subclass specification, but no per-subclass behavior.  

This is good if you believe in "shapes", and good for code reuse/separation.  It's similar to something I had in my old prototype, though I'd warn that you have to specify a lot.  For example, for distributions, you need 6 parameters:  center, spread, height, then how to normalize them.  More complex models either have a lot of parameters, or are just less flexible. 

I'm not personally a huge fan of this approach because I feel it papers over how many little tweaks I think you'll want to have for each subclass of Model.  (That is to say, if we come up with a good generic interface, then I am a fan of the above approach.)  If you're writing one for every subclass you might as well include it in the subclass IMO.  I agree there's a separation of concerns issue...but related to dependencies, I had a solution for this.  I'll warn you though, you might consider it blasphemy :)

So you define all of your model classes normally; no interactivity stuff.  Then you attempt to import the interactive dependencies; if it fails, you're done.  If it succeeds, then you define a bunch of "InteractiveGaussianMixin" "InteractiveVoigtMixin" classes.  Then pick your poison: `setattr`, `__bases__`, or `__mro__` on the "normal" Model classes (not class instances), dynamically adding in the interactive methods.  If this sounds upsetting, it could also be done "properly" via a metaclass.

Yes this is "hacky", but IMO Python exposes its internal machinery deliberately and for a reason.  This would let you separate the concerns by cleanly putting all the interactivity code in its own classes, while retaining the idea of "per-subclass" behavior.  Additionally, it enables good code reuse IMO more naturally and with less boilerplate than having to define a full-blown interface as above.  You'd write a method `distribution_model_to_poly(model_info, model, poly)`, then could, say, `setattr(GaussianModel, 'to_poly', partial(distribution_model_to_poly, gaussian_info))`.  Personally I really like this kind of idiom: you're not required to write complex, general, reusable code, but you can if you want, and it's super readable either way.  I find strong-defined interfaces tend to distribute code in several places and it becomes a mess to keep track of and/or refactor, making it unreadable.  This approach wouldn't have that problem.

> Printing out the fit report info once the fit is done would be useful.

Agreed.  Do you think there's value in interactivity there?  I was thinking of just printing ModelResult.fit_report() out in the GUI...

> It would be slick if the scrolling to adjust float values were sensitive to the position of the cursor in the FloatWidget. (I haven't used Panel before, don't know if cursor readout is available.)

Hmm...like the scrolling is faster if your mouse is at the edge of the widget?

The scrolling logic happens JS-side I'm pretty sure, and it'd probably be easier to implement something like that with a custom JS logic...Not sure off the top of my head if you need a full custom model or a `jscallback/CustomJS` sort of thing would do it.  Either way, it's probably doable, though I'm no JS expert.

> I didn't confirm behavior with a non-composite model, but that would be my main usage. I assume it's fine.

Yup.  Though, now I understand why you want a more generic drag interface!  Perhaps, as Matt asked me, it would help if you described your specific use-case in more detail/screenshots so I could understand what exactly you're trying to solve here.

> Thanks again for the prompting on all of this, Jonathan. Looks promising to me.

Of course, thanks again for working with me on it!  Glad someone else thinks I'm headed in the right direction.

~ Jonathan

To view this discussion on the web visit https://groups.google.com/d/msgid/lmfit-py/CAOUO9Y27r-X0YpoRawuvHnF7U-Dz8fcz5rz6CyNj-iA8AZkcNw%40mail.gmail.com.

[Matt Newville]

Hi Jonathon,

Thanks.  I'm trying to bridge optimism and realism here.  There is just no way that I will be able to respond to every part of your message, which sort of means this won't be "a conversation"  but more like "comments on a presentation".

My overall comments are: 

 a) I am not sure that expecting the user to write substantial code is likely to reach a large audience.

 b) I am equally unsure that people writing substantial code using lmfit will want to insert "break for interactively tweaking model parameter values using toolkit X", including "launch a browser".  

On Tue, Nov 24, 2020 at 12:50 PM Jonathan Okasinski <jo...@cornell.edu> wrote:

Hello everyone!

As promised I've cleaned up the code for my demo; it's here on GitHub (gist).  There's a quick guide included.  Let me know if you have any issues playing with it or other thoughts!  

Matt - 

As my response is admittedly quite long, I've divided it into sections.  

Meta: about this discussion itself 

Goals: identifying users, use-cases, justifying this approach

Implementation: responding to your comments on my prototype so far

 --- meta ---

> For sure, this sort of conversation can turn into "well, I do this" with a reply of "well, this other thing works for me".   And, I think that could be OK: we don't currently have a "general-purpose lmfit app", ad maybe the desire to have one is simply is not that strong.   OTOH, if that is the actual goal, I think it would be helpful to discuss this as if the intention was to consider working together on such a project.  I'll continue making that assumption for a few more rounds of this discussion. 

Apologies, I'm trying to avoid the "well, I do this" sort of conversation.  My intention is also to work together, but I know many people are already fairly oversubscribed.  Thus, I was planning to do most or all of the coding work, and wanted feedback & ideas for usability/ecosystem/interface/code style/strategy sort of thing.  I figured I'd come out with a more complete/polished/widely applicable solution that hopefully someone else could find useful.  If this model is not desirable to you, I'd be happy to discuss distributing the work in a different manner.     

Well, if you expect to do most of the coding work, you are more or less saying that you are not interested in improving an existing approach or in working with others on a shared codebase.  You have kind of announced that you already decided that, and that your tool choices are the right set, in distinction to all existing code.  Maybe that's OK - I definitely write and support a lot of GUI code that is mostly me writing code.  But, if your intention is to work with other people, maybe you want to view everything done so far (including all of your own work) as a proof-of-concept prototype that will be replaced. 

That said, it does seem like lots of multi-developer scientific Python projects deliberately steer clear of GUIs and scientifically-oriented GUI tools and libraries seem to have few (often one) developers.  Maybe that correlation is meaningful.

 

>  But I should also be realistic: I don't know that we'll agree on how to do this, even if the time and resources were available.

I could be wrong, but I really suspect we agree on almost all the major points.  With that being said, I'm having a hard time describing what's in my head, and I think that accounts for much of the apparent difference in opinion.  I hope that looking at the demo will help with this.  If this continues to be a problem I'm happy to set up a time to talk; if you're amenable to this, I've found it's a lot easier and faster to communicate clearly in real-time.  

Hm, maybe...  I would be more interested in a wider conversation about the potential for a lmfit-based GUI app (approximately "replace and enhance fityk") and what that might look like. 

 --- goals ---

> What is your target user and what do you expect them to have to do to be able to do a fit?  It might also be good to start with a use-case.

What is your target user?

Anyone who wants to fit complex models built out of simple components, and/or a lot of data to apply the same model to. Experimental physicists come to mind.

What do you have to do for a fit?

Call `model.interactive_guess(data)`, then edit your model to your satisfaction.  No more no less.

I think I see.  A user is expected to write code that defines a model, reads in and prepares data, and then would call this to tweak the model parameter values.

I'll talk about target users first.  Who's got...

1) Complex models from simple components?  While I don't know enough about experimental physics to argue it permeates the field, it seems packages that think primarily in this manner are pretty widespread.  Already 5 have been mentioned:

  - NexPy, as Ray mentioned  

  - XASViewer, as you mentioned on the wiki

  - HyperSpy, and its nearly 30 pre-built 1D model components

  - I can speak somewhat for the PAD group here at Cornell - this kind of modeling is the basis for many of our bread-and-butter measurements

  - Will's prototype GUI

One of the things I find interesting with your approach is that you acknowledge at least passing familiarity with these other approaches and have decided to use none of them and do your own thing.  As far as I can tell, there is close to zero overlap between these different approaches.

I think there are probably others if we went hunting.  So now:

a) if you believe that there are users who want to regress complex models that are simple components summed together

b) and that the simple components have a natural interactive interface, such as the line I showed on the Gaussian peak that you can drag to resize it

Then I think this is a good use-case!

To me, "regression" means a non-iterative process, probably on a linearizable problem.  What lmfit does are "fits" of parameter values for non-linear problems, not regression.

2) A lot of data to apply the same model to?  Part of what I'm selling with PyX is that even if you don't think you have a lot of data, you actually want & can have a lot of data, making this capability useful.  So I'd tell you that this is everyone in the hard sciences - but I know that's more conjectural, so I'll stick to:

  - Shane described this as exactly his use-case (in quantum computing)

  - Again speaking for my group, we'd love to be able to, say, automatically optimize detector performance, as measured by metrics derived from a fit

  - Materials scientists & co are also starting to think hard about high-throughput characterization & automated phase space exploration.  See for example SARA (recently brought in $15M) or this review in Joule.  One may consider those proposals relatively aggressive, but even in a conservative sense, deriving performance metrics from a fit seems perfectly reasonable to do with tons of phase-space data. 

I think Shane nailed the reasons why an interactive interface would be good for this use case:

> Since most of our fitting succeeds automatically, we would want to activate the interactivity on demand, less than 1% of the time. For example, when a fit fails it's often because our automated parameter initialization was bad, so we would interactively fix them.

 

OK. This implies that the user will write and run a script that sort of pauses in the middle for interactivity on selecting starting parameter values.

 

Now, on to the actual fitting:

`model.interactive_guess(data)` is really a replacement for the `guess`/`fit`/`plot` workflow.  You gather your data, and do all the preprocessing on the command line.  Then, rather than trying to guess reasonable initial parameters ala `guess`/`fit`/`plot`, you call `interactive_guess` to launch a non-blocking GUI.  You select your first guess, and can then fit, interactively in a live setting.  If the fit isn't good enough, you can change your initial guesses/fitting parameters as-needed until it is, either via the GUI or the command line.  If you want to change your preprocessing and thus the data you're fitting, you may do that via the command line (again, GUI's still open, no need to explicitly "save" model/parameters/etc, it doesn't block the command line).  Once you're happy with your data & fit you can close out the GUI.  You're left with a command line object that's a complete record of your data, model, parameters, and fit results.  This can be saved/processed however you like.

I think I see what you're going for.  That expects a lot of programming for the user, but that might be a useful way to initialize a model.

I like this workflow for small-scale fitting (1-10 datasets).  When you have multiple data sets and well-defined models with many parameters (e.g. sum of 5 Gaussians), this is a significant quality of life & speed bump in my opinion.  But I think this actually enables new possibilities in the 10-10000 dataset range (more on the importance of this below).  Shane's figure on doing many regressions was a <1% failure rate.  At 10000 datasets that's <100 that need fixing.  Fixing 100 regressions by tweaking parameters, repeatedly doing `fit`, tweaking `params`, and looking at `plot`, sounds pretty time-consuming to me.  I think it'd take me at least 4-5 minutes each.  500 minutes is 8 hours - an entire work day.  However, with `interactive_guess`, I could:

1) Easily come up with a good first guess to apply to 10k datasets, via the already described workflow

2) Write a script to apply it to all 10000 datasets, then identify failed regressions (~100)

3) Call `interactive_guess` sequentially on all 100 datasets.  As soon as the user is satisfied with the fit/closes the GUI, launch the next GUI with the next dataset.  

I think I could do that at about 1 minute per regression.  That's 1.6 hours total - half my morning.  While that's not great, it's a lot better than a full day!  While you can disagree with my specific numbers, my core argument is that I expect this workflow to be 4-5x faster per regression, and for a sufficiently large number of regressions, this becomes an enabling difference in what is possible.

And finally, a use-case:

My prototypical use case is in extracting gain & noise figures from x-ray detectors.  This essentially involves taking 3D image stacks, doing some preprocessing down to 1D, and then fitting to a sum of 5 or so Gaussians. In theory this is only one fit - but these are early stage detectors.  Given the current empirical understanding of variations in gain & noise, I actually need to do 6 separate, but similar, fits, to properly characterize one detector at one operating point.  This is a little painful to do without the interactive interface.

But now, I want to optimize this detector, let's say to decrease its performance variation.  I'll do this by changing operating parameters.  But for every single set of operating parameters I try, I've got to do another 6 regressions.  Two things here:

1) There are literally hundreds of operating parameters with maybe 6-10 steps each.  A complete exploration would require 200**7 = 1e16 sets of parameters.  With some intuition, maybe I only care about 2-5 parameters and 3-6 steps at a time.  That's between 2**3 = 8 and 5**6 = 15000 sets of parameters.  At every point I need 6 regressions.  So that's between 54 - 90000 regressions.   

2) The "6 regressions" figure is empirically determined.  What if the nature of the performance variation has been misunderstood?  A complete exploration would require 2*128*128 (every pixel) = 16000 fits, per set of operating parameters.  Realistically, with maybe 100 regressions I can be fairly confident my set of 6 is sufficient.

What's my point?  I've got a space that theoretically requires 1e20 regressions to explore, but I've only got the tools to do 1-10 regressions.  I've got to guess, via my best intuition, the behavior of my detector in this huge space from a limited set of data points.  If I could do more regressions, on the 100-10000 scale, relatively easily, that would be tremendously beneficial, and massively limit the guesswork required.

Traditionally people close this gap with intuition & experience.  That's hard.  Moreover, even experienced people can't be 100% certain, because they've had to make assumptions; they simply don't have the data.  And with regression capability in the 100-10000 range, we could have much, if not all, of the data required.

Don't forget that my task here is to come up with just one number, supposedly requiring just one regression.  Imagine what a more complex task might require!  

I've described a narrow use-case: radiation imaging detector development.  But I think this general case of having a large operating parameter space and metrics to be extracted from regression is common, perhaps much more common than is broadly admitted.  For example, of the past 3 major projects I've worked on, I would claim that at least 2 were majorly held back for lack of practical bulk (100-1000 fits) regression ability.  I had enough raw data, but not the tools to make sense of it.

To support my claim I'll speak briefly of another project.  I was essentially doing I-V characterization of some novel perovskites on the ~100fA scale.  I discovered that these perovskites had all kinds of nasty hysteretic behaviors to voltage (literally degrading during measurement), light, air, humidity, temperature, you name it.  I wanted to understand how the apparent trap density & carrier mobility changed (again - just two numbers), which required a fit to a piecewise linear & quadratic model.   I stared at those numbers for months, one dataset at a time, but there were just too many variables that I couldn't directly control.  I couldn't understand any of the trends.  One day I buckled down and wrote a highly kludgy auto-regression framework that barely worked - and literally had insights overnight (which I manually confirmed, of course!)  

If I could have had easy access to the ability to perform several hundred regressions and verify the output earlier, it would have absolutely been a game changer for me.

Um, OK.  It is easy for this sort of thread to have people talking past each other.   

I think "use-case" as being what you expect other **users** of this software to do with it.

  

 --- Implementation ---

> OK.  I think I cannot tell if you think the screenshots shown on the wiki pages interesting or worth trying to use or emulate. 

I do think it's interesting and worth emulating.  I've already taken the parameter table, and plan to implement the same kind of interface for `expr` & fit results.  Perhaps more uniquely, I really like the sidebar where it appears one can scroll through different files - I hadn't considered that yet, and I think that could be quite a valuable option to have especially in my & Shane's use cases.

Well, a GUI *could* include building up the model, reading in the data, and maybe some pre-processing steps,  and presenting the fit results. In fact, the existing GUIs do some of those steps - they may even focus on the reading/processing/visualization and have added on "fitting" as processing and analysis step and maybe present results in a way to compare fits using different model components.   I guess those might not be necessary as an interactive "guess and verify" part of writing a fitting script, but then you've definitely limited the user base to "people who write python scripts to read in, process, and fit their data and read the results".   That's is kind of a narrow view of "interactive front-end", but maybe it is enough.

> I'm afraid I may not fully understand you here.  I think you might mean that you want to click and drag on a plot of data and have a model function "follow the mouse" - redrawing.   I'm not certain of that.  I'm also not certain how you would like that to happen, at least for a truly general case.

Yes, I want the model to "follow the mouse".  I agree there's no true general solution, but recall that my use-case here is specifically models with a number of simple components.  I think this use is far more straightforward:

1) Select the component model you want to edit

2) Each component decides how it should "follow the mouse" - interpreting drags or scrolls

3) Only one component at a time is following the mouse; the model as a whole updates when any component changes

OK, that makes some sense and is similar to what I do.  But it seems a hard thing to do in general.  A mouse can move in 2 dimensions.  How do you tell a Gaussian model to move its center position, increase its sigma, and decrease its amplitude?   Let's say I have a decaying sine wave: Amplitude, Frequency, Phase Shift, and Decay.   How do you get those from mouse movements and gestures?   But, maybe it can work well enough.

Personally, I have not and would not recommend spending a lot of time on effort on "drag the mouse around to draw a model component of the right shape and place".  Good starting values are only part of what a GUI can do. 

One common interaction I use frequently in GUIs is to have the user click on a point on the plot of data (that is clear and hard for the user to mess up, even easier than mouse-down-and-drag) and then click on a small icon (I often use a pin / thumbtack icon) to mean "use that last selected point to set this parameter/meta-parameter".   That can work well to infer:

    fitting bounds, parameter bounds

    peak centers 

    peak half-widths (as the distance from center)

    peak amplitudes

    outlier points to ignore

 

IMO, anything more than that is too hard to infer what the user means. But, maybe you or someone else has a better understanding.

I've found this greatly speeds up the regression process, which as I discussed in "goals" is what I'm looking for.

As it turns out, this PolyEditTool didn't work for me at all.  Maybe I don't understand what it is supposed to do, but I never saw any line change when I moved around on the plot.  I guessed that I would select a component, click on the center and drag to ~hwhm then release the mouse.

Also, when I ran your example, the initial values for the parameters were pretty bad, with repeated values for center and sigma.  I did not look at any of your code.   FWIW,  I could change them in the boxes provided and that updated.

I suggest that you not have users enter values for "height" or "fwhm" parameters - these should be seen as outputs rather than inputs. 

This also relates to:

> How do you know what parameters to change on mouse or scroll-wheel events?

The component being actively edited decides for itself how to respond

OK.

Thus, each component in the model has to implement this interface.  However, many models are reusable, it takes very little code (~20 easy lines) to do so, and the interface is pretty reusable between shapes.  For example, the interface for a Gaussian distribution would be nearly identical to that for the Voigt or Lorentzian distributions, just with renamed parameters for center/spread/height.

Related:

> What is a polygon shape?

Each component should represent itself as a connected series of line segments (a "polygon shape").  When those line segments move, the component updates itself to match.

Each model component is represented by a polygon?  I really have no idea what that means.   Is this related to something specific to Bokeh? 

> Hm, that seems kind of fragile.  Would subclassing Parameters be helpful for this?

I agree that this is fragile and want to find a better way.  I've already had to subclass Parameter to hack in the callbacks.  I can think of a few possible workarounds, but none are great:

1) Put watchers on Parameters so whenever a Parameter is replaced, we transfer the callbacks.  However, Parameters subclasses OrderedDict.  Off the top of my head, one could replace a Parameter with bracket-indexing, via `.set`, or via `.update`.  I feel like there are probably others - I just don't know readily where to put a callback to catch all instances of a Parameter being replaced.  

2) When a user wants to replace a Parameter object, instead, just copy all the attributes over.  In addition to sharing the problem with 1), this sounds like it could have a lot of unintended side effects.

3) Watch for the Parameter attributes themselves changing.  However, I personally don't know of any way to just "watch" an arbitrary Python variable change.

Hm, I'm not sure what the callbacks on Parameters are meant to do...   I would suggest having a set of working parameters for the initial guesses and when you then do a fit give the user the option to update the "working parameters" with the current best-fit.  That is, don't blow away the guesses without user input. 

> Not sure I follow you there, but maybe that is specific to the toolkit choices you're making.  Should I understand what 'igs' is?

Apologies, I had an editing error.  `igs` is the InteractiveGuessSession instance - I'm saying, you could send new data to the front-end display via a method call from the Python terminal.

> Is this IPython or Jupyter?  Or is it in some browser?  I don't recognize it. 

This is not IPython or Jupyter and does not depend on them in any way.  This is Panel, which provides HTML/JS front-end tools for Python, and is primarily designed to run standalone with a web browser.   It is very high-level and built on top of Bokeh.  

OK.  It is not a choice I would make, but maybe that's ok.

> Wait, JSON doesn't support infinities?

That's what I said too!   As it turns out, Python's default implementation (`allow_nan=True`) is not officially JSON-compliant, as the JSON specification doesn't support NaN/Inf.  For this reason, Bokeh has had to disable Python's NaN/Inf serialization.  And forcibly enabling Bokeh's NaN/Inf serialization indeed causes errors JS-side for me.

Hm that seems like a downside of using javascript.  I would be reluctant to use a browser as a GUI framework for a desktop app.  

I hope this was informative and worth reading.  Thank you for working with me on this!

Again, I would suggest starting with use-cases and feature wish-list and treat any code you've written so far as a proof-of-concept.    But, maybe you and Shane are on the same wavelength and will come up with something that's useful for lots of people -- that would be great.

So, I'm going to have to say "good luck" and hope that you come up with something great.  I doubt I'll have much more to add here.

--Matt

[Jonathan Okasinski]

Hello everyone,

I have released a relatively mature demo here.  

Matt - I appreciate your feedback and comments; thank you for taking the time over the past few months to respond to my lengthy messages.  I did not realize exactly the scope you were considering, and with that in mind, I do agree that what I've put together is just a proof of concept.  I hope that it may be useful to others in the future.

Shane - I gave more thought to applying the parameters directly, and I've got an idea that I think is workable.  I'll reach out separately about this.

Thank you all for your time,

Jonathan

--

You received this message because you are subscribed to a topic in the Google Groups "lmfit-py" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/lmfit-py/oFv_6Y96qXc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to lmfit-py+u...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/lmfit-py/CA%2B7ESbqY93MrBOp88SwsjwOxAx-io%2B61%2BzLcrjZym9gxk3bEfQ%40mail.gmail.com.