Improving Speed in multifit()

[Andrew Herzing]

Hi Everyone,

I'm attempting to do model-based fitting of a hyperspectral dataset using the multifit(kind='smart') function as detailed in the documentation. This is understandably a time-consuming process.  However, even for fairly small datasets (say, 300 spectra with 2048 channels) and only using two or three elements, the fit can often take exceedingly long if the fine structure is enabled.  

Are there any parameters that I might utilize to speed this up a bit?  Even a improvement of 10% would be helpful.

Thanks,

Andy Herzing

[Francisco de la Pena]

* Are you using the low-loss to take into account plural scattering?

* Are the onset energies free? If they are, fixing them to reasonable values can speed things up dramatically.

* Is your data very noisy? Noisy data takes longer to fit.

Also, you could try decreasing the fine structurure smoothing parameter.

What do you consider exceedingly long fitting times? How long does it take per spectrum? 

Also, we have recently experienced some issues when using smart fit that we need to investigate. In particular it wasn't fitting properly the background in some cases. We have to investigate the issue further but, if that's what happening to you, this issue alone could explain longer fitting times and poorer results...

Francisco

--
You received this message because you are subscribed to the Google Groups "hyperspy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hyperspy-user...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Andrew Herzing]

Francisco,

Thank you for the quick reply.

My spectra are not very noisy, and when I fit a single spectrum it takes between 10 and 15 seconds which is great.  I think my problem is arising from the fact that the field of view in my dataset contains several different phases.  Because of this, it appears that multifit is getting bogged down in trying to fit edges that are not necessarily present in some spectra.  Does this make sense?

If so, other than sub-selecting different areas based on element content and fitting them independently, are there any workarounds for this?

As for the background fitting, I have noticed that in some datasets it works less well than others.  I haven't been able to identify what particular spectral features causes this, so I'd be interested in hearing what sorts of data produce the problem you've described.

Thanks very much,

Andy

[Andrew Herzing]

Francisco,

As a follow-on, I tried removing the background from the input spectrum using remove_background() prior to creating the model.  I then disabled the background and performed the fit.  In this case, the fit was 10X faster and appeared to fit very nicely.  I think this points towards the background fitting being the source of the problem.

If you need any help troubleshooting this, please don't hesitate to ask.

Thanks again,

Andy

[Tomas]

Hi Andy,

Another suggestion is to make sure that the plot of the model is closed when fitting - it slows down the fitting significantly.

The other option is to fit the fine structure in one pixel, and then fix it (using http://hyperspy.org/hyperspy-doc/current/api/hyperspy.models.html#hyperspy.models.eelsmodel.EELSModel.fix_fine_structure) for the other pixels. It may not be viable for your particular case or only viable just for some of the elements, though. Any way, the fine structure fitting is most often the bit that is computationally most difficult, so reducing that workload should go a long way.

Tomas

P.S. if you have a github account, you are welcome to have a chat in our (new and somewhat experimental) gitter chat: https://gitter.im/hyperspy/hyperspy