Creating spectrum from python arrays

[Magnus Nord]

Hey,

I got my EELS data in .npz files, and I'm wondering if there are any easy ways to "convert" numpy arrays into the hyperspy signal class?

- Magnus

[Magnus Nord]

Some more questions:

Is there a way to use hyperspy as a "normal" python library? For example:

import hyperspy
hyperspy.load("spectrum.msa")

Is it possible to do smart_fit() on a stack of EELSspectrums?

I ran fitting on a single spectrum from a .msa file, which gives me a nice fitting and it plots fine. However when I use export_results on the model I get:
"NavigationDimensionError: 'navigation dimension  = 0, >0 expected'"
Is there some other way of getting the fitting data?

- Magnus

[francisco]

Hi Magnus,

Yes, there is e.g. : s = signals.Spectrum({'data' : your_numpy_array_here})

You can obviously use any other signal.

Best,

Francisco

[Francisco de la Peña]

Is there a way to use hyperspy as a "normal" python library?

To import the same object that you get when starting hyperspy "the standard way" into a Python session you simply have to import hyperspy.hspy, e.g. : from hyperspy.hspy import *

 

Is it possible to do smart_fit() on a stack of EELSspectrums?

 Yes, you can pass the 'kind' kwarg to the multifit function e.g: m.multifit(kind='smart')

I ran fitting on a single spectrum from a .msa file, which gives me a nice fitting and it plots fine. However when I use export_results on the model I get:
"NavigationDimensionError: 'navigation dimension  = 0, >0 expected'"
Is there some other way of getting the fitting data?

For a single spectrum you use simply call the print_current_values Model method to print the result of the fit.

 

Best,

Francisco

[Magnus Nord]

Thanks for the quick replies.

Some more questions:

I'm having some problems using the multifit(kind="smart"), the following does not work:
s = load(imagefile).to_EELS() #stack
ll = load(lowlossfile).to_EELS()
s.add_elements((...))
s.set_microscope_parameters(...)
m = create_model(s, ll=ll)
m.enable_fine_structure()
m.multifit(kind="smart")
When running the multifit, I get the error: http://pastebin.com/PctYUD2K
If I do not include the lowloss spectrum in the model, everything works fine. Currently using the latest github-version.

Is there a way to "extract" a single spectrum from a stack of spectrums, while retaining all the metadata (energy-range, elements, instrument parameters, ...)?

Is there an easy way get the fine structure from a model as a spectrum, so that I can do further curve fitting on the fine structure (for example using voigt-profiles to further analyze the fine structure)?

I'm thinking of writing some examples of using hyperspy, would a good way of doing this be to write it and send a pull request to the hyperspy github project?

- Magnus

[Francisco de la Peña]

I'm having some problems using the multifit(kind="smart"), the following does not work:
s = load(imagefile).to_EELS() #stack
ll = load(lowlossfile).to_EELS()
s.add_elements((...))
s.set_microscope_parameters(...)
m = create_model(s, ll=ll)
m.enable_fine_structure()
m.multifit(kind="smart")
When running the multifit, I get the error: http://pastebin.com/PctYUD2K
If I do not include the lowloss spectrum in the model, everything works fine. Currently using the latest github-version.

I think that the problem may be that the low-loss navigation (i.e. spatial) dimensions are not the same as the core-loss navigation dimensions that you are trying to fit. Is it the case?
 

Is there a way to "extract" a single spectrum from a stack of spectrums, while retaining all the metadata (energy-range, elements, instrument parameters, ...)?

I have just added a new method to the Signal class to get the feature that you suggest. After pulling from github the following should work: s0 = s.get_current_signal()

Is there an easy way get the fine structure from a model as a spectrum, so that I can do further curve fitting on the fine structure (for example using voigt-profiles to further analyze the fine structure)?

I guess that the easiest way to do it is by generating the full model using
generate_data_from_model. e.g.:

>>> s = load('yourfile')
>>> m = create_model(s)

---- fit your data ---

>>> m.generate_data_from_model()
>>> sm = s.deepcopy()
>>> sm.data = m.model_cube

Now you can create a new model to fit sm.

I'm thinking of writing some examples of using hyperspy, would a good way of doing this be to write it and send a pull request to the hyperspy github project?

Great, the project desperately needs better documentation and many more examples! Would you like to add the examples to the main documentation or to the examples section of the website? To add them to the examples section of the website you just need to add the scripts to an appropriate folder in the examples folder of the project root folder. To add them to the documentation, as you know, you must edit the rst files in doc/user_guide. In any case, yes, sending a pull request is the way to put it in. Let me know if you need any help in the process and thank you very much for helping improve Hyperspy.


Francisco
 

[Magnus Nord]

I think that the problem may be that the low-loss navigation (i.e. spatial) dimensions are not the same as the core-loss navigation dimensions that you are trying to fit. Is it the case?

Yes, using a lowloss spectrum with the same spatial dimensions fixes the issue.
 

I have just added a new method to the Signal class to get the feature that you suggest. After pulling from github the following should work: s0 = s.get_current_signal()

Nice, thanks :)

I guess that the easiest way to do it is by generating the full model using
generate_data_from_model. e.g.:

>>> s = load('yourfile')
>>> m = create_model(s)

---- fit your data ---

>>> m.generate_data_from_model()
>>> sm = s.deepcopy()
>>> sm.data = m.model_cube

Now you can create a new model to fit sm.

This gives me the full model (with both powerlaw background and the edges), is it possible to get just the fine structure?

Great, the project desperately needs better documentation and many more examples! Would you like to add the examples to the main documentation or to the examples section of the website? To add them to the examples section of the website you just need to add the scripts to an appropriate folder in the examples folder of the project root folder. To add them to the documentation, as you know, you must edit the rst files in doc/user_guide. In any case, yes, sending a pull request is the way to put it in. Let me know if you need any help in the process and thank you very much for helping improve Hyperspy.

I think I'll add some examples of the stuff I've been doing. And maybe add some "workflow" documentation, which I find very helpful (such as http://hyperspy.org/hyperspy-doc/current/user_guide/model.html#eels-curve-fitting).

- Magnus

[Francisco de la Peña]

This gives me the full model (with both powerlaw background and the edges), is it possible to get just the fine structure?

At the moment it is not possible in an easy way, but it would be nice to have this feature so I will work on it and I will let you know when it is ready, probably next week. In the meantime you could get what you want by disabling what you don't want in your model before calling m.generate_data_from_model , e.g.

# To disable the power law background

>>> background.active = False

# To disable the edges of the elements that you are not interested it

>>> m.disable_edges((Sr,Ti))

# To get the single scattering distribution

>>> m.convolved = False

I think I'll add some examples of the stuff I've been doing. And maybe add some "workflow" documentation, which I find very helpful (such as http://hyperspy.org/hyperspy-doc/current/user_guide/model.html#eels-curve-fitting).

Great, thanks!

Francisco

 

- Magnus

[Magnus Nord]

At the moment it is not possible in an easy way, but it would be nice to have this feature so I will work on it and I will let you know when it is ready, probably next week. In the meantime you could get what you want by disabling what you don't want in your model before calling m.generate_data_from_model , e.g.

# To disable the power law background

>>> background.active = False

# To disable the edges of the elements that you are not interested it

>>> m.disable_edges((Sr,Ti))

# To get the single scattering distribution

>>> m.convolved = False

This gives me some strange results: I followed this procedure, and I've added the results as attachments.
 

I think I'll add some examples of the stuff I've been doing. And maybe add some "workflow" documentation, which I find very helpful (such as http://hyperspy.org/hyperspy-doc/current/user_guide/model.html#eels-curve-fitting).

Great, thanks!

I sent a pull request on github with two examples, and some small fixes.

 - Magnus

[Francisco de la Peña]

The noise amplication is due to a too high value of the fine_structure_smoothing parameter, decreasing its value should solve your problem. Note that I've just renamed this parameter so you'll need to pull from github for it to work.

To give you an extra motivation to pull from github I have just added the feature that you suggested: now there is a get_fine_structure_as_spectrum EELSCLEdge method that should give you what you want by simply typing:

>>> Ofs = O_K.get_fine_structure_as_spectrum()

If you test it, could you let me know if it works fine for you?

Francisco

2012/10/2 Magnus Nord <magn...@gmail.com>

[Magnus Nord]

get_fine_structure_as_spectrum works perfectly! Thanks a lot :)

Still having some problems with "noisy" fine structure, not using a lowloss spectrum solves the problem:
s = load("spectrum.msa").to_EELS()
s.add_elements(("Ti", "Mn", "O"))
s.set_microscope_parameters(beam_energy=300, convergence_angle=24.6, collection_angle=13.6)
m = create_model(s)
m.enable_fine_structure()
m.smart_fit()
OKfs = O_K.get_fine_structure_as_spectrum()

Gives a nice fine structure (see the attachment No_lowloss.png and No_lowloss_OKfs.png). Using a lowloss spectrum gives a noisy fine structure:

s = load("spectrum.msa").to_EELS()
s.add_elements(("Ti", "Mn", "O"))
s.set_microscope_parameters(beam_energy=300, convergence_angle=24.6, collection_angle=13.6)
ll = load("lowloss.msa").to_EELS()

m = create_model(s, ll=ll)
m.enable_fine_structure()

m.smart_fit()
OKfs = O_K.get_fine_structure_as_spectrum()

Gives a noisy fine structure (see With_lowloss.png and With_lowloss_OKfs.png). I also attached the .msa spectrums. This behaviour is the same when using spectrum-stacks and multifit(kind="smart"): using no lowloss spectrum gives a nice fine structure, while using a lowloss spectrum gives a noisy spectrum.

Another bug (I should maybe post stuff like this on github instead?):
when plotting a model (m.plot()) consisting of 1 spatial dimension, the spatial axis becomes very thin (see attachment model_plot.png). The plots from spectrums have the same issue (s.plot()). While the spectrums from the new get_fine_structure_as_spectrum does not have this issue.

 -Magnus

[Francisco de la Peña]

The "problem" is noise amplification similar to what you can get by iterating too much when using Richardson-Lucy deconvolution. The solution is to set the fine_structure_smoothing parameter to what you judge to be a good trade-off between noise amplification and detail. For example (git pull first):

s = load("spectrum.msa").to_EELS()
s.add_elements(("Ti", "Mn", "O"))
s.set_microscope_parameters(beam_energy=300, convergence_angle=24.6, collection_angle=13.6)
ll = load("lowloss.msa").to_EELS()

m = create_model(s, ll=ll, GOS="hydrogenic")
m.enable_fine_structure(O + Mn)
O_K.fine_structure_smoothing = 0.06
Mn_L3.fine_structure_smoothing = 0.06
Mn_L3.onset_energy = 643
Mn_L3.fine_structure_width = 20
m.smart_fit(weights=True)
O_K.get_fine_structure_as_spectrum().plot()
Mn_L3.get_fine_structure_as_spectrum().plot()

(figures attached)

Note that you get the fine structure without multiple scattering and probably with better resolution and SNR that you could achieve using fourier-ratio deconvolution.

In respect to the navigator aspect ratio problem, yes, it is a bug. While someone fixes it you can browse your line spectrum by:

• Using the left and right arrow keys when the spectrum figure is on focus
  
• Using the index slider  s.axes_manager.navigation_axes[0].edit_traits()

Francisco

2012/10/9 Magnus Nord <magn...@gmail.com>

[Magnus Nord]

Hey,

what would be the best way use fourier-ratio deconvolution on a fine structure spectrum which does not have the same size as the lowloss spectrum (as you get when using the get_fine_structure_as_spectrum() function)?

Magnus

[Francisco de la Peña]

Hi,

I've fixed a problem (spotted by Gäel) with the deconvolution algorithms and in addition they should now work with spectra of different sizes. The code lives in the fix_deconvolution branch.

Francisco

2012/11/14 Magnus Nord <magn...@gmail.com>

[Magnus Nord]

That seems to have fixed the problem.

Thanks :)

Magnus