failed segmentation with STICA

[Jack Waters]

I'm having difficulty convincing the STICA method to define ROIs.

I've had success deriving ROIs from the example data set using the example code (PlaneCA1PC method). However, I have failed to extract ROIs using any other data set or the STICA method. Segmentation results in a 'no valid ROIs found' error.

An example of a complete error message:

C:\Anaconda\lib\site-packages\sima\extract.py:346: UserWarning: Empty ROIs will return all NaN values: 75 empty ROIs found

"{} empty ROIs found".format(original_n_rois - n_rois))

Traceback (most recent call last):

File "<stdin>", line 1, in <module>

File "C:\Anaconda\lib\site-packages\spyderlib\widgets\externalshell\sitecustomize.py", line 682, in runfile

execfile(filename, namespace)

File "C:\Anaconda\lib\site-packages\spyderlib\widgets\externalshell\sitecustomize.py", line 71, in execfile

exec(compile(scripttext, filename, 'exec'), glob, loc)

File "C:/Users/jackw/Desktop/SIMA example code.py", line 109, in <module>

dataset.extract(signal_channel='GCaMP', label='GCaMP_signals')

File "C:\Anaconda\lib\site-packages\sima\imaging.py", line 726, in extract

demix_channel=demix_channel, save_summary=save_summary

File "C:\Anaconda\lib\site-packages\sima\extract.py", line 517, in save_extracted_signals

signal_channel=signal_channel, **kwargs)

File "C:\Anaconda\lib\site-packages\sima\extract.py", line 348, in extract_rois

raise ValueError('No valid ROIs found.')

ValueError: No valid ROIs found.

This error message occurs with a wide range of parameters, with example data set and with our own data sets. I've tried...
Example code, STICA segmentation using red and using green channels. mu= 0.01, 0.1, 0.5 and 1.
Our own data sets, mu= 0.01, 0.1, 0.5 and 1.
Our own data sets, PlaneCA1PC method (tutorial settings).

Does anyone have STICA settings that will segment the example data set? Any suggestions on what I may be doing that prevents segmentation?

Thanks

[Patrick Kaifosh]

Hi Jack,
Can you please send your exact code for one of the cases where it fails with STICA on the example dataset (the one downloaded from our website?) so that we can have a quick look.
Thanks,
Patrick

[Jack Waters]

Of course. Code is:

import sima

import sima.motion

import sima.segment

tiff_filenames = [['test_dir/Cycle{n1:02d}_Ch{n2}.tif'.format(n1=cycle, n2=channel) for channel in range(1, 3)] for cycle in range(1, 16)]

print "TIFF filenames:\n", tiff_filenames

sequences = [sima.Sequence.join(*[sima.Sequence.create('TIFF', chan) for chan in cycle]) for cycle in tiff_filenames]

dataset_path = 'test_dir/dataset.sima'

correction_approach = sima.motion.HiddenMarkov2D(num_states_retained=30, max_displacement=[20, 30])

dataset = correction_approach.correct(sequences, dataset_path, channel_names=['tdTomato', 'GCaMP'], trim_criterion=0.95)

output_filenames = [[[channel.replace('.tif', '_corrected.tif') for channel in cycle]] for cycle in tiff_filenames]

print "Output filenames:\n", output_filenames

dataset.export_frames(output_filenames, fill_gaps=True)

segmentation_approach = sima.segment.STICA(mu=0.5)

dataset.segment(segmentation_approach, 'auto_ROIs')

dataset.extract(signal_channel='GCaMP', label='GCaMP_signals')

dataset.export_signals('example_signals.csv', channel='GCaMP', signals_label='GCaMP_signals')

[Patrick Kaifosh]

Here is what I am getting (running your code but skipping the exporting step). Do you have any idea why there may be a difference? I'm running our latest code from the GitHub repository. Are you running that as well, or a released version? We've fixed some bugs -- sounds like we are due for another release.


In [1]: import sima

In [2]: import sima.motion

In [3]: import sima.segment

In [4]: tiff_filenames = [['workflow_data/Cycle{n1:02d}_Ch{n2}.tif'.format(n1=cycle, n2=channel) for channel in range(1, 3)] for cycle in range(1, 16)]

In [5]: tiff_filenames
Out[5]:
[['workflow_data/Cycle01_Ch1.tif', 'workflow_data/Cycle01_Ch2.tif'],
 ['workflow_data/Cycle02_Ch1.tif', 'workflow_data/Cycle02_Ch2.tif'],
 ['workflow_data/Cycle03_Ch1.tif', 'workflow_data/Cycle03_Ch2.tif'],
 ['workflow_data/Cycle04_Ch1.tif', 'workflow_data/Cycle04_Ch2.tif'],
 ['workflow_data/Cycle05_Ch1.tif', 'workflow_data/Cycle05_Ch2.tif'],
 ['workflow_data/Cycle06_Ch1.tif', 'workflow_data/Cycle06_Ch2.tif'],
 ['workflow_data/Cycle07_Ch1.tif', 'workflow_data/Cycle07_Ch2.tif'],
 ['workflow_data/Cycle08_Ch1.tif', 'workflow_data/Cycle08_Ch2.tif'],
 ['workflow_data/Cycle09_Ch1.tif', 'workflow_data/Cycle09_Ch2.tif'],
 ['workflow_data/Cycle10_Ch1.tif', 'workflow_data/Cycle10_Ch2.tif'],
 ['workflow_data/Cycle11_Ch1.tif', 'workflow_data/Cycle11_Ch2.tif'],
 ['workflow_data/Cycle12_Ch1.tif', 'workflow_data/Cycle12_Ch2.tif'],
 ['workflow_data/Cycle13_Ch1.tif', 'workflow_data/Cycle13_Ch2.tif'],
 ['workflow_data/Cycle14_Ch1.tif', 'workflow_data/Cycle14_Ch2.tif'],
 ['workflow_data/Cycle15_Ch1.tif', 'workflow_data/Cycle15_Ch2.tif']]

In [6]: sequences = [sima.Sequence.join(*[sima.Sequence.create('TIFF', chan) for chan in cycle]) for cycle in tiff_filenames]

In [7]: sequences
Out[7]:
[<sima.sequence._Joined_Sequence at 0x7f405baf8b10>,
 <sima.sequence._Joined_Sequence at 0x7f405baf8bd0>,
 <sima.sequence._Joined_Sequence at 0x7f405baf8b90>,
 <sima.sequence._Joined_Sequence at 0x7f405bab9e90>,
 <sima.sequence._Joined_Sequence at 0x7f405bab9550>,
 <sima.sequence._Joined_Sequence at 0x7f405ba4a090>,
 <sima.sequence._Joined_Sequence at 0x7f405ba4a250>,
 <sima.sequence._Joined_Sequence at 0x7f405ba4a310>,
 <sima.sequence._Joined_Sequence at 0x7f405ba4a3d0>,
 <sima.sequence._Joined_Sequence at 0x7f405ba4a490>,
 <sima.sequence._Joined_Sequence at 0x7f405ba4a550>,
 <sima.sequence._Joined_Sequence at 0x7f405ba4a610>,
 <sima.sequence._Joined_Sequence at 0x7f405ba4a6d0>,
 <sima.sequence._Joined_Sequence at 0x7f405ba4a790>,
 <sima.sequence._Joined_Sequence at 0x7f405ba4a850>]

In [8]: sequences[0].shape
Out[8]: (120, 1, 128, 256, 2)

In [9]: dataset_path = 'workflow_data/dataset.sima'

In [10]: correction_approach = sima.motion.HiddenMarkov2D(num_states_retained=30, max_displacement=[20, 30])

In [11]: dataset = correction_approach.correct(sequences, dataset_path, channel_names=['tdTomato', 'GCaMP'], trim_criterion=0.95)
Estimating model parameters.
Estimating displacements for cycle  0
Estimating displacements for cycle  1
Estimating displacements for cycle  2
Estimating displacements for cycle  3
Estimating displacements for cycle  4
Estimating displacements for cycle  5
Estimating displacements for cycle  6
Estimating displacements for cycle  7
Estimating displacements for cycle  8
Estimating displacements for cycle  9
Estimating displacements for cycle  10
Estimating displacements for cycle  11
Estimating displacements for cycle  12
Estimating displacements for cycle  13
Estimating displacements for cycle  14

In [12]:

In [12]:

In [12]: segmentation_approach = sima.segment.STICA(mu=0.5)

In [13]:

In [13]: dataset.segment(segmentation_approach, 'auto_ROIs')
Out[13]: [<ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252), <ROI: label=None, id=None, type=mask, im_shape=(1, 125, 252)]

[Jack Waters]

Thanks Patrick,

I updated SIMA recently (a few weeks ago, via pip install) so I doubt it's a version issue.

I'm using Anaconda. I don't know whether this might be a cause of trouble. I'll run the code again, maybe in iPython notebook on the off-chance that the outcome is different. If that doesn't work, I'll try another machine. Perhaps I can find one with a different distribution of Python.

How many ROIs did your code extract? (I can't see the end of the output you've copied above.)

Jack

[Patrick Kaifosh]

Hi Jack,

The code extracts quite a few. If you installed with pip, it probably is a version issue, since pip only installs released versions of our code, and we have accumulated some bug fixes that we haven't yet had a chance to release. I'll try to create a new release in the next day or so.

Cheers,
Patrick

--
You received this message because you are subscribed to the Google Groups "sima-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sima-users+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sima-users/9faab5eb-2e25-4b06-81f7-e4661f9f71c1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jack Waters]

I updated my version of SIMA from the Github. Still no ROIs.

[Patrick Kaifosh]

 So, if you run

>>> import sima
>>> sima.__version__

then the result is 1.2.1 ?

If that is the case, then that's odd, since it seems we are getting different results. If so, I'll try looking deeper to see if I can think of anything that could explain the discrepancy.

[Jack Waters]

Yes, my sima version is 1.2.1. I double-checked and still have no ROIs.

Incidentally, after updating to the newest version from Github, 'import sima' triggered errors. The solution was to rescue two files from the previous installation. I don't know whether this might indicate some sort of initialization issue with the new code? The files were _motion.pyd and _opca.pyd (in motion and segment directories, respectively).

[Rylan Larsen]

To try this code on an additional python distribution and computer, I ran Jack's code from above in WinPython-64bit-2.7.6.4 in Windows 7 (64 bit) and it produced the same errors. This is with Sima version 1.2.1. Error pasted in below.

1.2.1
Started at 2015-06-09 17:52:58.596000
TIFF filenames:
[['test_dir/Cycle01_Ch1.tif', 'test_dir/Cycle01_Ch2.tif'], ['test_dir/Cycle02_Ch1.tif', 'test_dir/Cycle02_Ch2.tif'], ['test_dir/Cycle03_Ch1.tif', 'test_dir/Cycle03_Ch2.tif'], ['test_dir/Cycle04_Ch1.tif', 'test_dir/Cycle04_Ch2.tif'], ['test_dir/Cycle05_Ch1.tif', 'test_dir/Cycle05_Ch2.tif'], ['test_dir/Cycle06_Ch1.tif', 'test_dir/Cycle06_Ch2.tif'], ['test_dir/Cycle07_Ch1.tif', 'test_dir/Cycle07_Ch2.tif'], ['test_dir/Cycle08_Ch1.tif', 'test_dir/Cycle08_Ch2.tif'], ['test_dir/Cycle09_Ch1.tif', 'test_dir/Cycle09_Ch2.tif'], ['test_dir/Cycle10_Ch1.tif', 'test_dir/Cycle10_Ch2.tif'], ['test_dir/Cycle11_Ch1.tif', 'test_dir/Cycle11_Ch2.tif'], ['test_dir/Cycle12_Ch1.tif', 'test_dir/Cycle12_Ch2.tif'], ['test_dir/Cycle13_Ch1.tif', 'test_dir/Cycle13_Ch2.tif'], ['test_dir/Cycle14_Ch1.tif', 'test_dir/Cycle14_Ch2.tif'], ['test_dir/Cycle15_Ch1.tif', 'test_dir/Cycle15_Ch2.tif']]
C:\WinPython-64bit-2.7.6.4\python-2.7.6.amd64\lib\site-packages\skimage\filter\__init__.py:6: skimage_deprecation: The `skimage.filter` module has been renamed to `skimage.filters`.  This placeholder module will be removed in v0.13.
  warn(skimage_deprecation('The `skimage.filter` module has been renamed '

Estimating model parameters.
Estimating displacements for cycle  0
Estimating displacements for cycle  1
Estimating displacements for cycle  2
Estimating displacements for cycle  3
Estimating displacements for cycle  4
Estimating displacements for cycle  5
Estimating displacements for cycle  6
Estimating displacements for cycle  7
Estimating displacements for cycle  8
Estimating displacements for cycle  9
Estimating displacements for cycle  10
Estimating displacements for cycle  11
Estimating displacements for cycle  12
Estimating displacements for cycle  13
Estimating displacements for cycle  14

Output filenames:
[[['test_dir/Cycle01_Ch1_corrected.tif', 'test_dir/Cycle01_Ch2_corrected.tif']], [['test_dir/Cycle02_Ch1_corrected.tif', 'test_dir/Cycle02_Ch2_corrected.tif']], [['test_dir/Cycle03_Ch1_corrected.tif', 'test_dir/Cycle03_Ch2_corrected.tif']], [['test_dir/Cycle04_Ch1_corrected.tif', 'test_dir/Cycle04_Ch2_corrected.tif']], [['test_dir/Cycle05_Ch1_corrected.tif', 'test_dir/Cycle05_Ch2_corrected.tif']], [['test_dir/Cycle06_Ch1_corrected.tif', 'test_dir/Cycle06_Ch2_corrected.tif']], [['test_dir/Cycle07_Ch1_corrected.tif', 'test_dir/Cycle07_Ch2_corrected.tif']], [['test_dir/Cycle08_Ch1_corrected.tif', 'test_dir/Cycle08_Ch2_corrected.tif']], [['test_dir/Cycle09_Ch1_corrected.tif', 'test_dir/Cycle09_Ch2_corrected.tif']], [['test_dir/Cycle10_Ch1_corrected.tif', 'test_dir/Cycle10_Ch2_corrected.tif']], [['test_dir/Cycle11_Ch1_corrected.tif', 'test_dir/Cycle11_Ch2_corrected.tif']], [['test_dir/Cycle12_Ch1_corrected.tif', 'test_dir/Cycle12_Ch2_corrected.tif']], [['test_dir/Cycle13_Ch1_corrected.tif', 'test_dir/Cycle13_Ch2_corrected.tif']], [['test_dir/Cycle14_Ch1_corrected.tif', 'test_dir/Cycle14_Ch2_corrected.tif']], [['test_dir/Cycle15_Ch1_corrected.tif', 'test_dir/Cycle15_Ch2_corrected.tif']]]
C:\WinPython-64bit-2.7.6.4\python-2.7.6.amd64\lib\site-packages\sima\extract.py:349: UserWarning: Empty ROIs will return all NaN values: 75 empty ROIs found
  original_n_rois - n_rois))

Traceback (most recent call last):

  File "C:\Users\rylanl\Desktop\segmantation test code.py", line 31, in <module>

    dataset.extract(signal_channel='GCaMP', label='GCaMP_signals')

  File "C:\WinPython-64bit-2.7.6.4\python-2.7.6.amd64\lib\site-packages\sima\imaging.py", line 710, in extract
    demix_channel=demix_channel, save_summary=save_summary
  File "C:\WinPython-64bit-2.7.6.4\python-2.7.6.amd64\lib\site-packages\sima\extract.py", line 520, in save_extracted_signals
    signal_channel=signal_channel, **kwargs)
  File "C:\WinPython-64bit-2.7.6.4\python-2.7.6.amd64\lib\site-packages\sima\extract.py", line 351, in extract_rois
    raise ValueError('No valid ROIs found.')

On Monday, June 8, 2015 at 9:41:16 PM UTC-7, Patrick Kaifosh wrote:

[Jack Waters]

An update:

We've now duplicated this error on multiple systems, each with similar results, namely no ROIs.

Windows / Anaconda / SIMA 1.2.1

Windows / Winpython / SIMA 1.2.1

Windows / Anaconda / SIMA 1.3.0

Linux / Anaconda / SIMA 1.3.0

To debug further, I used python debugger to extract principal and independent components immediately before the return statement at the end of _segment, within the STICA class (in stica.py). The scikitlearn ica routine (used by SIMA) returns a non-zero array (st_components, dimensions 33300 x 75), but the spatial masks aren't what I was expecting. I've attached tiff files containing the masks for the first spatial PC and the first IC. The subsequent PCs and ICs look similar (as do the temporal PCs). There's nothing in these IC masks that I'd expect to generate ROIs in the subsequent IC->ROI conversion steps (to pass the size filter, for example). Looking at the PC masks, I'd hazard a guess that the problem may originate with the PCA.

Patrick, how do these masks compare to those generated when you run SIMA? Is the STICA segmentation problem rooted in an erroneous PCA step perhaps?

[Patrick Kaifosh]

I'm running it again and will inspect the PC components that I get. It
would be fairly easy to try another PCA implementation (e.g. from scikit
learn). The one that we have implemented is designed to use the temporal
structure of the data to some advantage, but if it's giving you trouble,
then you could just try something else. You could submit a pull request
which allows for users to specify which PCA (or other dim reduction)
method they wish to use.

> --
> You received this message because you are subscribed to the Google
> Groups "sima-users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to sima-users+...@googlegroups.com

> <mailto:sima-users+...@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/sima-users/79493f28-3710-4494-8072-8f16f2460fda%40googlegroups.com
> <https://groups.google.com/d/msgid/sima-users/79493f28-3710-4494-8072-8f16f2460fda%40googlegroups.com?utm_medium=email&utm_source=footer>.

[Jack Waters]

I ran the example images through the scikitlearn PCA and they look fairly similar so probably the root cause of the problem is not the PCA itself, but the conversion of ICs to ROIs. I suspect the size filter (or perhaps some other post-ICA processing step) is excluding everything, hence no ROIs.

I'm using the default code, which doesn't specify size filters, etc. I'm assuming you also used the default code, with no specification of post-processing? Could the default post-ICA processing differ across our machines?

> <mailto:sima-users+unsub...@googlegroups.com>.

[Jack Waters]

Hi Patrick,

I've been looking into the the post-ICA steps to ROI extraction, in the hope that this is where the STICA code is failing. There appear to be several parameters in SparseROIsFromMasks. My attention was drawn to min_size=50, static_threshold=0.5, smooth_size=4 since these parameters are reminiscent of the Mukamel trinity of Gaussian convolution (smooth_size?), standard deviation threshold to create a binary mask (static_threshold?) and size filter (min_size). Unfortunately reducing the values of these parameters failed to recover ROIs (assuming, of course, that I was successful in changing the values). Is there a summary somewhere in the SIMA documentation of the post-ICA processing steps that are used for ROI extraction? Is this the best location at which to set these parameters? Would I be correct in thinking that reducing these numbers should lead to less stringent filtering and, probably, more ROIs?

Thanks,

Jack

[Patrick Kaifosh]

I think I figured out what was happening. You didn't specify which
channel to use for segmentation, and it defaulted to using channel 0,
which is the static red signal. There is nothing dynamic going on in
that signal, so that is why the PCs looked so bad, and thus you got ICs
that didn't pass the tests to be converted into ROIs. Please confirm
that specifying the channel fixes your problem.

Once that is cleared up, I'd be happy to take any suggestions about how
to clarify the documentation for the various parameters.

> --
> You received this message because you are subscribed to the Google
> Groups "sima-users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to sima-users+...@googlegroups.com

> <mailto:sima-users+...@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/sima-users/19b3afe4-d7b3-490f-bc56-07aae400bd48%40googlegroups.com
> <https://groups.google.com/d/msgid/sima-users/19b3afe4-d7b3-490f-bc56-07aae400bd48%40googlegroups.com?utm_medium=email&utm_source=footer>.

[Jack Waters]

I changed one line to address the channel problem. I switched

segmentation_approach = sima.segment.STICA()
to
segmentation_approach = sima.segment.STICA(channel='GCaMP')

Still no ROIs. Complete code and the error is below.

Is there anything else I need to change for STICA to operate? For example, do I need to state the number of PCs to use, specify post-ICA filters or append any post-processing steps? As far as I can tell (from inserting print statements in the segmentation code), no post-ICA filters are currently operating. This leads me puzzled as to how SIMA - as I'm running it currently - converts from ICs to ROIs.

 

Code:

from __future__ import print_function

from builtins import input

from builtins import range

import sima

import sima.motion

import sima.segment

 

tiff_filenames = [['workflow_data/Cycle{n1:02d}_Ch{n2}.tif'.format(n1=cycle, n2=channel) for channel in range(1, 3)] for cycle in range(1, 16)]

sequences = [sima.Sequence.join(*[sima.Sequence.create('TIFF', chan) for chan in cycle]) for cycle in tiff_filenames]

dataset_path = 'workflow_data/dataset.sima'

 

correction_approach = sima.motion.HiddenMarkov2D(num_states_retained=30, max_displacement=[20, 30])

dataset = correction_approach.correct(sequences, dataset_path, channel_names=['tdTomato', 'GCaMP'], trim_criterion=0.95)

 

output_filenames = [[[channel.replace('.tif', '_corrected.tif') for channel in cycle]] for cycle in tiff_filenames]

dataset.export_frames(output_filenames, fill_gaps=True)

 

segmentation_approach = sima.segment.STICA(channel='GCaMP')

dataset.segment(segmentation_approach, 'auto_ROIs')

Error:

C:\Anaconda\lib\site-packages\sima\extract.py:357: UserWarning: Empty ROIs will return all NaN values: 75 empty ROIs found

  original_n_rois - n_rois))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Anaconda\lib\site-packages\spyderlib\widgets\externalshell\sitecustomize.py", line 682, in runfile
    execfile(filename, namespace)
  File "C:\Anaconda\lib\site-packages\spyderlib\widgets\externalshell\sitecustomize.py", line 71, in execfile
    exec(compile(scripttext, filename, 'exec'), glob, loc)

  File "C:/Users/jackw/Desktop/testexample_STICA.py", line 134, in <module>

    dataset.extract(signal_channel='GCaMP', label='GCaMP_signals')

  File "C:\Anaconda\lib\site-packages\sima\imaging.py", line 743, in extract
    demix_channel=demix_channel, save_summary=save_summary
  File "C:\Anaconda\lib\site-packages\sima\extract.py", line 534, in save_extracted_signals
    signal_channel=signal_channel, **kwargs)
  File "C:\Anaconda\lib\site-packages\sima\extract.py", line 359, in extract_rois

[Jack Waters]

With this updated code, _segment (within the STICA class in stica.py) returns a 1 x 128 x 256 x 75 st_components array that's all zeros.

[Jack Waters]

I'm now confident that the masks are emptied somewhere between extraction of ICs and line ~350 of extract.py.

I inserted two print statements into extract.py, the first at line 350, to poll original_n_rois and n_rois. Code as follows:

350    print('original_n_rois = ' + str(original_n_rois))

351

352    rois_to_include = np.array(

353    [idx for idx, mask in enumerate(masks) if mask.nnz > 0])

354    n_rois = len(rois_to_include)

355
356    print('n_rois = ' + str(n_rois))

results:
original_n_rois = 75
n_rois = 0

The subsequent if loop appears to be the origin of the errors that print to the console:
'Empty ROIs will return all NaN values: 75 empty ROIs found'
and
'ValueError: No valid ROIs found.'

[Jack Waters]

Problem fixed. I now have ROIs with STICA and the example data set. Read on for an explanation.

 

The error originated from line 351 of extract.py and was related to the identification and elimination of overlapping pixels by _identify_overlapping_pixels (lines 327 and 232-250 of extract.py) and _remove_pixels (line 331 and 253-276 of extract.py).

 

The identification and elimination of overlapping pixels assumes that the incoming masks are binary and the overlapping pixels code identifies and eliminates pixels that are non-zero in more than one mask. The output from STICA is a list of weighed masks. Essentially all pixels in the weighed masks are non-zero. Hence all pixels in all of the masks were eliminated and the code returned no ROIs.

 

We resolved this by appending the sima.segment.SparseROIsFromMasks method to the STICA segmentation approach:

segmentation_approach = sima.segment.STICA()

segmentation_approach.append(sima.segment.SparseROIsFromMasks())

dataset.segment(segmentation_approach, 'auto_ROIs')

 

Patrick, I think you weren't able to replicate the error because you didn't call extract.py with your snippet of code.