large differences in spike amplitudes within clusters

[Stephen Whitmarsh]

Dear spike sorters,

I've been noticing that in some of my datasets the amplitudes within a cluster can be much larger than I expected, and that clusters do not separate well based on amplitude. For example, in the attached screenshot you can see the green cluster having a large variance in amplitude, while I would expect the large amplitude spikes to belong to a separate cluster (together with the "blue" spikes) and the tiny ones to fall into another cluster.

I am using version 1.0.7; Below are my relevant settings. I wonder if I should change cluster.dispersion, or fitting.amp_limits?

Thanks for any insights,

Stephen

[clustering]
extraction     = median-raw # Can be either median-raw (default) or mean-raw
sub_dim        = 10         # Number of dimensions to keep for local PCA per electrode
max_elts       = 10000      # Max number of events per electrode (should be compatible with nb_elts)
nb_elts        = 0.8        # Fraction of max_elts that should be obtained per electrode [0-1]
nb_repeats     = 3          # Number of passes used for the clustering
smart_search   = True       # Activate the smart search mode
merging_method = nd-bhatta  # Method to perform local merges (distance, dip, folding, nd-folding, bhatta, nd-bhatta)
merging_param  = default    # Merging parameter (see docs) (3 if distance, 0.5 if dip, 1e-9 if folding, 2 if bhatta)
sensitivity    = 3          # Single parameter for clustering sensitivity. The lower the more sensitive
cc_merge       = 0.95       # If CC between two templates is higher, they are merged
dispersion     = (5, 5)     # Min and Max dispersion allowed for amplitudes [in MAD]
fine_amplitude = True       # Optimize the amplitudes and compute a purity index for each template
make_plots     =            # Generate sanity plots of the clustering [Nothing or None if no plots]

[fitting]
amp_limits     = (0.3, 5)   # Amplitudes for the templates during spike detection [if not auto]
amp_auto       = True       # True if amplitudes are adjusted automatically for every templates
collect_all    = False      # If True, one garbage template per electrode is created, to store unfitted spikes
ratio_thresh   = 0.9        # Ratio of the spike_threshold used while fitting [0-1]. The lower the slower

[laitej...@gmail.com]

Hi,

The picture is rather small to see all the details, but maybe this could help you to sort this out: https://github.com/spyking-circus/spyking-circus/issues/329

Best,

Stan

[Stephen Whitmarsh]

Dear Stan,

Thanks, that does help a lot as I was not aware (or forgot about) the normalization, i.e. that partitioning doesn't use the amplitude.

Since I our data we typically only have several electrodes, this might result in similar problems as is described in that thread.

I will see how it behaves without normalization.

thanks again,

Stephen

Op ma 7 jun. 2021 om 09:08 schreef laitej...@gmail.com <laitej...@gmail.com>:

--
You received this message because you are subscribed to the Google Groups "SpyKING CIRCUS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to spyking-circus-u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/spyking-circus-users/5153fe1c-3799-4c4d-bc2e-e12c700bd21an%40googlegroups.com.

[Stephen Whitmarsh]

Dear all,

Following up on this thread;

It seems that the parameter templates_normalization is no longer a parameter in the default params file.

Perhaps it's an undocumented option now or has it somehow become irrelevant?

Cheers,

Stephen

Op ma 7 jun. 2021 om 09:46 schreef Stephen Whitmarsh <stephen....@gmail.com>:

[Pierre Yger]

Dear all

In theory, in recent versions, the code should automatically set this parameter to False if it detects that most of your templates are defined, on average, on less than 3 channels. This is because we do want this parameter to be active for dense layout (we want to merge similar templates up to an amplitude factor), but for low density probes, we want to keep amplitude as a discriminant factor, and thus set it to False. 

Hope this helps

Best

Pierre 

To view this discussion on the web visit https://groups.google.com/d/msgid/spyking-circus-users/CAFrxm%3DwNOUZwZoLxtfzC3EYUMeg-00FZwzrXk_gkOcXhLwzkFw%40mail.gmail.com.

[Stephen Whitmarsh]

Thanks Pierre,

That might then indeed be a bit tricky in those cases that are close to that (necessarily) somewhat arbitrary point of 3, as in this example, where the template includes 5 channels.

In our case, templates should only include a single electrode, something I try to enforce with the probe file (radius smaller than electrode distances).

Is there a way to check whether normalization is disabled or not? Is it outputted in the log file?

Cheers,

Stephen

Op ma 7 jun. 2021 om 10:40 schreef Pierre Yger <pierr...@gmail.com>:

[Pierre Yger]

Yes, should be put somewhere in the log file. You can also enforce the behaviour by setting templates_normalization = False in the [clustering] section.

Best

Pierre

[Stephen Whitmarsh]

Oh, great! I'll just set it to False in my analyses, that way I'm sure.

Cheers,

Stephen

Op ma 7 jun. 2021 om 11:01 schreef Pierre Yger <pierr...@gmail.com>:

To view this discussion on the web visit https://groups.google.com/d/msgid/spyking-circus-users/CAGGVdvR4kSoG_O6_L2LFjoJ8-F4YujxSLqNgWTkary0b4PMhzA%40mail.gmail.com.