template-based methods and phy

[Kevin Bolding]

Hi All,

I have a little information to share about Spyking-Circus and a question. 

First, we have extensively tested Spyking-Circus on 32-site polytrode recordings. I'm happy to share details but here is what we've learned. 

1) The instructions for installing on the website are out of date and there are some bugs. We needed to contact Pierre to get a password to download the code.

2) Spyking-Circus works as advertised in terms of processing speed (processing time is about the same as recording time).

3) We varied spike thresholds from 6 SD to 12 SD and found that different recordings had optimal results at different thresholds. We have not found a variable (overall spiking rate, average channel SD, a rudimentary measure of SNR etc..) that can predict which is appropriate so we will probably have to cluster at multiple thresholds for the time being.

4) We got better results using mean-pca extraction than the default median-raw.

5) Spyking-Circus makes a huge improvement in the data lost to overlapping spikes. 

6) It is very difficult to cut apart clusters in the Circus MATLAB GUI. 

Regarding this last point, I would very much like to be able to open a Circus file in Phy and refine the sorting. I can see this is possible or intended to be possible but the way to go about it is not entirely clear. Do I need to follow the instructions in the Circus docs and have a command called circus-gui-python? Or is it true as stated here, that once I have phy-contrib up and running I should be able to "work with data processed with Spyking Circus and KiloSort"? If so, could someone indicate to me which of the Spyking-Circus output files I should use in this command? "phy template-gui ?????"

Regarding KiloSort. 

We had our first succesful runthrough with KiloSort a few days ago. It was relatively easy to install, though we haven't attempted GPU processing yet. However, barring some user error on our part, it appears that the instructions for setting up phy and phy-contrib that are in the Phy repository worked for us while the instructions in the KiloSort repository did not. When we used the latter set of instructions I think something went wrong with the 'click' module so that the 'template-gui' subcommand was not created, and it's not obvious how to create the subcommand by hand (even though there is an example for a 'hello world' command). 

We are excited about the increase in accuracy and savings in time and computation KiloSort offers and will be testing it on our known data. The first results look very nice in terms of the quality of clustering. However, for the file we tested, we get in the range of 50-60 clusters using klustakwik or spyking-circus. KiloSort returned 31. I assume we need to tweak some parameters, since we just started with the defaults and didn't even make a proper channel map (although I'm not certain the channel map matters in this case?). We look forward to more guidance or info from other users about which parameters affect which outcomes. I currently have no intuition for how annealing template threshold parameters, for instance, should affect the results. We'll try different settings as time allows. 

Thanks as always for developing and sharing these tools. Hope this helps.

Kevin Bolding

Franks Lab

Duke University

[Nick Steinmetz]

Hi Kevin,

Thanks for sharing these observations!

To answer one question, Kilosort will fit as many templates as you ask it to - it does not yet have any mechanism to pick the "correct number". This is a parameter called Nfilt, which you might try putting at 4 times the number of channels you have (default is 32 as you saw). 

Could you help me understand, when you say for example that "different results had optimal results at different thresholds", what kind of things are you considering when you decide what are the "optimal" results? 

Thanks,

Nick

--
You received this message because you are subscribed to the Google Groups "phy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to phy-users+...@googlegroups.com.
To post to this group, send email to phy-...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/phy-users/fa115bfa-5e58-40da-8527-7fbe3d5d52b3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Kevin Bolding]

Hi Nick,

Thanks! I'll try that ASAP.

What I meant by optimal is simply that after all of the clustering and merging and manual inspection is done, we had more well-isolated "good" clusters. I believe the tradeoff is between getting small unsortable activity in which mucks up the algorithm and corrupts good sortable units and getting fewer spikes and losing small but sortable units. So if we plot the number of Good units vs threshold it's an inverted U and I can't yet predict where the max of that function will be based on any preliminary screening of the data. 

Kevin B.

[maxym myroshnychenko]

Hi Kevin, 

Please consider starting this discussion on spyking-circus boards , perhaps in the parameters threads. I think the devs would be very interested in what you have to say, as well as other circus users.

Best,
Max

[Pierre Yger]

Hi all

Thanks for the feedback !! I just realized there is a discussion here going on, and maybe I could shine some light on some points about our software. To answer your points in details:

- we'll release very soon 0.4 and will update all instructions on the website. Some may be out-dated, sorry for that, we are doing our best to try to develop the code and docs at the same time, but this is not always easy. However, as a pure python package, we believe the install should not be too difficult

- There is no way we can find the "best" threshold, and indeed, this is something we are working on. Currently, 6MAD is a decent value for most datasets, that we never changed, but we can discuss about that.

- mean-PCA as an extraction method will produce smoother templates, but likely to be also slightly distorted. Again, this depend on the size of your data. If you have a lot of spikes in clusters, median-raw will be better. If only few, then median-pca could help. But from several benchmarks performed here, this should not lead to a drastic change

- The MATLAB GUI is not intended to be made for cutting clusters... 

- You can clearly export your data from SpyKING CIRCUS to phy. With 0.3, you just need to launch the command circus-gui-python. This is well explained in the documentation, on the website. This command will automatically perform the export, and launch phy appropriately. With 0.4, there will be a slight change of workflow and you'll have to convert them before launching that command. But everything is explained and will be documented, and this should be very easy and straightforward to use

Let me know if you have more feedback or thoughts. The publication on the software should be online soon, showing methods/performance with real ground truth neurons/scaling up to 4225 channels with virtually no limit in the number of templates

Best

Pierre

[Pierre Yger]

Just to mention, as I was speaking about it: 0.4 version of SpyKING CIRCUS is now out, and the preprint describing the algorithm/performance can be found at

http://biorxiv.org/content/early/2016/08/04/067843.article-metrics

I did my best to update the documentation (http://spyking-circus.rtfd.org), but do not hesitate to give us feedback for issues or errors with the software. Export to phy have been consolidated, even if this is still not the default output of the algorithm.

Best

Pierre