False positive determination in Stimfit paper

[Anand Kulkarni]

Hi Guys,

              I could not find how false positive percentage was determined in the Stimfit Frontiers paper? What was the denominator? 

Best,

Anand

[Christoph Schmidt-Hieber]

I’ve just added the code that we used for the manuscript to our GitHub repository.

If you scan for the string “FP” you should be able to reproduce our false positive computation:

https://github.com/neurodroid/stimfit/blob/master/manuscript/events.py

> --
>
> ---
> You received this message because you are subscribed to the Google Groups "stimfit" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to stimfit+u...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Anand Kulkarni]

Thanks Christoph!

Here is what I gather. Can you confirm it?

Definitions:

positives events =  events identified by the algorithm

true events =  events identified by an expert labeler 

In your code here is how the fractions are calculated:

% TP = # of TP / # of positive events

% FP = # of FP / # of positive events

% FN = # of FN / # of true events

Is that accurate?

[Christoph Schmidt-Hieber]

iirc we ran simulations to obtain ground truth data, i.e. the events are determined by the simulation and there is no expert labeller.

[Anand Kulkarni]

I see. In which case consider true events as those generated in the simulation. 

Therefore, is my description of the calculations accurate? 

[Christoph Schmidt-Hieber]

Seems reasonable, though I’d have to re-run the code to be sure.