example AUC calculation

[Francois Berenger]

Hello,

I just noticed the auc function in

biocaml/src/lib/biocaml_roc.mli

Can someone give an example
on how to use it?

Let's say I have a list of score labels, sorted by decreasing scores:

[ (score_1, true); (score_2, false); ...]

true means it is of the same class than the thing
I was looking for. False means it is not.

Thanks a lot,
F.

[Ashish Agarwal]

I'm not sure how you could with that data. A ROC plot requires sensitivity vs false-positive-rates, and the AUC is the area under such a curve.

Maybe Philippe knows more. He's the author of the Roc module.

--
You received this message because you are subscribed to the Google Groups "biocaml" group.
To unsubscribe from this group and stop receiving emails from it, send an email to biocaml+u...@googlegroups.com.
To post to this group, send email to bio...@googlegroups.com.
Visit this group at http://groups.google.com/group/biocaml.
To view this discussion on the web visit https://groups.google.com/d/msgid/biocaml/3f0e84df-a73c-48fa-af84-af298a2530cc%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Philippe Veber]

Hi François (and Ashish!)

Ashish's right, you need to have the score of the positive examples separated from the scores of the negative examples (here your dichotomy is more like true predictions vs false predictions). I have a better version of this Roc module that I lasted too long integrating to biocaml. I can do it by wednesday if you need it.

ph.

To view this discussion on the web visit https://groups.google.com/d/msgid/biocaml/CAMu2m2Lyfdk%2BY7_ZRt5GPY%3DZ-JOo5Dop5Ox9Qe3P6wJKV%3DSd0g%40mail.gmail.com.

[Francois Berenger]

On Tue, Mar 18, 2014 at 3:35 AM, Philippe Veber
<philipp...@gmail.com> wrote:
> Hi François (and Ashish!)
>
> Ashish's right, you need to have the score of the positive examples
> separated from the scores of the negative examples (here your dichotomy is
> more like true predictions vs false predictions). I have a better version of
> this Roc module that I lasted too long integrating to biocaml. I can do it
> by wednesday if you need it.

If the list I shown is filtered based on the boolean, we can
separate the positive examples from the negative ones, no?

> You received this message because you are subscribed to a topic in the
> Google Groups "biocaml" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/biocaml/j7-H8np8sTE/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to

> biocaml+u...@googlegroups.com.
> To post to this group, send email to bio...@googlegroups.com.
> Visit this group at http://groups.google.com/group/biocaml.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/biocaml/CAOOOohSyF2E4C3XO1NyMykpVQNuC9_oPb2zXoYQCR_u8YYj5cg%40mail.gmail.com.

[Philippe Veber]

Le 18 mars 2014 10:38, "Francois Berenger" <francois.ber...@gmail.com> a écrit :
>
> On Tue, Mar 18, 2014 at 3:35 AM, Philippe Veber
> <philipp...@gmail.com> wrote:
> > Hi François (and Ashish!)
> >
> > Ashish's right, you need to have the score of the positive examples
> > separated from the scores of the negative examples (here your dichotomy is
> > more like true predictions vs false predictions). I have a better version of
> > this Roc module that I lasted too long integrating to biocaml. I can do it
> > by wednesday if you need it.
>
> If the list I shown is filtered based on the boolean, we can
> separate the positive examples from the negative ones, no?

Yes, if you know which score threshold was used to get your prediction. BTW I committed a better version of the ROC stuff. See the new Bin_pred module:

https://github.com/biocaml/biocaml/blob/master/src/lib/biocaml_bin_pred.mli

Cheers,

PH.

> To view this discussion on the web visit https://groups.google.com/d/msgid/biocaml/CAMQaOgNufBKZYgXTTVTrfiyVd8F979UrcCOFGmbT-uF8vfHFYA%40mail.gmail.com.

[Francois Berenger]

There is a rather simple auc function in there:
https://github.com/EigenDog/dawg/blob/master/roc.ml
I could adapt it to my case easily (list of score-labels).
I could also cross validate it against the Python CROC package
implementation.

> https://groups.google.com/d/msgid/biocaml/CAOOOohQ7UjQSk4VZgL-ay77_8Of57_C9tmUrF3hX1ViMPOh7_w%40mail.gmail.com.