Small clarification regarding task 2 evaluation

[Luca]

As I optimize my models, I am looking at how task 2 (vandalism
evaluation) performance will be evaluated.
I have two small questions:

* I am reading the ROC101.pdf paper on ROC. The "positive" label will
be used for "vandalism", and "negative" label for "non-vandalism", I
imagine, right?

* Regarding the measurement of the area under the curve (AUC), if I
turn in a boolean classification (rather than one that e.g. has a
confidence value), you will use the segment-curve as illustrated for
classifier A of Figure 7, right?

Thank you for the confirmations!
Best,

Luca

[Martin Potthast]

Hi Luca,

> * I am reading the ROC101.pdf paper on ROC.  The "positive" label will
> be used for "vandalism", and "negative" label for "non-vandalism", I
> imagine, right?
>
> * Regarding the measurement of the area under the curve (AUC), if I
> turn in a boolean classification (rather than one that e.g. has a
> confidence value), you will use the segment-curve as illustrated for
> classifier A of Figure 7, right?

Both right.

Martin

--
Martin Potthast
Bauhaus-Universität Weimar
www.webis.de --- www.netspeak.cc