Sexual Predator Identification (partial) Results

[giacomo...@usi.ch]

Dear all,
in the authorship page of the pan website (http://pan.webis.de/) you can now find the results for the first problem (identifying the predators) of the Sexual Predator Identifications subtask. Results for the second problem (identifying the distinctive chat lines of the predator behavior) have been delayed by one week due to the fact that the manual evaluation is taking more time than expected.

Thanks for participating and stay tuned for the other results.

Giacomo

------------------------------------------------------------
Giacomo Inches
PhD Student
Faculty of Informatics
Università della Svizzera italiana, USI
Via G. Buffi 13
CH - 6904 Lugano
(e) giacomo...@usi.ch
(w) http://www.giacomo.inches.ch
(t) +41 (0)58.666.4699
(f) +41 (0)58.666.4536
------------------------------------------------------------

[David Losada]

thanks giacomo for your efforts

can we have now the qrels file?

this would help us to start analyzing our other runs

d

> --
> You received this message because you are subscribed to the Google Group "PAN".
> Visit this group at http://groups.google.com/group/pan-workshop-series
> To unsubscribe send email to pan-workshop-se...@googlegroups.com.



--
David E. Losada, PhD. ACM Senior Member
Associate Professor (Prof. Titular de Universidad)
Dept. Electrónica y Computación.
Universidad de Santiago de Compostela, Campus Vida s/n.
15782 Santiago de Compostela, SPAIN
tel:+34 881813570, fax:+34 881 813602
david....@usc.es http://www.gsi.dec.usc.es/ir
Colegiado num. 292 del C.P. Enx. Informáticos de Galicia
http://maps.google.com/maps/ms?ie=UTF8&hl=gl&t=h&om=1&doflg=ptk&msa=0&ll=42.876373,-8.554809&spn=0.004599,0.011373&z=17&msid=118004855173432402891.0004632efb9ebb10eee34

[giacomo...@usi.ch]

Hi David,
I just sent around the evaluation of the additional runs submitted to whom who submitted more than one.

I am still computing qrels for the problem 2, once I'll have all (problem 1 and problem 2), I'll make them available.

Bests
Giacomo

[Tim Snider]

thanks to everybody for all your work in running this competition ... it's been extremely instructive for us ... and will continue to be so once we get the ground truth ... i expect to learn a lot by figuring out how we managed to miss almost a third of the predators :-(

c u o, tim

[giacomo...@usi.ch]

Hi all,
unfortunately the evaluation for the second problem (identifying the distinctive chat lines of the predator behavior) it's taking really more time than expected. You have to allow us another week to conclude the assessment and compute the results.

Thanks for understanding.

Giacomo

Begin forwarded message:

> --
> You received this message because you are subscribed to the Google Group "PAN".
> Visit this group at http://groups.google.com/group/pan-workshop-series
> To unsubscribe send email to pan-workshop-se...@googlegroups.com.

[tim gollub]

Dear participants of the detailed comparison task,

I'm happy to announce that the final performance results are now
published on the PAN website [1] for ten of the eleven software
submissions we received. We try to get the remaining submission running
for another couple of days, and will then officially announce the winner.

As a personal note, let me tell you that the competition was a great
experience for me, and I'd like to thank all of you for participating.
In the remaining months until the conference, we'll continue our work on
the TIRA platform to supply you with additional information about the
performance of your system in different categories. So stay tuned.

Looking forward to meeting you at the conference,
Tim.

[1]
http://www.uni-weimar.de/medien/webis/research/events/pan-12/pan12-web/plagiarism.html

[giacomo...@usi.ch]

Hi Gunnar, hi all,

I'm planning to release the list at the beginning of coming week. For the second part I apologize but I still need more time (I underestimate the overall time).


Bests
Giacomo

On Jul 18, 2012, at 3:46 PM, Gunnar Eriksson wrote:

>
>
> Hi Giacomo,
>
> Do you have any idea when the correct predator list for the test set will be made public? As I understood it, your work with evaluation of the first part is completed, and I am eager to start the analysis of our contribution ... :-)
>
>
> Best
>
> Gunnar

[Cristian Grozea]

Dear Tim,

> to supply you with additional information about the performance of
> your system in different categories. So stay tuned.

Is the information about the performance of our systems in different
categories ready?
We would much appreciate to see it, as there is really very few time
left until the conference.

best regards,
Cristian

--
Dr. Cristian Grozea
Visual Computing
Tel: +49 30 6392 � 1893
Fax: +49 30 6392 � 1805
cristia...@first.fraunhofer.de

Fraunhofer-Institut fuer Offene Kommunikationssysteme FOKUS
Standort Adlershof
Kekulestrasse 7
12489 Berlin
Germany
http://www.fokus.fraunhofer.de

[Martin Potthast]

Hi Cristian,

> Is the information about the performance of our systems in different
> categories ready?
> We would much appreciate to see it, as there is really very few time left
> until the conference.

Just today, we've finished our write-up of the plagiarism overview
paper, including the data on everyone's detection performance per
obfuscation. The paper will be published soon, but I can already share
the table with you.

Best,
Martin


--
Martin Potthast
Bauhaus-Universität Weimar
www.webis.de --- www.netspeak.org

[Tartessos]

Thanks, Martin. 

I miss a runtime column however, detailed for every subcorpus too.

¿Is it possible? (at least in spreadsheet .odc .xls format as last year)

¿Will be analized this year the performance depending of the lenght cases, plagiarism rate, ...?

Thanks.

[Martin Potthast]

> I miss a runtime column however, detailed for every subcorpus too.

Runtime is part of another table.

> ¿Is it possible? (at least in spreadsheet .odc .xls format as last year)

It is possible, but not at the moment, since Tim is on his well-earned
vacation right now.

> ¿Will be analized this year the performance depending of the lenght cases,
> plagiarism rate, ...?

We haven't yet done so, for time reasons. Also, we wanted to first
review which of these variables actually make a difference, and drop
those which don't.