Re: PAN 2012

[giacomo...@usi.ch]

> 2. In the second part of the Sexual Predator Identification task we are to identify the lines of the predator conversations, which are the most distinctive of the predator bad behavior. Firstly, is there going to be an extra data release for this task (e.g. annotations?) and, secondly, how will the results of this part of the task be evaluated?

Hi Claudia,
thanks for the interest in participating in the predator identification task. I suggest you to follow the public mailing list: http://groups.google.com/group/pan-workshop-series where you can find updates on the tasks (I am going to send one) and look for previous discussion. For example, on April 2 there was an answer to the same question:

"As you noticed, there is not ground truth for the second part of the task. Being this the first year of the task, we would like to see how this (difficult) problem could be approach and solved without much evidence (as in a real scenario).
We will later evaluate based on your (the participants) consensus plus some external user evaluation."

Good work!

Giacomo

------------------------------------------------------------
Giacomo Inches
PhD Student
Faculty of Informatics
Università della Svizzera italiana, USI
Via G. Buffi 13
CH - 6904 Lugano
(e) giacomo...@usi.ch
(w) http://www.giacomo.inches.ch
(t) +41 (0)58.666.4699
(f) +41 (0)58.666.4536
------------------------------------------------------------

[Tim Snider]

this seems like an overly subjective decision-making process ... "based on ... consensus" ... to someone like me who hasn't drunk the google crowd-sourcing kool-aid :-) our system does unsupervised training, so short of reading the transcripts and subjectively deciding which lines are 'predatory' we can't enter this part of the competition ... and my hat is off to people who actually have to work with this kind of data ... the few conversations i've had to read are quite disturbing ... on the first part, identifying predatory conversations, where we have been given training data with ground truth, we're proceeding well and looking forward to the testing release ... as with last year's competition, we do appreciate the work you've done to set this up ... and the opportunity it gives us to see how our approach compares to others in a concrete way

c u o, tim

--
You received this message because you are subscribed to the Google Group "PAN".
Visit this group at http://groups.google.com/group/pan-workshop-series
To unsubscribe send email to pan-workshop-se...@googlegroups.com.