[WePS3] AE task

[IPCC]

Hello everybody,

I quote, concerning the AE task in WePS3 (README.txt file) :

"The WePS-3 Attribute Extraction (AE) task is different from WePS-2 in
that systems are requested to relate each
attribute to a person (cluster of documents) instead of just listing
the attributes obtained from each document.
This is the reason why participants in the AE task are required to
participate in the Clustering task too.
Systems are expected to output no more than one attribute of each type
in each cluster of documents (i.e. only
one affiliation, only one occupation, etc. for each person)."

I would like to come back on the last sentence : does it mean that if
a person had two jobs, we have to affect him only the most recent one?
And what about two differents mails for the same person ? There's
probably something I'm missing.

Best Regards,

Nicolas.

[Satoshi Sekine]

Hi Nicolas,

Thanks for the question.

Even a people has more than one job, we ask system to output one job.
So it means we will not judge the system by the absolute recall.
It is for the evaluation's reason (we can't afford to create the data with many many
system output). I understand it is not ideal situation, but we have to live with this.

Thanks,
Satoshi

> --
> You received this message because you are subscribed to the Google Groups "Web People Search Task" group.
> To post to this group, send email to web-people-...@googlegroups.com.
> To unsubscribe from this group, send email to web-people-search...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/web-people-search-task?hl=en.
>

------------------------------
Satoshi Sekine
sek...@cs.nyu.edu

[IPCC]

Hi,

Thanks for the answer.

Let me just ask you one last thing : if one person has two jobs, I can
understand that for the evaluations
you only need one output, but how to decide which one among the two
jobs?

> > For more options, visit this group athttp://groups.google.com/group/web-people-search-task?hl=en.
>
> ------------------------------
> Satoshi Sekine
> sek...@cs.nyu.edu

[Satoshi Sekine]

Hi Nicolas,

You can choose either one.

We will evaluate the output based on pooling method (i.e. we will search the correct answers
among the ones submitted by the participants. The answers gathered by this method is the
one we will use for the evaluation). So, it does not matter. We understand this is
not going to make the perfect reference data for the future, but based on the limited budget,
this is what we are planning to do.

Thanks,
Satoshi

> For more options, visit this group at http://groups.google.com/group/web-people-search-task?hl=en.
>

------------------------------
Satoshi Sekine
sek...@cs.nyu.edu