PETE answers and system scores available

2 views
Skip to first unread message

Deniz Yuret

unread,
Apr 4, 2010, 4:45:05 AM4/4/10
to semeval-pete
Dear PETE Participants,

The gold answers and system scores for the PETE task have been
released and are available for download from http://pete.yuret.com
(also attached to this message).  There were 301 questions in the test
dataset with 156 YES and 145 NO answers. This makes the "always YES"
baseline 51.83%. About half of the 19 participating systems were
above this baseline with the best system scoring 72.43%. We are
looking forward to your analyses of the task and your systems. Please
do not hesitate to send any questions, suggestions or feedback about
the task to our mailing list.

Best Regards,
deniz

PETE_gold.zip

Deniz Yuret

unread,
Apr 4, 2010, 4:46:25 AM4/4/10
to semeval-pete
I forgot to mention: the system scores are also available at

http://spreadsheets.google.com/pub?key=tDUXEL8YSSL7S2vz_Jn_ckQ&single=true&gid=0&output=html

On Apr 4, 11:45 am, Deniz Yuret <denizyu...@gmail.com> wrote:
> Dear PETE Participants,
>
> The gold answers and system scores for the PETE task have been

> released and are available for download fromhttp://pete.yuret.com


> (also attached to this message).  There were 301 questions in the test
> dataset with 156 YES and 145 NO answers.  This makes the "always YES"
> baseline 51.83%.  About half of the 19 participating systems were
> above this baseline with the best system scoring 72.43%.  We are
> looking forward to your analyses of the task and your systems.  Please
> do not hesitate to send any questions, suggestions or feedback about
> the task to our mailing list.
>
> Best Regards,
> deniz
>

>  PETE_gold.zip
> 45KViewDownload

Deniz Yuret

unread,
Apr 4, 2010, 4:05:52 PM4/4/10
to semeval-pete
You can see the system descriptions and user/system ID numbers at:

http://docs.google.com/View?docid=d2jm3f3_1437z88dk3fj


On Apr 4, 11:46 am, Deniz Yuret <denizyu...@gmail.com> wrote:
> I forgot to mention: the system scores are also available at
>

> http://spreadsheets.google.com/pub?key=tDUXEL8YSSL7S2vz_Jn_ckQ&single...

Reply all
Reply to author
Forward
0 new messages