The gold answers and system scores for the PETE task have been
released and are available for download from http://pete.yuret.com
(also attached to this message). There were 301 questions in the test
dataset with 156 YES and 145 NO answers. This makes the "always YES"
baseline 51.83%. About half of the 19 participating systems were
above this baseline with the best system scoring 72.43%. We are
looking forward to your analyses of the task and your systems. Please
do not hesitate to send any questions, suggestions or feedback about
the task to our mailing list.
Best Regards,
deniz
http://spreadsheets.google.com/pub?key=tDUXEL8YSSL7S2vz_Jn_ckQ&single=true&gid=0&output=html
On Apr 4, 11:45 am, Deniz Yuret <denizyu...@gmail.com> wrote:
> Dear PETE Participants,
>
> The gold answers and system scores for the PETE task have been
> released and are available for download fromhttp://pete.yuret.com
> (also attached to this message). There were 301 questions in the test
> dataset with 156 YES and 145 NO answers. This makes the "always YES"
> baseline 51.83%. About half of the 19 participating systems were
> above this baseline with the best system scoring 72.43%. We are
> looking forward to your analyses of the task and your systems. Please
> do not hesitate to send any questions, suggestions or feedback about
> the task to our mailing list.
>
> Best Regards,
> deniz
>
> PETE_gold.zip
> 45KViewDownload
http://docs.google.com/View?docid=d2jm3f3_1437z88dk3fj
On Apr 4, 11:46 am, Deniz Yuret <denizyu...@gmail.com> wrote:
> I forgot to mention: the system scores are also available at
>
> http://spreadsheets.google.com/pub?key=tDUXEL8YSSL7S2vz_Jn_ckQ&single...