new version of the scorer - please update your files!

8 views
Skip to first unread message

Lucia Specia

unread,
Mar 2, 2012, 11:06:13 AM3/2/12
to semeval2012_lexi...@googlegroups.com
Hi all,

We've uploaded a new version of the scorer script on the task's
website, please download and use this version. We modified the way the
compute the agreement by chance. If you are familiar with the kappa
metric, you will know that this component can be interpreted in
different ways and essentially our previous interpretation made the
overall kappa scores unnecessarily over pessimistic. We'll explain the
way the computation was done in the task description paper.

Best,

Lucia

Reply all
Reply to author
Forward
0 new messages