About the scorer

[Fabio Ticconi]

Hi there, I'm a student doing a project of CL-WSD, using resources and
constraints of this SemEval task.

I'm doing the first subtask with italian translation of the target
word. I've a doubt about the scorer (best translation, verbose
options).

I paste here some "results" of the up-to-date scorer.pl:

coach.n.it Item 46 mode 'treno' : system 'autobus' wrong
coach.n.it Item 46 credit 0 guesses 1 human responses 4: score is 0
coach.n.it Item 47 credit 0.285714285714286 guesses 1 human responses
7: score is 0.285714285714286
coach.n.it Item 48 mode 'autobus' : system 'autobus' correct
coach.n.it Item 48 credit 0.375 guesses 1 human responses 8: score is
0.375
coach.n.it Item 49 credit 0.166666666666667 guesses 1 human responses
6: score is 0.166666666666667
coach.n.it Item 50 credit 0.222222222222222 guesses 1 human responses
9: score is 0.222222222222222

As you can see, some instances don't print the "correct/wrong" string,
but only the "credit/score" one. Is this a correct behaviour or not?
If yes, why 47 and 49, both correct, have different scores?

Thank you in advance, any help will be greatly appreciated.

[els lefever]

Hi Fabio,

I'm on holidays, and I don't have access to my SemEval data right now.

I'll have a look at it as soon as I'm back at the office (last week of August).

All the best,

Els Lefever.

> Date: Wed, 11 Aug 2010 15:36:42 -0700
> Subject: About the scorer
> From: fabio....@gmail.com
> To: cross-li...@googlegroups.com

[Fabio Ticconi]

Hi Els,

Thank you for your kindness. For now, have good holidays! :)