Re: Question regarding manual evaluation:
Josh Schroeder
You can distribute the scores however you want amongst the 5 ranks,
and ties are allowed.
Our official metric is based on one system being ranked higher than
another, not on the absolute value of the ranks.
-Josh
On 12 Jan 2009, at 20:33, Gregor Leusch wrote:
> Hi,
>
> I wanted to post this to the mailing list, but unfortunately, google
> groups refused to accept my mail, so please feel free to forward the
> answer.
>
> ***
>
> Hi,
>
> a question regarding the ranking:
>
> does the ranking always have to start with 1, and fill all
> consequent integers?
>
> In other words, if I have one good translation, three absolutely
> (and equally)
> mediocre, and one that is really bad, can I express this by scoring
> 1 3 3 3 5
> or do I have to score
> 1 2 2 2 3
> ?
>
> Gregor
>
>
>
> --
> Dipl.-Inform. Gregor Leusch Tel
> +49-241-80-21618
> Chair of Computer Science 6 Fax
> +49-241-80-22219
> RWTH Aachen University leu...@informatik.rwth-aachen.de
> D-52056 Aachen, Germany www-i6.informatik.rwth-aachen.de/
> ~leusch/
>
>
>
--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.