The new task shows the source and the reference along with some
context. You should supply simple Yes/No judgments about whether the
edited translations represent fully fluent and meaning-equivalent
alternatives to the reference.
The deadline for the manual evaluation is the end of Friday. If you
participated in the shared task, and your group has not yet completed
your 8 hour time commitment, then please log in and judge some
translations. The acceptability judgments are highest priority, but
the ranking statistics are still useful.
Here is the URL for the manual evaluation:
http://statmt.org/wmt09/judge/
Thanks again for your participation!
Chris Callison-Burch
Philipp Koehn
Christof Monz
Josh Schroeder