Call for Participation: WMT 2009 Shared Task
Philipp Koehn
We are organizing again next year the
Workshop on Statistical Machine Translation
http://www.statmt.org/wmt09/
which will be take place under the umbrella of the EACL 2009 conference
in Athens, Greece, March 30 and 31, 2009.
As in the previous year, the workshop will include a shared task on translation
of European language pairs. Translation quality will be evaluated on a shared,
unseen test set of news stories. We provide a parallel corpus as training data,
a baseline system, and additional resources for download. Participants may
augment this system or use their own system.
The testing period for the shared task is December 8-12.
The training and development data is now online.
Please register for further announcements on the workshop web page.
No formal registration is required, but it is highly recommended.
** Changes this Year **
After a number of extensions over the last years, this year's translation task
will be more focused. The motivation for this is to have a clearly defined task
and to be able to gather sufficient human judgement data to make as many
statistically significant distinctions as possible.
* News Translation Only: We will only evaluate performance on a set of
news stories that we prepared for this evaluation. As in the previous year, the
news stories are taken from major news outlets as the BBC, Der Spiegel,
Le Monde, etc. during the time period of September-October 2008.
Last year's test set will serve as the development set, which we
re-release in a
slightly cleaned form.
* Official Metric Manual Sentence Ranking: While we continue to experiment
with different forms of manual and automatic metrics, we use as official metric
human judgment of preference on a sentence-by-sentence basis.
* More Training Data: The parallel corpora Europarl and News Commentary
are extended, a large French-English corpus (currently 200 million words) was
added, and we also proved 50-500 million word monolingual training data
from the test domain of news stories.
* Constraint vs. Unconstraint: You may use additional resources
(training data,
knowledge bases such as existing translation systems), but you have to
flag that
your system uses these. We will distinguish system submissions that used the
provided training data (constraint) from submissions that used significant
additional data resources. Note that basic linguistic tools such as taggers,
parsers, or morphological analyzers are allowed in the constraint condition.
We would welcome your participation in the shared task.
Chris Callison-Burch (Johns Hopkins University)
Philipp Koehn (University of Edinburgh)
Christof Monz (University of London)
Josh Schroeder (University of Edinburgh)
Organizers of the Workshop