WMT 2024 General MT announcement

[Tom Kocmi]

Hello,

we’re thrilled to share that the WMT General MT Shared Task for 2024 is now officially open!

This year, we've introduced several innovative changes to further advance the field of machine translation (MT) research and align with LLMs shift. Here’s what’s new:

 

Paragraph-level Test Set: Moving away from sentence-level to paragraph-level evaluations to ensure more natural context and coherence in translations.

 

Redefining the Constrained Track: We've updated restrictions in constrained track, separating away closed systems into own category (such as ONLINE), and allowing several open LLMs in the constrained track to support fine-tuning.

 

Introduction of Multimodality: For the first time, we're embracing multimodal translations by incorporating a speech domain with both audio and automatic transcripts. Traditional text-to-text MT remains a possible approach, however, the original audio may improve translation quality.

 

Literary Domain: We're adding a literary domain to push the boundaries of creative and contextual translation work.

 

Robustness Evaluation through Social Domain: we evaluate translation of non-polished data in the context of social media and informal communications.

 

New language pairs: we newly introduce translation for Japanese into Chinese, English into Hindi, Icelandic, and Spanish.

 

Redefining Human Evaluation: We're going to redefine the human evaluation to support all changes.

 

System Breakers - Test-suite Subtask: Help us identify the limits of current MT systems by preparing challenge testsets.

 

Dive into the details, spread the word, and join us in pushing the frontier of MT research to new heights: https://www2.statmt.org/wmt24/translation-task.html

 

On behalf of WMT organizers,

Tom Kocmi

(in Germany, he/him)