WMT 2025 Shared Task on Automated Translation Quality Evaluation: Now Open for Participation!

[Alon Lavie]

Dear colleagues,

This year’s shared task on Automated Translation Quality Evaluation brings together the strengths of the previous Metrics and Quality Estimation tasks under a single, unified evaluation framework.

https://www2.statmt.org/wmt25/mteval-subtask.html

We are pleased to announce that the shared task is now open to participants:

1. Task 1: Segment-level quality score prediction
   Predict a quality score for each source–target segment pair, leveraging document-level context and either ESA or MQM annotations.
   Full description: https://www2.statmt.org/wmt25/mteval-subtask1.html
   Submission platform: https://www.codabench.org/competitions/9794

2. Task 2: Span-level error detection
   Identify and locate translation errors within each segment (start/end indices) and classify their severity.
   Full description: https://www2.statmt.org/wmt25/mteval-subtask2.html Submission platform: https://www.codabench.org/competitions/9283

3. Task 3: Quality-informed segment-level error correction
   Automatically post-edit machine-translated text using quality annotations to generate minimal and accurate corrections.
   Full description: https://www2.statmt.org/wmt25/mteval-subtask3.html Submission platform: https://www.codabench.org/competitions/8371

Submissions must be received on the Codabench platform by no later than July 31 (AoE time)!

Please note that each participant is limited to submitting one primary system and two contrastive systems for each of the three tasks. Furthermore, the Codabench platform will enforce a limit of no more than ten submissions per participant per task (limited to no more than five submissions per day). See the task descriptions and instructions for details.

Note on the "Gambit" challenge set
While we encourage participants to cover as many language pairs as possible, we understand that the Gambit challenge set is extensive and may be too large for all participants to tackle in full. In this case, participants are encouraged to primarily focus on the following language pairs: en–el, en–fr, fi–el, fi–fr, tr–el, tr–fr.

If you have any questions or encounter any issues, please contact us at wmt-qe-metri...@googlegroups.com. We look forward to your participation and contributions!

On behalf of the organisers,

Alon Lavie

Greg Hanneman

Sweta Agarwal

Diptesh Kanojia

[Alon Lavie]

The deadline for participating in this year's WMT Shared Task on Automated Translation Quality Evaluation has been extended by six days to August 6, 2025 (AoE time).  Submissions are being accepted on the Codabench Platform (see details below).

It's not too late to register and participate by submitting a system over the next week!

We look forward to your participation and contributions!

On behalf of the task organizers,

Alon Lavie

Greg Hanneman

Sweta Agrawal

Diptesh Kanojia