General MT: unofficial scores and system submission deadline

[Tom Kocmi]

Hi All,

 

Thank you for participating in the WMT General MT Shared Task. We have compiled unofficial ranks for systems based on automatic evaluation: https://github.com/wmt-conference/wmt23-news-systems/blob/master/scores/unofficial_wmt23_general.pdf (you can find tables in TeX format in the same folder to be used in the system description papers).

Please, keep in mind that automatic evaluation is NOT the official ranking of WMT and will likely differ from the final human evaluation. Metrics such as BLEU and ChrF are not reliable in modern MT. Lastly, for Hebrew, refA is likely post-edited from ONLINE-B, so we also provide refB.

 

All systems and references can be found in the same Github repo.

 

The deadline for the system description papers submission is next week, on the 5th September.

 

You may use following the bibtex when citing the General MT shared task:

 

@inproceedings{kocmi-etal-2023-findings,

    title = "Findings of the 2023 Conference on Machine Translation ({WMT}23)",

    author = "Kocmi, Tom and Avramidis, Eleftherios and Bawden, Rachel and Bojar, Ond{\v{r}}ej and Dvorkovich, Anton and Federmann, Christian and Fishel, Mark and Freitag, Markus and Gowda, Thamme and Grundkiewicz, Roman and Haddow, Barry and Koehn, Philipp and Marie, Benjamin and Monz, Christof and Morishita, Makoto and Murray, Kenton and Nagata, Masaaki and Nakazawa, Toshiaki and Popel, Martin and Popovi{\'c}, Maja",

    booktitle = "Proceedings of the Seventh Conference on Machine Translation (WMT)",

    month = dec,

    year = "2023",

    address = "Singapore, Singapore (Hybrid)",

    publisher = "Association for Computational Linguistics",

}

 

 

Have a lovely day, on behalf of the organizing team,

Tom

(in Germany, he/him)

 

[Tom Kocmi]

One clarification about the results, those are only General MT results, testsuite challenge sets will be evaluated in isolation (you may find the testsuite translations in txt-ts and in xml files)

 

I also apologize for an error; this is the corrected bibtex:

 

@inproceedings{kocmi-etal-2023-findings,

    title = "Findings of the 2023 Conference on Machine Translation ({WMT}23)",

    author = "Kocmi, Tom and Avramidis, Eleftherios and Bawden, Rachel and Bojar, Ond{\v{r}}ej and Dvorkovich, Anton and Federmann, Christian and Fishel, Mark and Freitag, Markus and Gowda, Thamme and Grundkiewicz, Roman and Haddow, Barry and Koehn, Philipp and Marie, Benjamin and Monz, Christof and Morishita, Makoto and Murray, Kenton and Nagata, Masaaki and Nakazawa, Toshiaki and Popel, Martin and Popovi{\'c}, Maja and Shmatova, Mariya",

    booktitle = "Proceedings of the Seventh Conference on Machine Translation (WMT)",

    month = dec,

    year = "2023",

    address = "Singapore, Singapore (Hybrid)",

    publisher = "Association for Computational Linguistics",

}

 

Have a lovely day,

Tom

(in Germany, he/him)

 

--
You received this message because you are subscribed to the Google Groups "WMT: Workshop on Machine Translation" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wmt-tasks+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wmt-tasks/AS4PR83MB05238E0FC3B3132A2A807668FAE7A%40AS4PR83MB0523.EURPRD83.prod.outlook.com.