WMT 2022 Large Scale Machine Translation Evaluation for African Languages track Queries

[Sunil Kumar Sahu]

Dear Evaluators,

We are a team planning to make a submission to the WMT 2022 Large Scale Machine Translation Evaluation for African Languages track. Specifically, our submission will be to the constrained translation track.

Towards this, we are facing a few confusions, listed below. We hope you can help us to clarify them.

1. Number of models allowed per submission: We are allowed a maximum of two submissions. In each submission, we have planned to submit 5 checkpoint files to address all 52+48 language pairs– where each checkpoint will address a different subset of the language pairs. The inference code wrapping our model inference will automatically route each language pair to the corresponding model. Please confirm if this methodology is okay for the evaluation.
2. Using Flores dev as validation set: As per the instructions on the track’s homepage, we have used Flores101 dev as our validation set. Noting that our submission is to the constrained translation track, please confirm if this is okay.
3. Assumptions of the evaluation environment: We have assumed that the machine for running model inference will have (A) CUDA version 10.2/11.1 and (B) Anaconda already installed. Please confirm if this is safe to assume.

Thanks, 

[Antonis Anastasopoulos]

Hi,

responses below:

1. Yes, multiple checkpoints are ok, as long as your wrapper handles that appropriately.

2. flores for validation: that's ok

3. yup, these are safe to assume.

Looking forward to your submission!

--
You received this message because you are subscribed to the Google Groups "Workshop on Statistical Machine Translation" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wmt-tasks+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wmt-tasks/4dd92dbd-c4ca-4638-94e2-f6cce1a4a5c6n%40googlegroups.com.