Questions for Large-scale Multilingual Task

[Blaise Cruz]

Hello! A few questions regarding the Large-scale Multilingual Task.

1. The statmt website says that the evaluation dashboard for the task will open on Jun 28, but the website has not been updated yet. Is the dashboard up? Will we be using dynabench like last year?

2. Are pretrained models allowed for the constrained track or will it only allow translation models trained from scratch?

Apologies and thanks in advance!

Best regards,

Blaise Cruz

[Antonis Anastasopoulos]

Hi Blaise,

we're working on the evaluation dashboard and we're putting the finishing touches (additional metrics etc). It will open soon, we hope by next week! And yes, we're working with MLCommons to use dynabench like last year.

Regarding pre-trained models: they're allowed, as long as they were open-sourced before May 10 2022. So, for example, mBERT/XLM-R/mBART are OK to use, since they were open-sourced before that. But a model like say Minerva that was published (I think?) a couple of days ago wouldn't be allowed.

Best, looking forward to your submission!

Antonis Anastasopoulos

--
You received this message because you are subscribed to the Google Groups "Workshop on Statistical Machine Translation" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wmt-tasks+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wmt-tasks/3701fe9f-1e44-4813-b935-b3fede43ac5an%40googlegroups.com.