Non-English summarization

[bin]

Hi folks,

Thank you for creating the wonderful GEM benchmark!

The shared task contains the MLSum dataset used for evaluating multi-lingual summarization. However, our NLG model is applicable to English text only. Would it be possible for us to only submit English results on MLSum to the shared task, ignoring non-English ones?

Thanks!

Best regards,

Bin

[Simon Mille]

Hi Bin,

as far as I know you can submit for whichever dataset you want; note however that, in your case, you'd be asked to generate outputs for all the English MLSum test sets (official test set and special test sets) with your model. Here's some text from the official website:

While we highly encourage participation in the shared task even for a single dataset, we ask you to please submit outputs for all possible challenge sets to help us assess your submission.

hope this helps!
simon  

--
You received this message because you are subscribed to the Google Groups "gem-benchmark" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gem-benchmar...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gem-benchmark/d38b11cb-761c-4ff5-ac26-d642cceb268fn%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.