Re: ROUGE Evaluation Metric

[Ashok Urlana]

Dear Organizers,

Please help with the following questions
1. Which metric is used to compute the ROUGE score for Hindi and Gujarati? If possible, release the evaluation script.
2. Are we allowed to use different models in the testing phase that are not used in the validation phase?

Regards

Ashok 

[Parth Mehta]

Dear Ashok,

Same rouge metrics will be used for evaluation of all languages, as ROUGE is language independent. The only language dependent component is a stemmer which we will not be using.

We will release evaluation script after test submission closes, meanwhile we encourage all teams to evaluate using the submission platform and leaderboard.

And yes, you are free to use any new models during test phase irrespective of what you submitted in validation phase. However, the maximum submissions in test phase will be limited to 3, so we suggest choosing your models wisely. 

Let us know if you have any other questions.

Regards

Parth

--
You received this message because you are subscribed to the Google Groups "ILSUM" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ilsum+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ilsum/55d7556e-9769-47a5-a5da-e142ab0ff4f8n%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.