Gold Summaries for Validation

[Niranjan Ramesh Rao]

Hello Sir,

Is there any reason why gold summaries for validation data is not released, as this would be extremely useful in the Test Phase.

And to be clear, there are only 3 submissions possible for test data right? The submission platform mentions 

• For every subtask, maximum 10 submissions per day are allowed.

Please clarify this point.

Regarding paper publication, will gold summaries or evaluation script for test set be released? This would be useful to carry out experiments which are not entirely focused on improving ROUGE accuracy.

Regards

Niranjan Rao

[Parth Mehta]

Dear Niranjan,

We do not plan to release validation / test set gold standard as of now as we are contemplating about keeping the leaderboard active beyond FIRE 2022.

I understand that there will be experiments for which ROUGE is not a focus, and we will try to figure out a way to make that possible. We will reach a decision soon and let you know.

The evaluation script we will surely make public and you can reuse it.

> And to be clear, there are only 3 submissions possible for test data right? The submission platform mentions
> For every subtask, maximum 10 submissions per day are allowed.

Yes you are right, there are at most three submissions per language. There is a mistake on the webpage and we will correct it soon. Thanks for pointing it out.

Regards,

Parth Mehta

--
You received this message because you are subscribed to the Google Groups "ILSUM" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ilsum+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ilsum/9fc0d8f6-06e0-4c46-a1f1-946ab359efafn%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.