Final Scores and Paper submission deadline

[sandeep]

Hi Scott,

Thank you (and other organizers) for this competition and patiently answering all our questions. It's almost the end of the competition and I have last couple of questions : 

1. Are the current leaderboard scores the final scores or have they not yet been updated?

2. According to https://semeval.github.io/SemEval2022/ the system paper deadline is February 28? Is the paper submission deadline for this task still February 23?

Thanks in advance for your reply and have a nice day!

S 

[Tony]

Hi Scott,

Thanks for everything you done for this competition. I am also concerned about the deadline for the system paper, as said in the official website of SemEval2022, it is February 28.

Best,

Tony

[Scott Hale]

Hi Sandeep, Tony,

Apologies for the delayed response. The SemEval website was updated last Wednesday to reflect a new, extended deadline of 28 Feb. I've just emailed specifically on that. We're still waiting on final submission details, but I'll send those as soon as I have them (they should also be posted on https://semeval.github.io/SemEval2022/ once available).  

The final leaderboard rankings are now under "Task Results" on the CodaLab website per the email yesterday.

We're excited to get your papers,

Best wishes,

Scott

--
You received this message because you are subscribed to the Google Groups "semeval-2022-task-8-multilingual-news" group.
To unsubscribe from this group and stop receiving emails from it, send an email to semeval-2022-task-8-mult...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/semeval-2022-task-8-multilingual-news/92da46da-8e09-4bac-8780-eebea14978ccn%40googlegroups.com.

--

Dr Scott A. Hale

http://scott.hale.us
computer...@gmail.com

[Михаил Сергеевич Куимов]

Hi Scott,

I would like to ask, why has the leader board changed. I've checked it several days ago and the place of our team was 13th. Now it 15th. 

Respectfully,

Mikhail

суббота, 12 февраля 2022 г. в 13:27:20 UTC+3, computer...@gmail.com:

[David Jurgens]

Hi Mikhail,

  You might be looking at the post-evaluation period results, which will change over time since teams can still submit new results. The results of the task are finalized here (with some explanation of how the ranks were decided). There was an earlier email sent to this task email list going into some details on this as well.

  Thanks,

  David

To view this discussion on the web visit https://groups.google.com/d/msgid/semeval-2022-task-8-multilingual-news/c87e9dfd-e575-4cbc-8fd5-4e9bc0530862n%40googlegroups.com.

[Scott Hale]

Hi Mikhail,

Which page are you referring to? The final rankings of the shared task are under "Task Results" on the "Learn the Details" tab of the CodaLab website. That page has not changed as far as I know. 

If you look at the post-evaluation results, that is a live leaderboard that is after the evaluation period and can change. It is not the task rankings for SemEval.

Best wishes,

Scott

To view this discussion on the web visit https://groups.google.com/d/msgid/semeval-2022-task-8-multilingual-news/c87e9dfd-e575-4cbc-8fd5-4e9bc0530862n%40googlegroups.com.

[Михаил Сергеевич Куимов]

My mistake. I’ve referred to the wrong list. Thank you for your answer.

However, I wonder why my personal nickname (kmisterios) instead of the team name (SkoltechNLP) is listed in the final ranking. In the post-evaluation table, it is correct.

With respect,

Mikhail

[EMM]

Dear organizers,

we used one codalab account to submit contributions for 2 different teams using totally different approaches due to impossibility to get a second account activated before deadline. I wonder whether this has been taken into account when computing the final score, as it seems there is an assumption that all contributions originate from one algorithm (team names were clearly made different when submitting).

As Mikhail, I also noted that the team name in the ranking is not the one declared in the challenge but the name of the codalab account.

Best,

Nicolas