Results do not appear on the leaderboard

63 views
Skip to first unread message

yingjins

unread,
Jan 20, 2022, 1:06:20 AM1/20/22
to semeval-2022-task-2-MWE
Hi,

I I submitted the submission.zip file, and the status is "finished". But I can not see my result on the leaderboard.Could you please help to check what is wrong

Thank you.

Harish Tayyar Madabushi

unread,
Jan 20, 2022, 8:10:06 AM1/20/22
to semeval-2022-task-2-MWE
Hi, 

tl;dr: 
  1. The evaluation phase leaderboard is hidden, results will be released after the evaluation phase is complete (see reasons below)
  2. If you have submitted the same model results twice (by mistake) you can post here or email me and I will remove one of them so they are not counted towards your 5 submissions. 

The evaluation phase leaderboard is hidden - so you won't be able to see the results until after the evaluation phase. The reason for this is to make sure that there is no hyperparameter optimisation based on submissions to the test data. If you want to optimise your model, you should do this on either the development data (and associated gold labels or using the evaluation phase which is still active. You can continue to submit your results to the evaluation phase and the leaderboard for that phase is active (you can use the tabs on top of the "participate" and "results" sections to switch between the phases).

About your submissions, I see that you've made three submissions to the evaluation phase: One has failed, but two of them are the same (one with NaNs) - the one with NaNs should not have passed but is in the "Finished" state because of a CodaLab error. As a result it will also be counted towards your 5 submissions. I can fix this by deleting that submission - let me know if you'd like me to do that for you (either by reply here or by email). 

Harish

yingjins

unread,
Jan 24, 2022, 9:38:56 PM1/24/22
to semeval-2022-task-2-MWE
Hi Harish,

Could you please help to delete all my 3 submissions? 
Thank you.

Harish Tayyar Madabushi

unread,
Jan 25, 2022, 11:26:19 AM1/25/22
to semeval-2022-task-2-MWE
Hi, 

I've now deleted the three submissions. Please also see this conversions on NaNs  - I've decided not to block those submissions to avoid confusing participants. Don't hesitate to get in touch if you require any further clarifications. 

General note to everyone: You are only allowed 5 submissions to the test phase. Any submission that "fails" is not counted towards this. Any submission in the "finished" state is counted towards this limit and CodaLab will not allow you to make more than 5 submissions. So if you want me to delete a submission, please get in touch with me (either here or by email) along with your CodaLab username and the submission time associated with the submission that you want me to delete. 

Best
Harish

Reply all
Reply to author
Forward
0 new messages