CodaLab Downtime: Submissions Stuck in 'submitting' (25 Jan 2022)

[Harish Tayyar Madabushi]

Dear all, 

Unfortunately, CodaLab seems to be experiencing some difficulties affecting all competitions hosted on the site. CodaLab is working on resolving this bug and I will keep you posted on the progress. More information can be found on the CodaLab groups here. 

Meanwhile, you can use the development set (which has associated gold labels) to continue to improve your models. 

To reiterate: 

If there is a problem with CodaLab just before the deadline (and you cannot submit there for some reason), please email me your submission (exactly as you would upload it) to h.tayyar...@sheffield.ac.uk with the subject "Submission: SemEval 2022 Task 2 <your team name>". This does not extend the deadline, I must receive your email before 11:59pm UTC on the 31st of January 2022. 

Please feel free to reach out if there is something you need clarification with.

Best wishes, 

Harish

[Harish Tayyar Madabushi]

Dear all, 

CodaLab is back up again and all submissions have been processed. If your submission has failed, please check the following: 

1. For both subtasks, predictions on the development set (i.e for the file dev.csv) should be tested using the gold labels (dev.gold.csv) using the evaluation script (SubTask1Evaluator.py for Subtask A and SubTask2Evaluator.py for Subtask B). 
2. For both subtasks, predictions on the evaluation set (i.e for the file eval.csv) should be uploaded to the practice phase of CodaLab. The default phase when you click on "Participate" is not the practice phase. Make sure you select this phase if you want to submit to it. Your results will be available on the leaderboard in the "Results" section of CodaLab. 
3. Evaluation Phase: Your predictions on the test data (i.e for the file test.csv, available in the TestData folders for each subtask on GitHub) must be submitted to the Evaluation phase. 
4. If you do not submit the correct file to the correct phase, you will get the error "ERROR: IDs in Submission file do not match IDs in gold file!"
   
5. Your submission must consist of a zip file, which contains a single folder called "submission" containing a separate file for each subtask (task2_subtaska.csv and task2_subtaskb.csv).  See here for more information on the submission format.

Best 

Harish

[Rob v]

My submissions keep reporting failed, but there is no error (except for WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.)

I am planning to only participate in subtask-a one shot. A sample of how I uploaded the file is attached

[Harish Tayyar Madabushi]

Hi, 

This was our mistake and has since been fixed. You can now resubmit your submission and everything should work as expected. I'll keep an eye on things, but if you encounter any other problems, don't hesitate to get in touch. 

Harish