Submission website and evaluation code are now online
33 views
Skip to first unread message
SemEval 2026 Task 5 Participants
Oct 10, 2025, 4:06:47 PMOct 10
to SemEval 2026 Task 5 Participants
Hello to all who are interested in participating in SemEval 2026 Task 5,
today we share some news:
1) The CodaBench Task website is now live at https://www.codabench.org/competitions/10877/
The development phase will run until January 10th. Anyone can register to the task, and you can already submit scores for the validation set.
2) An evaluation script is now also online and can be accessed here: https://github.com/Janosch-Gehring/semeval26-05-scripts
See the README for usage instructions. Aside from outputting Spearman and Accuracy scores, the script also includes a prediction format validator and two simple baselines (majority and random). What is currently still missing is a model baseline, but we plan to add one in the near future.
Thank you for your interest and have fun!
Best regards,
Janosch Gehring
today we share some news:
1) The CodaBench Task website is now live at https://www.codabench.org/competitions/10877/
The development phase will run until January 10th. Anyone can register to the task, and you can already submit scores for the validation set.
2) An evaluation script is now also online and can be accessed here: https://github.com/Janosch-Gehring/semeval26-05-scripts
See the README for usage instructions. Aside from outputting Spearman and Accuracy scores, the script also includes a prediction format validator and two simple baselines (majority and random). What is currently still missing is a model baseline, but we plan to add one in the near future.
Thank you for your interest and have fun!
Best regards,
Janosch Gehring
Reply all
Reply to author
Forward
0 new messages