Announcements from the organisers: scorer script, CodaLab, and test data release dates

[Doris Hoogeveen]

Dear participants of SemEval Task 3,

We have three announcements for you:

1. There's a new scorer available from the Data and Tools page on the Task Website (v2.3). The only thing that is different about this version is that it can be used for subtask E too. For the other subtasks it doesn't matter whether you use version 2.2 or version 2.3. For subtask E only version 2.3 works.

2. We have set up competitions on CodaLab for each of the subtasks. Here you can submit your results on the development set and the test set (when it becomes available).
Each competition contains a leaderboard, so you can see how your system's results compare to those of the other teams. For the development phase you can make as many submissions as you like. Only one of them will show up on the leaderboard, and you need to choose which one (if any). On the test set only two submissions are allowed, of which the primary shows up on the leaderboard. Again, it is up to you to choose which one that is.
The scores on the leaderboard are calculated using the same scorer script as the one on the Task Website, and CodaLab leaderboard will be used to determine the winners of the competitions.
If you run into trouble with CodaLab, don't hesitate to get in touch.

Subtask A: https://competitions.codalab.org/competitions/15632
Subtask B: https://competitions.codalab.org/competitions/15635
Subtask C: https://competitions.codalab.org/competitions/15636
Subtask D: https://competitions.codalab.org/competitions/15637
Subtask E: https://competitions.codalab.org/competitions/15653

3. The test data for Subtask E will be released later than for the other tasks, due to the specific nature of the data and subtask. The test data for subtasks A, B, C, and D will be released on the 9th of January, but for subtask E we will release it on the 21st of January instead.

Kind regards,
Doris Hoogeveen