Important: Evaluation phase submissions

[Sapna Negi]

Dear participants

We have been receiving a number of common queries regarding the evaluation phase. Please be reminded of the following information:

1. Only 3 maximum attempts are allowed to submit the prediction file and maximum 2 attempts per day. This excludes failed uploads due to an incorrect file format, which can be seen during the time of upload.

2. Leaderboard will be hidden until the end of the evaluation phase. This is to avoid tuning your systems on the test dataset. Please use validation data / trial test data for this purpose.

3. At the end of the evaluation phase, only the best out of your 3 attempts will be considered for team ranking.

4. Please make sure you submit the full prediction file, i.e. without excluding any rows from the original file, otherwise your score will be 0.

5. The evaluation phase will end on Jan 26th, 2019 (GMT-12), 11.59 pm. CodaLab may show you a different date due to configuration issues.

6. You are not allowed to use any same-domain hand labeled dataset for subtask B.


Regards
Organisers

[Jean-Marc Marty]

Hello,

Just to be totally clear, we can't use the SubtaskB_Trial_Test_Labeled.csv data for training nor development ?

Regards,

JM 

[Sapna Negi]

Hi,

Correct, you can’t use it for SemEval evaluation.

Regards

Organisers

--
You received this message because you are subscribed to the Google Groups "SemEval 2019: Task 9" group.
To unsubscribe from this group and stop receiving emails from it, send an email to semeval-2019-ta...@googlegroups.com.
To post to this group, send email to semeval-2...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/semeval-2019-task-9/35facdf5-971e-4efd-9b93-dd0f95725de5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Sapna Negi]

We will re-consider/re-evaluate the use of subtask B for development, and make an announcement today.

Regards

Organisers

[Sapna Negi]

Dear participants,

Please be reminded that the evaluation phase ends today i.e. Jan 26th, 2019 (GMT-12), 11.59 pm.

Regards
Organisers