Test data for subtask D
25 views
Skip to first unread message
Doris Hoogeveen
Jan 31, 2017, 1:12:44 AM1/31/17
to SemEval-2017 Task 3 CQA
Dear participants,
Some of you reported having problems with submitting test results for subtask D. The reason was a few missing lines in the goldstandard data. I have sent some of you the exact list of id pairs that we expect in the predictions file, but in the meantime I have adjusted the scorer, so it should now accept any submission, with or without the extra id pairs.
But of course, don't hesitate to let me know if you run into problems anyway.
Kind regards,
Doris
Some of you reported having problems with submitting test results for subtask D. The reason was a few missing lines in the goldstandard data. I have sent some of you the exact list of id pairs that we expect in the predictions file, but in the meantime I have adjusted the scorer, so it should now accept any submission, with or without the extra id pairs.
But of course, don't hesitate to let me know if you run into problems anyway.
Kind regards,
Doris
Preslav Nakov
Jan 31, 2017, 9:02:15 AM1/31/17
to Doris Hoogeveen, SemEval-2017 Task 3 CQA
The reason is that 19 lines in the inout file are duplicated.
We will make sure we remove these duplicates before we score.
They were already removed from the GOLD file, which caused the problem for some of the submissions.
--
Task website: http://alt.qcri.org/semeval2016/task3/
---
You received this message because you are subscribed to the Google Groups "SemEval-2017 Task 3 CQA" group.
To unsubscribe from this group and stop receiving emails from it, send an email to semeval-cqa+unsubscribe@googlegroups.com.
To post to this group, send email to semev...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/semeval-cqa/0c3841a8-820f-41cc-8901-97ea366ebc2e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply all
Reply to author
Forward
0 new messages