Results released in CodaLab (but still issues with 4D and 4E Arabic)

[Preslav Nakov]

Dear SemEval-2017 Task 4 participants,

The results have been released in CodaLab.

However, there are still issues with 4D and 4E Arabic. We are working on the issue.

-- The task organizers

[Mathieu Cliche]

Thanks!  

I think the scoring script for subtask B is an old one.  If you look at the average recall, most scores are in the 50s, that's because the average recall is divided by 3 instead of 2.  (an old version of the scoring script was expecting a Neutral class, so it divided by 3 for get the averages)

[Preslav Nakov]

Good point.

There are also some other issues with CodaLab. We will score all systems out of CodaLab to double-check.

Also, we are missing some team names.

Preslav

--
You received this message because you are subscribed to the Google Groups "SEMEVALTWEET" group.
To unsubscribe from this group and stop receiving emails from it, send an email to semevaltweet+unsubscribe@googlegroups.com.
To post to this group, send email to semeva...@googlegroups.com.
Visit this group at https://groups.google.com/group/semevaltweet.
For more options, visit https://groups.google.com/d/optout.

[Sara Rosenthal]

Regarding Subtask B - English and Arabic. Thank you Mathieu for pointing this out. Codalab was still pointing to the old version for the testing phase. I updated it and reran all the submissions. The recall's are all now higher. The accuracy and F-score are unchanged.

If anyone sees an issue with their results please let me know.

Thanks,

Sara

[Ali Hamdi]

What about the results of the subtasks 4D and 4E Arabic ?

[Preslav Nakov]

subtask D:

SamB 0.127 0.170 2.462

ELiRF-UPV 1.183 0.537 11.434

OMAM 0.202 0.238 4.835

subtask E: 

OMAM 0.548

ELiRF-UPV 0.564

--