RumourEval 2019 results

[Leon Derczynski]

Dear RumourEval participants,

Below are the results of this year's evaluation. Congratulations on your submissions! We look forward to hearing how you each did it; system description papers are due 28 Feb 2019. Details to follow; see the SemEval site http://alt.qcri.org/semeval2019/ for updates.

Results are also visible on CodaLab, where you can sort through columns as you like. The gold labels are not quite yet available, as they are being used in another evaluation exercise - but will be distributed to participating teams privately to assist in writing system description papers. Contact me directly about that.

User	Verif	RMSE	SDQC	Verif Eng	RMSE Eng	SDQC Eng
quanzhi	0.5765 (1)	0.6078 (15)	0.5776 (4)	0.5765 (1)	0.6078 (15)	0.5776 (4)
ukob-west	0.2856 (2)	0.7642 (14)	0.3740 (15)	0.2856 (2)	0.7642 (14)	0.3740 (15)
ukob-west	0.2763 (3)	0.7913 (13)	0.3214 (23)	0.2763 (3)	0.7913 (13)	0.3214 (23)
sardar	0.2620 (4)	0.8012 (12)	0.4352 (10)	0.2620 (4)	0.8012 (12)	0.4352 (10)
BLCU-nlp	0.2525 (5)	0.8179 (8)	0.6187 (1)	0.2525 (5)	0.8179 (8)	0.6187 (1)
BLCU-nlp	0.2434 (6)	0.8225 (7)	0.6187 (1)	0.2434 (6)	0.8225 (7)	0.6187 (1)
kochkinael	0.2350 (7)	0.8047 (11)	0.4702 (7)	0.2350 (7)	0.8047 (11)	0.4702 (7)
shaheyu	0.2284 (8)	0.8081 (10)	0.3053 (24)	0.2284 (8)	0.8081 (10)	0.3053 (24)
ShivaliGoel	0.2244 (9)	0.8623 (3)	0.3625 (17)	0.2244 (9)	0.8623 (3)	0.3625 (17)
mukundyr	0.2244 (9)	0.8623 (3)	0.3404 (20)	0.2244 (9)	0.8623 (3)	0.3404 (20)
mukundyr	0.2244 (9)	0.8623 (3)	0.3404 (20)	0.2244 (9)	0.8623 (3)	0.3404 (20)
Xinthl	0.2238 (10)	0.8623 (4)	0.2297 (25)	0.2238 (10)	0.8623 (4)	0.2297 (25)
lzr	0.2238 (11)	0.8678 (2)	0.3404 (20)	0.2238 (11)	0.8678 (2)	0.3404 (20)
sardar	0.2076 (12)	0.8089 (9)	0.3460 (19)	0.2076 (12)	0.8089 (9)	0.3460 (19)
Bilal.ghanem	0.1996 (13)	0.8264 (6)	0.4895 (5)	0.1996 (13)	0.8264 (6)	0.4895 (5)
Bilal.ghanem	0.1697 (14)	0.8292 (5)	0.4391 (8)	0.1697 (14)	0.8292 (5)	0.4391 (8)
NimbusTwoThousand	0.0950 (15)	0.9148 (1)	0.1272 (26)	0.0950 (15)	0.9148 (1)	0.1272 (26)
deanjjones	0.0000 (16)	0.0000 (16)	0.3267 (22)	0.0000 (16)	0.0000 (16)	0.3267 (22)
jurebb	0.0000 (16)	0.0000 (16)	0.3537 (18)	0.0000 (16)	0.0000 (16)	0.3537 (18)
z.zojaji	0.0000 (16)	0.0000 (16)	0.3875 (14)	0.0000 (16)	0.0000 (16)	0.3875 (14)
lec-unifor	0.0000 (16)	0.0000 (16)	0.4384 (9)	0.0000 (16)	0.0000 (16)	0.4384 (9)
lec-unifor	0.0000 (16)	0.0000 (16)	0.3879 (13)	0.0000 (16)	0.0000 (16)	0.3879 (13)
magc	0.0000 (16)	0.0000 (16)	0.3927 (12)	0.0000 (16)	0.0000 (16)	0.3927 (12)
Martin	0.0000 (16)	0.0000 (16)	0.6028 (3)	0.0000 (16)	0.0000 (16)	0.6028 (3)
ShivaliGoel	0.0000 (16)	0.0000 (16)	0.3625 (17)	0.0000 (16)	0.0000 (16)	0.3625 (17)
Martin	0.0000 (16)	0.0000 (16)	0.6067 (2)	0.0000 (16)	0.0000 (16)	0.6067 (2)
shaheyu	0.0000 (16)	0.0000 (16)	0.3359 (21)	0.0000 (16)	0.0000 (16)	0.3359 (21)
jacobvan	0.0000 (16)	0.0000 (16)	0.4792 (6)	0.0000 (16)	0.0000 (16)	0.4792 (6)
wshuyi	0.0000 (16)	0.0000 (16)	0.3699 (16)	0.0000 (16)	0.0000 (16)	0.3699 (16)
cjliux	0.0000 (16)	0.0000 (16)	0.4298 (11)	0.0000 (16)	0.0000 (16)	0.4298 (11)

All the best,

The RumourEval 2019 team

--

Natural Language Processing, Department of Computer Science

IT University of Copenhagen  http://nlp.itu.dk/  +45 5157 4948

[ghan]

Dear organizers,

Please consider the "team name" in the coda lab settings instead of the first and the second name of the participant.

Best,

[hunnyis...@gmail.com]

Dear organizers,

Could you please make seperate leaderboards on CodaLab for subtask A and subtask B, so that we can see our rankings for each subtask clearly? Thanks!

Best,