WNUT17 Shared task results

[Leon Derczynski]

Hi all,

Thanks for your submissions. In total, seven teams submitted results:

Team	F (entity)	F (surface)
MIC-CIS	36.90%	50.38%
Arcada	40.09%	56.60%
Drexel-CCI	26.81%	59.92%
SJTU-Adapt	41.22%	60.00%
FLYTXT	38.10%	57.64%
SpinningBytes	41.76%	57.98%
UH Ritual	41.90%	66.59%

The gold standard file for the evaluation is here: http://noisy-text.github.io/2017/files/emerging.test.annotated

Congratulations to the top scorer, UH Ritual in both categories!

Good luck writing your system description papers. These should follow the same guidelines as the main workshop. The submission deadline remains Friday 7 July and the link is https://www.softconf.com/emnlp2017/w-nut/user/

All the best,

Leon

[Bill Yuchen Lin]

Hi Leon,

    Thanks for you effort in this wonderful shared task.  

    I'd like to make sure the page limit of the system description paper. Is that the same as emnlp short paper 4+2references?

    By the way, should we be anonymized? 

Thanks and regards,

Bill 

[Leon Derczynski]

Hi Bill,

You're welcome!

For limits, 4+2 is the target, 5+2 is probably fine, 6+2 is right out. And no, you don't need to anonymize your paper, but there is no penalty for doing so either. WIth system descriptions like these, or large dataset and other language resource papers, the effort required to anonymise effectively is often just too much and not very efficient, so we're not going to ask for that.

Best,

Leon