CodaLab Competitions pages now available

[Tristan Miller]

Dear all,

The evaluation criteria for our three subtasks have now been posted on
the task website at
<http://alt.qcri.org/semeval2017/task7/index.php?id=evaluation-criteria>.
This page explains the scoring metrics we will be using, as well as how
to properly format and submit your results files.

All SemEval tasks this year will be evaluated using the CodaLab
Competitions platform. This means that to participate in our shared
task on puns, you need to sign up for a free CodaLab account at
<https://competitions.codalab.org/>, and then register for one or more
of our three subtasks via the "Participate" tab on our subtask pages:

Subtask 1 (pun detection):
https://competitions.codalab.org/competitions/15705

Subtask 2 (pun location):
https://competitions.codalab.org/competitions/15706

Subtask 3 (pun interpretation):
https://competitions.codalab.org/competitions/15707

You can then submit your system results through the "Submit / View
Results" link on the "Participate" tab. Your results file will be
automatically checked for syntactic correctness and then scored against
the manually annotated gold standard.

Each subtask has three evaluation phases: a "Trial" phase where you can
practice submitting results, and two "Test" phases, one each for the
homographic and heterographic data sets.

Results submitted to the "Trial" phases are scored against the trial
data posted at
<http://alt.qcri.org/semeval2017/task7/data/uploads/semeval2017_pun_task.tar.xz>.
(Note that we have recently updated this data with some extra contexts,
so if you previously downloaded it, please download a new copy now.)

Please practice submitting your results in the "Trial" phases now! It's
important that you get the formatting of your results files correct,
because the "Test" phases allow you a maximum of two submissions. If
you encounter any problems or think you've found a bug in the scorer,
please get in touch with us.

Regards,
Tristan

--
Tristan Miller, Research Scientist
Ubiquitous Knowledge Processing Lab (UKP-TUDA)
Department of Computer Science, Technische Universität Darmstadt
Tel: +49 6151 162 5296 | Web: https://www.ukp.tu-darmstadt.de/