STS interpretable - call for participation

[Interpretable STS Semeval Task]

[Apologies for duplication]


Call for Participation SEMEVAL 2016 Task 2 -- Interpretable Semantic Textual Similarity

Semantic textual similarity (STS) has received an increasing amount of attention in recent years, culminating with the Semeval/*SEM tasks organized in 2012, 2013, 2014 and 2015, bringing together more than 60 participating teams. Please check http://ixa2.si.ehu.es/stswiki/ for details on previous tasks.

Given two sentences of text, s1 and s2, the systems participating in STS compute how similar s1 and s2 are, returning a similarity score. Although the score is useful for many tasks, it does not allow to know which parts of the sentences are equivalent in meaning (or very close in meaning) and which not.

The 2015 STS task offered a pilot subtask on interpretable STS. With the pilot task we wanted to explore whether STS systems are able to explain WHY they think the two sentences are related / unrelated, adding an explanatory layer to the similarity score. For 2016, the pilot subtask has been updated into a standalone task, for which we plan to provide new training and evaluation sets.

Please join the mailing list for updates at http://groups.google.com/group/iSTS-semeval. Check out the task's website at http://alt.qcri.org/semeval2016/task2/ for more details.


Important dates:

Trial data ready: June 30, 2015
Training data ready: August 30, 2015
Test data ready: December 15, 2015
Evaluation start: January 10, 2015
Evaluation end: January 31, 2015
Paper submission due: February 28, 2016 [TBC]
Paper reviews due: March 31, 2016 [TBC]
Camera ready due: April 30, 2016 [TBC]
SemEval workshop: Summer 2016


Organizers:

Eneko Agirre, Aitor Gonzalez-Agirre, Iñigo Lopez-Gazpio, Montse Maritxalar, German Rigau and Larraitz Uria.


References:

Agirre, E.  and  Banea, C.  and  Cardie, C.  and  Cer, D.  and Diab, M.  and  Gonzalez-Agirre, A.  and  Guo, W.  and Lopez-Gazpio, I. and Maritxalar, M. and Mihalcea, R.  and  Rigau, G.  and  Uria, L. and Wiebe, J. (2015). SemEval-2015 task 2: Semantic textual similarity, English, Spanish and pilot on interpretability. In Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), June.[pdf]