CFP: Shared Task on Semantic Similarity and Textual Entailment in Portuguese

[Erick Fonseca]

ASSIN Shared Task - PROPOR 2016

Second Call For Participation

ASSIN - Avaliação de Similaridade Semântica e Inferência Textual

Shared task on Semantic Similarity and Recognition of Text Entailment in Portuguese

16 July 2016

Tomar, Portugal (Held in conjunction with PROPOR 2016)

http://propor2016.di.fc.ul.pt/?page_id=381

ASSIN is an evaluation forum for two related tasks: semantic

similarity and textual entailment recognition. It introduces

a large-scale dataset annotated for both phenomena in 

Portuguese, allowing the development of machine learning-based

NLP systems to solve them.

The ASSIN workshop will promote discussions on the subject,

its difficulties and their possible solutions, as well as a

comparison of the contribution of different computational

techniques, tools and linguistic resources by researchers

taking part in the shared task.

Task Description

UPDATE: All training data is now available

The ASSIN dataset contains 10,000 sentence pairs collected

from Google News, half from Brazilian sources and half from

Portuguese ones. 6,000 pairs were already released for training

and the remaining will serve as a blind test set. Each pair

is annotated for both semantic relatedness and textual

entailment. 

Semantic relatedness is measured on a scale from 1 to 5 (see

the website for detailed information). Text entailment occurs

when a sentence T (the text) entails another sentence H (the

hypothesis) if, after reading both and knowing that T is true,

a person concludes that H must also be true. ASSIN also

distinguishes bidirectional entailment cases, or paraphrases.

The training data is available at http://nilc.icmc.usp.br/assin

Important dates

Release of the test data to registered participants: February 27, 2016.

Evaluation period: February 27 – March 8, 2016

Paper submission: April 15, 2016

Reviews due: May 5, 2016

Camera ready papers: June 5, 2016

Workshop: July 16, 2016 (after PROPOR main conference)

Organizers

Erick Fonseca (ICMC/University of São Paulo, Brazil)

Sandra Aluísio (ICMC/University of São Paulo, Brazil)

Marcelo Criscuolo (ICMC/University of São Paulo, Brazil)

Leandro Santos (ICMC/University of São Paulo, Brazil)