Task Description Paper: Additional Information

[Harish Tayyar Madabushi]

Dear all, 

It's great to see the wide variety of methods that have been used for this task - if you haven't already done so, please fill in this form with methods used: https://forms.gle/REVPQ9Z88AJsLQCp7

Please do this by Thursday, 10th of February 2022. 

If you have trouble accessing it, please email me with details of what you have done (and, of course, your CodaLab username). For details of what to fill in, see this thread at: https://groups.google.com/g/semeval-2022-task-2-mwe/c/F2Hrd8gM5B0

We are extremely interested in learning about the methods that were used in solving this task, so we strongly urge you to consider submitting a paper. If you have any questions regarding submitting a paper or you think you need some kind of support please let me know by email and I will see what we can do to help. 

We are not just interested in deep learning methods or methods that use pre-trained language models, but are interested in any method that can be applied to this problem regardless of how it performs. NO method is too simple and no score is not good enough. Like we said before, please note that negative results (such as a model not performing well) are also of interest to the NLP community, and are worth writing a paper about. This will help others decide what to continue to build on. For more details on what is expected in the task description papers, see this thread: https://groups.google.com/g/semeval-2022-task-2-mwe/c/TAPR2-LsaGE

New Information

Use the following to cite the task: 

@inproceedings{tayyarmadabushi-etal-2022-semeval,
    title={{SemEval-2022 Task 2}: {Multilingual Idiomaticity Detection and Sentence Embedding}},
    author={Tayyar Madabushi, Harish and Gow-Smith, Edward and Garcia, Marcos and Scarton, Carolina and Idiart, Marco and Villavicencio, Aline },
    booktitle={Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022)},
    year={2022},
    publisher={Association for Computational Linguistics},
}

Use the following to cite the paper about the resources that were used in this task: 

@inproceedings{tayyar-madabushi-etal-2021-astitchinlanguagemodels-dataset,
    title = "{AS}titch{I}n{L}anguage{M}odels: Dataset and Methods for the Exploration of Idiomaticity in Pre-Trained Language Models",
    author = "Tayyar Madabushi, Harish  and
      Gow-Smith, Edward  and
      Scarton, Carolina  and
      Villavicencio, Aline",
    booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2021",
    month = nov,
    year = "2021",
    address = "Punta Cana, Dominican Republic",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2021.findings-emnlp.294",
    doi = "10.18653/v1/2021.findings-emnlp.294",
    pages = "3464--3477",
}

Baselines

We have now updated the task description website with the baselines on the evaluation set. You can the breakdown of the baselines along with a description of the baselines used at: https://sites.google.com/view/semeval2022task2-idiomaticity/baselines

As always, if you have any questions at all, please feel free to either post here or email me. 

Best wishes, 

Harish