RETECO SemEval-2027: training and development data now available

85 views
Skip to first unread message

Abdelrahman Abdallah

unread,
Sep 15, 2026, 5:29:02 PMSep 15
to semeval-2027-reteco
Dear all,

Welcome to the RETECO mailing list, and thank you for joining.

RETECO is a SemEval-2027 shared task on reasoning-oriented retrieval, with two tracks and five sub-tracks:

Track 1 - Temporal Grounded Retrieval (13 domains)
  1a Temporal Retrieval
  1b Step-wise Temporal Retrieval

Track 2 - Reasoning-Intensive Conversational Retrieval (11 domains)
  2a Conversational Retrieval
  2b Gold-Passage Generation
  2c Full Conversational RAG

You can enter any subset of sub-tracks, including just one.

What is available now:
- Training and development data for all 24 domains, with gold judgments for both splits:
  https://huggingface.co/datasets/DataScience-UIBK/RETECO-SemEval2027
- Starter kit with a BM25 baseline, the official scorer, and a submission format checker:
  https://github.com/DataScienceUIBK/RETECO/tree/main/starter_kit
- Full participation guide:
  https://datascienceuibk.github.io/RETECO/participate.html

Upcoming:
- 10 January 2027: evaluation window opens
- 31 January 2027 (at the latest): evaluation window closes
- February 2027: system description papers due (tentative)

The competition platform, registration form, and submission limits will be announced here, so please stay subscribed.

If you run into any issues with the data or the scorer, or something in the guide is unclear, just reply to this list. We are happy to help, and your question will likely help other teams too.

We look forward to your participation!

Best regards,
Abdelrahman Abdallah, Mohammed Ali, Muhammad Abdul-Mageed, Kevin Duh, and Adam Jatowt
RETECO organizers
Reply all
Reply to author
Forward
0 new messages