Invitation to Participate in SemTab 2025 – Semantic Web Challenge on Tabular Data to Knowledge Graph Matching

42 views
Skip to first unread message

Marco Cremaschi

unread,
Jun 8, 2025, 12:57:41 PMJun 8
to Sem-Tab Challenge

Dear Community,

We would like to invite you to take part in this year's SemTab challenge. SemTab 2025 will be hosted by the International Semantic Web Conference (ISWC 2025), to be held in Nara, Japan, from November 2 to 6, bringing together researchers and practitioners to advance the field of semantic technologies.

This year's challenge mandates the use of Large Language Models (LLMs) for all tasks. Participants are expected to employ LLM-based approaches, either through fine-tuning or Retrieval-Augmented Generation (RAG) strategies, to tackle the semantic annotation tasks. This focus aims to explore and benchmark the capabilities of LLMs in interpreting complex and realistic tabular data.

Challenge Tracks and Datasets

The 2025 challenge features two main tracks, each designed to test different aspects of semantic table interpretation:

  • MantisTable: This track utilises a carefully selected subset of 870 tables from the new version of the MammoTab dataset, comprising a total of 84,907 cell annotations. Participants are expected to address challenges such as disambiguation, homonymy, alias resolution, NIL detection, noise robustness, and collective inference. Only approaches based on Large Language Models are allowed, either in fine-tuning settings or using Retrieval-Augmented Generation strategies. The evaluation will focus exclusively on the Cell Entity Annotation (CEA) task.

  • Secu-table: This dataset involves security data extracted from Common Vulnerability and Exposure (CVE) and Common Weakness Enumeration (CWE) data sources. It comprises 1,554 tables, with 20% being error-free and 80% containing various challenges, such as ambiguity, NIL, missing context, and misspelt data. Participants are invited to utilise open-source large language models (LLMs) to address all STI tasks: Cell Entity Annotation (CEA), Column Type Annotation (CTA), and Column Property Annotation (CPA), using the SEPSES Computer Security Knowledge Graph, and the CEA task using the Wikidata Knowledge Graph.

Tentative Schedule

  • Release of datasets and instructions: June 9th, 2025 

  • Round 1 submission deadline: July 9th, 2025
    Note: To be invited to the conference for a presentation, you must submit to Round 1

  • Evaluation: from July 10th, 2025

  • Paper submission deadline: July 16th, 2025

  • Invitation to present at ISWC: July 23rd, 2025

  • Paper camera-ready submission deadline: September 15th, 2025

  • Round 2 (Final) submission deadline: October 20th, 2025

  • Final results to be announced at the conference: November 2-6, 2025

For more details and updates, please visit the official SemTab 2025 website: https://sem-tab-challenge.github.io/2025/

We look forward to your participation in advancing the field of Semantic Table Interpretation.

Best regards,

SemTab Chairs

Reply all
Reply to author
Forward
0 new messages