Invitation to participate in WikiKGQA 2026 Challenge @ ISWC 2026

[Debayan Banerjee]

Dear members,

[Apologies for cross-postings]

We are pleased to announce WikiKGQA: Wiki-Based Knowledge Graph Question
Answering Challenge co-located with the 25th International Semantic Web
Conference (https://iswc2026.semanticweb.org/), Bari, Italy.

Challenge website: https://wikikgqa.org
The competition takes place on the Codabench platform:
https://www.codabench.org/competitions/15358/

Important links and dates:
- Training data + challenge platform:
https://www.codabench.org/competitions/15358/
- Challenge solution submission: June 19, 2026 (11:59pm, AoE)
- Challenge results: July 10, 2026
- Paper submissions: July 17, 2026 (11:59pm, AoE)
- Paper acceptance notification: August 21, 2026
- Workshop days: October 25 OR 26, 2026

Motivation:

KGQA has a long tradition in the Semantic Web community, particularly
under the Question Answering over Linked Data (QALD) moniker and
challenge series. With the rise of LLMs, the performance of
state-of-the-art systems has improved dramatically. However, benchmark
datasets in the space exhibit well-known limitations, primarily due to
the manual cost of creating parallel corpora of natural language
questions and structured queries. To address these issues, QAWiki
(https://qawiki.org) proposes a collaboratively-edited community
resource of hand-crafted question-query pairs, focusing currently on
English & Spanish questions answered over Wikidata's SPARQL query
service. The motivation is two-fold: (1) to evaluate KGQA systems over a
hand-crafted collection of multilingual question-query pairs over
Wikidata, and (2) to build a community centered around QAWiki, to expand
and improve it, and thus expand and improve the resources available for
training and benchmarking of KGQA systems.

Task Description:

Knowledge Graph Question Answering (KGQA) offers a promising alternative
to end-to-end QA systems by obtaining truthful answers from trusted and
well-maintained knowledge sources, such as Wikidata, without losing the
flexibility of natural language input. Nevertheless, the creation of
large high-quality KGQA benchmarks remains a challenge, with many
benchmarks being small, containing errors or repetitive questions that
are rarely fixed. Therefore, building on QAWiki, this challenge aims to
make the creation of KGQA benchmarks a collaborative effort, just like
Wikidata itself.

For evaluation, a private test dataset will be created, which would be
integrated with QAWiki after the challenge. Over the years, we aim to
build an evolving high-quality KGQA dataset of increasing size.

English and Spanish queries will be ranked separately, each participant
can choose to participate only for one or for both languages. The task
itself consists of two sub-tasks for each language:
1. KGQA using only the input question
2. KGQA using the input question together with the metadata QAWiki
provides, i.e., entity and property annotations

In both sub-tasks, the goal is to generate an answer set for that
question based on the version of Wikidata accessible via the WikiKGQA
SPARQL endpoint or using the available Wikidata dump (more details at
the Codabench competition page). As an evaluation metric, Macro QALD F1
scores will be used to evaluate the result sets. The full training data
is available under https://www.codabench.org/competitions/15358/ as an
extended version of QALD_JSON
(https://www.nliwod.org/challenge#task-1-multilingual-question-answering-over-knowledge-graphs).

Submission Guidelines:
We invite submissions of English challenge papers (up to 8 pages
excluding references). All submissions should be formatted in the CEUR
layout
https://www.overleaf.com/latex/templates/template-for-submissions-to-ceur-workshop-proceedings-ceur-ws-dot-org/wqyfdgftmcfw


A prerequisite for submitting a challenge paper is to submit challenge
solutions of your approach to at least one of the tracks on Codabench:
https://www.codabench.org/competitions/1535.
Additionally, each participant/team has to let the organizers know their
team name such that Codabench submissions can be linked to paper
submissions later.
Submission link: https://easychair.org/conferences/?conf=wikikgqa2026
Codabench: https://www.codabench.org/competitions/15358/
Google Groups list: https://groups.google.com/g/wikikgqa-challenge-list/

The central communication platform for the challenge (aside from the
website) is the corresponding Google Groups list:
https://groups.google.com/g/wikikgqa-challenge-list/
For private concerns or questions, please contact David Maria Schmidt
(david....@uni-bielefeld.de) or any other organizer directly.

Follow us on social media!
LinkedIn: https://www.linkedin.com/company/wikikgqa/
Bluesky: https://bsky.app/profile/wikikgqa.bsky.social
Mastodon: https://mastodon.social/@wikikgqa

Workshop Organizers:
- Shakeeb Arzoo, CRISIL Ltd., S&P Global Ratings
- Debayan Banerjee, Leuphana University Lüneburg, Germany
- Philipp Cimiano, CITEC, Bielefeld University, Germany
- Aidan Hogan, DCC, Universidad de Chile
- Alberto Moya Loustaunau, DCC, Universidad de Chile
- David Maria Schmidt, CITEC, Bielefeld University, Germany
- Ricardo Usbeck, Leuphana University Lüneburg, Germany

regards,

Debayan