NTCIR-18 - CALL FOR TASK PARTICIPATION - June 2025, Tokyo, Japan
CHUNG-CHI CHEN
[apologies if you receive multiple copies of this call]
NTCIR-18 tasks: https://research.nii.ac.jp/ntcir/ntcir-18/tasks.html
NTCIR-18 registration: https://research.nii.ac.jp/ntcir/ntcir-18/howto.html
____________________________________________________________
NTCIR-18 (June 10-13, 2025, Tokyo, Japan) now calls for task participation of anyone interested in research on information access technologies and their evaluation, such as retrieval from a large amount of document collections, question answering and natural language processing. We welcome students, young researchers, professors who supervise students, researchers working for a company, and anyone who is interested in informatics.
== What is NTCIR? ==
Development of Information Access technologies based on techniques of Information Retrieval, Natural Language Processing, and Database Management becomes increasingly more important for many applications, (e.g., providing effective access to Web resources and text archives, and analyzing big data obtained from various kinds of sensors). It is indispensable for developing such technologies to experimentally evaluate them by using test collections constructed under collaborations of many researchers. Over the past 20 years, NTCIR has been formulating the infrastructure for the evaluation, and contributing to development of the Information Access technologies. A total of over 80 "evaluation tasks" have been organized, attracting over 1,000 participant research groups worldwide so far. Furthermore, over 4,600 research groups have signed up to use the NTCIR test collections in their research. Consequently, NTCIR has been a major forum for researchers to intensively discuss the evaluation methodology of emerging information access technologies.
== NTCIR-18 Tasks ==
NTCIR-18 Program Committee has selected 10 tasks.
The overview slide of each task is available at the
kick-off page: https://research.nii.ac.jp/ntcir/ntcir-18/kickoffcfp.html
For more details, please visit the websites of each task:
Task Overview page: https://research.nii.ac.jp/ntcir/ntcir-18/tasks.html
----------------------------------------------------------------------------
NTCIR-18 Tasks
Core Tasks
AEOLLM: Automatic Evaluation of LLMs
FairWeb-2: The Second Fair Web Task
FinArg-2: Temporal Inference of Financial Arguments
Lifelog-6: Personal Lifelog Organisation & Retrieval Task
RadNLP: Natural Language Processing for Radiology
MedNLP-CHAT Medical Natural Language Processing for AI Chat
Transfer-2: Resource Transfer Based Dense Retrieval
Pilot Tasks
HIDDEN-RAD: Hidden Causality Inclusion in Radiology Report Generation
SUSHI: Searching Unseen Sources for Historical Information
U4: Unifying, Understanding, and Utilizing Unstructured Data in Financial Reports
== How to Participate ==
1. Please carefully read "How to Participate to NTCIR-18 Task(s)":
https://research.nii.ac.jp/ntcir/ntcir-18/howto.html
2. Please register at https://ntcir.nii.ac.jp/index.php/ntcir-18-registration-form/
The datasets of each task will be delivered to the team after registration (the date may vary depending on the task).
Registration Due (Depend on the task):
Nov. 1st, 2024 (Passed)
FairWeb-2: The Second Fair Web Task
Nov. 15th, 2024
SUSHI Searching Unseen Sources for Historical Information
Dec. 1st, 2024
RadNLP: Natural Language Processing for Radiology
Dec. 16th, 2024
Transfer-2: Resource Transfer Based Dense Retrieval
Dec. 28th, 2024
U4: Unifying, Understanding, and Utilizing Unstructured Data in Financial Reports
Jan 8th, 2025
HIDDEN-RAD: Hidden Causality Inclusion in Radiology Report Generation
Jan 15th, 2025
AEOLLM: Automatic Evaluation of LLMs
FinArg-2: Temporal Inference of Financial Arguments
Lifelog-6: Personal Lifelog Organisation & Retrieval Task
Jan 17th, 2025
MedNLP-CHAT: Medical Natural Language Processing for AI Chat
== Schedule ==
(*Schedule can be different for different tasks. Please visit
webpages of each task for the details.)
Sep 2024-Feb 2025 Formal run*
Feb 1, 2025 Evaluation results return
Feb 1, 2025 Task overview release (draft)
Mar 1, 2025 Submission due of participant papers (draft)
May 1, 2025 Camera-ready participant paper due
Jun 10-13, 2025 NTCIR-18 Conference
(At the NTCIR-18 Conference, an online presentation will be available)
== Questions? ==
For information regarding the task specifications etc.,
please contact the NTCIR Program Chairs: ntc1...@nii.ac.jp or task organizers.
For information regarding the online registration and previous NTCIR
test collections, please contact the NTCIR office: ntc-sec...@nii.ac.jp
We are looking forward to your participation!
NTCIR-18 Program Co-Chairs:
Qingyao Ai (Tsinghua University, China)
Chung-Chi Chen (AIST, Japan)
Shoko Wakamiya (NAIST, Japan)
NTCIR-18 General Co-Chairs:
Charles Clarke (University of Waterloo, Canada)
Noriko Kando (National Institute of Informatics, Japan)
Makoto P. Kato (University of Tsukuba, Japan)
Yiqun Liu (Tsinghua University, China)