2nd Call For Participation: Automatic Minuting Shared Task - AutoMin @ SIGDIAL 2025

[Ondrej Bojar]

Third Run of the Automatic Minuting Shared Task - AutoMin @ SIGDIAL 2025

A great opportunity to assess LLMs' ability to summarize and retrieve information from long contexts

We would like to invite you to take part in AutoMin 2025, the third instance of bi-annual shared task on meeting summarization into structured meeting minutes. We build upon our past experience from two previous AutoMins and add a new challenge to facilitate personalized access: answering questions about meeting content.

As in the previous editions, the tasks are run in English and Czech while the new question-answering task will be run monolingually (English-only) and cross-lingually (Czech questions about English meetings).

This challenge is highly relevant for assessing LLMs' ability to summarize and retrieve information from long contexts, equivalent to the length of an hour-long meeting.

A dedicated workshop is scheduled for August 28th, 2025, as part of SIGDIAL 2025: https://2025.sigdial.org.

 

Challenge Website

https://ufal.github.io/automin-2025/ 

 

Tasks

We propose two main tasks.

• Task A (minuting): The task consists of automatically creating minutes from multiparty meeting transcripts. The generated minute would be evaluated both via automatic and manual metrics.

• Task B (question answering): The task consists in answering questions based on meeting transcripts. For this challenge, participants will be tasked with answering these questions either monolingually (English questions on English transcripts) or cross-lingually (Czech questions on English transcripts). Answers will be evaluated using a LLM-as-a-judge metric.

Participant registration

All participants should register using the following link. In particular, please provide your github login name to get access to the repository for making your submission.

Important Dates

Registration and CfP	From March 1st, 2025
Training and Dev Data Available	March 1st, 2025
Test Data Available	April 15th, 2025
System Run and Output Submission	April 29th, 2025
System Paper Submission	May 5th, 2025
Result Announcement	May 22d, 2025
Acceptance Notification of System Papers	June 2025 (TBA)
Camera Ready for System Papers	TBA
Event Date	August 28th, 2025

 

Publication

All teams will be required to submit a brief technical report describing their method. Details will be given on the challenge website soon.

 

Organizers

Laurent Besacier, Naver Labs Europe, France

Ondřej Bojar, UFAL, Charles University, Czech Republic

Tirthankar Ghosal, Oak Ridge National Laboratory, USA

Thibaut Thonet, Naver Labs Europe, France

Contact

The organizers can be contacted via the following email address: aut...@ufal.mff.cuni.cz

--

Ondrej Bojar (mailto:o...@cuni.cz / bo...@ufal.mff.cuni.cz)
http://www.cuni.cz/~obo