[CfP] Second Shared Task on Automatic Minuting at INGL

[Ondrej Bojar]

(apologies for multiple copies; this is not a WMT task but related)

*** Second Shared Task on Automatic Minuting ***
*** AutoMin 2023 as INLG Challenge ***

Automatic summarization of meetings into meeting minutes (aka
`minuting') would be an amazing tool. Superficially, large language
models like BART, GPT* and friends may seem to have solved it already,
but serious concerns still remain regarding completeness and factual
correctness.

We would like invite you to take part in AutoMin 2023, the second
instance of the shared on automatic minuting, i.e. summarization of
English and Czech meeting transcripts into meeting minutes and their
evaluation. See the details on task options below.

AutoMin 2023 is a registered INLG Challenge this year.

*** Website ***

https://ufal.github.io/automin-2023/

*** Tasks ***

We invite you to take part in one or both main tasks (Task A and D), and
optionally in two subsidiary tasks (Task B and C):

- Main Task A: The main task consists of automatically creating minutes
from multiparty meeting transcripts. The generated minute will be
evaluated both via automatic and manual metrics.

- Task B: Given a pair of meeting transcripts and minutes, the task is
to identify whether the minutes belong to the transcript. During our
data preparation from meetings on similar topics, we found that this
task could be challenging given repetition and not straightforward
correspondences between transcripts and minutes.

- Task C: Given a pair of minutes, the task is to identify whether the
two minutes belong to the same or different meetings. This sub-task is
important as we want to uncover how minutes created by two different
persons for the same meeting may differ in content and coverage.

- Task D: Given a meeting transcript, a candidate minutes, and a set of
one or more reference minutes, assign a score indicating the quality
of the candidate. Task D will not be evaluated by a single criterion.
All submissions to Task D will be evaluated in terms of Pearson
correlation against all manual and all automatic evaluation scores
available in AutoMin 2023, including other submissions to Task D.

*** Registration ***

All participants should register using the following link:

https://forms.office.com/e/jdvsdtV5Tm

Registration and participation is free of charge.
Please get in touch in case you would be willing to contribute to manual
evaluation of AutoMin submissions.

*** Important Dates ***

Feb 6, 2023 All Training and Dev Set Data Available
Feb 20, 2023 Test Data Available
Apr 15, 2023 System Submission Deadline
May 3, 2023 Report Submission Deadline
TBA Camera Ready Submission; in line with INLG
September, 2023 AutoMin @ INLG 2023

*** Publication ***

All teams are required to submit a brief technical report describing
their method. Please submit using the INLG paper template (to be
announced) All reports must be a minimum of 2 pages and a maximum of 4
pages excluding references. Reports must be written in English.
The proceedings will be published in ACL Anthology.

*** Organizers ***

Tirthankar Ghosal, UFAL, Charles University, Czech Republic
Muskaan Singh, UFAL, Charles University, Czech Republic
Anja Nedoluzhko, UFAL, Charles University, Czech Republic
Ondřej Bojar, UFAL, Charles University, Czech Republic

*** Contact ***

For inquiries, contact <aut...@ufal.mff.cuni.cz>.

--
Ondrej Bojar (mailto:o...@cuni.cz / bo...@ufal.mff.cuni.cz)
http://www.cuni.cz/~obo