[Shared Task 2023] Launch of Multi-lingual Document-grounded Dialogue Challenge

91 views
Skip to first unread message

Haiyang Yu

unread,
Feb 21, 2023, 10:31:02 AM2/21/23
to dialdoc
Dear ALL:

Welcome to our competition co-located with the DialDoc workshop at ACL 2023! The competition is hosted on the Tianchi platform at https://tianchi.aliyun.com/competition/entrance/532063/information?lang=en-us We invite the participants to join us to tackle a challenging Document-grounded Dialogue task in a multilingual setting! In this competition, you will be given a conversational query and a set of domain documents in Vietnamese and French. Your goal is to generate a piece of text that answers the query in the target language. 

To assess your performance, we will use the evaluation metrics such as token-level F1, SacreBleu and Rouge-L. You will be scored based on how accurately your generated response matches the ground-truth answer and how well it aligns with the target language. We will provide training data for this task, including 3,446 turns in Vietnamese and 3,510 turns in French. We have also organized the currently available Chinese and English document-grounded dialogue data. We hope that participants can leverage the linguistic similarities, for example, a large number of Vietnamese words are derived from Chinese, and English and French both belong to the Indo-European language family, to improve their models' performance in Vietnamese and French. Please note that no additional human annotated data is allowed. 

Cash prizes will be awarded to the top-performing participants. The winners will be determined based on their scores on the evaluation metrics. Additionally, we will require all winners to submit a technical paper describing their methods and approaches. The prize pool is $7000 in total: 1st Place: $3000; 2nd Place: $1600; 3rd Place: $1000; 4th Place: $800; 5th Place: $600. 

We have also provided a baseline model, which can be found at https://github.com/AlibabaResearch/DAMO-ConvAI/tree/main/acl23doc2dial

The Important dates for this competition are as follows: 

Training data & Dev data release: February 17, 2023 
Test data release: March 25, 2023 
Winners announced: April 07, 2023 
Technical paper submission deadline: April 24, 2023 

We look forward to your participation in this exciting competition!

Nico Daheim

unread,
May 26, 2023, 5:34:34 AM5/26/23
to dialdoc
Dear all,

I was wondering if it would be possible to provide the dev set labels for the shared task now that it is finished?
This would allow researchers to continue working with the dataset even though the leaderboard is closed.

Best Regards,

Nico Daheim
Reply all
Reply to author
Forward
0 new messages