[Shared Task 2023] Launch of Multi-lingual Document-grounded Dialogue Challenge
91 views
Skip to first unread message
Haiyang Yu
Feb 21, 2023, 10:31:02 AM2/21/23
to dialdoc
Dear ALL:
Welcome to our competition co-located with the DialDoc workshop at ACL 2023! The competition is hosted on the Tianchi platform at https://tianchi.aliyun.com/competition/entrance/532063/information?lang=en-us We invite the participants to join us to tackle a challenging Document-grounded Dialogue task in a multilingual setting! In this competition, you will be given a conversational query and a set of domain documents in Vietnamese and French. Your goal is to generate a piece of text that answers the query in the target language.
To assess your performance, we will use the evaluation metrics such as token-level F1, SacreBleu and Rouge-L. You will be scored based on how accurately your generated response matches the ground-truth answer and how well it aligns with the target language. We will provide training data for this task, including 3,446 turns in Vietnamese and 3,510 turns in French. We have also organized the currently available Chinese and English document-grounded dialogue data. We hope that participants can leverage the linguistic similarities, for example, a large number of Vietnamese words are derived from Chinese, and English and French both belong to the Indo-European language family, to improve their models' performance in Vietnamese and French. Please note that no additional human annotated data is allowed.
Cash prizes will be awarded to the top-performing participants. The winners will be determined based on their scores on the evaluation metrics. Additionally, we will require all winners to submit a technical paper describing their methods and approaches. The prize pool is $7000 in total: 1st Place: $3000; 2nd Place: $1600; 3rd Place: $1000; 4th Place: $800; 5th Place: $600.
We have also provided a baseline model, which can be found at https://github.com/AlibabaResearch/DAMO-ConvAI/tree/main/acl23doc2dial.
The Important dates for this competition are as follows:
Training data & Dev data release: February 17, 2023
Test data release: March 25, 2023
Winners announced: April 07, 2023
Technical paper submission deadline: April 24, 2023
We look forward to your participation in this exciting competition!
To assess your performance, we will use the evaluation metrics such as token-level F1, SacreBleu and Rouge-L. You will be scored based on how accurately your generated response matches the ground-truth answer and how well it aligns with the target language. We will provide training data for this task, including 3,446 turns in Vietnamese and 3,510 turns in French. We have also organized the currently available Chinese and English document-grounded dialogue data. We hope that participants can leverage the linguistic similarities, for example, a large number of Vietnamese words are derived from Chinese, and English and French both belong to the Indo-European language family, to improve their models' performance in Vietnamese and French. Please note that no additional human annotated data is allowed.
Cash prizes will be awarded to the top-performing participants. The winners will be determined based on their scores on the evaluation metrics. Additionally, we will require all winners to submit a technical paper describing their methods and approaches. The prize pool is $7000 in total: 1st Place: $3000; 2nd Place: $1600; 3rd Place: $1000; 4th Place: $800; 5th Place: $600.
We have also provided a baseline model, which can be found at https://github.com/AlibabaResearch/DAMO-ConvAI/tree/main/acl23doc2dial.
The Important dates for this competition are as follows:
Training data & Dev data release: February 17, 2023
Test data release: March 25, 2023
Winners announced: April 07, 2023
Technical paper submission deadline: April 24, 2023
We look forward to your participation in this exciting competition!
Nico Daheim
May 26, 2023, 5:34:34 AM5/26/23
to dialdoc
Dear all,
I was wondering if it would be possible to provide the dev set labels for the shared task now that it is finished?
This would allow researchers to continue working with the dataset even though the leaderboard is closed.
Best Regards,
Nico Daheim
Reply all
Reply to author
Forward
0 new messages