CHiME 2023 Challenge - call for proposals / deadline 26th Sept

[Emmanuel Vincent]

https://www.chimechallenge.org

Submission deadline - 26 Sept 2022

---

CHiME is a challenge series whose purpose is to produce scientific advances in the field of distant microphone speech processing. The challenge’s six previous iterations from 2011 until 2020 have resulted in major improvements in word error rate and diarisation error rate for single-speaker voice commands in various environments and for multi-speaker conversations in homes. These six previous iterations have been centrally organised.

We now wish to introduce a more open model, where the annual CHiME Challenge features independent tasks organised by teams who work within a schedule defined by the CHiME Steering Group. Teams will be responsible for managing the tasks, i.e., providing data and instructions, scoring submissions and results, etc. The CHiME Steering Group will then work to support teams in the development of their task. The Steering Group will also organise a CHiME workshop where the tasks will be presented.

The CHiME Steering Group is now inviting task proposals for the CHiME 2023 Challenge. 

Proposed tasks will be evaluated according to the following criteria:

- relevance to the CHiME goals

- methodology - ie. likelihood to lead to robust and novel scientific findings

- accessibility - i.e. how easy is it for teams to engage with

- feasibility - likelihood to complete on schedule, risk management

- reproducibility - e.g. openness of data, baseline

- diversity - amongst organisers and expected participants, e.g. academia vs industry, geography, gender etc.

We wish to foster a collaborative proposal process that minimises overlap between proposals. To this end, if you are considering proposing a task, please sign up to the CHiME mailing list and feel free to discuss your plans. Or if you wish information to be shared more privately, i.e., amongst other bidders only, then you may share details or initial plans with the CHiME Steering Group who will then share with others whose plans may overlap.

Proposal

Task bidders should submit a task proposal for review by the Steering Group. The proposal should be a maximum of two pages (pdf format preferred) including the following information:

- Task name

- Coordinators

- Keywords (e.g. enhancement, diarization, speaker recognition, etc)

- Definition (one sentence)

- Short description (incl. research question the task is tackling)

- Dataset description: development, evaluation (short description, how much data is already available and prepared, how long would it - take to prepare the rest, mention if you allow external data/transfer learning or not)

- Evaluation method / metric

- Baseline system (brief description, planned method if you do not have one from a previous challenge)

- List of expected participants

- Contact person (for main communication, website)

Submission

The proposal should be sent to the CHiME Steering Group (chimech...@gmail.com) by 26th September. The Steering Group will review the proposals and provide feedback by 10th October. In case task amendments are requested, the task acceptance decision will be finalized by 4th November.

Important dates

- 26 Sept 2022 - Submission of task proposals

- 10 Oct 2022 - Feedback from Steering Group

- 21 Oct 2022 - Resubmission of conditionally accepted proposals

- 4 Nov 2022 - Notification of acceptance

- 1 Feb 2023 - Challenge opening

- 30 June 2023 - Challenge Submission deadline

- 25 Aug 2023 - Interspeech satellite workshop (details TBC)

For further information visit https://www.chimechallenge.org

CHiME Steering Group 

- Jon Barker Univ. of Sheffield, UK
- Emmanuel Vincent Inria, France
- Shinji Watanabe Carnegie Mellon Univ., USA
- Michael Mandel CUNY/META, USA
- Marc Delcroix NTT, Japan
- Leibny Paola Garcia Perera Johns Hopkins Univ., USA