[CFP] Announcing the ML Reproducibility Challenge 2022

135 views

Skip to first unread message

Koustuv Sinha

unread,

Aug 24, 2022, 6:15:16 PM8/24/22

to ml-...@googlegroups.com

Dear Instructors/Researchers/Students,

We are happy to announce the official launch of the ML Reproducibility Challenge 2022 (RC2022)!

We will be accepting reproducibility reports of papers published this year in relevant conferences or journals, including the conferences: NeurIPS 2022, ICML 2022, ICLR 2022, ACL 2022, EMNLP 2022, CVPR 2022, ECCV 2022, AAAI 2022, IJCAI-ECAI 2022, ACM FAccT 2022, SIGIR 2022, and also for papers published in top ML journals in 2022, including JMLR, TACL and TMLR.

Key Dates

Notification of the challenge: August 19th, 2022
Deadline to submit reports: February 3rd, 2023
Notification of acceptance: May 15th, 2023

Over the last five years we have hosted a series of Machine Learning reproducibility challenges (v1, v2, v3, v4, v5). These challenges have helped raise visibility on the importance of producing papers that support reproducibility, sound scientific methodology, and robust results. We have been having active participation from several course instructors who have incorporated the ML reproducibility challenge as a component of their course, typically as the final course project. The challenges have led to several reproducibility reports, some of which were published in four volumes of the journal ReScience (see J1, J2, J3, J4), and have helped emphasize the importance of reproducibility, sound scientific methodology, and robust results. We are also happy to announce a poster session for RC2021 at NeurIPS this year, which would add more incentives for participating in the challenge.

If you are teaching a course where this may be appropriate, we strongly encourage you to participate (please fill out this form to feature your course on our website)! If you are a student in an ML course and want to participate in it, you are more than welcome, and we encourage you to share this information to your instructor/TAs. Submissions are also most welcome from the industry as well.

We hope you will participate in this challenge, include this in your course, and encourage your colleagues and students to participate.

Best regards,

Koustuv Sinha, Jesse Dodge, Jessica Forde, Robert Stojnic, Samarth Bhargav, Maurits Bleeker, Sharath Chandra, Joelle Pineau