Hi all,
We are pleased to announce the commencement of the
Machine Learning Reproducibility Challenge 2023 (
MLRC2023). This is a unique online conference, which encourages the community to investigate the reproducibility, replicability and generalisability of published claims in top conferences in the literature. We invite submissions which investigate the recently published claims, add novel insights to them and enable reproducible research spanning various topics in the ML literature. New from this edition, submissions must be first accepted at
TMLR to be considered in the MLRC 2023 Proceedings. Please read the
author guidelines and
submission guidelines from
TMLR to get the submission format and to understand more about the reviewing process. Please read our announcement
blog post for more motivation, retrospectives and roadmap for the challenge.
Scope
We invite thorough reproducibility studies, including but not limited to:
- Generalisability of published claims: novel insights and results beyond what was presented in the original paper. We recommend you choose any paper(s) published in the 2023 calendar year from the top conferences and journals (NeurIPS, ICML, ICLR, ACL, EMNLP, ICCV, CVPR, TMLR, JMLR, TACL) to run your reproducibility study on.
- Meta-reproducibility studies on a set of related papers.
- Research on tools enabling reproducible research.
- Meta-analysis on the state of reproducibility in various subfields in Machine Learning.
Important Dates- Challenge goes live: October 23, 2023
- Submit to TMLR OpenReview.
- Deadline to share your intent to submit a TMLR paper to MLRC: February 16th, 2024 in the following form.
- This form requires that you provide a link to your TMLR submission. Once it gets accepted (if it isn’t already), you should then update the same form with your paper camera ready details.
- We aim to announce the accepted papers by May 31st, 2024, pending decisions of all papers.
Important LinksPlease consider sharing this information with your colleagues and circles. We hope you participate in this conference and contribute to the growing literature of reproducible research in Machine Learning.
Many thanks,
Koustuv Sinha, Jessica Zosa Forde, Mandana Samiei, Arna Ghosh, Lintang Sutawika, Siba Smarak Panigrahi
Organizing Committee, MLRC 2023
https://reproml.org/