resuming weekly conf-calls for the MLCommons Task Force on Automation and Reproducibility

[Grigori Fursin]

Dear all,

Following our recovery from COVID, we are restarting the weekly conf-calls for the Automation and Reproducibility TF. 

The goal of the TF is still the same: making it easier to run, optimize, reproduce, compare and extend ML(Perf) benchmarks (inference, training, tiny) across any software and hardware from any vendor using the common and technology-agnostic MLCommons CM interface (natively or inside automatically generated containers), modular C++ implementation of MLPerf inference benchmarks and CK playground. Another important goal is to support reproducibility initiatives at ML and Systems conferences.

You can find the agenda for the upcoming conf-call this Thursday at 10am PT in this shared doc including CK community awards, collaboration with other MLCommons WGs and TFs, planned automation extensions for MLPerf inference, training and tiny benchmarks, and CM support for the Student Cluster Competition’23.

Please, use this doc to suggest your topics of interest and request new features to automate MLPerf inference v4.0 benchmarks, reproduce past results from different vendors and support new benchmarking and reproducibility initiatives.

For now, we plan to keep the old time (10am PT on Thursdays starting from October 26th) but if this time is not suitable, please tell us and we will try to accommodate it. You can find and accept the current conf-call invite using this Google Calendar link. You can also join our community via this public Discord server.

Looking forward to collaborating with all of you this Fall,

Arjun and Grigori

MLCommons Task Force on Automation and Reproducibility

cTuning foundation