successful validation of the new CM (CK2) automation to run MLPerf inference at SuperComputing'22!
2 views
Skip to first unread message
Grigori Fursin
Nov 17, 2022, 1:47:25 PM11/17/22
to artifact-...@googlegroups.com, Collective Knowledge, inference
Dear All,
We are very excited to share with you our news that the latest version of the MLCommons CM automation meta-framework v1.1.1 was successfully validated at the Student Cluster Competition at SuperComputing'22.
Around 10 teams from all over the world had just 1 hour to run MLPerf inference with RetinaNet, Open Images, ONNX runtime and CPU or GPU and prepare their submission (using reduced data set) using this tutorial. There were no major issues and all teams submitted their results to the live W&B dashboard using our CM automation! Furthermore, these teams are interested to work with our MLCommons taskforce on education and reproducibilty to continue optimizing MLPerf and submit results to v3.0!
Many thanks to Hai Ah Nam, Steve Leak, Vijay Janappa Reddi, Tom Jablin, Ramesh N Chukka, Peter Mattson, David Kanter, Pablo Gonzalez Mesa, Thomas Zhu, Thomas Schmid and Gaurav Verma for their help, feedback and suggestions!
If you have 20 minutes, please help us test our CM automation on your system using this CM-MLPerf tutorial and report any issues here - you help will be very appreciated by the community!
Please find our CM-MLPerf development roadmap here and the CLI/Docker description of our CM script for modular MLPerf inference here .
One of our priorities is to add thorough CM tests and Docker containers for all reference implementations of the MLPerf inference benchmark to the inference repo. We have added the first test here and we have developed a prototype of a modular CM-MLPerf container here.
Feel free to join our weekly conf-calls, provide your feedback and participate in developments!
We are looking forward to continue working with you to modularize MLPerf inference benchmark and automate submissions!
Thank you,
Grigori and Arjun
-------------------
Grigori Fursin
* Vice President at OctoML.ai (MLOps)
* Author of the MLCommons automation meta-framework (CM/CK)
* A founding member of the ACM taskforce on reproducibility
We are very excited to share with you our news that the latest version of the MLCommons CM automation meta-framework v1.1.1 was successfully validated at the Student Cluster Competition at SuperComputing'22.
Around 10 teams from all over the world had just 1 hour to run MLPerf inference with RetinaNet, Open Images, ONNX runtime and CPU or GPU and prepare their submission (using reduced data set) using this tutorial. There were no major issues and all teams submitted their results to the live W&B dashboard using our CM automation! Furthermore, these teams are interested to work with our MLCommons taskforce on education and reproducibilty to continue optimizing MLPerf and submit results to v3.0!
Many thanks to Hai Ah Nam, Steve Leak, Vijay Janappa Reddi, Tom Jablin, Ramesh N Chukka, Peter Mattson, David Kanter, Pablo Gonzalez Mesa, Thomas Zhu, Thomas Schmid and Gaurav Verma for their help, feedback and suggestions!
If you have 20 minutes, please help us test our CM automation on your system using this CM-MLPerf tutorial and report any issues here - you help will be very appreciated by the community!
Please find our CM-MLPerf development roadmap here and the CLI/Docker description of our CM script for modular MLPerf inference here .
One of our priorities is to add thorough CM tests and Docker containers for all reference implementations of the MLPerf inference benchmark to the inference repo. We have added the first test here and we have developed a prototype of a modular CM-MLPerf container here.
Feel free to join our weekly conf-calls, provide your feedback and participate in developments!
We are looking forward to continue working with you to modularize MLPerf inference benchmark and automate submissions!
Thank you,
Grigori and Arjun
-------------------
Grigori Fursin
* Vice President at OctoML.ai (MLOps)
* Author of the MLCommons automation meta-framework (CM/CK)
* A founding member of the ACM taskforce on reproducibility
Reply all
Reply to author
Forward
0 new messages