MLCommons task force on automation and reproducibility

Contact owners and managers

1–30 of 254

Welcome to the public mailing list for the open MLCommons task force on automation and reproducibility. Please check our mission, developments, weekly notes and news here. Looking forward to collaborating with you!

0 selected

11/9/23

CM automation update for October (MLPerf training, ACM YouTube video; SCC'23, MICRO'23 AE, etc)

Dear all, It was an exciting week for the adoption of our CM automation language: please find the

unread,

CM automation update for October (MLPerf training, ACM YouTube video; SCC'23, MICRO'23 AE, etc)

Dear all, It was an exciting week for the adoption of our CM automation language: please find the

11/9/23

10/25/23

resuming weekly conf-calls for the MLCommons Task Force on Automation and Reproducibility

Dear all, Following our recovery from COVID, we are restarting the weekly conf-calls for the

unread,

resuming weekly conf-calls for the MLCommons Task Force on Automation and Reproducibility

Dear all, Following our recovery from COVID, we are restarting the weekly conf-calls for the

10/25/23

9/22/23

[CM task force update] automating, visualizing and comparing upcoming MLPerf inference v4.0

The MLCommons Task Force on Automation and Reproducibility is pleased to announce the release of the

unread,

[CM task force update] automating, visualizing and comparing upcoming MLPerf inference v4.0

The MLCommons Task Force on Automation and Reproducibility is pleased to announce the release of the

9/22/23

9/5/23

urgently looking for 1..2 volunteers to help us evaluate complex MICRO'23 artifact

Dear all, We are urgently looking for 1..2 volunteers to help us evaluate a very interesting but

unread,

urgently looking for 1..2 volunteers to help us evaluate complex MICRO'23 artifact

Dear all, We are urgently looking for 1..2 volunteers to help us evaluate a very interesting but

9/5/23

8/11/23

Call for participation: HiPEAC Reproducibility Student Challenge – register by 15 September

hipeac.net View this email in your browser HiPEAC Reproducibility Student Challenge HiPEAC conference

unread,

Call for participation: HiPEAC Reproducibility Student Challenge – register by 15 September

hipeac.net View this email in your browser HiPEAC Reproducibility Student Challenge HiPEAC conference

8/11/23

8/11/23

Urgent: MICRO'23 is looking for artifact evaluators

Hi, The ACM/IEEE MICRO'23 conference is looking for motivated artifact evaluations to help the

unread,

Urgent: MICRO'23 is looking for artifact evaluators

Hi, The ACM/IEEE MICRO'23 conference is looking for motivated artifact evaluations to help the

8/11/23

7/20/23

CFP for MLS/H'23

Call for Papers International Workshop on Machine Learning for Software Hardware Co-Design (MLSH) in

unread,

CFP for MLS/H'23

Call for Papers International Workshop on Machine Learning for Software Hardware Co-Design (MLSH) in

7/20/23

7/19/23

weekly conf-call tomorrow (20/07) at 10am PT

Dear all, Just a reminder that we will have our weekly conf-call tomorrow 20/07 at 10am PT (1 hour)

unread,

weekly conf-call tomorrow (20/07) at 10am PT

Dear all, Just a reminder that we will have our weekly conf-call tomorrow 20/07 at 10am PT (1 hour)

7/19/23

7/18/23

Announcing the 1st Collective Knowledge Cup to benchmark and optimize AI systems (latency, throughput, energy, accuracy, cost)

Dear all, I would like to announce the 1st Collective Knowledge Cup - a set of open challenges

unread,

Announcing the 1st Collective Knowledge Cup to benchmark and optimize AI systems (latency, throughput, energy, accuracy, cost)

Dear all, I would like to announce the 1st Collective Knowledge Cup - a set of open challenges

7/18/23

6/21/23

[MLCommons Task Force on Automation and Reproducibility] agenda for our weekly conf-call tomorrow to prepare reproducibility and optimization challenges for MLPerf inference v3.1

Hi all, Please find the agenda for our tomorrow's weekly conf-call here (June 22, 10am PT, Google

unread,

[MLCommons Task Force on Automation and Reproducibility] agenda for our weekly conf-call tomorrow to prepare reproducibility and optimization challenges for MLPerf inference v3.1

Hi all, Please find the agenda for our tomorrow's weekly conf-call here (June 22, 10am PT, Google

6/21/23

6/1/23

[MLCommons task force on automation and reproducibility] conf-call agenda for today

Dear all, We plan to continue techical discussions about CM automation for MLPerf, research papers

unread,

[MLCommons task force on automation and reproducibility] conf-call agenda for today

Dear all, We plan to continue techical discussions about CM automation for MLPerf, research papers

6/1/23

5/10/23

[MLCommons task force on automation and reproducibility] agenda for tomorrow's conf-call (May 11)

Hi all, A reminder that our task force on automation and reproducibility will have a weekly conf-call

unread,

[MLCommons task force on automation and reproducibility] agenda for tomorrow's conf-call (May 11)

Hi all, A reminder that our task force on automation and reproducibility will have a weekly conf-call

5/10/23

5/3/23

[MLCommons task force on automation and reproducibility] agenda for tomorrow's conf-call

Hi all, A reminder that our task force on automation and reproducibility will have a weekly conf-call

unread,

[MLCommons task force on automation and reproducibility] agenda for tomorrow's conf-call

Hi all, A reminder that our task force on automation and reproducibility will have a weekly conf-call

5/3/23

4/27/23

CK playground presentation and an open panel on "Empowering Your Research with Open Source"

Dear all, We were asked to prepare a presentation with our vision and development plans for the

unread,

CK playground presentation and an open panel on "Empowering Your Research with Open Source"

Dear all, We were asked to prepare a presentation with our vision and development plans for the

4/27/23

4/25/23

Participating in upcoming optimization challenges to co-design Pareto-efficient AI/ML systems using the MLCommons automation platform

Dear all, Following our presentation at the MLCommons community meeting about our new MLCommons CK-

unread,

Participating in upcoming optimization challenges to co-design Pareto-efficient AI/ML systems using the MLCommons automation platform

Dear all, Following our presentation at the MLCommons community meeting about our new MLCommons CK-

4/25/23

4/17/23

the outcome of the 1st community submission to MLPerf inference v3.0 and asking for your feedback about our v3.1 plans

Dear all, We would like to give you an update about the progress of our MLCommons taskforce on

unread,

the outcome of the 1st community submission to MLPerf inference v3.0 and asking for your feedback about our v3.1 plans

Dear all, We would like to give you an update about the progress of our MLCommons taskforce on

4/17/23

4/12/23

Fwd: [HiPEAC Publicity] ACM Europe School

FYI -------- Forwarded Message -------- Subject: [HiPEAC Publicity] ACM Europe School Date: Wed, 12

unread,

Fwd: [HiPEAC Publicity] ACM Europe School

FYI -------- Forwarded Message -------- Subject: [HiPEAC Publicity] ACM Europe School Date: Wed, 12

4/12/23

3/14/23

Fwd: [HiPEAC Publicity] MICRO 2023 Call for Papers

-------- Forwarded Message -------- Subject: [HiPEAC Publicity] MICRO 2023 Call for Papers Date: Fri,

unread,

Fwd: [HiPEAC Publicity] MICRO 2023 Call for Papers

-------- Forwarded Message -------- Subject: [HiPEAC Publicity] MICRO 2023 Call for Papers Date: Fri,

3/14/23

3/13/23

Fwd: MLBench'23 Call for Papers

FYI -------- Forwarded Message -------- Subject: MLBench'23 Call for Papers Date: Mon, 13 Mar

unread,

Fwd: MLBench'23 Call for Papers

FYI -------- Forwarded Message -------- Subject: MLBench'23 Call for Papers Date: Mon, 13 Mar

3/13/23

Grigori Fursin2

3/7/23

Canceled: Helping to automate MLPerf inference v3.0 submissions

Following successful MLPerf inference v3.0 submission we will organize new conf-calls for our

unread,

Canceled: Helping to automate MLPerf inference v3.0 submissions

Following successful MLPerf inference v3.0 submission we will organize new conf-calls for our

3/7/23

2/15/23

Showcasing a new visualizer of MLPerf inference results and asking for feedback

Dear all, Our MLCommons taskforce has developed a prototype to visualize and compare private and

unread,

Showcasing a new visualizer of MLPerf inference results and asking for feedback

Dear all, Our MLCommons taskforce has developed a prototype to visualize and compare private and

2/15/23

2/9/23

Porting of old CK scripts to new CK/CM format

Hi all, We are about to complete a year since we started working on CK2/CM and we have already ported

unread,

Porting of old CK scripts to new CK/CM format

Hi all, We are about to complete a year since we started working on CK2/CM and we have already ported

2/9/23

Grigori Fursin, Grigori Fursin2

2/1/23

the latest update on automating MLPerf inference and resuming weekly conf-calls to help inference v3.0 submitters

Extra spam warning: please do not use ChatGPT to learn how to run MLPerf using the CK framework - the

unread,

the latest update on automating MLPerf inference and resuming weekly conf-calls to help inference v3.0 submitters

Extra spam warning: please do not use ChatGPT to learn how to run MLPerf using the CK framework - the

2/1/23

1/10/23

Happy New Year and a questionnaire to select a better time slot for our weekly conf-call

Dear colleagues, We wish you Happy New Year with many exciting achievements! We are searching for a

unread,

Happy New Year and a questionnaire to select a better time slot for our weekly conf-call

Dear colleagues, We wish you Happy New Year with many exciting achievements! We are searching for a

1/10/23

Grigori Fursin, David Kanter3

12/23/22

"reproducibility" discord channel

Hi David, I would like to use MLC discord server but we urgently needed a channel for discussions ;)

unread,

"reproducibility" discord channel

Hi David, I would like to use MLC discord server but we urgently needed a channel for discussions ;)

12/23/22

12/13/22

please fill in this questionnaire in you plan to submit to MLPerf inference v3.0

Dear all, As you know, we have created an open MLCommons taskforce to help you automate and validate

unread,

please fill in this questionnaire in you plan to submit to MLPerf inference v3.0

Dear all, As you know, we have created an open MLCommons taskforce to help you automate and validate

12/13/22

Grigori Fursin3

11/30/22

weekly notes and new agenda for the MLCommons "education and reproducibility" conf-call

Hi All, Please find our agenda for the conf-call tomorrow (Dec 1, 2022): https://docs.google.com/

unread,

weekly notes and new agenda for the MLCommons "education and reproducibility" conf-call

Hi All, Please find our agenda for the conf-call tomorrow (Dec 1, 2022): https://docs.google.com/

11/30/22

11/17/22

successful validation of the new CM (CK2) automation to run MLPerf inference at SuperComputing'22!

Dear All, We are very excited to share with you our news that the latest version of the MLCommons CM

unread,

successful validation of the new CM (CK2) automation to run MLPerf inference at SuperComputing'22!

Dear All, We are very excited to share with you our news that the latest version of the MLCommons CM

11/17/22

11/11/22

Asking for a favor to test modular benchmark automation for the Student Cluster Competition'22

Dear All, I would like to ask you for a favor: we have prepared a tutorial to help students run the

unread,

Asking for a favor to test modular benchmark automation for the Student Cluster Competition'22

Dear All, I would like to ask you for a favor: we have prepared a tutorial to help students run the

11/11/22

11/4/22

PLDI'23 is looking for artifact evaluation committee members

PLDI'23 is looking for artifact evaluation committee members - please help our colleagues with

unread,

PLDI'23 is looking for artifact evaluation committee members

PLDI'23 is looking for artifact evaluation committee members - please help our colleagues with

11/4/22

Search

Clear search

Close search

Google apps

Main menu