Groups
Sign in
Groups
Artifact Evaluation
Conversations
About
Send feedback
Help
Artifact Evaluation
Contact owners and managers
1–30 of 61
Mark all as read
Report group
0 selected
Grigori Fursin
11/9/23
CM automation update for October (MLPerf training, ACM YouTube video; SCC'23, MICRO'23 AE, etc)
Dear all, It was an exciting week for the adoption of our CM automation language: please find the
unread,
CM automation update for October (MLPerf training, ACM YouTube video; SCC'23, MICRO'23 AE, etc)
Dear all, It was an exciting week for the adoption of our CM automation language: please find the
11/9/23
Grigori Fursin
10/25/23
resuming weekly conf-calls for the MLCommons Task Force on Automation and Reproducibility
Dear all, Following our recovery from COVID, we are restarting the weekly conf-calls for the
unread,
resuming weekly conf-calls for the MLCommons Task Force on Automation and Reproducibility
Dear all, Following our recovery from COVID, we are restarting the weekly conf-calls for the
10/25/23
Grigori Fursin
9/22/23
[CM task force update] automating, visualizing and comparing upcoming MLPerf inference v4.0
The MLCommons Task Force on Automation and Reproducibility is pleased to announce the release of the
unread,
[CM task force update] automating, visualizing and comparing upcoming MLPerf inference v4.0
The MLCommons Task Force on Automation and Reproducibility is pleased to announce the release of the
9/22/23
Grigori Fursin
9/5/23
urgently looking for 1..2 volunteers to help us evaluate complex MICRO'23 artifact
Dear all, We are urgently looking for 1..2 volunteers to help us evaluate a very interesting but
unread,
urgently looking for 1..2 volunteers to help us evaluate complex MICRO'23 artifact
Dear all, We are urgently looking for 1..2 volunteers to help us evaluate a very interesting but
9/5/23
Grigori Fursin
8/11/23
Call for participation: HiPEAC Reproducibility Student Challenge – register by 15 September
hipeac.net View this email in your browser HiPEAC Reproducibility Student Challenge HiPEAC conference
unread,
Call for participation: HiPEAC Reproducibility Student Challenge – register by 15 September
hipeac.net View this email in your browser HiPEAC Reproducibility Student Challenge HiPEAC conference
8/11/23
Grigori Fursin
8/11/23
Urgent: MICRO'23 is looking for artifact evaluators
Hi, The ACM/IEEE MICRO'23 conference is looking for motivated artifact evaluations to help the
unread,
Urgent: MICRO'23 is looking for artifact evaluators
Hi, The ACM/IEEE MICRO'23 conference is looking for motivated artifact evaluations to help the
8/11/23
Grigori Fursin
7/19/23
weekly conf-call tomorrow (20/07) at 10am PT
Dear all, Just a reminder that we will have our weekly conf-call tomorrow 20/07 at 10am PT (1 hour)
unread,
weekly conf-call tomorrow (20/07) at 10am PT
Dear all, Just a reminder that we will have our weekly conf-call tomorrow 20/07 at 10am PT (1 hour)
7/19/23
Grigori Fursin
7/18/23
Announcing the 1st Collective Knowledge Cup to benchmark and optimize AI systems (latency, throughput, energy, accuracy, cost)
Dear all, I would like to announce the 1st Collective Knowledge Cup - a set of open challenges
unread,
Announcing the 1st Collective Knowledge Cup to benchmark and optimize AI systems (latency, throughput, energy, accuracy, cost)
Dear all, I would like to announce the 1st Collective Knowledge Cup - a set of open challenges
7/18/23
Grigori Fursin
6/21/23
[MLCommons Task Force on Automation and Reproducibility] agenda for our weekly conf-call tomorrow to prepare reproducibility and optimization challenges for MLPerf inference v3.1
Hi all, Please find the agenda for our tomorrow's weekly conf-call here (June 22, 10am PT, Google
unread,
[MLCommons Task Force on Automation and Reproducibility] agenda for our weekly conf-call tomorrow to prepare reproducibility and optimization challenges for MLPerf inference v3.1
Hi all, Please find the agenda for our tomorrow's weekly conf-call here (June 22, 10am PT, Google
6/21/23
Grigori Fursin
6/1/23
[MLCommons task force on automation and reproducibility] conf-call agenda for today
Dear all, We plan to continue techical discussions about CM automation for MLPerf, research papers
unread,
[MLCommons task force on automation and reproducibility] conf-call agenda for today
Dear all, We plan to continue techical discussions about CM automation for MLPerf, research papers
6/1/23
Grigori Fursin
4/4/23
links to setup AE
Hi Liuyi, You can find and reuse info about how to set up Artifact evaluation here: * https://cTuning
unread,
links to setup AE
Hi Liuyi, You can find and reuse info about how to set up Artifact evaluation here: * https://cTuning
4/4/23
Grigori Fursin
12/23/22
"reproducibility" discord channel
Hi All, As you suggested, we have created a discord channel dedicated to our reproducibility
unread,
"reproducibility" discord channel
Hi All, As you suggested, we have created a discord channel dedicated to our reproducibility
12/23/22
Grigori Fursin
11/17/22
successful validation of the new CM (CK2) automation to run MLPerf inference at SuperComputing'22!
Dear All, We are very excited to share with you our news that the latest version of the MLCommons CM
unread,
successful validation of the new CM (CK2) automation to run MLPerf inference at SuperComputing'22!
Dear All, We are very excited to share with you our news that the latest version of the MLCommons CM
11/17/22
Grigori Fursin
11/11/22
Asking for a favor to test modular benchmark automation for the Student Cluster Competition'22
Dear All, I would like to ask you for a favor: we have prepared a tutorial to help students run the
unread,
Asking for a favor to test modular benchmark automation for the Student Cluster Competition'22
Dear All, I would like to ask you for a favor: we have prepared a tutorial to help students run the
11/11/22
Grigori Fursin
11/4/22
PLDI'23 is looking for artifact evaluation committee members
PLDI'23 is looking for artifact evaluation committee members - please help our colleagues with
unread,
PLDI'23 is looking for artifact evaluation committee members
PLDI'23 is looking for artifact evaluation committee members - please help our colleagues with
11/4/22
Grigori Fursin
2
10/20/22
weekly notes and new agenda for the MLCommons "education and reproducibility" conf-call
Hi all, Based on your requests, we are planning to make a demo of the CM automation workflows with
unread,
weekly notes and new agenda for the MLCommons "education and reproducibility" conf-call
Hi all, Based on your requests, we are planning to make a demo of the CM automation workflows with
10/20/22
Grigori Fursin
9/23/22
preparing a new working group in MLCommons to support collaborative and reproducible R&D
Just to let you know that we are preparing a new working group at MLCommons to automate benchmarking,
unread,
preparing a new working group in MLCommons to support collaborative and reproducible R&D
Just to let you know that we are preparing a new working group at MLCommons to automate benchmarking,
9/23/22
Grigori Fursin
9/16/22
CGO'23 is looking for volunteers to evaluate artifacts
Please help the community reproduce experimental results from CGO'23 papers - see this
unread,
CGO'23 is looking for volunteers to evaluate artifacts
Please help the community reproduce experimental results from CGO'23 papers - see this
9/16/22
Fabrice Rastello
, …
Michel Steuwer
9
1/20/22
Re: AE. Artifact available
Don't you think we can just, as a conference, decide not to attribute a given badge to any paper
unread,
Re: AE. Artifact available
Don't you think we can just, as a conference, decide not to attribute a given badge to any paper
1/20/22
Grigori Fursin
12/11/20
seeking evaluators for artifact evaluation at ASPLOS'21
Dear all, We expect a record number of artifact submissions at ASPLOS'21 and need more evaluators
unread,
seeking evaluators for artifact evaluation at ASPLOS'21
Dear all, We expect a record number of artifact submissions at ASPLOS'21 and need more evaluators
12/11/20
Grigori Fursin
11/13/20
AI is wrestling with a replication crisis
FYI: https://www.technologyreview.com/2020/11/12/1011944/artificial-intelligence-replication-crisis-
unread,
AI is wrestling with a replication crisis
FYI: https://www.technologyreview.com/2020/11/12/1011944/artificial-intelligence-replication-crisis-
11/13/20
Grigori Fursin
10/6/20
New Changes to Badging Terminology
Dear colleagues, If you are not aware yet, ACM recently updated the terminology used for its artifact
unread,
New Changes to Badging Terminology
Dear colleagues, If you are not aware yet, ACM recently updated the terminology used for its artifact
10/6/20
Grigori Fursin
2
9/8/20
ISPASS'20 slides and the new CK documentation
By the way, feel free to participate in the related discussion on Reddit: www.reddit.com/r/
unread,
ISPASS'20 slides and the new CK documentation
By the way, feel free to participate in the related discussion on Reddit: www.reddit.com/r/
9/8/20
Grigori Fursin
8/12/20
performance analysis of machine learning systems
Dear colleagues, I just wanted to draw your attention to the FastPath workshop on August 23: https://
unread,
performance analysis of machine learning systems
Dear colleagues, I just wanted to draw your attention to the FastPath workshop on August 23: https://
8/12/20
Grigori Fursin
,
Davide Del Vento
3
7/6/20
The prototyping phase of the CK technology is over!
Thank you very much, Davide, for your nice words! And thank you so much for the great discussions we
unread,
The prototyping phase of the CK technology is over!
Thank you very much, Davide, for your nice words! And thank you so much for the great discussions we
7/6/20
Grigori Fursin
5/15/20
Reproduced papers from ASPLOS'20 ad MLSys'20
Hi all, I finally found some time to add all reproduced papers from ASPLOS'20 (25th International
unread,
Reproduced papers from ASPLOS'20 ad MLSys'20
Hi all, I finally found some time to add all reproduced papers from ASPLOS'20 (25th International
5/15/20
Grigori Fursin
2/28/20
meeting at MLSys'20 in Austin next week?
Hi all, We have just completed the artifact evaluation process at MLSys'20. You can see the list
unread,
meeting at MLSys'20 in Austin next week?
Hi all, We have just completed the artifact evaluation process at MLSys'20. You can see the list
2/28/20
zoe...@gmail.com
,
Grigori Fursin
2
2/7/20
How to communicate with author(s) on artifact issues
Ups, seems like there was a typo in the reviewing guide - thanks for noting! You can communicate with
unread,
How to communicate with author(s) on artifact issues
Ups, seems like there was a typo in the reviewing guide - thanks for noting! You can communicate with
2/7/20
Grigori Fursin
2/1/20
ImageNet 2012 is no longer available - a problem for reproducibility
Dear colleagues, Since ImageNet 2012 train and validation sets are not publicly available anymore,
unread,
ImageNet 2012 is no longer available - a problem for reproducibility
Dear colleagues, Since ImageNet 2012 train and validation sets are not publicly available anymore,
2/1/20
Grigori Fursin
1/28/20
the release of the open CodeReef platform
Dear colleagues, I wish you a very happy New Year and would like to share some exciting news. We have
unread,
the release of the open CodeReef platform
Dear colleagues, I wish you a very happy New Year and would like to share some exciting news. We have
1/28/20