Let me add to this discussion the links I provided in the Zoom chat:
which was the result of a similar but longer discussion at the POS'20 workshop:
which actually was captured on Video
This also contains some discussion why we came at the end to finally start something
like that (we were really kind of starting to feel bad about how evaluation are done in SAT'16).
The idea to organize this through GITHUB (from Daniel) is nice.
P.S. The SAT manifesto is the result of the lightning talk of Kuldeep. I talked in that session on CDF vs. Cactus and
blamed Laurent Simon to have invented the Cacuts plot, but then during the talk had to learn that a CASC paper
with Geoff provided the original idea.