HWMCC 2011 Follow-up Analysis

[Jason Baumgartner]

Hi Armin. I spoke today with the Berkeley ABC team, and they somehow
were unaware of your Google group about the HWMCC. I forwarded the
link for them to subscribe.

We and they definitely would be interested in seeing some post-HWMCC
discussion, particularly on benchmarks that were uniquely solved by
one tool and on those that were unsolved in the competition. Eg, it
would be valuable to hear details of the uniquely-solving algorithm;
if someone after the fact solves an unsolved one that would be
interesting; others not participating in the competition (e.g. my
team) could post information on how they solved some of these.

I'm not sure the best format for this. I would think maybe something
like a 2D table with 1 row per benchmark, summarizing per-tool data as
columns and with a comment/discussion type link for the more
interesting subset. I think that merely posting emails may not be as
good in that we would want discussion organized per benchmark - and it
would be nice to see this per-benchmark discussion organized in a
meaningful way. What do you think about the best way to manage this?

Thanks

[Armin Biere]

There is a lot of information on the results page.
The 'joined' tables might be a good starting point:

http://fmv.jku.at/hwmcc11/single-details-joined.xls

I am not sure it is necessary to set up links. Why
can't we just start discussing individual benchmarks
(or benchmark sets) as new threads?

Of course if you can share some experience with
IBM tools on these benchmarks that would be
great. We could add addtional spread sheets or
merge this info with existing ones.

Armin