Polyhedra benchmarks

[David Edelsohn]

By the way, the IBM XL team are using what they say is a collection of
benchmarks that the Polyhedra community normally uses:

jac1d
jac2d
tmm
corcol
covcol
strmm
dsyrk
dsyr2k
ssymm
strsm
trisolv
seidel
sor

They have seen good performance improvements using the Polyhedral
model for loop tiling.

Also, they are enhancing support for conservative analysis of
"blackbox" blocks of code as well as conservative analysis of
irregular array references, e.g., A[B[k]] .

David

[Tobias Grosser]

On Tue, 2009-06-30 at 12:41 -0400, David Edelsohn wrote:
> By the way, the IBM XL team are using what they say is a collection of
> benchmarks that the Polyhedra community normally uses:
>
> jac1d
> jac2d
> tmm
> corcol
> covcol
> strmm
> dsyrk
> dsyr2k
> ssymm
> strsm
> trisolv
> seidel
> sor

These are new to me. However it would be nice to test Graphite on them.
Does anybody have a link where to get them?

At the moment we run test on the polyhedral benchmark suite:

http://www.polyhedron.com/650573512125810

However as I have not seen any failures since a while maybe the tester
does not work.

> They have seen good performance improvements using the Polyhedral
> model for loop tiling.

> Also, they are enhancing support for conservative analysis of
> "blackbox" blocks of code as well as conservative analysis of
> irregular array references, e.g., A[B[k]] .

It should not be too hard to allow this kind of array references. And it
would allow our code to be even better tested. It would be nice to have
it sooner than later. ;-)

Tobi

[David Edelsohn]

I have uploaded the Polyhedral benchmark kernels to the Google Group files area.

The benchmarks are written in C. There is no Makefile or driver
program or input data. Using matrices of 2K x 2K or 4K x 4K was
recommended so that the benchmark would run long enough for
statistically meaningful results.

The tar file also includes PolyScript options for the benchmarks.
PolyScript is the language that the XL team has invented for
specifying Polyhedral transformations and parameters. The scripts
list the optimal transformations that the IBM Research team found
through empirical testing. The loop blocking is based on their neural
network feedback.

David

[Albert Cohen]

Thank you David. It will be extremely useful in the coming weeks when we
move forward more performance evaluation results. Especially if we fail
to get good numbers on SPEC (for very good reasons), we could at least
demonstrate Graphite's impact on those benchmarks.

Albert

[Sebastian Pop]

Thanks David for these benchmarks.

I wonder if we could put these in a public git and continue adding
the missing Makefiles and scripts around to automatize their tests.
Do you see something against putting these benchmarks on
http://repo.or.cz ?

The plan is to include these tests in the daily tests of the graphite
patches on the gcc farm.

Thanks again,
Sebastian

[David Edelsohn]

On Fri, Jul 3, 2009 at 3:15 AM, Sebastian Pop<seb...@gmail.com> wrote:
>
> Thanks David for these benchmarks.
>
> I wonder if we could put these in a public git and continue adding
> the missing Makefiles and scripts around to automatize their tests.
> Do you see something against putting these benchmarks on
> http://repo.or.cz ?
>
> The plan is to include these tests in the daily tests of the graphite
> patches on the gcc farm.

Adding the individual C source file benchmarks should be fine. I
would appreciate if the PolyScript files were omitted and not
installing the tar file itself.

Thanks, David

[David Edelsohn]

I have uploaded a newer version of the benchmarks to the Google Groups
file area as polybench2.tgz. This also includes corcol and covcol.

These versions apparently are easier to test and time. Apparently one
must notice and define the SELFTEST macro to prevent compilers from
removing the entire function as unused.

David