Performance Statistics (and Graphs!)
Matt Diephouse
an automated build system for parrot that tracks performance (with
help from Dan's box). We have collected benchmark data for regular and
optimized builds with and without JIT from June 1st through October.
With some help from Perl and GD, we've up several pages of graphs:
http://www.sidhe.org/~timeparrot/graphs/
Both the build process and the creation of the graphs will (hopefully)
be run daily (automated). There are plans to do some other things with
the data as well (some notification scripts).
If anyone would like to use this data or the scripts to do something
interesting (and hopefully useful too), contact either me or Joshua
and we'll get it to you.
--
matt diephouse
http;//matt.diephouse.com
Jeff Clites
> Joshua Gatcomb and I have been working a little under a week to set up
> an automated build system for parrot that tracks performance (with
> help from Dan's box). We have collected benchmark data for regular and
> optimized builds with and without JIT from June 1st through October.
>
> With some help from Perl and GD, we've up several pages of graphs:
>
> http://www.sidhe.org/~timeparrot/graphs/
>
> Both the build process and the creation of the graphs will (hopefully)
> be run daily (automated). There are plans to do some other things with
> the data as well (some notification scripts).
Very cool. A useful thing to do would be to preserve all of the builds,
so that newly-created benchmarks could be back-filled. (That way, if we
realize that we've not been effectively testing something, we can
devise an appropriate benchmark, and evaluate the consequences of past
design changes against it.)
JEff
Leopold Toetsch
> Joshua Gatcomb and I have been working a little under a week to set up
> an automated build system for parrot that tracks performance (with
> help from Dan's box). We have collected benchmark data for regular and
> optimized builds with and without JIT from June 1st through October.
> With some help from Perl and GD, we've up several pages of graphs:
> http://www.sidhe.org/~timeparrot/graphs/
Great. That's really helpful.
Some remarks WRT various {in,de}creases:
* ~2 weeks in Oct including the release, JIT was accidentally disabled
(best seen in primes*_i)
* ~ Sep 22th the broken return continuation recycling was abandoned
(fib)
* end of Oct the indirect register addressing was working (fib)
* the increase in e.g. fib in the beginning of Sept could be related
to disabling ARENA_DOD_FLAGS during incremental GC implementation
Thanks a lot,
leo
Dan Sugalski
That's doable through the joys of timestamped CVS checkouts. (Which
is how the graphs were built in the first place) Granted, it hammers
the build machine, but... :)
--
Dan
--------------------------------------it's like this-------------------
Dan Sugalski even samurai
d...@sidhe.org have teddy bears and even
teddy bears get drunk
Joshua Gatcomb
thinking more along the lines of people wanting to
check other run-time cores (such as computed goto).
To save on disk space, the executables are stripped
and bzip2'd, but we can go back easily to any previous
point in the 5 month window if we want.
Cheers
Joshua Gatcomb
a.k.a. Limbic~Region
__________________________________
Do you Yahoo!?
Check out the new Yahoo! Front Page.
www.yahoo.com
Jerome Quelin
> We have collected benchmark data for regular and
> optimized builds with and without JIT from June 1st through October.
What about comparing against perl*, python and ruby?
Nice work,
Jerome
--
jqu...@mongueurs.net
Matt Diephouse
> What about comparing against perl*, python and ruby?
What about it? Many of the benchmarks are parrot only: the gc tests,
for example. The others should remain mostly static, unless we do
daily checkouts, which is a lot of work, even for parrot. Presumably
you can just run the perl, python, and ruby scripts yourself and
compare. Unless you're asking for them to be on the graphs?
> Nice work,
Thanks.
--
matt