Real World program behavior

3 views
Skip to first unread message

Isaac Gouy

unread,
Jan 10, 2010, 12:15:22 PM1/10/10
to Ruby Benchmark Suite
Although this 50 page report is about JavaScript web apps, it still
seems like a nice reminder about micro-benchmarking's lack of
ecological validity -

"We measure three specific areas of JavaScript runtime behavior: 1)
functions and code; 2) heap-allocated objects and data; 3) events and
handlers. We find that the benchmarks are not representative of many
real websites and that conclusions reached from measuring the
benchmarks may be misleading.

Specific examples of such misleading conclusions include the
following: that web applications have many loops, that non-string
objects in web applications are extremely short-lived, and that web
applications handle few events."


JSMeter: Measuring JavaScript Web Applications

http://research.microsoft.com/en-us/projects/jsmeter/

Roger Pack

unread,
Jan 11, 2010, 11:42:24 AM1/11/10
to ruby-bench...@googlegroups.com
> "We measure three specific areas of JavaScript runtime behavior: 1)
> functions and code; 2) heap-allocated objects and data; 3) events and
> handlers. We find that the benchmarks are not representative of many
> real websites and that conclusions reached from measuring the
> benchmarks may be misleading.


Totally agree. Despite how cool the micro benchmarks are, the RBS
benchmarks which have most accurately reflected real world performance
are the rails and rdoc ones [which show 1.9 at 2x the speed of 1.8,
jruby somewhere between].

http://blog.pluron.com/2009/05/ruby-19-performance.html

is interesting, as well.
-r

Isaac Gouy

unread,
Jan 12, 2010, 3:25:14 PM1/12/10
to Ruby Benchmark Suite

iirc even on microbenchmarks there seemed more difference between
1.8.6 and 1.9 than between 1.8.7 and 1.9 ?

Roger Pack

unread,
Jan 12, 2010, 8:12:29 PM1/12/10
to ruby-bench...@googlegroups.com
> iirc even on microbenchmarks there seemed more difference between
> 1.8.6 and 1.9 than between 1.8.7 and 1.9 ?

For me on doze 1.8.7 was a tidge slower than 1.8.6.
I'd imagine they're pretty close with 1.8.7 a tidge faster in linux.
-r

Reply all
Reply to author
Forward
0 new messages