Akka benchmark in akka-actor-tests

[Patrik Nordwall]

I have included parts of the akka-sample-trading benchmark into akka-actor-tests and added some useful reporting features. Charts are generated on the fly, or actually it is URL:s to Google Chart API that is included in a simple html report.

The purpose of having the benchmarks in akka-actor-tests is to facility tuning and see how changes affects performance with quick turn around. A more long term goal is also to have the tests running automatically on some stable server to be able to see trend deviations.

The generated html result look like the one attached. I hope it is self explaining what it shows, otherwise ask me. This was a comparison of akka 1.2 and 2.0 snapshots. The results are saved to disk to be able to compare the results between test runs. Up to 7 historical results are displayed in the report. If you want to have one test run as a reference point that is always included, you can rename the stored .ser files to start with the magic "baseline-" naming convention.

I run like this. I created a separate sbt launch script, which includes the following jvm parameters:

-server -Xms1024M -Xmx1024M -Xss1M -XX:MaxPermSize=256m -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled -Dbenchmark=true -Dbenchmark.minClients=1 -Dbenchmark.maxClients=40 -Dbenchmark.useDummyOrderbook=true -Dbenchmark.repeatFactor=300 -Dbenchmark.resultDir=/Users/patrik/dev/akka-benchmark -Dakka.config=/Users/patrik/dev/akka-benchmark/config/akka.conf

sbt

> project akka-actor-tests

> test-only akka.performance.trading.oneway.OneWayPerformanceTest

Never, ever, draw conclusions from runs with default settings. It is to short run period (low repeatFactor).

There is one more feature. It is possible to compare different tests with each other. Then you override compareResultWith in the test.

override def compareResultWith = Some("RspPerformanceTest")

override def compareResultWith = Some("OneWayPerformanceTest")

run both tests:

> test-only akka.performance.trading.oneway.OneWayPerformanceTest akka.performance.trading.response.RspPerformanceTest

I hope you find it useful. Happy tuning!

--

Patrik Nordwall
Typesafe - Enterprise-Grade Scala from the Experts
Twitter: @patriknw

[√iktor Ҡlang]

Very nice work Patrik!

Really useful

--
Viktor Klang

Akka Tech Lead

Typesafe - Enterprise-Grade Scala from the Experts

Twitter: @viktorklang

[Jonas Bonér]

Great job Patrik.

--
Jonas Bonér
CTO

Typesafe - Enterprise-Grade Scala from the Experts

Phone: +46 733 777 123
Twitter: @jboner

> Typesafe <http://typesafe.com/> - Enterprise-Grade Scala from the Experts
> Twitter: @patriknw