Akka http vs Spray performance

2,163 views
Skip to first unread message

zergood

unread,
May 27, 2015, 7:44:49 AM5/27/15
to akka...@googlegroups.com
I've done little benchmarks to compare spray and akka-http performance. I use default jvm and akka settings. So you can see that there is an significant performance difference between it.

code:

versions:
spray 1.3.3 
akka-http 1.0-RC3
scala 2.11.6
java 1.8

wrk output for spray:
  30 threads and 64 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     2.14ms    9.82ms  78.22ms   98.22%
    Req/Sec     2.55k   609.68     4.22k    78.12%
  4322357 requests in 1.00m, 614.20MB read
Requests/sec:  72044.97
Transfer/sec:     10.24MB

wrk output for akka-http:
  30 threads and 64 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     5.39ms    6.82ms 108.07ms   92.80%
    Req/Sec   454.43    126.73   679.00     77.77%
  811836 requests in 1.00m, 115.36MB read
Requests/sec:  13531.62
Transfer/sec:      1.92MB

Is there any akka-http config options to increase performance to the same level as spray?

Endre Varga

unread,
May 27, 2015, 7:49:10 AM5/27/15
to akka...@googlegroups.com
Hi,

At the current point the aim of streams 1.0 was to reach a desired functionality level. Basically there is zero performance work done at this point. We will improve performance in later versions though. 

We basically refactored streams API in roughly every second release, so we are currently happy that the current API looks usable and most of the bugs are ironed out. 

-Endre

--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+...@googlegroups.com.
To post to this group, send email to akka...@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

zergood

unread,
May 27, 2015, 7:53:12 AM5/27/15
to akka...@googlegroups.com
Ok. I understand. Thank you for your work, good luck.

среда, 27 мая 2015 г., 14:49:10 UTC+3 пользователь drewhk написал:

Giovanni Alberto Caporaletti

unread,
May 27, 2015, 4:45:19 PM5/27/15
to akka...@googlegroups.com
I'm already using http/streams in a minor component, but since you're talking about this: do you have a rough idea of when the performance improvement work will start? Of course I'm not talking about the exact day but it would be very useful to know if we can expect something by the end of the year, more or less. I mean, if I had a 75% chance to have 50% the performance of spray by the end of the year, I'd start using akka-http right away for my project... it's an investment ;)

thanks
G

Endre Varga

unread,
May 28, 2015, 5:56:10 AM5/28/15
to akka...@googlegroups.com
On Wed, May 27, 2015 at 10:45 PM, Giovanni Alberto Caporaletti <para...@gmail.com> wrote:
I'm already using http/streams in a minor component, but since you're talking about this: do you have a rough idea of when the performance improvement work will start? Of course I'm not talking about the exact day but it would be very useful to know if we can expect something by the end of the year, more or less. I mean, if I had a 75% chance to have 50% the performance of spray by the end of the year, I'd start using akka-http right away for my project... it's an investment ;)

I think it should start soon, since we are mostly happy with the API. Also, since M5 the DSL layer, the representation layer and the implementation layer are highly decoupled, so we can much more easily hack around with the internals than before.

I think two main architectural areas are to be covered:
 - Dynamic streams usage: the current DSL is very rigid as in the layout of the streams are mostly static after being materialized. We need to solve this in a way that is not a collection of ad-hoc hacks and has similar safety properties than the most common stream stages. 
 - Graph interpreter support: this will allow us to expose much richer APIs than the current FlexiMerge/FlexiRoute and unify it with the PushPullStage infrastructure. It will allso allow users to demarcate regions of a graph they want to map to an actor. I.e. you will be able to execute a complex graph/flow inside one actor instead one actor for each stage. I believe this will also improve performance.

Performance optimizations should not interfere with the above goals, therefore we are not jumping right into them, but we are very eager!

-Endre

Giovanni Alberto Caporaletti

unread,
May 28, 2015, 11:11:41 AM5/28/15
to akka...@googlegroups.com
Nice, thank you Endre.

I guess knowing that the project will stay under active development after 1.0.0 is enough to choose it over spray, once the api is (roughly) stable with 1.0.0. 
I think that for most people it's important to know that the project will be supported and the current limitations will be gradually removed. I personally see less "danger" in using akka stream/http as experimental modules compared to something like persistence (back then) because -correct me if I'm wrong- it's less likely that something unrecoverable that completely destroys application data or requires a huge migration could happen, i.e. experimenting with data is riskier.
As I said I've been using http/stream in a small component here at my startup and I am pretty happy with the results. Some workarounds because of missing features are ok, as long as the job is done. I already "migrated" some of those workarounds to production code during the MX-RCX phases (e.g. I had written my own custom "SynchronousFileSource" before using the current one). Using an incomplete/changing api can be worked around, slow (1 order of magnitude) internals cannot, that's why it's very important to know that performance improvement is a priority. 

If the optimization work is supposed to show its first results in a matter of months (not year+), I could choose to use the new api in a more important component because I like it (a lot) and see a lot of potential for future extensions.


Cheers 
G

Jakub Kahovec

unread,
Oct 8, 2015, 3:59:31 PM10/8/15
to Akka User List
Hi,

first thank you so much for making such a great piece of software. As the 2.4.0 is out, have you made any progress on improving performance of akka http or at least is there any roadmap for this ?

Thanks

Jakub

Roland Kuhn

unread,
Oct 8, 2015, 4:11:27 PM10/8/15
to akka-user
Hi Jakub,

we are currently working on the performance improvements, but there is nothing to see yet since the first (and biggest) step requires the internal framework that is necessary for internal optimizations (operation fusion). We are about to finish the preparations and will hopefully have good news in a small number of weeks.

Regards,

Roland

--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+...@googlegroups.com.
To post to this group, send email to akka...@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.



Dr. Roland Kuhn
Akka Tech Lead
Typesafe – Reactive apps on the JVM.
twitter: @rolandkuhn

Kahovec Jakub

unread,
Oct 8, 2015, 4:21:34 PM10/8/15
to akka...@googlegroups.com
That’s a good news, looking  forward to see more ;-)

Thank you Roland

Jakub

You received this message because you are subscribed to a topic in the Google Groups "Akka User List" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/akka-user/XB7H2n2r-9A/unsubscribe.
To unsubscribe from this group and all its topics, send an email to akka-user+...@googlegroups.com.

Владимир Морозов

unread,
Feb 18, 2016, 2:52:10 PM2/18/16
to Akka User List
I take server’s code from first message and run both benchmarks  on Akka-http 2.4.2

My results, akka-http serve more connections than spray 1.3.3. Each server started in Run mode from 'intellij idea 15 CE' with root logging level INFO. I run WRK command thee times without server shutdown (because we use JWM and have JIT)

  30 threads and 64 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    20.63ms   99.19ms   1.18s    96.17%
    Req/Sec     1.49k   770.37     2.95k    61.72%
  420300 requests in 10.03s, 59.72MB read
Requests/sec:  41886.83
Transfer/sec:      5.95MB

  30 threads and 64 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    11.67ms   62.36ms 910.17ms   97.23%
    Req/Sec     2.13k   618.76     6.12k    77.79%
  619516 requests in 10.07s, 88.03MB read
Requests/sec:  61519.04
Transfer/sec:      8.74MB

  30 threads and 64 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     7.20ms   30.98ms 529.91ms   95.50%
    Req/Sec     2.07k   583.44     7.44k    71.47%
  613762 requests in 10.06s, 87.21MB read
Requests/sec:  61006.76
Transfer/sec:      8.67MB

  30 threads and 64 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     8.00ms   20.62ms 168.02ms   92.19%
    Req/Sec     1.58k   745.87     3.05k    67.38%
  465741 requests in 10.08s, 66.18MB read
Requests/sec:  46201.19
Transfer/sec:      6.57MB

  30 threads and 64 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     5.41ms   12.92ms 147.92ms   90.83%
    Req/Sec     1.74k   600.65     7.59k    67.28%
  520192 requests in 10.08s, 73.92MB read
Requests/sec:  51620.57
Transfer/sec:      7.34MB

  30 threads and 64 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     5.82ms   13.55ms 150.83ms   91.05%
    Req/Sec     1.65k   617.64     4.33k    69.76%
  497039 requests in 10.10s, 70.63MB read
Requests/sec:  49220.11
Transfer/sec:      6.99MB

четверг, 8 октября 2015 г., 23:21:34 UTC+3 пользователь Jakub Kahovec написал:

Roland Kuhn

unread,
Feb 18, 2016, 3:11:27 PM2/18/16
to akka-user
Wow, this is quite interesting. I take it that your machine is a bit slower than the original poster’s machine, because the Spray results are ca. 30% lower.

Thanks for sharing!

Regards,

Roland


For more options, visit https://groups.google.com/d/optout.
Message has been deleted

Владимир Морозов

unread,
Aug 4, 2016, 5:55:41 AM8/4/16
to Akka User List
I perform some test with Akka 2.4.8


  30 threads and 64 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    19.19ms   86.05ms   1.20s    96.02%
    Req/Sec     1.53k   716.29     4.49k    60.40%
  435786 requests in 10.08s, 61.92MB read
Requests/sec:  43221.21
Transfer/sec:      6.14MB
  30 threads and 64 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    20.33ms   95.59ms   1.13s    96.66%
    Req/Sec     1.95k   723.17     8.15k    75.32%
  557516 requests in 10.06s, 79.22MB read
Requests/sec:  55426.85
Transfer/sec:      7.88MB
  30 threads and 64 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     8.83ms   37.21ms 614.55ms   96.12%
    Req/Sec     1.76k   647.83     5.66k    71.19%
  519199 requests in 10.07s, 73.78MB read
Requests/sec:  51580.71
Transfer/sec:      7.33MB

And then with: Akka 2.4.9-RC1

  30 threads and 64 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    10.29ms   42.81ms 685.72ms   95.91%
    Req/Sec     1.65k   721.12     5.46k    61.92%
  478914 requests in 10.07s, 69.88MB read
Requests/sec:  47574.64
Transfer/sec:      6.94MB

  30 threads and 64 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     8.63ms   35.87ms 556.24ms   96.45%
    Req/Sec     1.81k   663.09     6.11k    70.47%
  534884 requests in 10.07s, 78.05MB read
Requests/sec:  53123.50
Transfer/sec:      7.75MB

  30 threads and 64 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     4.56ms   11.40ms 205.91ms   90.97%
    Req/Sec     1.91k   556.78     5.98k    62.47%
  569945 requests in 10.06s, 83.16MB read
Requests/sec:  56663.71
Transfer/sec:      8.27MB

четверг, 18 февраля 2016 г., 23:11:27 UTC+3 пользователь rkuhn написал:

Konrad 'ktoso' Malawski

unread,
Sep 2, 2016, 4:19:03 AM9/2/16
to Akka User List
Figured I should link to the 2.4.9 annoucement from this old thread (as it shows up high in google).
In 2.4.9 we did quite some improvements, annoucement here:

Highlights of the performance improvements include:
  • Overall Akka HTTP throughput and transfer rate has been improved by 30-40%
  • Performance is on-par or better than Spray.
    • Matching it both in raw throughput as well as latency distribution across a spectrum of request rates.
    • We measured an overall improvement of ~14% over Spray
  • Short lived connections throughput, which remains to be the worst-case scenario for Akka HTTP thought remains rare in the real world (due to connection pooling), has been doubled.
  • Given our EC2 infrastructure (m4.2xlarge instances) the server currently reaches a maximum of ~160.000 “ping / pong” requests per second (measured using 100 concurrent connections).
Happy hakking!

-- Konrad

Владимир Морозов

unread,
Sep 2, 2016, 4:38:05 AM9/2/16
to Akka User List
I perform some test for 2.4.9 release

  30 threads and 64 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    10.00ms   37.94ms 640.12ms   94.49%
    Req/Sec     1.66k   738.92     4.73k    63.21%
  487404 requests in 10.07s, 69.26MB read
Requests/sec:  48405.49
Transfer/sec:      6.88MB

  30 threads and 64 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     6.58ms   18.10ms 212.50ms   92.59%
    Req/Sec     1.94k   672.59     7.34k    69.63%
  577713 requests in 10.08s, 82.09MB read
Requests/sec:  57308.72
Transfer/sec:      8.14MB
  30 threads and 64 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     7.87ms   28.93ms 456.46ms   95.19%
    Req/Sec     1.89k   639.70     4.98k    74.75%
  561358 requests in 10.09s, 79.77MB read
Requests/sec:  55630.70
Transfer/sec:      7.90MB

пятница, 2 сентября 2016 г., 11:19:03 UTC+3 пользователь Konrad 'ktoso' Malawski написал:

Olga Gorun

unread,
Oct 5, 2016, 12:56:34 AM10/5/16
to Akka User List

I tried to repeat this benchmark and would be glad to see comments to results I got.  
Tested porjects: gists from original post with the following versions: 

akka-http, 2.4.11 vs spray, 1.3.1 (akka - 2.3.6). In both cases scala 2.11.8, jvm 8.

The tests were done on AWS machine of c4.2xlarge type (8 cores CPU, 16GB memory, ulimit - 500000). As I can see from jvm and system monitoring, it is far from being limited by CPU, memory, disk IO or networking. OS: Ubuntu 14.04


Starting from the fact that my results are different (spray gives better results than akka-http). In addition I see interesting effects when I use different benchmark tools. I also started from wrk, but in addition to throughput I wanted to see failures if exist, and latency percentiles, so I added  tests with weighttp and ab. 

Throughput reported by ab is much different than the rest and shows drastic difference between akka-http and spray. Reported latency also don't have much in common. I can think that ab (as a single-threaded tool) can be bottleneck itself. But how to explain such a difference in throughput for akka-htttp and spray? And how to explain difference in reported latency.

Regards,
Olga Gorun



   

Roland Kuhn

unread,
Oct 5, 2016, 1:47:54 AM10/5/16
to akka...@googlegroups.com
Hi Olga,

establishing a connection is a very expensive operation, which is why all modern clients/browsers reuse them for multiple requests. The ab tool needs the -k switch to enable this behavior.

Regards, Roland 

Sent from my iPhone

Olga Gorun

unread,
Oct 5, 2016, 11:18:20 AM10/5/16
to Akka User List
Hi rkurn,

Thank you for reply. You are right. keep-alive makes the difference. I updated in google spreadshit.

Olga Gorun

unread,
Oct 6, 2016, 12:08:57 PM10/6/16
to Akka User List
I continue playing with given example projects and see strange results. If I increase string length of static response from 7 to 2490 characters throughput is reduced to ~24K rps for both spray and akka-http. 
Does anybody else see such results? How can they be explained? 



On Wednesday, October 5, 2016 at 8:47:54 AM UTC+3, rkuhn wrote:

Christian Schmitt

unread,
Oct 6, 2016, 3:03:47 PM10/6/16
to Akka User List
garbadge collection I guess. also I didn't see if you've set Xms/Xmx you should set them equal. And also try the following: -XX:+UseNUMA -XX:+UseG1GC -XX:+AlwaysPreTouch -XX:+PerfDisableSharedMem -XX:+ParallelRefProcEnabled paramters, not all of them will be helpful, but you should try some of them.

Olga Gorun

unread,
Oct 6, 2016, 3:26:48 PM10/6/16
to Akka User List
Hi Christian. Thank you for response. I checked that equal Xms/Xmx both set to more than enough memory (4GB while used heap is not more than 1.5GB during load test with or without  explicitly defined memory) don't influence results. I'll try other JVM settings you propose. But generally speaking I don't understand why I should see something different in GC behavior. String used as response defined as val in object. It's not created in some class instantiated per request. So GC should not clear it. When GC should make more work if response length is increased?

Olga Gorun

unread,
Oct 6, 2016, 3:42:04 PM10/6/16
to Akka User List
For more information, I give here versions I test with.

ubuntu 14.04, scala 2.11.8, oracle jdk 1.8

akka-http: 2.4.11

spray: 1.3.1 with akka 2.3.6

Olga Gorun

unread,
Oct 8, 2016, 3:35:38 PM10/8/16
to Akka User List
I added scalatra (scalatra 2.4.1 on embedded jetty) to tests set.  Two testing tools from three (ab and weighttp) give me approximately the same results for all three frameworks in ~70MB/s throughput when I test long static result. So  it is similar to network bandwidth saturation. On the other hand in case of scalatra wrk is able to get to 86MB/s and iperf from client to server with default parameters can give me ~88MB/s.

Can it be networking problem? How do I check it?

Reply all
Reply to author
Forward
0 new messages