[1.3.0] Embedded server VS Tomcat performance test - HELP

[Ivan Viragine]

Hi there folks!

I'm stress testing an application built on Play Framework 1.3.0 and getting some strange results (or not!).

For example: when running a test with siege (https://www.joedog.org/siege-home/) I get slight better performance with Tomcat deployed app over embedded server.

From what I saw in documentation and forums, built in server should be better.

An example of how I'm testing it:

I'm using Tomcat 7, default configuration, most of the time:

Transactions:        1585 hits

Availability:      100.00 %

Elapsed time:        9.76 secs

Data transferred:        9.39 MB

Response time:        0.11 secs

Transaction rate:      162.40 trans/sec

Throughput:        0.96 MB/sec

Concurrency:       17.32

Successful transactions:        1585

Failed transactions:           0

Longest transaction:        1.42

Shortest transaction:        0.00

(it peaked 1700 hits at some runs!)

When using Play's "play run", my best was

Transactions:        1566 hits

Availability:      100.00 %

Elapsed time:        9.65 secs

Data transferred:        9.33 MB

Response time:        0.11 secs

Transaction rate:      162.28 trans/sec

Throughput:        0.97 MB/sec

Concurrency:       18.10

Successful transactions:        1566

Failed transactions:           0

Longest transaction:        0.39

Shortest transaction:        0.00

I see that difference is minimal, the CPU usage on play is almost 10% less (which is great, but, still both uses not more than 60-70% of CPU), the memory usage varies from 100% less to 50% less (which is great, but, still both uses minimum memory).

I see another difference that Tomcat's performance varies a lot from user to user, while Play maintains the request time close to the mean.

My question is: is this the expected result or am I doing something wrong? I remember a lot of people sayin' the built server was way faster...

[Scott Rippee]

Hi Ivan,

I'm curious, did you do a play precompile before running the tests?  Also are you running play in production mode (I think it needs to be in production mode when precompiling, but I'm not sure)?

[Ivan Viragine]

Hi Scott!

Yes, I'm running with "play run --%prod"!

[Ivan Viragine]

Adding an important information that I left out:

I'm using an external file with URLs simulating a real usage scenario, and I use the same file in every test, so, it shouldn't be any cache or other stuff that might be the cause (at least I think so).

[Scott Rippee]

Did you also execute play precompile before running?

[Gavin Baumanis]

Just curious - are your results different at all - if you use a more current version of Play?

(and with a caveat of : I have a "vague recollection" of something....)

Also - I am "pretty sure" that after watching the scala days presentations from 2014 or 2015 I saw mention of the fact the resources "reported" to be in use were a bit misleading.

In that what was reported to be used and the actual amount of work being carried out were not equal values, where the reported value was higher. I will try and find the presentation and add some clarity about what I am talking about.

-Gavin.

[Ivan Viragine]

@Scott:

I haven't precompiled, I thought --%prod would do this.

I'll check about it and let you know! Thanks!

[Ivan Viragine]

@Gavin;

Haven't had the chance to test on other (older) versions.

I'll take a look and let you know!

Thanks!

[Ivan Viragine]

@Scott:

Tried now, precompiling, almost the same results.