Techempower Round 3

[Adis]

No comment, just a link: http://www.techempower.com/benchmarks/#section=data-r3

[m e]

Needs to be better!

On Fri, Apr 19, 2013 at 8:46 AM, Adis <ad...@live.nl> wrote:

No comment, just a link: http://www.techempower.com/benchmarks/#section=data-r3

--
You received this message because you are subscribed to the Google Groups "play-framework" group.
To unsubscribe from this group and stop receiving emails from it, send an email to play-framewor...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[konstant...@gmail.com]

Just compare the results for play:

Round2    Round3

DB access

1,595        1,311            - ec2

6,870        7,472            - hard

DB access (multiple queries)

188           246               - ec2

509           2,125            - hard

JSON serialization

4,831        5,411            - ec2

26,872      26,396          - hard

I don't believe that upgrade turn out to be so slight...

[James L.]

For the interested, Round 4 planning,:

https://github.com/TechEmpower/FrameworkBenchmarks/issues/133

[Martin Grotzke]

Has anybody a clue why play java is still that slow? @huntchr?
I thought it should be quite similar to the scala version now.

Cheers,
Martin

--

[Paul C]

These benchmarks are unsettling for both Scala and Java.  I thought Play 2 had a solid and efficient foundation for core stuff.  Now I'm only hearing comments dismissing the poor performance because these tests don't represent real world scenarios.  Now I'm even wondering if the template rendering performance in round 4 will be any good.

[krispo]

So, is it the end of the fight?

[Adis]

I hope it is not the end of the PlayFlight!

But sure I hope that the devs will do whatever they can to let Play perform better and that there should be minimal difference between Scala and Java.

Also when I compare Node.js with MySql (raw!) the Play 2 is performing ok. I think we all hoped for more (especially in round 2), but it is not bad.  But when I comapre it to Wicket or Tapestry I am dissapointed that play is that 'slow'...

What do real world applications say about the performance of Play? Like The Guardian or LinkedIN?

[Bing Ran]

We all know that benchmarks cheat. But we also know that they have very strong marketing power/influence. Stats like this does much harm than what the developers have seemed to care about. 

I wish that Play team take this opportunity to do whatever it takes to optimize the framework just for better numbers in benchmarks like this. Consider it as marketing dollars well spent for Play. 

Bing

  

2013/4/23 krispo <konstant...@gmail.com>

[Brian Hauer]

Krispo, this is not the end!  We don't know how many rounds of these tests we will run, but we presently have no plans of stopping.  As for the Play tests in particular, I am confident that the performance of Play will improve and that the Java and Scala versions will become more consistent.

Bing, we are trying not to cheat, but rather show representative small workloads that are typical for many web applications.  Our fourth test (introduced in the next round) will include work with collections and server-side templates.  We have tuned the tests to-date to be small-payload so that we don't end up exercising the speed of gigabit Ethernet.  In fact, with our plain JSON serialization test and its very small payload, on i7 hardware with the fastest frameworks (over 200,000 requests per second), we do run into the gigabit Ethernet wall.  Frameworks that set fewer HTTP response headers appear marginally faster because their payload per request is several bytes smaller.  We're attempting to formalize and normalize the set of required response headers a bit for the next round to minimize this effect.

[Bing Ran]

Sorry Brian. I was not saying that the particular benchmarks cheated. I was just saying benchmarks reflects numbers in very particular circumstances.

What you guys have picked are pretty representative circumstances and I urge the Play team to find out what has stopped Play to perform comparatively well in those circumstances. 

Bing

2013/4/23 Brian Hauer <teona...@gmail.com>

[Brian Hauer]

Hi Bing,

Absolutely no need to apologize!  I understood the point you were making, and it's valid.  I just wanted to reiterate that while it's true that the benchmarks can never be as valid as testing the actual work your application needs to do, we're attempting to select workloads that are representative of the kinds of things many web applications need to do.  As long as we can keep improving them, we want to aim for an ideal of being a useful input into readers' decision-making processes, understanding that there is a lot more to consider than just performance charts.

All things considered, Play is performing quite well in this set of benchmarks.  But like you I'd love to see it get even better.  Some of the most rewarding observations we've had in coordinating the benchmarks effort is seeing several frameworks and platforms improve as a result.  We also like the increased emphasis on having clear documentation about production deployments.

In our evolution away from posting on our blog to a stand-along site, we've lost the embedded code samples, but we plan to bring those back in some form.  That is one area where I believe Play will really shine since the code is quite readable.

[Christopher Hunt]

Hi guys

We will continue to improve Play's performance. It is an important goal.

Zenexity may have found a JDK issue relating to Fork Join Pool implementations. More info to come.

Kind regards
Christopher

[Brian Hauer]

Hi Christopher,

Interesting!  I'm excited to hear where that leads.

Out of curiosity, where in the Play framework are fork-join pools being used?  Is a fork-join pool being exercised even by our simple JSON serialization test?

[Aliquip]

Personally i don't see value in these spedific tests. For example, the JSON decoding test shows nettty as one of the fastest, play at about 30% of netty performance. Still, play is build on top of netty, it just does more under the hoods than just serving static content. To make play perform near netty in theory all it'd take is slightly change the netty pipeline used, intercept requests to a certain url, and handle them directly, not moving them through all processing play does. Name it FastAction or something, one that bypasses the router, session handling, async stuff, everything.

You'd also want to change the routes syntax, add FASTPOST/FASTGET so that at compile time FastAction can be directly added to the netty pipeline.

Of course, the practical value is low, but well, yo could sure market it as a new feature. PLAY2^2, now with two request handling path, on for serving near static content at breakneck speed, one for more dynamic content.

Op vrijdag 19 april 2013 09:46:48 UTC+2 schreef Adis het volgende:

[Paul C]

As a user,  as long as the Play community can see these area's of mediocre performance as challenges to rise back to the top then I'm not so concerned.  I expect Play to perform at the top for these kinds of tests because it ditched the whole java servlets layer to become lean and mean with netty.  Spring did not ditch backward compatibility with prior versions and supports servlet api's and still performs well in these tests.

--
You received this message because you are subscribed to a topic in the Google Groups "play-framework" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/play-framework/tiXQjgwBn0A/unsubscribe?hl=en-US.
To unsubscribe from this group and all its topics, send an email to play-framewor...@googlegroups.com.

[Slim Slam]

Why is Play at the bottom of the results and it says "Did not complete"?

https://www.techempower.com/benchmarks/

J

[virtualeyes]

I filed an issue a few days ago, but alas, if the benchmark does not complete it looks like one has to wait until the next round?

Hopefully not a full year, as the gap between rounds was last year o_O

[alex s]

воскресенье, 26 апреля 2015 г., 17:09:27 UTC+3 пользователь Slim Slam написал:

Why is Play at the bottom of the results and it says "Did not complete"?

https://www.techempower.com/benchmarks/

https://github.com/TechEmpower/TFB-Round-10/search?utf8=%E2%9C%93&q=port+not+available
https://github.com/TechEmpower/TFB-Round-10/search?utf8=%E2%9C%93&q=bind

I think Play falls in the same category, but it looks like the benchmark runner doesn't capture stderr output (!), so there is only 'Oops, cannot start the server' log entry.

Note:
1. A bunch of failed attempts (dropwizard, unfiltered, etc) for some reason are absent from the techempower site, that's really unfair to the frameworks at the bottom of the results table;
2. the jetty-servlet sample was somehow 'fixed' between preview4 and preview5, but it did not receive any updates in that period.

So, there is a general stability problem with the techempower benchmark, of which the benchmark maintainers are somewhat in denial. I mean they obviously know about the issue https://github.com/TechEmpower/FrameworkBenchmarks/commit/04a6c14005d6bc501415de42cd2838f5c670c357, but that didn't stop them from releasing round 10.