Py2.7

[Joshua Smith]

In today's blog:

We think the Python 2.7 runtime for App Engine is a great step forward for our developers.  First, it allows applications to take advantage of concurrent requests, allowing you to build more performant and efficient applications. If your application wasn't fully utilizing the CPU, chances are that you'll be able to use concurrent requests to reduce the total number of instances and serve more with less.

That doesn't at all gibe with the stats people have been posting.

From what I've read on these lists, 2.7 is slow at RPCs. But RPCs are the main way to not fully utilize the CPU. So for most apps, what you gain in concurrency, you lose in performance per thread.

Is google announcing some performance breakthrough, or is this just reality-distortion-field stuff?

-Joshua

[pdknsk]

To be honest, it seems a bit early to declare it stable. It still has
many minor bugs, and performance problems.

[Brandon Wirtz]

I’m on 2.7 it made a huge difference. Likely if your app is Data Store Heavy you will see Massive improvements. If you are computation heavy you will not.

Look at your dashboard the CPU Seconds Use/Second will tell you the ratio of Instance CPU Cycles to API CPU cycles. On 2.5  I ran at 1/10 the CPU as the API. On 2.7 I’m at closer to 1/3  

That didn’t equate to 1/3 fewer instances but it did equate to half as many.

2.7 seems to be hit harder by the random “everything is taking 3 times as long” bug.  And 2.7 instance spin up seems to be a bit slower than 2.5 instances.
Also because you can have more than one request per instance you have to be more aware of your memory usage because if you have an app that uses 50 megs of memory and it is serving 3 requests, you will see your instance killed more often for hitting the soft memory limit.

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

[Cayden Meyer]

The App Engine team have made many recent changes which have fixed bugs and improved performance. We will continue to make improvements and add new libraries as time goes on. 

We have seen many applications which have seen significant reductions in the number of instances when using concurrent requests, however concurrent requests will offer the greatest reduction in instance usage when applications are not CPU bound. 

I encourage you to try Python 2.7 with your own applications and look forward to hearing your questions, comments and suggestions for new libraries. 

Cayden Meyer

Product Manager, Google App Engine

[Brian Quinlan]

Hi,

On Tue, Feb 28, 2012 at 6:06 AM, pdknsk <pdk...@googlemail.com> wrote:
> To be honest, it seems a bit early to declare it stable. It still has
> many minor bugs, and performance problems.

Not to be flip but all software has bugs and the Python 2.7 runtime
(and the Python 2.5 runtime and the Java runtime and the Go
runtime...) are no exceptions. We aren't aware of any unfixed bugs
that would prevent someone from using the runtime but, if you
encounter a bug, please file it on the issue tracker and we will fix
it as soon as possible.

Now onto performance :-) There are definitely instances where the
Python 2.5 runtime outperforms the Python 2.7 runtime. The reverse is
also true.

It is not feasible for us to outperform the Python 2.5 runtime in
every case. To pick a trivial example, Python 2.7 supports
user-created threads and concurrent requests so some APIs now use
locks when mutating global state and copy some data structures when
previously that wasn't necessary. There is obviously some cost
associated with doing this but it is hard to avoid paying it. We think
that the benefits outweigh the costs.

Brandon mentioned a 50% reduction in billed instances when moving to
the Python 2.7 runtime and enabling threadsafe. We have seen a similar
reduction in other applications. But, as Cayden said in another
message, applications will benefit from concurrent requests to varying
degrees. This is not only an App Engine property - any Python
application modified to use threads may become more or (usually
slightly) less performant. Given the way that App Engine is normally
used, we expect concurrent requests to provide a cost benefit for most
(but not all) non-trivial-volume [1] applications.

Some of our larger customers have already migrated to reduce their
instance costs, to use features not available in Python 2.5, or to
gain access to higher-performance modules like cPickle and json. But,
if you think that the Python 2.7 runtime is too risky to use right now
due to bugs or performance problems, you can always wait a few months
and re-evaluate.

Now I'm going to reactivate my reality-distortion-field and eat some
more cake :-)

Oh, and thanks for starting the discussion!

Cheers,
Brian

[1] An application serving one request per minute doesn't have any
concurrent requests so it can't benefit.

[Brandon Wirtz]

I should also mention that Py2.7 is MUCH more exciting when you run F2 and
F4 instances. The benefits are much more profound. And if you are averaging
more than 100 billed instance hours a day you should try using F2 or F4's.

-Brandon

[Tom Willis]

A recent version of webob in the sdk would be nice. :)

http://code.google.com/p/googleappengine/issues/detail?id=6507

On Monday, February 27, 2012 6:05:40 PM UTC-5, Cayden Meyer wrote:

The App Engine team have made many recent changes which have fixed bugs and improved performance. We will continue to make improvements and add new libraries as time goes on. 

We have seen many applications which have seen significant reductions in the number of instances when using concurrent requests, however concurrent requests will offer the greatest reduction in instance usage when applications are not CPU bound. 

I encourage you to try Python 2.7 with your own applications and look forward to hearing your questions, comments and suggestions for new libraries. 

Cayden Meyer

Product Manager, Google App Engine

On 28 February 2012 06:20, Brandon Wirtz <dra...@digerat.com> wrote:

I’m on 2.7 it made a huge difference. Likely if your app is Data Store Heavy you will see Massive improvements. If you are computation heavy you will not.

Look at your dashboard the CPU Seconds Use/Second will tell you the ratio of Instance CPU Cycles to API CPU cycles. On 2.5  I ran at 1/10 the CPU as the API. On 2.7 I’m at closer to 1/3  

That didn’t equate to 1/3 fewer instances but it did equate to half as many.

2.7 seems to be hit harder by the random “everything is taking 3 times as long” bug.  And 2.7 instance spin up seems to be a bit slower than 2.5 instances.
Also because you can have more than one request per instance you have to be more aware of your memory usage because if you have an app that uses 50 megs of memory and it is serving 3 requests, you will see your instance killed more often for hitting the soft memory limit.

 

From: google-appengine@googlegroups.com [mailto:google-appengine@googlegroups.com] On Behalf Of Joshua Smith
Sent: Monday, February 27, 2012 11:57 AM
To: google-appengine@googlegroups.com
Subject: [google-appengine] Py2.7

 

In today's blog:

 

We think the Python 2.7 runtime for App Engine is a great step forward for our developers.  First, it allows applications to take advantage of concurrent requests, allowing you to build more performant and efficient applications. If your application wasn't fully utilizing the CPU, chances are that you'll be able to use concurrent requests to reduce the total number of instances and serve more with less.

That doesn't at all gibe with the stats people have been posting.

 

From what I've read on these lists, 2.7 is slow at RPCs. But RPCs are the main way to not fully utilize the CPU. So for most apps, what you gain in concurrency, you lose in performance per thread.

 

Is google announcing some performance breakthrough, or is this just reality-distortion-field stuff?

 

-Joshua

 

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.

To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to google-appengine+unsubscribe@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.

To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to google-appengine+unsubscribe@googlegroups.com.

To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to google-appengine+unsubscribe@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.

To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to google-appengine+unsubscribe@googlegroups.com.

[Anand Mistry]

We're starting to put more love into the dev_appserver, eventually fixing some of these issues. Also, we've had a self-imposed freeze on third-party library versions for the past couple of months. Now that we're GA, we can lift that freeze and start adding newer library versions. We're already adding webapp 2.5.1 and django 1.3 in 1.6.3.

[Tom Willis]

Glad to hear it. As I'm sure the webob maintainers will be also. :)

I was unaware there was a freeze myself. 

[Anand Mistry]

Now I'm going to reactivate my reality-distortion-field and eat some
more cake :-)

And here is said cake:

https://plus.sandbox.google.com/u/1/114844683991917629614/posts

[pdknsk]

What! Google has App Engine plushies!

[Brian Quinlan]

On Tue, Feb 28, 2012 at 3:02 PM, pdknsk <pdk...@googlemail.com> wrote:
> What! Google has App Engine plushies!

But of course:
http://www.googlestore.com/Fun/App+Engine+Squishable.axd

Cheers,
Brian

[Andrin von Rechenberg]

Re-Post from google-appengine-python:

Hey there

I spent yesterday & today performance testing the different Python versions.

First of all I have to say that the accuracy of my results is probably only around 90%.

https://docs.google.com/a/miumeet.com/spreadsheet/ccc?key=0AnoF3gOQDUcMdGZjbFVmUFdJRnRsMDZ2YjJZZm90R3c#gid=0

DISCLAIMER:

Data accuracy, fluctuation and suspicious data points

As stated above I think the data is about 90% accurate. I had to read it out from the

dashboard graphs (I scaled them up and interpolated the data points in photoshop). Yuck.

The results between Round 1 and Round 2 differ quite a lot for Python 2.7 with

threadsafe = true. I can not explain that. Round 1 was a test of about 1 hour

and Round 2 lasted about 12 hours. Therefor Round 2 should be more accurate.

I think that D6-D7 and H6-H7 are suspicious results that I would not use to draw

conclusions. Python 2.7 with threadsafe=true performed much better yesterday from

17:00-18:00 UTC than from 21:00-09:00 UTC. I can't explain the why.

CONCLUSIONS:

Max Idle Instances set to Auto vs very low

Never set Max-Idle instances to Auto. Auto is more than 4 times as expensive.

However, with Max-Idle Instances set to Auto, the product felt much faster,

it felt so great to use it. I think that 90% of the request had about half the

latency. But since I can only measure avg latency and not the lower 90

percentile of the requests, I can not show this in numbers. 10% of my

requests make up the majority of my avg latency.

Python 2.5 vs Python 2.7 threadsafe=false

2.5 outperforms 2.7 slightly. Cost seems to be the same (B12 vs C12)

but 2.7 has higher latency. About 15% (B13-B15 vs C13-C15).

I can say that latency of 2.7 is 30% higher for one specific handler:

Our search requests (which are a very essential part of our product).

Our search requests do the following:

- About 60 key_only=true datastore queries of which about 52% are

  memcached.

- Get about 500 entities by key_name from datastore of which 50%

  are memcached. Fetch 25 entities per RPC to datastore.

All in all its about 100 RPCs for one search request.

Latency to do this in Python 2.5 is about 3000ms (B15) and in

Python 2.7 about 3900ms (C15).

Doing a couple of calculations I was able to see that the increased

overall latency (B14 vs C14) really mostly come from my search requests.

All my other requests do about 5 RPCs. Only my search requests

do 100 RPCs. So it seems to me that Python 2.5 and Python 2.7

perform exactly at the same level except for RPCs, where 2.7 is

noticeably slower.

Python 2.7 threadsafe false vs true

It is very obvious that with threadsafe=true latency will go up.

If one instance handles only one request it does it as fast as it can.

But if one instance handles two requests and both would be

ready to be processed (RPC result is available), one thread has

to wait until the instance finished handling the other thread.

When setting threadsafe=true, the overall latency of my RPC

heavy search requests went through the roof! See C15 vs D15.

Compared to Python 2.5 the search requests took more than

double the amount of time. (This is a deal breaker for our use-case.

One option would be to deploy a threadsafe=false version that handles

our search requests and for all the other stuff we would use

threadsafe=true. It would be nice to have this functionality built into

AppEngine)

Another thing that I would expect is that with threadsafe=true

the amount of instances needed should be lower. This was

clearly the case in Round 1, see C4 vs D4. However, in Round2

with lower system load, the amount of instances needed were

the same, see C11 vs D11.

Our wishlist for Google

- Improve Python2.7 RPC perfomance.

- Being able to specify threadsafe=true|false per handler.

- Traffic splitting will help me a lot in the future to do these kind of tests.

  You will have to fix this bug first though (everyone please star):

  http://code.google.com/p/googleappengine/issues/detail?id=6726

- Better graphs in the dashboard: Please use the latest version of the

  Google chart API which allows one to however over charts and read

  out data points.

Final note

If you have read this email down to here, you owe me a case

of beers. Cheers,

-Andrin

[Johan Euphrosine]

FYI Anan commented on the google-appengine-python thread.

https://groups.google.com/d/msg/google-appengine-python/BZV4I11qP0g/ekifteZqEdUJ

Johan Euphrosine (proppy)
Developer Programs Engineer
Google Developer Relations