Poor performance on GAE

[Ted G]

I'm wondering if anyone has tips on where I should start in trying to
track down performance problems running my web2py application on GAE.
Initially I created an application using only the api's supplied by
GAE. This app was very fast in terms of response times.

I've since recreated and updated the app within web2py and have been
frustrated with very poor performance. I experience long delays when
loading pages and in particular, when loading pages that contain small
images (where the image source is using web2py's download() handler to
retrieve an image stored in the db as a web2py upload field).

As mentioned, my previous GAE API version of the app also stored these
same images as blobs in the db, but experienced none of the
performance issues I see under web2py.

Due to the delay experienced loading pages, my first thought, after
reading threads in this group, was that maybe my code was being
recompiled on every request. I updated to 1.61 this morning in the
hope that the GAE cache issue it was addressing would solve my problem
- but running a side to side comparison between a 1.59 version and
1.61 on GAE I did not see any difference.

Any tips on how I can narrow down the source of the performance
problems would be greatly appreciated.

[mdipierro]

I do not know how to profile on GAE.

There are some things to consider:
0) make sure you use the provided gaehandler.py
1) you get the benefit of chaching bytecode compiler models and
actions only if the page if accessed often
2) you will get a speed up if you use memcache session instead of
datastore based sessions (the default)
3) Use

if request.function in ['one','two','three']:....

to conditionally execute code in models so that you do not define
tables that your actions do not need
4) Avoid too many {{ }} in templates.
5) Use memcache as much as you can.

Massimo

[Robin B]

Some ideas:

There is a log_stats decorator in gaehandler.py that can be
uncommented and enabled in production to give a more precise timing
profile than the log coarse timing.

The main caching mechanism is in compileapp.py and the compiled code
should be cached in a dict called 'cfs' which can be printed for
debugging.

If the caching mechanism is working, and the responses are not fast
enough, then memcach'ing the responses could be used to reduce
database accesses.

Robin

On Apr 14, 2:38 pm, Ted G <tedg...@gmail.com> wrote:

[mdipierro]

Another trick is put as much code you can in a module (in yourapp/
modules/) as and just call that from your models and controllers.

Massimo

[Ted G]

Thank you for the tips. I'll work through these and determine if I can
find a specific root cause of the performance problem.

I've been testing on a particular page of the application that lists a
number of items along with thumbnail images for those items. The
thumbnails are images placed into an upload field of the item record
and then displayed within the page using the following as the image
src:

{{ =URL(r=request,f='download',args=[channel.image]) }}

The page itself takes a while to initially respond, and then each
thumbnail in turn takes a while to download.

As a test, I replaced the above image src with the following:

/img/channel?id={{ =channel.id }}

and used App.yaml to direct the /img to a request handler using GAE's
native wsgihandler where I perform the following to download the
thumbnail image instead of using the web2py download():

record = channel.get_by_id(int(self.request.get
('id')))
self.response.headers['Content-Type'] = contenttype
(record.image)
self.response.out.write(record.image_data)

While the delay before the page first loads has not changed, the
subsequent display of the thumbnails on the page is now very fast
(which are the only requests now not being handled by web2py).

Not sure what to make of this at this point, but I wanted to first
ensure there was not a GAE or DB issue before digging deeper into what
part of my web2py implementation is slowing things down.

I have not modified the gaehandler.py and as a first step did move
sessions to ram cache instead of db, but have not seen any noticeable
performance increase.

[mdipierro]

Whatever you discover please let us know. It will help us improve.

Massimo

[Robin B]

I am also not happy with the CGI performance of GAE.

Some interesting numbers:

welcome/default/index:

cold request time: ~1500ms
warm request time: ~150ms (about 1/10 time of a cold request)

a warm process is reaped in just *60 sec* if there is no activity.

If you have a low activity site, where requests are less frequent than
1/minute then every request will be a cold request (~1.5 sec/request
not including datastore access).

In other words, on low activity sites, byte code caching does not
help, since everything is recompiled on every request...

I wonder if cron can be used to maintain a warm process. The min cron
interval is 1 minute, so it would be possible to hit a dummy handler
with cron that does nothing just to keep a warm process.

Robin

[Markus Gritsch]

Hi,

has anyone tried using web2py on Java-GAE using Jython? Does it
work? Do the request times get improved?

Kind regards,
Markus

On Apr 15, 3:19 pm, Robin B <robi...@gmail.com> wrote:
> I am also not happy with the CGI performance ofGAE.
>
> Some interesting numbers:
>
> welcome/default/index:
>
> cold request time: ~1500ms
> warm request time: ~150ms  (about 1/10 time of a cold request)
>
> a warm process is reaped in just *60 sec* if there is no activity.
>
> If you have a low activity site, where requests are less frequent than
> 1/minute then every request will be a cold request (~1.5 sec/request
> not including datastore access).
>
> In other words, on low activity sites, byte code caching does not
> help, since everything is recompiled on every request...
>

> I wonder ifcroncan be used to maintain a warm process.  The mincron

> interval is 1 minute, so it would be possible to hit a dummy handler

> withcronthat does nothing just to keep a warm process.

>
> Robin
>
> On Apr 14, 9:23 pm, mdipierro <mdipie...@cs.depaul.edu> wrote:
>
> > Whatever you discover please let us know. It will help us improve.
>
> > Massimo
>
> > On Apr 14, 8:26 pm, Ted G <tedg...@gmail.com> wrote:
>
> > > Thank you for the tips. I'll work through these and determine if I can
> > > find a specific root cause of the performance problem.
>
> > > I've been testing on a particular page of the application that lists a
> > > number of items along with thumbnail images for those items. The
> > > thumbnails are images placed into an upload field of the item record
> > > and then displayed within the page using the following as the image
> > > src:
>
> > > {{ =URL(r=request,f='download',args=[channel.image]) }}
>
> > > The page itself takes a while to initially respond, and then each
> > > thumbnail in turn takes a while to download.
>
> > > As a test, I replaced the above image src with the following:
>
> > > /img/channel?id={{ =channel.id }}
>

> > > and used App.yamlto direct the /img to a request handler usingGAE's

> > > native wsgihandler where I perform the following to download the
> > > thumbnail image instead of using the web2py download():
>
> > >             record = channel.get_by_id(int(self.request.get
> > > ('id')))
> > >             self.response.headers['Content-Type'] = contenttype
> > > (record.image)
> > >             self.response.out.write(record.image_data)
>
> > > While the delay before the page first loads has not changed, the
> > > subsequent display of the thumbnails on the page is now very fast
> > > (which are the only requests now not being handled by web2py).
>
> > > Not sure what to make of this at this point, but I wanted to first

> > > ensure there was not aGAEor DB issue before digging deeper into what