Long Response times?

[Magnus O.]

Hi,

I just started developing with python and google app engine and my
first project is to build a small easy proxy server to enable cross
domain xml requests from the client. I'm experiencing logn response
times in all python handlers. Static files as being served fast but
python handlers are loading slow 800ms - 1.5 seconds. For instance,
here is my default page that (for now) just loads an index.html static
file as template:

import cgi
import urllib
import os
import wsgiref.handlers
from google.appengine.ext import webapp
from google.appengine.api import urlfetch
from google.appengine.ext.webapp import template

class MainController(webapp.RequestHandler):

def get(self):

template_values = {}

path = os.path.join(os.path.dirname(__file__), 'index.html')
self.response.out.write(template.render(path, template_values))

def main():
application = webapp.WSGIApplication([('/',
MainController)],debug=True)
wsgiref.handlers.CGIHandler().run(application)

if __name__ == "__main__":
main()




Anyone that can tell me why the load times are so slow? You can see it
in action here: http://blirdetsol.appspot.com/

[Ikai L (Google)]

What part of this application is slow? Is it just slow in development mode? The link you sent out seems to load reasonably fast.

What may be happening is that your application may be cycling out. We do this to dynamically adjust the number of instances to match the level of load your application is experiencing. If your application is cycled out, when a request comes in, we cycle it back in, but to do this, we need to load your environment and fire up an application instance.

--

You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=.

--
Ikai Lan
Developer Programs Engineer, Google App Engine

[Magnus O.]

Hi!,

I feel it's slow in both the development environment and live. I have
run some profiling on the site with firebug and I can see that the
"waiting for response" part on all request to script handlers is about
800ms. This seems like a very long time. The site I have developed is
a weather service and I fetcht the weather data using ajax requests. I
takes a few requests to load all the weather.

In a test it took 24 requests to load all the weatherdata and the load
time for this was 13.24 seconds. This is a long time to fetch these
request. About 99% of this time is "waiting for response". I'm using
memcached to cache the response using this code:

import cgi
import urllib

import wsgiref.handlers
from google.appengine.ext import webapp
from google.appengine.api import urlfetch

from google.appengine.api import memcache

class YrController(webapp.RequestHandler):

def get(self):
self.response.headers['Content-Type'] = "application/xml"

endpoint = 'http://api.yr.no/weatherapi/locationforecast/1.6/'

params = self.request.GET
apiquery = urllib.urlencode(params)

weather = memcache.get(apiquery)

if weather is not None:
self.response.out.write(weather)
return
else:

result = urlfetch.fetch(url=endpoint + '?' + apiquery,
method=urlfetch.GET)
memcache.add(key=apiquery, value=result.content, time=3600)
self.response.out.write(result.content)

def main():
application = webapp.WSGIApplication([('/api/',
YrController)],debug=True)

wsgiref.handlers.CGIHandler().run(application)

if __name__ == "__main__":
main()

This handler works like a proxy server against a xml api. I can
understand that it takes some time to load when the data is fetched
from the api but once the data is in memcached it should not take
800ms for the server to handle the response. This webbapp used to be
written in ASP .NET also using memcached and there the average
"waiting for response" time for a request is about 100ms.

I will run some profiling on the handler and see what it is that takes
time. Or could anyone else se what it is? Anyway I could optimize this
for shorter load times?

//Magnus

> > google-appengi...@googlegroups.com<google-appengine%2Bunsubscrib e...@googlegroups.com>
> > .

[Ikai L (Google)]

Are you measuring the request from the client side or the server side? What do your server logs say?

When it comes to client side optimizations, there are a ton of things you can do to improve the performance. What you'll want to do is benchmark on the server using the admin console, then do benchmarking on the client side using a tool such as YSlow:

http://developer.yahoo.com/yslow/

This will tell you if you need to set cache headers, minify JavaScript, etc. Can you post some numbers from both the server and the client?

To unsubscribe from this group, send email to google-appengi...@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

[Magnus O.]

I did some profiling and this is the results:

I used this url as reference: http://blirdetsol.appspot.com/api/?lat=57.34145354230627;lon=113.60809999999998

Using the logging tool in python it seems like this request takes 95ms
to handle on the server:

11-26 12:02AM 52.792 /api/?
lat=57.34145354230627;lon=113.60809999999998 200 95ms 96cpu_ms 66kb
Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.5) Gecko/
20091102 Firefox/3.5.5,gzip(gfe)
212.247.53.124 - - [26/Nov/2009:00:02:52 -0800] "GET /api/?
lat=57.34145354230627;lon=113.60809999999998 HTTP/1.1" 200 66326 -
"Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.5) Gecko/
20091102 Firefox/3.5.5,gzip(gfe)" "blirdetsol.appspot.com"

11-26 12:02AM 52.863 Start api main function
11-26 12:02AM 52.866 Start api response handling
11-26 12:02AM 52.880 Returned content from mechached

Firebug told me the following data about the same request:

DNS Lookup: 0ms
Connecting: 0ms
Queuing: 0ms
Waiting for response: 244ms
Recieving data: 253ms


These numbers are actually pretty good but what is it that makes the
server to take 244ms before it actually sends any data to the client
when the request only took 95ms to handle?

Another question a little bit off topic, why is this response not
gzipped? From searching I found that it seems like xml content is not
gzipped, why?

//Magnus

> > > > google-appengi...@googlegroups.com<google-appengine%2Bunsubscrib e...@googlegroups.com><google-appengine%2Bunsubscrib

> > e...@googlegroups.com>
> > > > .
> > > > For more options, visit this group at
> > > >http://groups.google.com/group/google-appengine?hl=.
>
> > > --
> > > Ikai Lan
> > > Developer Programs Engineer, Google App Engine
>
> > --
>
> > You received this message because you are subscribed to the Google Groups
> > "Google App Engine" group.
> > To post to this group, send email to google-a...@googlegroups.com.
> > To unsubscribe from this group, send email to
> > google-appengi...@googlegroups.com<google-appengine%2Bunsubscrib e...@googlegroups.com>
> > .
> > For more options, visit this group at

[Ikai L (Google)]

Where are you seeing that the content is not gzipped? Your client has to specify that it can accept "gzip".

As far as your client side results go, this could simply be a result of geographic latency. The 95ms is the time it takes to render the page, and what you are seeing in Firebug is the full round trip time.

To unsubscribe from this group, send email to google-appengi...@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

[Ikai L (Google)]

Interesting, I just checked that link and it is not coming back in gzip format. Have you seen that it will gzip if you change the format to plain text?

[Magnus O.]

It will gzip if I set the content type to text/xml instead of
application/xml. So that's good enough :)

About the latency you might be right that it is a geographic latency
issue. I compared it to the static files and they take about the same
time to load.

Thanks!

Are the servers of app engine all hosted in the US? There is
geographic spread of the servers so the client always will send the
request to a geographic close server? That would have been awesome!

//Magnus

On Nov 30, 9:08 pm, "Ikai L (Google)" <ika...@google.com> wrote:
> Interesting, I just checked that link and it is not coming back in gzip
> format. Have you seen that it will gzip if you change the format to plain
> text?
>
> On Mon, Nov 30, 2009 at 12:07 PM, Ikai L (Google) <ika...@google.com> wrote:
>
>
>
> > Where are you seeing that the content is not gzipped? Your client has to
> > specify that it can accept "gzip".
>
> > As far as your client side results go, this could simply be a result of
> > geographic latency. The 95ms is the time it takes to render the page, and
> > what you are seeing in Firebug is the full round trip time.
>
> > On Thu, Nov 26, 2009 at 12:11 AM, Magnus O. <
> > magnus.ottos...@magnusottosson.se> wrote:
>
> >> I did some profiling and this is the results:
>
> >> I used this url as reference:

> >>http://blirdetsol.appspot.com/api/?lat=57.34145354230627;lon=113.6080...

> >> > > google-appengi...@googlegroups.com<google-appengine%2Bunsubscrib e...@googlegroups.com><google-appengine%2Bunsubscrib

> >> e...@googlegroups.com>
> >> > > .
> >> > > For more options, visit this group at
> >> > >http://groups.google.com/group/google-appengine?hl=en.
>
> >> > --
> >> > Ikai Lan
> >> > Developer Programs Engineer, Google App Engine
>
> >> --
>
> >> You received this message because you are subscribed to the Google Groups
> >> "Google App Engine" group.
> >> To post to this group, send email to google-a...@googlegroups.com.
> >> To unsubscribe from this group, send email to
> >> google-appengi...@googlegroups.com<google-appengine%2Bunsubscrib e...@googlegroups.com>
> >> .
> >> For more
>

> ...
>
> read more »

[Ikai L (Google)]

Our data centers are distributed for redundancy and failover, but we're not doing anything with regards to intelligently serving content from data centers close to users. The good news is that we definitely have the technology and the know-how to do so!

To unsubscribe from this group, send email to google-appengi...@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.