Variable - High/Low CPU usage for the same simple page cache scenario

[Sharp-Developer.Net]

Hi,

I've enabled complete page content caching and surprisingly see high
CPU for requests that utilize the memcache with a single call to the
memcache.

As you could see from log & code below there are just single call to
memcache and a call to users.get_current_user()

I have a Django middleware handler as well. But as I understand it
should work on every request and it's fairly simple - doing
configuration basing on request parameters - no DB calls.

Any thouts on reason of such a big difference in numbers?
--
Alex
http://sharp-developer.net/

Example from log:
===========================================================
1. 12-06 09:11AM 17.294 / 200 1097ms 3725mcycles 27kb
86.41.125.175 - - [06/12/2008:09:11:18 -0800] "GET / HTTP/1.1"
200 27103 - -

2. D 12-06 09:11AM 18.388
Page taken from MEMCACHE by key: page5/
--------------------------------------------------------------------------------------------------------------------
what strange is that sometimes I get very good results for the same
request:
--------------------------------------------------------------------------------------------------------------------
1. 12-06 09:22AM 40.639 / 200 15ms 14mcycles 27kb
86.41.125.175 - - [06/12/2008:09:22:40 -0800] "GET / HTTP/1.1"
200 27103 - -

2. D 12-06 09:22AM 40.650
Page taken from MEMCACHE by key: page5/
===========================================================


where code for caching is:

====================
@cache.memorize_page()
def index(request):
# some code here
return render_to_response('root_index.html',
payload,
context_instance=RequestContext
(request))
def memorize_page(name=None, time=60*60*12):
def memorize_fn(fn):
def new_fn(*a, **kw):
request = a[0]
# We are caching whole page just for GET requests
if request.method != "GET":
return fn(*a, **kw)
prefix = "page5"
key = "%s%s" % (prefix, request.path)
user = get_current_user()
if user:
key = "u=%s/%s" % (user.key(), key)
if request.GET.has_key("cache") and request.GET["cache"]
== "reset":
memcache.delete(key)
LOG.debug("Page delete from MEMCACHE by key: %s", key)
else:
response_content = memcache.get(key)
if response_content is not None:
LOG.debug("Page taken from MEMCACHE by key: %s",
key)
return HttpResponse(content=response_content)
#return res

res = fn(*a, **kw)
memcache.set(key, res.content, time=time)
LOG.debug("Page stored to MEMCACHE with key: %s" % key)
return res
return new_fn
return memorize_fn
----------------------------------------
def get_current_user():
guser = users.get_current_user()
if not guser:
return None
#my code here for authorized users
# in cache use case we work with unauthorized request
====================

[Marzia Niccolai]

Hi Alex,

Does your script load up a lot of modules when it starts?  It's possible that these are the CPU warnings you are seeing.  You may be seeing them inconsistently since this would only happen when a new interpreter for your script is fired up.  The basic process is described here:
http://code.google.com/appengine/docs/python/appcaching.html

-Marzia

[Sharp-Developer.Net]

Thanks Marzia,

I've put some logging and seems yes - spikes happens when new instance
is starting.

I've noticed Django helper installation takes solid amount of time and
I guess CPU.

See log extract:

# 12-11 10:42AM 27.040 - Loading django helper...
# 12-11 10:42AM 27.862 - Successfully loaded the Google App Engine
Helper for Django.

I know GAE was providing Django 0.96.

Is it providing Django 1.0 and appengine_django helper now (on server
side)?

If yes, would it fire up the cold starts if I exclude local version of
Django & appengine_django?

I have not found any news regarding updates of GAE production from
Django 0.96 to release 1.0

I think it would be interesting to many developers.
--
Alex
http://sharp-developer.net/

On Dec 8, 7:26 pm, Marzia Niccolai <ma...@google.com> wrote:
> Hi Alex,
>
> Does your script load up a lot of modules when it starts?  It's possible
> that these are the CPU warnings you are seeing.  You may be seeing them
> inconsistently since this would only happen when a new interpreter for your
> script is fired up.  The basic process is described here:http://code.google.com/appengine/docs/python/appcaching.html
>
> -Marzia
>
> On Sat, Dec 6, 2008 at 9:30 AM, Sharp-Developer.Net <
>