GAE Caching and Views

[johntynan]

I am using the following decorator:

@cache(request.env.path_info, time_expire=5, cache_model=cache.ram)

To cache some db queries, however, when I go to deploy this on Google
App Engine, I receive a "PicklingError: Can't pickle <type
'function'>: attribute lookup __builtin__.function failed" You can see
the traceback here:

https://gist.github.com/1384892

I have a feeling the error has to do with caching a for loop in a
view. You can see the view here:

http://code.google.com/p/pledgedrivetracker/source/browse/pledgedrives/views/refresh/thank_yous.html?spec=svnf8cd17ed0367521b8bd726c48ac829e920207c11&r=f8cd17ed0367521b8bd726c48ac829e920207c11

You can also see the "thank_yous" function here:

http://code.google.com/p/pledgedrivetracker/source/browse/pledgedrives/controllers/refresh.py?spec=svnf8cd17ed0367521b8bd726c48ac829e920207c11&r=f8cd17ed0367521b8bd726c48ac829e920207c11#74

I think the solution lies with creating the view within the function
using return response.render() as described in the web2py book here:

http://www.web2py.com/book/default/chapter/04?search=cache+view

but I'm at a loss for how to do this. Does anyone have any clear
examples for using response.render and an html template.

Thanks!

[Anthony]

How about:

    return response.render(dict(pledges=pledges))

An alternative is just to cache the select instead of the whole view:

    pledges=db((db.pledge.segment==segment_id) &

(db.pledge.read == False)).select(orderby=~db.pledge.created_on,

cache=(cache.ram, 5))

In that case, you would remove the @cache decorator from the function.

See http://www.web2py.com/book/default/chapter/06#Caching-Selects.

Anthony

[howesc]

actually, unless something changed on on GAE, select caching does not work because the rows object is not picklable.

the @cache decorator caches the dict that you return to the view, not the rendered view.  if something in that dict is not picklable (say a rows object) the cache will fail.  anthony's first suggestion should work in all cases.

[Massimo Di Pierro]

select(cache) does not serielizes Rows but the dictionary returned by
the select. I have never tried it on GAE. For sure you cannot cache in
run but it should be possible to cache.memcache. If that does not work
it is because the datastore returns a type that is not serializible.

[howesc]

hmmm, i've never known GAE to cache selects.  at one point i thought the book said as much, now it simply says:

"The results of a select are complex, un-pickleable objects; they cannot be stored in a session and cannot be cached in any other way than the one explained here."

and i believe that un-pickleable means not memcacheable.

reading dal.py, select() in the GAEAdaptor does not seem to do anything with cache.  (maybe i'm mis-reading it)

[Bruno Rocha]

On Tue, Nov 22, 2011 at 8:34 PM, Massimo Di Pierro <massimo....@gmail.com> wrote:

. I have never tried it on GAE

I have a select cache on my app, works locally, but fails on GAE. It does not raise any exception but does not cache anything, and I only realized it does not works when I see my "clean cache" function raising an error.

My code is: https://github.com/rochacbruno/Movuca/blob/master/modules/config.py#L41

I have not tried other cache methods for GAE.

--

Bruno Rocha

[http://rochacbruno.com.br]

[johntynan]

Thanks everyone for your excellent help!

I posted an account of enabling caching with web2py on GAE in a blog
post here:

http://opensourcebroadcasting.blogspot.com/2011/11/web2py-caching-and-google-app-engine.html

All the best, JT

[Constantine Vasil]

>not picklable

there is a way to serialize with protocol buffers.  I am using it in my code but it better be implemented in the web2py core as a 'gae' dependency.

[howesc]

Constantine, do you mind sharing what and how you are serializing things with protocol buffers?  perhaps we can adapt what you are doing into the web2py GAE caching core.

JT - thanks for the nice writeup.  i'm working on optimizing some web2py apps for customers now, and should be using your technique where i can.