how to use GAE deferred

80 views
Skip to first unread message

Richard

unread,
Mar 7, 2010, 5:44:56 PM3/7/10
to web2py-users
I am adding some background work to my web2py GAE app with the
deferred library:
http://code.google.com/appengine/articles/deferred.html

As the article suggests I have added this to app.yaml:
- url: /_ah/queue/deferred
script: $PYTHON_LIB/google/appengine/ext/deferred/deferred.py
login: admin

The article also says "You can't call a method in the request handler
module" - what does that correspond to in web2py?

Any other advice about using Deferred / Task Queues through web2py
would be welcome.

Richard

waTR

unread,
Mar 7, 2010, 8:49:31 PM3/7/10
to web2py-users
I too am interested in this...

Richard

unread,
Mar 7, 2010, 11:44:36 PM3/7/10
to web2py-users
found they need to go in modules/models rather than controllers.
However one model function can't defer another model function...

Richard

unread,
Mar 9, 2010, 6:25:34 AM3/9/10
to web2py-users
Actually it seems only module functions can be deferred. If I try
deferring a controller of model function then I get this error:
PicklingError: Can't pickle <function test at 0xa743ca4>: it's not
found as __main__.test

However I need my deferred function to interact with the database. Any
ideas??

Richard

unread,
Mar 9, 2010, 8:00:40 AM3/9/10
to web2py-users
I guess I could use gql directly to access the database, but I would
prefer to have a single syntax for accessing the database.

(controller of model -> controller or model)

Richard

unread,
Mar 10, 2010, 8:10:33 AM3/10/10
to web2py-users
didn't figure this out so have changed (rather awkwardly) to using
cron.
Would be interested to hear if anyone else has better luck.

howesc

unread,
Mar 19, 2010, 6:00:05 PM3/19/10
to web2py-users
Hi all,

So i need to do an GAE bigtable upgrade today, and add a new field and
i need its value to be set properly....so i too want to do this.
Turns out deferred is not what we web2py people want. just use the
taskqueue directly. it's trivial (well i think it is, i have not
deployed yet, but it runs on dev_appserver):

in default.py add the following 2 methods:

def test():
from google.appengine.api.labs import taskqueue
taskqueue.add(url='/default/task', params={'key': "bob"})
return

def task():
logging.info("in da task")
logging.info(repr(request.vars))
logging.info("rec count: %d" % db(db.recording.id>0).count())
return

then, if you hit /default/test in your browser it will add a task to
the default queue - keep in mind that the default queue executes tasks
at 5 per second rate, create a separate queue if you want to configure
it more. See TaskClass and QueueClass subsections of
http://code.google.com/appengine/docs/python/taskqueue/overview.html.
Then watch your logs (if using dev_appserver remember to tell the task
to run - access the default queue at http://127.0.0.1:8080/_ah/admin/tasks?queue=default).
Note that if you blindly copied my code and don't have a table named
recording then you'll probably get an exception.

important bits:
- pass params to your taskqueue.add() call, they show up as request
vars. free. no pickling involved. whoo-hooo
- now that it's handled just like any other web request in web2py you
have the full environment at your disposal. not sure about getting a
login or session to the queue call though.
- i don't have to figure out how to get my db.py imported into a
module. :)

good luck!

Christian


On Mar 10, 6:10 am, Richard <richar...@gmail.com> wrote:
> didn't figure this out so have changed (rather awkwardly) to using
> cron.
> Would be interested to hear if anyone else has better luck.
>
> On Mar 10, 12:00 am, Richard <richar...@gmail.com> wrote:
>
> > I guess I could use gql directly to access the database, but I would
> > prefer to have a single syntax for accessing the database.
>
> > (controller of model -> controller or model)
>
> > On Mar 9, 10:25 pm, Richard <richar...@gmail.com> wrote:
>
> > > Actually it seems only module functions can be deferred. If I try
> > > deferring a controller of model function then I get this error:
> > > PicklingError: Can't pickle <function test at 0xa743ca4>: it's not
> > > found as __main__.test
>
> > > However I need my deferred function to interact with the database. Any
> > > ideas??
>
> > > On Mar 8, 3:44 pm, Richard <richar...@gmail.com> wrote:
>
> > > > found they need to go in modules/models rather than controllers.

> > > > However one model function can'tdeferanother model function...


>
> > > > On Mar 8, 12:49 pm, waTR <r...@devshell.org> wrote:
>
> > > > > I too am interested in this...
>
> > > > > On Mar 7, 2:44 pm, Richard <richar...@gmail.com> wrote:
>

> > > > > > I am adding some background work to my web2pyGAEapp with the

Richard

unread,
Dec 10, 2010, 7:06:09 AM12/10/10
to web...@googlegroups.com
thanks - just what I wanted
Reply all
Reply to author
Forward
0 new messages