Using MapReduce as a zipimport

31 views
Skip to first unread message

Brent Washburne

unread,
Aug 14, 2011, 4:00:23 PM8/14/11
to Google App Engine
Hi all,

I'm trying out the MapReduce library as a way to handle long-
running tasks. The very first thing in the Getting Started page
( http://code.google.com/p/appengine-mapreduce/wiki/GettingStartedInPython
) is to download the library into the application directory. This
adds about 100 files to my application.

I have read about using zipimport to combine the Django library
into a single zip file, and then adding the file to Python's path. I
tried to do this with MapReduce.

The first problem is that MapReduce is meant to be called from
app.yaml, not from a Python module, e.g.:

handlers:
- url: /mapreduce(/.*)?
script: mapreduce/main.py

There is no way to tell app.yaml to use zipimport, is there? Are
there any plans to make mapreduce into a "native" handler so I don't
need to download the library at all?

In general, has anyone successfully used zipimport on any library
besides Django? There doesn't seem to be any working examples out
there.

~Brent

Tim Hoffman

unread,
Aug 15, 2011, 12:47:35 AM8/15/11
to google-a...@googlegroups.com
Hi Brent

Just create your own mapreduce handler  that imports the default one and do the imports there before 
you import any of your code.

This goes for any system supplied handler, such as ereporter, deferred etc...

Rgds

T

Robert Kluin

unread,
Aug 15, 2011, 12:48:41 AM8/15/11
to google-a...@googlegroups.com
Why don't you write a custom 'main' handler for the mapreduce library?
There isn't much to it:
http://code.google.com/p/appengine-mapreduce/source/browse/trunk/python/src/mapreduce/main.py

Robert

> --
> You received this message because you are subscribed to the Google Groups "Google App Engine" group.
> To post to this group, send email to google-a...@googlegroups.com.
> To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
>
>

Reply all
Reply to author
Forward
0 new messages