Python 2.7 multithreaded: ereporter & mapreduce compatibility

[Pol]

Hi,

What are the plans to get these 2 extensions compatible with Python
2.7 multithreaded?

$PYTHON_LIB/google/appengine/ext/mapreduce/main.py
$PYTHON_LIB/google/appengine/ext/ereporter/report_generator.py

Thanks,

- Pol

[Brian Quinlan]

I assume that you are seeing an error during the parsing of your appcfg.py file?

The error message should say that you can't use a CGI with threadsafe.
The solution is to use WSGI in your handlers instead i.e.

google.appengine.ext.mapreduce.application
google.appengine.ext.ereporter/report_generator.application

Cheers,
Brian

> Thanks,
>
> - Pol
>
> --
> You received this message because you are subscribed to the Google Groups "Google App Engine" group.
> To post to this group, send email to google-a...@googlegroups.com.
> To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
>
>

[Pol]

Indeed the problem is the parsing of app.yaml:
- url: /mapreduce(/.*)?  script: $PYTHON_LIB/google/appengine/ext/
mapreduce/main.py  login: admin- url: /ereporter.*  script:
$PYTHON_LIB/google/appengine/ext/ereporter/report_generator.py  login:
admin
So what do I use instead for script paths? Looking into the SDK
source, these extensions are not 2.7 multithreaded ready.

[Brian Quinlan]

Hi Pol,

On Sat, Nov 12, 2011 at 7:46 AM, Pol <p...@everpix.net> wrote:
> Indeed the problem is the parsing of app.yaml:
> - url: /mapreduce(/.*)?  script: $PYTHON_LIB/google/appengine/ext/
> mapreduce/main.py  login: admin- url: /ereporter.*  script:
> $PYTHON_LIB/google/appengine/ext/ereporter/report_generator.py  login:
> admin
> So what do I use instead for script paths?

script: google.appengine.ext.mapreduce.application
script: google.appengine.ext.ereporter.report_generator.main.APP

> Looking into the SDK
> source, these extensions are not 2.7 multithreaded ready.

How so?

Cheers,
Brian

[Pol]

Ah OK it works, thanks. Didn't know had to replace path entirely...
Was wondering about compatibility since there were still some CGI
leftovers in these files.

[working]

Hi Brian,

As for
google.appengine.ext.mapreduce.application

Does it mean the SDK/runtime already has its mapreduce? Right now, I
still supply an additional copy in my app folder. If SDK/runtime
already has it, I will delete my own copy.

How about the pipeline?

Thanks,
coronin

On Nov 10, 6:10 pm, Brian Quinlan <bquin...@google.com> wrote:

[Brian Quinlan]

Hi coronin,

On Sun, Nov 13, 2011 at 8:05 AM, working <cor...@gmail.com> wrote:
> Hi Brian,
>
> As for
> google.appengine.ext.mapreduce.application
>
> Does it mean the SDK/runtime already has its mapreduce? Right now, I
> still supply an additional copy in my app folder. If SDK/runtime
> already has it, I will delete my own copy.
>
> How about the pipeline?

The SDK and runtime does include a copy of mapreduce but it is
recommended that you include your own copy because mapreduce is
experimental and incompatible changes may still be made. Including
your own copy will protect your from this.

Cheers,
Brian

[Pol]

Hi Brian,

Although we can deploy to production using 2.7, accessing /mapreduce
fails:

Traceback (most recent call last):
File "/base/python27_runtime/python27_lib/versions/1/google/
appengine/runtime/wsgi.py", line 168, in Handle
handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
File "/base/python27_runtime/python27_lib/versions/1/google/
appengine/runtime/wsgi.py", line 217, in _LoadHandler
__import__(cumulative_path)
ImportError: No module named application

The app.yaml is set like this:

- url: /mapreduce(/.*)?
script: google.appengine.ext.mapreduce.application
login: admin

Thanks for your help!

On Nov 11, 1:26 pm, Brian Quinlan <bquin...@google.com> wrote:
> Hi Pol,
>
> On Sat, Nov 12, 2011 at 7:46 AM, Pol <p...@everpix.net> wrote:
> > Indeed the problem is the parsing of app.yaml:
> > - url: /mapreduce(/.*)?  script: $PYTHON_LIB/google/appengine/ext/
> >mapreduce/main.py  login: admin- url: /ereporter.*  script:
> > $PYTHON_LIB/google/appengine/ext/ereporter/report_generator.py  login:
> > admin
> > So what do I use instead for script paths?
>
> script: google.appengine.ext.mapreduce.application
> script: google.appengine.ext.ereporter.report_generator.main.APP
>
> > Looking into the SDK

> > source, these extensions are not2.7multithreaded ready.

[Brian Quinlan]

On Sat, Nov 12, 2011 at 8:26 AM, Brian Quinlan <bqui...@google.com> wrote:
> Hi Pol,
>
> On Sat, Nov 12, 2011 at 7:46 AM, Pol <p...@everpix.net> wrote:
>> Indeed the problem is the parsing of app.yaml:
>> - url: /mapreduce(/.*)?  script: $PYTHON_LIB/google/appengine/ext/
>> mapreduce/main.py  login: admin- url: /ereporter.*  script:
>> $PYTHON_LIB/google/appengine/ext/ereporter/report_generator.py  login:
>> admin
>> So what do I use instead for script paths?
>
> script: google.appengine.ext.mapreduce.application
> script: google.appengine.ext.ereporter.report_generator

Oops, I meant:

script: google.appengine.ext.mapreduce.main.APP
script: google.appengine.ext.ereporter.report_generator.application

Cheers,
Brian