Python 2.7 multithreaded: ereporter & mapreduce compatibility

Showing 1-9 of 9 messages
Python 2.7 multithreaded: ereporter & mapreduce compatibility Pol 11/10/11 7:44 AM
Hi,

What are the plans to get these 2 extensions compatible with Python
2.7 multithreaded?

$PYTHON_LIB/google/appengine/ext/mapreduce/main.py
$PYTHON_LIB/google/appengine/ext/ereporter/report_generator.py

Thanks,

- Pol
Re: [google-appengine] Python 2.7 multithreaded: ereporter & mapreduce compatibility Brian Quinlan 11/10/11 6:10 PM

I assume that you are seeing an error during the parsing of your appcfg.py file?

The error message should say that you can't use a CGI with threadsafe.
The solution is to use WSGI in your handlers instead i.e.

google.appengine.ext.mapreduce.application
google.appengine.ext.ereporter/report_generator.application

Cheers,
Brian


> Thanks,
>
> - Pol
>
> --
> You received this message because you are subscribed to the Google Groups "Google App Engine" group.
> To post to this group, send email to google-a...@googlegroups.com.
> To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
>
>

Re: Python 2.7 multithreaded: ereporter & mapreduce compatibility Pol 11/11/11 12:46 PM
Indeed the problem is the parsing of app.yaml:
- url: /mapreduce(/.*)?  script: $PYTHON_LIB/google/appengine/ext/
mapreduce/main.py  login: admin- url: /ereporter.*  script:
$PYTHON_LIB/google/appengine/ext/ereporter/report_generator.py  login:
admin
So what do I use instead for script paths? Looking into the SDK
source, these extensions are not 2.7 multithreaded ready.
Re: [google-appengine] Re: Python 2.7 multithreaded: ereporter & mapreduce compatibility Brian Quinlan 11/11/11 1:26 PM
Hi Pol,

On Sat, Nov 12, 2011 at 7:46 AM, Pol <p...@everpix.net> wrote:
> Indeed the problem is the parsing of app.yaml:
> - url: /mapreduce(/.*)?  script: $PYTHON_LIB/google/appengine/ext/
> mapreduce/main.py  login: admin- url: /ereporter.*  script:
> $PYTHON_LIB/google/appengine/ext/ereporter/report_generator.py  login:
> admin
> So what do I use instead for script paths?

script: google.appengine.ext.mapreduce.application
script: google.appengine.ext.ereporter.report_generator.main.APP

> Looking into the SDK
> source, these extensions are not 2.7 multithreaded ready.

How so?

Cheers,
Brian

Re: Python 2.7 multithreaded: ereporter & mapreduce compatibility Pol 11/11/11 6:09 PM
Ah OK it works, thanks. Didn't know had to replace path entirely...
Was wondering about compatibility since there were still some CGI
leftovers in these files.
Re: Python 2.7 multithreaded: ereporter & mapreduce compatibility working 11/12/11 1:05 PM
Hi Brian,

As for
google.appengine.ext.mapreduce.application

Does it mean the SDK/runtime already has its mapreduce? Right now, I
still supply an additional copy in my app folder. If SDK/runtime
already has it, I will delete my own copy.

How about the pipeline?

Thanks,
coronin


On Nov 10, 6:10 pm, Brian Quinlan <bquin...@google.com> wrote:
Re: [google-appengine] Re: Python 2.7 multithreaded: ereporter & mapreduce compatibility Brian Quinlan 11/12/11 1:10 PM
Hi coronin,

On Sun, Nov 13, 2011 at 8:05 AM, working <cor...@gmail.com> wrote:
> Hi Brian,
>
> As for
> google.appengine.ext.mapreduce.application
>
> Does it mean the SDK/runtime already has its mapreduce? Right now, I
> still supply an additional copy in my app folder. If SDK/runtime
> already has it, I will delete my own copy.
>
> How about the pipeline?

The SDK and runtime does include a copy of mapreduce but it is
recommended that you include your own copy because mapreduce is
experimental and incompatible changes may still be made. Including
your own copy will protect your from this.

Cheers,
Brian

Re: Python 2.7 multithreaded: ereporter & mapreduce compatibility Pol 11/12/11 10:10 PM
Hi Brian,

Although we can deploy to production using 2.7, accessing /mapreduce
fails:

Traceback (most recent call last):
  File "/base/python27_runtime/python27_lib/versions/1/google/
appengine/runtime/wsgi.py", line 168, in Handle
    handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
  File "/base/python27_runtime/python27_lib/versions/1/google/
appengine/runtime/wsgi.py", line 217, in _LoadHandler
    __import__(cumulative_path)
ImportError: No module named application

The app.yaml is set like this:

- url: /mapreduce(/.*)?
  script: google.appengine.ext.mapreduce.application
  login: admin

Thanks for your help!

On Nov 11, 1:26 pm, Brian Quinlan <bquin...@google.com> wrote:
> Hi Pol,
>
> On Sat, Nov 12, 2011 at 7:46 AM, Pol <p...@everpix.net> wrote:
> > Indeed the problem is the parsing of app.yaml:
> > - url: /mapreduce(/.*)?  script: $PYTHON_LIB/google/appengine/ext/
> >mapreduce/main.py  login: admin- url: /ereporter.*  script:
> > $PYTHON_LIB/google/appengine/ext/ereporter/report_generator.py  login:
> > admin
> > So what do I use instead for script paths?
>
> script: google.appengine.ext.mapreduce.application
> script: google.appengine.ext.ereporter.report_generator.main.APP
>
> > Looking into the SDK
> > source, these extensions are not2.7multithreaded ready.
Re: [google-appengine] Re: Python 2.7 multithreaded: ereporter & mapreduce compatibility Brian Quinlan 11/12/11 10:20 PM
On Sat, Nov 12, 2011 at 8:26 AM, Brian Quinlan <bqui...@google.com> wrote:
> Hi Pol,
>
> On Sat, Nov 12, 2011 at 7:46 AM, Pol <p...@everpix.net> wrote:
>> Indeed the problem is the parsing of app.yaml:
>> - url: /mapreduce(/.*)?  script: $PYTHON_LIB/google/appengine/ext/
>> mapreduce/main.py  login: admin- url: /ereporter.*  script:
>> $PYTHON_LIB/google/appengine/ext/ereporter/report_generator.py  login:
>> admin
>> So what do I use instead for script paths?
>
> script: google.appengine.ext.mapreduce.application
> script: google.appengine.ext.ereporter.report_generator

Oops, I meant:

script: google.appengine.ext.mapreduce.main.APP
script: google.appengine.ext.ereporter.report_generator.application

Cheers,
Brian