Pure Python .egg files?

[Bradley Kite]

Hi all,

I'd like to use a pure-python module, however when ever I run my app
it complains that the python interpreter cannot find the module?

What is the correct procedure for including 3rd party python modules
within app-engine applications?

Thanks in advance
--
Brad.

[Marzia Niccolai]

Hi Brad,

Google App Engine doesn't support egg modules.  You will need the source files for the pure python module to upload with App Engine.

-Marzia

[Bradley Kite]

Hi Marzia,

OK - I have the source too - but I'm not sure where to put it. I've
tried including it in my project's src directory but it still
complains.

I'm trying to use google's protocol buffers modules:

from google.protobuf import descriptor

I have the following in my source directory:

google/protobuf/descriptor.py (plus a bunch of other related files/directories)
myapp.py - my application which imports the above module

But its still not right. I'm sure its something silly but I've been
trying many different things so far without success.

Your help is much appreciated!

Regards
--
Brad

[Dan Sanderson]

Do you have __init__.py files in your google/ and google/protobuf/ directories?  That's how Python knows those directories are packages that contain modules.  (The files can be empty.)

-- Dan

[Bradley Kite]

Hi Dan,

I've checked and these files are present.

How are other people using 3rd party python modules? Do you have to
give any special arguments when starting dev_appserver.py ?

Regards
--
Brad.

[Marce (Google)]

Hmmmm, my email post doesn't seem to be showing up, so I'm going to
try it through the web, this will probably end up being a re-post :)

Hi Brad,

This is a conflict with package names. Our packages are
google.appengine, and you are trying to use a packaged named
google.protobuf.

You should include the google.protobuf files in your application's
source directory, but you need to do a bit of sys.modules magic to get
it to work.

First, make a file called import_fixer.py. The contents should be:

import os
import sys

BASE_PACKAGE = 'google'

def FixImports(*packages):


topdir = os.path.dirname(__file__)
def ImportPackage(full_package):
"""Import a fully qualified package."""
imported_module = __import__(full_package, globals(), locals())

# Check if the override path already exists for the module; if it
does,

# that means we've already fixed imports.
original_module = sys.modules[full_package]
lib_path = os.path.join(topdir, full_package.replace('.', '/'))

if lib_path not in original_module.__path__:
# Insert after runtime path, but before anything else
original_module.__path__.insert(1, lib_path)

ImportPackage(BASE_PACKAGE)

for package in packages:
# For each package, we need to import all of its parent packages.
dirs = package.split('.')
full_package = BASE_PACKAGE
for my_dir in dirs:
full_package = '%s.%s' % (full_package, my_dir)
ImportPackage(full_package)

~~~

Then, in your application file, do the following after importing all
your appengine modules:

import import_fixer
import_fixer.FixImports('protobuf')

from google.protobuf import descriptor

Then your imports should work.

-Marzia

On Oct 30, 12:08 pm, "Bradley Kite" <bradley.k...@gmail.com> wrote:
> Hi Dan,
>
> I've checked and these files are present.
>
> How are other people using 3rd party python modules? Do you have to
> give any special arguments when starting dev_appserver.py ?
>
> Regards
> --
> Brad.
>

> On 30/10/2008, Dan Sanderson <dansander...@google.com> wrote:
>
> > Do you have __init__.py files in your google/ and google/protobuf/
> > directories?  That's how Python knows those directories are packages that
> > contain modules.  (The files can be empty.)
>
> > -- Dan
>

> > On Thu, Oct 30, 2008 at 10:46 AM, Bradley Kite <bradley.k...@gmail.com>

> > wrote:
>
> > > Hi Marzia,
>
> > > OK - I have the source too - but I'm not sure where to put it. I've
> > > tried including it in my project's src directory but it still
> > > complains.
>
> > > I'm trying to use google's protocol buffers modules:
>
> > > from google.protobuf import descriptor
>
> > > I have the following in my source directory:
>
> > > google/protobuf/descriptor.py (plus a bunch of other related
> > files/directories)
> > > myapp.py - my application which imports the above module
>
> > > But its still not right. I'm sure its something silly but I've been
> > > trying many different things so far without success.
>
> > > Your help is much appreciated!
>
> > > Regards
> > > --
> > > Brad
>
> > > On 30/10/2008, Marzia Niccolai <ma...@google.com> wrote:
> > > > Hi Brad,
>
> > > > Google App Engine doesn't support egg modules.  You will need the source
> > > > files for the pure python module to upload with App Engine.
>
> > > > -Marzia
>

> > > > On Thu, Oct 30, 2008 at 8:02 AM, Bradley Kite <bradley.k...@gmail.com>

[Bradley Kite]

Genius!

Thanks very much Marce, I think I'm nearly there now!

Just one more thing now tho, I'm getting this error now:

<type 'exceptions.NameError'>: name 'true' is not defined
args = ("name 'true' is not defined",)
message = "name 'true' is not defined"

This is from here:

application = webapp.WSGIApplication(
[('/', MainPage),
('/pbmessage', PBMessageHandler)],
debug=true)

The definition of true (and probably other def's) seems to have
disappeared (as a result of the import_fixer ?)

Regards
--
Brad.

[Dan Sanderson]

That should be True with a capital T, the Python Boolean literal.  I'm not sure what may have defined "true" if that had been working before.

-- Dan

[Bradley Kite]

Hmm - me neither, but I'm learning Python as we go along so its probably me...

Seems to work like a charm now tho. Many thanks for your help guys,
its very much appreciated.

Is there any chance of having the protocol buffers core modules
included in the SDK in future? This would open up the possibilities of
writing protobuf based RPC-over-HTTP applications without the learning
curve I've just had to go through :-)

Kind Regards
--
Brad.

[Marzia Niccolai]

A great (and existing) feature request!  If this is something you are interested in, I encourage you to star it: http://code.google.com/p/googleappengine/issues/detail?id=571

-Marzia