Pure Python .egg files?

465 views
Skip to first unread message

Bradley Kite

unread,
Oct 30, 2008, 11:02:14 AM10/30/08
to google-a...@googlegroups.com
Hi all,

I'd like to use a pure-python module, however when ever I run my app
it complains that the python interpreter cannot find the module?

What is the correct procedure for including 3rd party python modules
within app-engine applications?

Thanks in advance
--
Brad.

Marzia Niccolai

unread,
Oct 30, 2008, 1:37:15 PM10/30/08
to google-a...@googlegroups.com
Hi Brad,

Google App Engine doesn't support egg modules.  You will need the source files for the pure python module to upload with App Engine.

-Marzia

Bradley Kite

unread,
Oct 30, 2008, 1:46:19 PM10/30/08
to google-a...@googlegroups.com
Hi Marzia,

OK - I have the source too - but I'm not sure where to put it. I've
tried including it in my project's src directory but it still
complains.

I'm trying to use google's protocol buffers modules:

from google.protobuf import descriptor

I have the following in my source directory:

google/protobuf/descriptor.py (plus a bunch of other related files/directories)
myapp.py - my application which imports the above module

But its still not right. I'm sure its something silly but I've been
trying many different things so far without success.

Your help is much appreciated!

Regards
--
Brad

Dan Sanderson

unread,
Oct 30, 2008, 3:01:06 PM10/30/08
to google-a...@googlegroups.com
Do you have __init__.py files in your google/ and google/protobuf/ directories?  That's how Python knows those directories are packages that contain modules.  (The files can be empty.)

-- Dan

Bradley Kite

unread,
Oct 30, 2008, 3:08:14 PM10/30/08
to google-a...@googlegroups.com
Hi Dan,

I've checked and these files are present.

How are other people using 3rd party python modules? Do you have to
give any special arguments when starting dev_appserver.py ?

Regards
--
Brad.

Marce (Google)

unread,
Oct 30, 2008, 3:13:22 PM10/30/08
to Google App Engine
Hmmmm, my email post doesn't seem to be showing up, so I'm going to
try it through the web, this will probably end up being a re-post :)

Hi Brad,

This is a conflict with package names. Our packages are
google.appengine, and you are trying to use a packaged named
google.protobuf.

You should include the google.protobuf files in your application's
source directory, but you need to do a bit of sys.modules magic to get
it to work.

First, make a file called import_fixer.py. The contents should be:

import os
import sys

BASE_PACKAGE = 'google'

def FixImports(*packages):


topdir = os.path.dirname(__file__)
def ImportPackage(full_package):
"""Import a fully qualified package."""
imported_module = __import__(full_package, globals(), locals())

# Check if the override path already exists for the module; if it
does,

# that means we've already fixed imports.
original_module = sys.modules[full_package]
lib_path = os.path.join(topdir, full_package.replace('.', '/'))

if lib_path not in original_module.__path__:
# Insert after runtime path, but before anything else
original_module.__path__.insert(1, lib_path)

ImportPackage(BASE_PACKAGE)

for package in packages:
# For each package, we need to import all of its parent packages.
dirs = package.split('.')
full_package = BASE_PACKAGE
for my_dir in dirs:
full_package = '%s.%s' % (full_package, my_dir)
ImportPackage(full_package)

~~~

Then, in your application file, do the following after importing all
your appengine modules:

import import_fixer
import_fixer.FixImports('protobuf')

from google.protobuf import descriptor

Then your imports should work.

-Marzia

On Oct 30, 12:08 pm, "Bradley Kite" <bradley.k...@gmail.com> wrote:
> Hi Dan,
>
> I've checked and these files are present.
>
> How are other people using 3rd party python modules? Do you have to
> give any special arguments when starting dev_appserver.py ?
>
> Regards
> --
> Brad.
>
> On 30/10/2008, Dan Sanderson <dansander...@google.com> wrote:
>
> > Do you have __init__.py files in your google/ and google/protobuf/
> > directories?  That's how Python knows those directories are packages that
> > contain modules.  (The files can be empty.)
>
> > -- Dan
>
> > On Thu, Oct 30, 2008 at 10:46 AM, Bradley Kite <bradley.k...@gmail.com>
> > wrote:
>
> > > Hi Marzia,
>
> > > OK - I have the source too - but I'm not sure where to put it. I've
> > > tried including it in my project's src directory but it still
> > > complains.
>
> > > I'm trying to use google's protocol buffers modules:
>
> > > from google.protobuf import descriptor
>
> > > I have the following in my source directory:
>
> > > google/protobuf/descriptor.py (plus a bunch of other related
> > files/directories)
> > > myapp.py - my application which imports the above module
>
> > > But its still not right. I'm sure its something silly but I've been
> > > trying many different things so far without success.
>
> > > Your help is much appreciated!
>
> > > Regards
> > > --
> > > Brad
>
> > > On 30/10/2008, Marzia Niccolai <ma...@google.com> wrote:
> > > > Hi Brad,
>
> > > > Google App Engine doesn't support egg modules.  You will need the source
> > > > files for the pure python module to upload with App Engine.
>
> > > > -Marzia
>
> > > > On Thu, Oct 30, 2008 at 8:02 AM, Bradley Kite <bradley.k...@gmail.com>

Bradley Kite

unread,
Oct 30, 2008, 3:30:26 PM10/30/08
to google-a...@googlegroups.com
Genius!

Thanks very much Marce, I think I'm nearly there now!

Just one more thing now tho, I'm getting this error now:

<type 'exceptions.NameError'>: name 'true' is not defined
args = ("name 'true' is not defined",)
message = "name 'true' is not defined"

This is from here:

application = webapp.WSGIApplication(
[('/', MainPage),
('/pbmessage', PBMessageHandler)],
debug=true)

The definition of true (and probably other def's) seems to have
disappeared (as a result of the import_fixer ?)

Regards
--
Brad.

Dan Sanderson

unread,
Oct 30, 2008, 4:09:43 PM10/30/08
to google-a...@googlegroups.com
That should be True with a capital T, the Python Boolean literal.  I'm not sure what may have defined "true" if that had been working before.

-- Dan

Bradley Kite

unread,
Oct 30, 2008, 4:18:35 PM10/30/08
to google-a...@googlegroups.com
Hmm - me neither, but I'm learning Python as we go along so its probably me...

Seems to work like a charm now tho. Many thanks for your help guys,
its very much appreciated.

Is there any chance of having the protocol buffers core modules
included in the SDK in future? This would open up the possibilities of
writing protobuf based RPC-over-HTTP applications without the learning
curve I've just had to go through :-)

Kind Regards
--
Brad.

Marzia Niccolai

unread,
Oct 30, 2008, 4:21:49 PM10/30/08
to google-a...@googlegroups.com
A great (and existing) feature request!  If this is something you are interested in, I encourage you to star it: http://code.google.com/p/googleappengine/issues/detail?id=571

-Marzia
Reply all
Reply to author
Forward
0 new messages