Pylons on Google App Engine article
Jason S.
I've just finished (the first draft of) an article on getting Pylons
running on GAE with a mostly "stock" setup. I've focused on using a
typical virtualenv and stock paster template and attempted to find the
minimum number of changes required to get things running, rather than
creating a new setup.
Thus, things like "paster controller" still work, you still use an INI
file for configuration, and theoretically an existing virtualenv/app
could be converted (though the necessary changes to models, etc., mean
that reusability is poor nonetheless.)
http://countergram.com/articles/pylons-google-app-engine/
Feedback is welcome and I'll try to update the article promptly if
there are any show-stoppers or better approaches. I've done several
runs through the steps provided, though.
Jason
Matt Feifarek
This seems much better than my own recent work... I think it's a mistake to start with "appengine-homedir"... better to just start from stock Pylons and hack it till it works on GAE. That's been my discovery over the last few days, but I haven't patched my Pylons Wiki article yet.
I'll give you more detailed/useful feedback as soon as I can.
Mike Orr
appengine-homedir was to make it easy to get started, by handling the
virtualenv and directory structure for you. But if it turns out that
another way is easier, we can deprecate it. I do like the idea of a
pylons-appengine template for Paster.
--Mike
Mike Orr <slugg...@gmail.com>
Ian Bicking
> This seems much better than my own recent work... I think it's a mistake to
> start with "appengine-homedir"... better to just start from stock Pylons and
> hack it till it works on GAE. That's been my discovery over the last few
> days, but I haven't patched my Pylons Wiki article yet.
I personally got too annoyed with testing and general interactivity
(there's a nose extension though, which I cribbed from -- but I never
actually got it to work myself).
--
Ian Bicking | http://blog.ianbicking.org | http://topplabs.org/civichacker
Matt Feifarek
I personally got too annoyed with testing and general interactivity
(there's a nose extension though, which I cribbed from -- but I never
actually got it to work myself).
This process has been EXTREMELY annoying. I've spent many hours trying to get things to work that I should have been spending working on my project. Mostly, its my own ignorance of current magical techniques:
I still don't get setuptools. I don't understand namespace packages, or rather their deployment. I don't get why I can't move (for example) paste, and the Paste* items to another directory that is ON sys.path and have it be import-able. All of my frustrations with GAE have been of this sort.
Basically, the challenge to getting Pylons on GAE is to manage libraries. Some need to be there for Pylons but not for GAE. Some need to be there fore GAE by not Pylons. Some need to be both. Some need to be uploaded with the "app", some cannot be, some simply should-not be. It's not clear when egg-info directories need to go up to GAE, and when they are wasting space.
And one cannot simply add things to sys.path because of setuptools. And when setuptools can't find something, I have no idea how to fix it. I wish I could just ditch all of this stuff and manually manage sys.path, but as is, Paste especially seems to rely on some magic to find itself. Though, I noticed that if I put an __init__.py in paste/ then it starts acting like a normal package again (duh). Basically, packaging is blowing my mind, and I wish I was just using python modules and packages.
STILL, I've made progress, and seemingly Jason has too. I think though, that it's never going to be "automagic"... at least not on all OS's. GAE is too hackety.
My status: I can take a stock pylons application in a virtualenv, wedge-in the google libraries for yaml, datastore and the like, and run the app with paster serve. If one moves the pylons-required libraries from the standard site-packages to the GAE app's lib/python and add that location to sys.path, you're almost there. Hacking in some ENVIRONMENT variables at boot-strap time and you are there.
I didn't need to patch paste like Jason did, but maybe I'm missing something. I'll be writing up my new recipe asap.
Jason S.
>
> > I personally got too annoyed with testing and general interactivity
> > (there's a nose extension though, which I cribbed from -- but I never
> > actually got it to work myself).
>
> This process has been EXTREMELY annoying. I've spent many hours trying to
> get things to work that I should have been spending working on my project.
> Mostly, its my own ignorance of current magical techniques:
limitations with a few features. But I suppose we have to get things
running on it to do a hands-on evaluation.
> I still don't get setuptools. I don't understand namespace packages, or
> rather their deployment. I don't get why I can't move (for example) paste,
> and the Paste* items to another directory that is ON sys.path and have it be
> import-able. All of my frustrations with GAE have been of this sort.
easy_install.pth. If you move the egg somewhere else, it won't work
unless it's added to the path somehow. BTW, I don't think GAE
uses .pth files, which is why sys.path manipulation is necessary.
When you install Paste packages with easy_install, they go into
separate eggs and in each one, paste/__init__.py has code that
modifies the path to make the namespacing work. It's not something
that just works because of setuptools. Took me a while to find that.
When you install them with pip, I think they overlap (i.e. one paste/
directory in site-packages/).
> And one cannot simply add things to sys.path because of setuptools.
entail, but my app.py *prepends* to sys.path, which works a little
better because it gets around installed versions of libraries in the
GAE environment.
Here's part of app.py:
sys.path = [appdir, libdir] + ['%s/%s' % (libdir, n) for n in
os.listdir(libdir) if n.endswith('.egg')] + sys.path
Performance would probably be better with a hard-coded list of
packages, but this is more flexible of course. On the other hand, I
think a switch to pip would make the listdir part unnecessary.
> My status: I can take a stock pylons application in a virtualenv, wedge-in
> the google libraries for yaml, datastore and the like, and run the app with
> paster serve. If one moves the pylons-required libraries from the standard
> site-packages to the GAE app's lib/python and add that location to sys.path,
> you're almost there. Hacking in some ENVIRONMENT variables at boot-strap
> time and you are there.
specific libs should be provided without moving anything. Can you use
datastore, etc, locally without the SDK?
> I didn't need to patch paste like Jason did, but maybe I'm missing
> something. I'll be writing up my new recipe asap.
getting DistributionNotFound from pkg_resources and figured that it
would be better to work around eggs/the egg loader.
Jason
Matt Feifarek
> I still don't get setuptools. I don't understand namespace packages, or> rather their deployment. I don't get why I can't move (for example) paste,When you install eggs with easy_install, it adds the egg's path to
> and the Paste* items to another directory that is ON sys.path and have it be
> import-able. All of my frustrations with GAE have been of this sort.
easy_install.pth. If you move the egg somewhere else, it won't work
That part I get. In fact, I've often modified easy_install.pth manually.
But I'm using pip. And pip doesn't mess with that file, nor it seems any other meta-data anywhere on sys.path. But still, I couldn't move Paste* around.
I was considering trying an install with easy_install and then hand-modifying easy_install.pth next.
unless it's added to the path somehow. BTW, I don't think GAE
uses .pth files, which is why sys.path manipulation is necessary.
It seems like this isn't true. If I omit pth files when uploading to GAE, my app is failing with a 500 error. But I wouldn't trust that yet... can you do a test on your setup and see if you need or don't need the .pth files? (up to and including easy_install)
When you install them with pip, I think they overlap (i.e. one paste/
directory in site-packages/).
Yes, that's true, but I don't understand why moving those directories from one site-packages to another should matter. I can't find whatever cruft is telling python to look in that and only that sys.path location for paste.
I'm not sure what all the problems you're having with setuptools
entail, but my app.py *prepends* to sys.path, which works a little
better because it gets around installed versions of libraries in the
GAE environment.
You mean to have the upgraded webob for example?
I use the GAE SDK instead of paster serve, because then all the GAE-
specific libs should be provided without moving anything. Can you use
datastore, etc, locally without the SDK?
Yes. I simply add the path to the google libraries to sys.path; I'm running DataStore (well, whatever emulation that the SDK provides on my "paster serve development.ini" pylons app. Which means I can use the debugger, webtest, etc.
Can you use the debugger (weberror)?
Interesting. Maybe that step could be eliminated. I think I was
getting DistributionNotFound from pkg_resources and figured that it
would be better to work around eggs/the egg loader.
I think I got around that by dropping an empty __init__.py file into paste/
Mike Orr
> This process has been EXTREMELY annoying. I've spent many hours trying to
> get things to work that I should have been spending working on my project.
> Mostly, its my own ignorance of current magical techniques:
All of us have been annoyed. That's why it has taken us 1 1/2 years
to write a simple, failsafe howto, and we're still not there yet.
Philip and Ian did the original work in April 2008 which led to
Appengine-Monkey and patches to get Mako and Beaker to run on GAE. I
came in later to do more installation and testing. All of us have
spent weeks pulling our hair out trying to get various things to work.
If you just want to get going quickly with GAE, you may be better off
using the mini framework that comes with it, or one of the smaller
WSGI frameworks. Django has also been ported to it. (And they had
similar problems to what we had, BTW.)
> I still don't get setuptools. I don't understand namespace packages, or
> rather their deployment. I don't get why I can't move (for example) paste,
> and the Paste* items to another directory that is ON sys.path and have it be
> import-able. All of my frustrations with GAE have been of this sort.
First some vocabulary to make sure we're talking about the same things.
A "package" is a directory under sys.path containing Python modules
and an __init__.py module. It's importable.
A "distribution" is a tarball or egg packaged so that it can be
distributed on PyPI. Distributions thus contain packages. However,
many people including me also call distributions "packages", so the
term package is ambiguous.
A "namespace package" is a Setuptools trick that allows multiple
distributions to be installed under the same package namespace. Paste
uses this so that 'paste.deploy' and 'paste.script' can be distributed
seprately from 'paste', for those who want to use one but not the
others. Zope used this to split the 'zope' namespace into 100+
distributions. This was necessary to make Zope pythonic and
non-monolithic.
So the answer to your question is, when you moved the packages, you
did not keep everything in sync. Either you missed some egg-info
directories or .pth files, or there were paths encoded in the egg-info
files that were no longer valid. Or something like that.
Easy_install and Pip do manage namespace packages differently, so the
techniques for moving them would be different. It's normally easier
to reinstall the packages rather than to move them. If you don't want
to go to the network every time, you can build your own archive of
source files and install from that.
> Basically, the challenge to getting Pylons on GAE is to manage libraries.
> Some need to be there for Pylons but not for GAE. Some need to be there fore
> GAE by not Pylons. Some need to be both. Some need to be uploaded with the
> "app", some cannot be, some simply should-not be. It's not clear when
> egg-info directories need to go up to GAE, and when they are wasting space.
Yes, that is all true. The answer is complicated, and we're not sure
of all the details.
> And one cannot simply add things to sys.path because of setuptools. And when
> setuptools can't find something, I have no idea how to fix it. I wish I
> could just ditch all of this stuff and manually manage sys.path, but as is,
> Paste especially seems to rely on some magic to find itself.
Paste, and thus Pylons, was built using Setuptools' features. At the
time, Setuptools was seen as the package management solution for
Python, and would eventually be added to the standard library. That
didn't happen due to an ongoing controversy over Setuptools and
Distutils, which you can read about in the pylons-discuss archive and
web-sig archive.
The Python core developers did not like Setuptools and did not use it.
Some of those developers (Guido in particular) wrote Google App
Engine. So, unsurprisingly, it was Setuptools-unaware. This led to
the majority of problems for using Pylons on it. Basically, Pylons
and Paste was written for a standard Python runtime environment
(CPython and everything available in that world), and GAE is not that.
> My status: I can take a stock pylons application in a virtualenv, wedge-in
> the google libraries for yaml, datastore and the like, and run the app with
> paster serve. If one moves the pylons-required libraries from the standard
> site-packages to the GAE app's lib/python and add that location to sys.path,
> you're almost there. Hacking in some ENVIRONMENT variables at boot-strap
> time and you are there.
I used Appengine-Monkey/Appengine-Homedir because my paramount goal
was a simple install procedure I could explain to others. Those
packages seemed to offer that, by automating things that would
otherwise have to be done by hand. (Particularly installing
virtualenv and setuptools.) But maybe starting from a stock Pylons
installation and app is a better way to go? That's what your and
Jason's research will help answer.
--
Mike Orr <slugg...@gmail.com>
Jason S.
> > uses .pth files, which is why sys.path manipulation is necessary.
>
> do a test on your setup and see if you need or don't need the .pth files?
> (up to and including easy_install)
barebones directory structure:
app.py
app.yaml
test.pth
mydir/
myfile.py
test.pth contains ./mydir. app.py tries to "import myfile" and fails
to do so. Whereas if I add "mydir" to sys.path in app.py it works
immediately.
Matt Feifarek
All of us have been annoyed. That's why it has taken us 1 1/2 years
On Thu, Oct 15, 2009 at 9:59 AM, Matt Feifarek <matt.f...@gmail.com> wrote:
> This process has been EXTREMELY annoying. I've spent many hours trying to
> get things to work that I should have been spending working on my project.
> Mostly, its my own ignorance of current magical techniques:
to write a simple, failsafe howto, and we're still not there yet.
Philip and Ian did the original work in April 2008 which led to
Appengine-Monkey and patches to get Mako and Beaker to run on GAE. I
came in later to do more installation and testing. All of us have
spent weeks pulling our hair out trying to get various things to work.
When I said "hours" for me, I knew it meant "weeks" for a few of you. I would have no hope without the work that you-all have already done.
Thanks.
(I don't have much hair left to pull)
Jason S.
considering it, please visit and "star" this bug report. More below.
http://code.google.com/p/googleappengine/issues/detail?id=2272
On Oct 15, 2:01 pm, Matt Feifarek <matt.feifa...@gmail.com> wrote:
> > When you install them with pip, I think they overlap (i.e. one paste/
> > directory in site-packages/).
>
> Yes, that's true, but I don't understand why moving those directories from
> one site-packages to another should matter. I can't find whatever cruft is
> telling python to look in that and only that sys.path location for paste.
stuff, and it looks like there is a .pth file that comes with each
Paste package (ending with the string "nspkg.pth") that contains
executable Python (!) (I didn't know pth files could have code in
them) that jiggles with the path. Weird stuff. Anyway, it wouldn't
work without those pth files, or, as you noted, simply adding an
__init__.py to paste/
> > Interesting. Maybe that step could be eliminated. I think I was
> > getting DistributionNotFound from pkg_resources and figured that it
> > would be better to work around eggs/the egg loader.
>
> I think I got around that by dropping an empty __init__.py file into paste/
GAE's os.path.exists() pretends that a directory that (a) ends in
".egg-info" and (b) is NOT located in the deployment root, i.e. is a
subdirectory, does not exist. Naturally, this screws around with the
default paster create template, which has an .egg-info directory in
it, and anything installed using pip, which uses .egg-info
directories. EGG-INFO directories are unaffected.
With this knowledge I got a stock app working without patching Paste
Deploy, and without changing the INI file, in several different ways.
The "best" way so far is to rename the outer myapp directory to
myapp.egg and rename the inner myapp.egg-info directory to EGG-INFO.
You can also leave the app template as-is and install it inside the
virtualenv with easy_install, but you'd have to do this every time you
make a change.
I have filed a bug in the GAE issue tracker about this. Please visit
this page and add a star to encourage getting this fixed/changed:
http://code.google.com/p/googleappengine/issues/detail?id=2272
Jason
Qiangning Hong
> I just did a fresh pip install of Pylons and looked over the Paste
> stuff, and it looks like there is a .pth file that comes with each
> Paste package (ending with the string "nspkg.pth") that contains
> executable Python (!) (I didn't know pth files could have code in
> them) that jiggles with the path. Weird stuff. Anyway, it wouldn't
> work without those pth files, or, as you noted, simply adding an
> __init__.py to paste/
Here comes yet another approach to Pylons on GAE problem, without virtualenv:
1. Write a script, to analysis dependencies of the working Pylons
project (via setup.py or .egg-info dir), and gather all the
dependencies of dependencies, recursively.
2. ln -s (or copy) all the dependencies python package to the root
dir, so that GAE can import them.
3. If the dependence package has a .pth associated with it, append its
content to a module located in the GAE app root dir (the same dir with
app.yaml), named pthes.py. At the beginning of runner.py, import
pthes. So that the paste package without __init__.py can be imported,
because its .pth appended 'paste' dir in sys.path.
4. ln -s (or copy) all the dependencies .egg-info dir to the root dir,
so that pkg_resource.load_entry_point could work.
5. (optional) To avoid .egg-info dirs take too much quote of file
count, we can package (and compress) all of them into a single file,
e.g. a tar.gz file or a pickle file, and patch pkg_resource to know
how to read original entry_points.txt from that file. Add some cache
mechanism if need.
This approach makes all the pkg_resource needed information available
in GAE environment, so that no any further modification needed for the
Pylons package or project, including the .ini file.
As I'm not a setuptools guru, I'd like to hear some opinions on it or
suggestions before get hands dirty.
--
Qiangning Hong
Ian Bicking
> On Thu, Oct 15, 2009 at 12:41 PM, Jason S. <jas...@tuffmail.com> wrote:
>>
>> > I still don't get setuptools. I don't understand namespace packages, or
>> > rather their deployment. I don't get why I can't move (for example)
>> > paste,
>> > and the Paste* items to another directory that is ON sys.path and have
>> > it be
>> > import-able. All of my frustrations with GAE have been of this sort.
>>
>> When you install eggs with easy_install, it adds the egg's path to
>> easy_install.pth. If you move the egg somewhere else, it won't work
>
> That part I get. In fact, I've often modified easy_install.pth manually.
>
> But I'm using pip. And pip doesn't mess with that file, nor it seems any
> other meta-data anywhere on sys.path. But still, I couldn't move Paste*
> around.
>
> I was considering trying an install with easy_install and then
> hand-modifying easy_install.pth next.
pip will respect easy_install'd packages. So if you easy_install
something the pip install it, pip will see the easy_install'd package
and do nothing. You should be sure to use virtualenv
--no-site-packages. Also pip install --ignore-installed will force a
fresh install.
I remember at some point having a problem with pip installing the
Paste packages, and paste/__init__.py was missing. I think I was
working on an actual app so I punted and created the file manually,
and didn't try to reproduce and fix it.
Note that to get .pth files respected you can't add a directory to
sys.path, instead you have to do "import site; site.addsitedir(dir)"
which also looks for .pth files. runner.py does this; if nothing else
you should look at what runner.py does. It also goes through the
necessary steps to get rid of the system webob (or other installed
libraries) if you want to use an upgraded version. It's slightly
different on the SDK and the live environment.
>> unless it's added to the path somehow. BTW, I don't think GAE
>> uses .pth files, which is why sys.path manipulation is necessary.
>
> It seems like this isn't true. If I omit pth files when uploading to GAE, my
> app is failing with a 500 error. But I wouldn't trust that yet... can you do
> a test on your setup and see if you need or don't need the .pth files? (up
> to and including easy_install)
At one time .pth files weren't being read on GAE (or maybe
appengine-monkey messed something up, I can't remember). This hasn't
been a problem for at least the past six months, or maybe a year.
>> When you install them with pip, I think they overlap (i.e. one paste/
>> directory in site-packages/).
>
> Yes, that's true, but I don't understand why moving those directories from
> one site-packages to another should matter. I can't find whatever cruft is
> telling python to look in that and only that sys.path location for paste.
>
>
>>
>> I'm not sure what all the problems you're having with setuptools
>> entail, but my app.py *prepends* to sys.path, which works a little
>> better because it gets around installed versions of libraries in the
>> GAE environment.
>
> You mean to have the upgraded webob for example?
I found prepending alone to be insufficient in some cases, instead I
scan and remove any webob directory, in addition to prepending.
>> I use the GAE SDK instead of paster serve, because then all the GAE-
>> specific libs should be provided without moving anything. Can you use
>> datastore, etc, locally without the SDK?
>
> Yes. I simply add the path to the google libraries to sys.path; I'm running
> DataStore (well, whatever emulation that the SDK provides on my "paster
> serve development.ini" pylons app. Which means I can use the debugger,
> webtest, etc.
I don't use paster serve, and wouldn't recommend it. You can/should
use paste.deploy.loadapp('development.ini') in your main.py or
runner.py or whatever script, and then use dev_appserver.
> Can you use the debugger (weberror)?
I have gotten it to work, but by overwriting
environ['wsgi.multiprocess'] (to set it to False) -- without that the
debugger won't work. But in the SDK generally it won't be
multiprocess until you edit a file, so the debugger works fairly
reliably.
Generally, I'll note one advantage to GAE that you should make use of:
once a set of libraries is working, everything to make them work is in
a set of portable files (.py files and directories and whatnot).
After installing I check *all* the files into version control. I
don't do this elsewhere because there are C extensions and other
things that aren't portable, but GAE disallows all those so they
aren't an issue. It would be relatively easy to create an actual
tar/zip of everything for a working Pylons setup, and anyone could use
it as a starting point, and it would be 100% reliable. Unfortunately
that's only good enough for a specific set of libraries, and either
you have to repackage a collection of different library combinations
or you have to support installation in some manner. appengine-homedir
is one attempt to get installation working reasonably. The intent of
appengine-homedir is also that you check app/ into version control
(but not the enclosing virtualenv environment). It would be kind of
nice if app/ had some script included to recreate the enclosing
virtualenv environment.
Another detail I guess is what your app is named -- each person might
name it differently, and that effects the file locations, so makes a
simple tar/zip file harder. Are app names worth it? I don't know...
could you name them all "app" and be done with it? If you start
combining multiple apps it might matter, but people haven't done that
much (though it is possible, and several GAE tools are distributed
that way). Also I mostly got rid of configuration, as GAE apps aren't
generally "configured" (due to the static nature of the deployment).
But that broke paster controller and paster shell (though with what
sitecustomize does paster shell isn't as necessary). Restoring
development.ini, or allowing it as an option, would be easy enough to
do.
Matt Feifarek
I remember at some point having a problem with pip installing the
Paste packages, and paste/__init__.py was missing. I think I was
working on an actual app so I punted and created the file manually,
and didn't try to reproduce and fix it.
Yes, it seems to be necessary in all of my experiments to create that file to make paste/ into a true package. No big deal.
I don't use paster serve, and wouldn't recommend it. You can/should
use paste.deploy.loadapp('development.ini') in your main.py or
runner.py or whatever script, and then use dev_appserver.
Would you mind expanding on that a little bit? Why don't you recommend it?
> Can you use the debugger (weberror)?I have gotten it to work, but by overwriting
environ['wsgi.multiprocess'] (to set it to False) -- without that the
debugger won't work. But in the SDK generally it won't be
multiprocess until you edit a file, so the debugger works fairly
reliably.
Interesting. I'll have to try that.
Generally, I'll note one advantage to GAE that you should make use of:
once a set of libraries is working, everything to make them work is in
a set of portable files (.py files and directories and whatnot).
After installing I check *all* the files into version control.
Cool. Good concept. But it seems like once you do something like that (or like moving your libraries into the GAE "app") you can't manage the virtualenv install anymore (like adding new distributions with pip). I suppose one can always manually add them.
It would be relatively easy to create an actual
tar/zip of everything for a working Pylons setup, and anyone could use
it as a starting point, and it would be 100% reliable. Unfortunately
that's only good enough for a specific set of libraries, and either
you have to repackage a collection of different library combinations
or you have to support installation in some manner.
Still, a "stock" pylons app could be hosted on BitBucket; just clone and run. Cool idea.
appengine-homedir
is one attempt to get installation working reasonably. The intent of
appengine-homedir is also that you check app/ into version control
(but not the enclosing virtualenv environment). It would be kind of
nice if app/ had some script included to recreate the enclosing
virtualenv environment.
It *seemed* to me that once appengine-homedir got running, the VE is basically moot; you no longer run the devappserver from it, right? Since your runner.py script does all of the sys.path manipulations, there's no point in keeping the VE bin and lib and what-not around, especially if you don't recommend using paster serve.
Another detail I guess is what your app is named -- each person might
name it differently, and that effects the file locations, so makes a
simple tar/zip file harder. Are app names worth it? I don't know...
Since each GAE is insulated in its own namespace, if the whole world named their apps "app" it wouldn't matter, IMO.
that way). Also I mostly got rid of configuration, as GAE apps aren't
generally "configured" (due to the static nature of the deployment).
But that broke paster controller and paster shell (though with what
sitecustomize does paster shell isn't as necessary). Restoring
development.ini, or allowing it as an option, would be easy enough to
Why isn't paster shell necessary? Seems like sitecustomize.py is mostly about fixing imports and shim'ing things into sys.path.
Mike Orr
> On Fri, Oct 16, 2009 at 1:04 PM, Ian Bicking <ia...@colorstudy.com> wrote:
>> I don't use paster serve, and wouldn't recommend it. You can/should
>> use paste.deploy.loadapp('development.ini') in your main.py or
>> runner.py or whatever script, and then use dev_appserver.
>
> Would you mind expanding on that a little bit? Why don't you recommend it?
I'm note sure if he meant "never use paster serve" or "just don't use
it for App Engine apps".
> It *seemed* to me that once appengine-homedir got running, the VE is
> basically moot; you no longer run the devappserver from it, right?
It's mainly used for installing packages. You may realize later you
need a different package, or want to upgrade the ones you have.
Without virtualenv, you'll have to have a global copy of Setuptools
installed, and will have to specify the destination path every time
you install something (because you're not installing into the default
site-packages any more). Or create a distutils.cfg specifiying the
install location.
--
Mike Orr <slugg...@gmail.com>
Ian Bicking
>> I don't use paster serve, and wouldn't recommend it. You can/should
>> use paste.deploy.loadapp('development.ini') in your main.py or
>> runner.py or whatever script, and then use dev_appserver.
>
> Would you mind expanding on that a little bit? Why don't you recommend it?
On GAE specifically, dev_appserver does all the fixups and security
restrictions to make it a mostly representative environment. So you
should just use dev_appserver.
>> Generally, I'll note one advantage to GAE that you should make use of:
>> once a set of libraries is working, everything to make them work is in
>> a set of portable files (.py files and directories and whatnot).
>> After installing I check *all* the files into version control.
>
> Cool. Good concept. But it seems like once you do something like that (or
> like moving your libraries into the GAE "app") you can't manage the
> virtualenv install anymore (like adding new distributions with pip). I
> suppose one can always manually add them.
pip install works with appengine-homedir. A couple things are
installed outside of the app/ directory, but once it's setup there's a
file in lib/python2.5/distutils/distutils.cfg that makes everything
install into app/lib/python.
When I update a library or install something new, I just run another
commit after.
>> appengine-homedir
>> is one attempt to get installation working reasonably. The intent of
>> appengine-homedir is also that you check app/ into version control
>> (but not the enclosing virtualenv environment). It would be kind of
>> nice if app/ had some script included to recreate the enclosing
>> virtualenv environment.
>
> It *seemed* to me that once appengine-homedir got running, the VE is
> basically moot; you no longer run the devappserver from it, right? Since
> your runner.py script does all of the sys.path manipulations, there's no
> point in keeping the VE bin and lib and what-not around, especially if you
> don't recommend using paster serve.
I recommend the venv for running python interactively, running tests,
and installing things.
>> that way). Also I mostly got rid of configuration, as GAE apps aren't
>> generally "configured" (due to the static nature of the deployment).
>> But that broke paster controller and paster shell (though with what
>> sitecustomize does paster shell isn't as necessary). Restoring
>> development.ini, or allowing it as an option, would be easy enough to
>
> Why isn't paster shell necessary? Seems like sitecustomize.py is mostly
> about fixing imports and shim'ing things into sys.path.
It also sets up the data store, so you can access and use your models.
It doesn't set up the entire configuration of your app, but the most
important configuration is generally persistence, which sitecustomize
handles. And maybe it could even handle more if it was useful to do
so.