Setting PYTHONPATH in a per-VirtualHost basis
Alberto
I've been playing with mod_wsgi recently and looks very exciting. :)
I've been wondering if there is an apache config directive to set a
PYTHONPATH in a per VirtualHost basis or there are plans to implement
it.
This feature is/would-be great in shared hosting environments in order
to better isolate client's installations so installed libaries can
differ in version.
Thanks for mod_wsgi!
Alberto
Graham Dumpleton
>
> Hi,
>
> I've been playing with mod_wsgi recently and looks very exciting. :)
>
> I've been wondering if there is an apache config directive to set a
> PYTHONPATH in a per VirtualHost basis or there are plans to implement
> it.
FYI, unlike mod_python, the default in mod_wsgi is one Python sub
interpreter per WSGI application script. This means that each WSGI
application is isolated from each other. If you wanted all WSGI
applications hosted under the one VirtualHost to run within the same
Python sub interpreter, you would need to set:
WSGIApplicationGroup %{SERVER}
Note though that if multiple ports are being listened to under the one
VirtualHost, they are still separate. Only exception there is that
80/443 ports are seen as the one. See configuration directives
documentation for more detail.
> This feature is/would-be great in shared hosting environments in order
> to better isolate client's installations so installed libaries can
> differ in version.
Because each WSGI application script is isolated by default, the
script itself can set sys.path to what ever it wants. Thus at global
scope within the script you can use:
import sys
sys.path.append('/usr/local/django')
Thus each script has total control over the path already.
Do note though that if the script file itself is part of the
application and would be changed on a regular basis, with it being
reloaded when changes occur, you shouldn't append to sys.path without
checking if the path is already in there, else every reload will see
it added again. Thus use:
import sys
if not '/usr/local/django' in sys.path:
sys.path.append('/usr/local/django')
Thus there is no need for an Apache directive for setting the path as
the script can do it. Trying to do it using an Apache directive
actually gets quite messy and has caused lots of problems in
mod_python at times, so there is not intention of adding a directive.
Have to run now, but if you have more questions let me know.
Graham
Alberto Valverde
Thanks for the detailed explanation. I've managed to write a startup
script for a Pylons app inside a (slightly hacked) workingenv and got
it to run under mod_wsgi.
Now time to submit some patches and writer a recipe...
Thanks :)
Alberto
Alberto Valverde
FYI, I've just written a short howto explaining how I've done it:
http://docs.pythonweb.org/pages/viewpage.action?pageId=5439610
Thanks for the help
Alberto
Graham Dumpleton
Some corrections to your page.
You said:
"""mod_wsgi provides an independent interpreter for each VirtualHost
by default"""
As I mentioned in my original followup, the default for mod_wsgi is a
separate Python sub interpreter for each WSGI application script. Thus
your statement about one sub interpreter per VirtualHost is wrong.
Thus, if you actually had:
<VirtualHost *:80>
WSGIScriptAlias /mysite-1 /Users/alberto/wsgi/myapp-1.wsgi
WSGIScriptAlias /mysite-2 /Users/alberto/wsgi/myapp-2.wsgi
</VirtualHost>
the WSGI applications for /mysite-1 and /mysite-2 run in different
Python sub interpreters automatically without you doing anything.
You could automate the above a bit by also using:
<VirtualHost *:80>
WSGIScriptAliasMatch /[^/]+ /Users/alberto/wsgi/$1.wsgi
</VirtualHost>
Thus, without even restarting Apache, you can add another application
to your site just by placing the .wsgi file in your directory.
Importantly, even though the one directive is used to set up
potentially many sites in one go, each WSGI application script file
still runs in a separate sub interpreter.
Thus, unless you specifically go out to override the default behaviour
of mod_wsgi, each WSGI application script is in a separate sub
interpreter and is isolated from all other applications. They thus
each have your own sys.path, own separate Python modules and thus
potentially different versions of a particular Python egg.
I don't really understand what your workingenv.py module is all about,
but because of the above isolation which you get without doing
anything, it may well be redundant as you just need to create a new
.wsgi script for any additional application and in that script set
sys.path etc specific to that application as needs be.
Your patch to workingenv.py seems to be potentially wrong anyway as it
assumes that PYTHONPATH contains only a single directory when it can
have more than one separated by ':'.
Can you perhaps point me at documentation on what this workingenv.py
is all about?
I'd like to get this sorted out rather than that page be up there and
possibly give the impression that using mod_wsgi is harder than it is
for what you are trying to do?
As to commodity hosting, even mod_wsgi is still not capable enough for
that as everything still runs in the Apache child processes. I am
though working on extra capabilities at the moment which will allow
farming off different users applications into distinct child process.
Ie., like mod_fastcgi and mod_scgi but much simpler as no need to
install any separate software as all done by mod_wsgi itself with only
the addition of a couple of extra directives to the Apache
configuration file.
Graham
Graham Dumpleton
> I don't really understand what your workingenv.py module is all about,
> but because of the above isolation which you get without doing
> anything, it may well be redundant as you just need to create a new
> .wsgi script for any additional application and in that script set
> sys.path etc specific to that application as needs be.
Okay, not perhaps totally redundant now that I have found some
documentation on it, but can you post a copy of an "activate" script
that it generates so I can see what it looks like. I can't generate
one myself as use Python 2.3.5 and workingenv.py requires newer
version of Python to run.
Overall, workingenv.py looks like it should be able to work hand in
hand with mod_wsgi quite well as long as you have the ability to
activate it from inside Python script as your patch tries to do.
Better still because of fact as I have pointed out that each script
has its own sub interpreter by default.
Just a pain that workingenv.py will not work with older version of
Python as have no way of trying it myself as don't want to screw my
standard OS install of Python on Mac OS X. :-(
Graham
Alberto Valverde
On Apr 9, 2007, at 4:32 AM, Graham Dumpleton wrote:
>
> On 09/04/07, Graham Dumpleton <graham.d...@gmail.com> wrote:
>> I don't really understand what your workingenv.py module is all
>> about,
>> but because of the above isolation which you get without doing
>> anything, it may well be redundant as you just need to create a new
>> .wsgi script for any additional application and in that script set
>> sys.path etc specific to that application as needs be.
>
> Okay, not perhaps totally redundant now that I have found some
> documentation on it, but can you post a copy of an "activate" script
> that it generates so I can see what it looks like. I can't generate
> one myself as use Python 2.3.5 and workingenv.py requires newer
> version of Python to run.
I've changed the doc I've mentioned correcting my sub-interpreter
confusion you've pointed out in your previous post and describing an
alternate setup which doesn't require patching workingenv.py at all.
I've found a script that can activate a workingenv without any
patches here [1].
>
> Overall, workingenv.py looks like it should be able to work hand in
> hand with mod_wsgi quite well as long as you have the ability to
> activate it from inside Python script as your patch tries to do.
> Better still because of fact as I have pointed out that each script
> has its own sub interpreter by default.
>
> Just a pain that workingenv.py will not work with older version of
> Python as have no way of trying it myself as don't want to screw my
> standard OS install of Python on Mac OS X. :-(
I wasn't aware of that :( Haven't touched a 2.3 for since a long time
ago... If you want to try it out email me privately and I can set you
up an account at a dev. server where I have mod_wsgi + python2.4
installed so you can tinker.
Alberto
[1] http://swapoff.org/wiki/blog/2007-03-20-activating-a-workingenv-
from-python
Graham Dumpleton
Looks good. Thanks for putting the instructions together. I have added
link from the Pylons page on the mod_wsgi wiki:
http://code.google.com/p/modwsgi/wiki/IntegrationWithPylons
Now if we can only get this issue sorted out for when using Pylons:
http://code.google.com/p/modwsgi/issues/detail?id=12&can=2&q=
The 'Interpreter' reload option if you haven't worked it out from
documentation allows you to touch the script file, changing its
modification time, and mod_wsgi will delete the sub interpreter and
reload the whole WSGI application without needing to restart Apache.
Unfortunately as the issue describes, the pyrex generated code in
PyProtocols seems to cause problems. :-(
When I finish implementing the separate daemon process feature will
not be such a big deal as then you can dedicate a whole process just
to the application and just send it a SIGHUP to have the whole process
recycled.
Graham
Graham Dumpleton
>
> >
> > On 09/04/07, Graham Dumpleton <graham.d...@gmail.com> wrote:
> >> I don't really understand what your workingenv.py module is all
> >> about,
> >> but because of the above isolation which you get without doing
> >> anything, it may well be redundant as you just need to create a new
> >> .wsgi script for any additional application and in that script set
> >> sys.path etc specific to that application as needs be.
> >
> > Okay, not perhaps totally redundant now that I have found some
> > documentation on it, but can you post a copy of an "activate" script
> > that it generates so I can see what it looks like. I can't generate
> > one myself as use Python 2.3.5 and workingenv.py requires newer
> > version of Python to run.
>
> I've changed the doc I've mentioned correcting my sub-interpreter
> confusion you've pointed out in your previous post and describing an
> alternate setup which doesn't require patching workingenv.py at all.
> I've found a script that can activate a workingenv without any
> patches here [1].
One more point about your document. Where you say:
"""Uses the activate_workingenv script which you'll need to install
somewhere in your PYTHONPATH""
You may want to describe this a bit differently. The reason is that
Apache generally runs as a special user and so an individuals
PYTHONPATH as may be set in their user environment will not be used.
Thus, would be best to explicitly say that it needs to have been
installed into Python site-packages directory. Alternatively, give
example of extending sys.path within the script file itself to give
the location of where the users additional modules are held.
Graham
Alberto Valverde
Done. I've also modified the activate_workingenv.py script to no-op
if the workingenv lib dir is already in sys.path so it doesn't get
"re-activated" when the script is reloaded.
Thanks,
Alberto