mod_python
uuellbee
and the Django documentation, I finally got a mod_python setup working.
In the end, it's pretty simple, but figuring it out wasn't much fun. I
added an example to the bottom of the wiki page. I did that instead of
posting it here because I figure that is the first place most people
will look.
The example there doesn't run at the site root. I figured out that you
can run a project at the site root by removing all the DocumentRoot
stuff (leaving it in was causing infinite redirection) and just having
something like this in the top-level of the VirtualHost container:
SetHandler mod_python
PythonHandler mod_python.wsgi
PythonPath "['/path/to/project'] + sys.path"
PythonOption wsgi.application startup::app
With this setup you don't even have to do anything special to get at
stuff in the Pylons public directory. In my template I just have things
like href="/css/base.css". In the wiki example, I created an alias to
the public dir and the equivalent href is
"/project/public/css/base.css".
The only problem with this is when I go to http://example.com/ I get an
extra slash added at the end: http://example.com//. If I go to
something below the root, http://example.com/whatever, I don't get the
extra slash. The slash doesn't seem to affect anything, but it doesn't
look right.
References:
http://pylonshq.com/project/pylonshq/wiki/ModPython
http://groups.google.com/group/pylons-discuss/browse_thread/thread/1edda8b34f13511d/2aed674d12a9e53f
http://www.djangoproject.com/documentation/modpython/
Ben Bangert
> After reading the wiki page about mod_python, threads in this group,
> and the Django documentation, I finally got a mod_python setup working.
> In the end, it's pretty simple, but figuring it out wasn't much fun. I
> added an example to the bottom of the wiki page. I did that instead of
> posting it here because I figure that is the first place most people
> will look.
I agree, I switched the PylonsHQ site over yesterday, and there were
some quirks getting it to run at the site root.
> The example there doesn't run at the site root. I figured out that you
> can run a project at the site root by removing all the DocumentRoot
> stuff (leaving it in was causing infinite redirection) and just having
> something like this in the top-level of the VirtualHost container:
Here's what my eventual setup looked like. This setup assumes you have
Apache 2.0+ and mod_python.
First, the mod_python.wsgi script the Wiki points to needs a minor
change. This is because it doesn't properly handle the case when
SCRIPT_NAME isn't set in your mod_python config. The following lines
need to be added to the file around line 110 so that the section
dealing with SCRIPT_NAME looks like this:
111 if 'SCRIPT_NAME' not in options:
112 options['SCRIPT_NAME'] = ''
113
114 if 'SCRIPT_NAME' in options:
115 # Override SCRIPT_NAME and PATH_INFO if requested.
116 env['SCRIPT_NAME'] = options['SCRIPT_NAME']
117 env['PATH_INFO'] =
req.uri[len(options['SCRIPT_NAME']):]
118
119 env['wsgi.input'] = InputWrapper(req)
This way the script will properly deal with being at the root. I
decided to avoid the .htaccess thing, so this is what the full
VirtualHost looks like for PylonsHQ. Keep in mind I'm also serving
non-Pylons content, which you might want to do as well (svn, trac,
etc):
<VirtualHost *:80>
ServerAdmin EMAIL_ADDRESS
ServerName pylonshq.com
DocumentRoot /home/pylonshq/pylonshq # ROOT OF THE PYLONSHQ
PROJECT
CustomLog "/var/log/httpd/pylonshq.com/access_log" clf
ErrorLog "/var/log/httpd/pylonshq.com/error_log"
Alias /WebHelpers "/usr/local/www/pylonshq/WebHelpers"
Alias /trac "/usr/local/share/trac/htdocs"
Alias /files "/usr/local/www/pylonshq/files"
Alias /project/pylonshq
/usr/local/share/trac/cgi-bin/trac.fcgi/pylonshq
Alias /project /usr/local/share/trac/cgi-bin/trac.fcgi/pylonshq
# REMOVE THIS IF YOU ARENT USING mod_cache + mod_disk_cache
CacheRoot /tmp/apachecache
CacheSize 51200
CacheEnable disk /style
CacheEnable disk /img
CacheEnable disk /scripts
CacheEnable disk /trac/js
CacheEnable disk /trac/css
<Location />
SetHandler mod_python
PythonHandler mod_python.wsgi
PythonPath
"['/home/pylonshq/pylonshq','/home/pylonshq/lib/python2.4/'] +
sys.path"
PythonAutoReload Off
PythonOption wsgi.application startup::app
# Insert filter, REMOVE THIS IF YOU ARENT USING mod_deflate
SetOutputFilter DEFLATE
BrowserMatch ^Mozilla/4 gzip-only-text/html
BrowserMatch ^Mozilla/4\.0[678] no-gzip
BrowserMatch \bMSI[E] !no-gzip !gzip-only-text/html
SetEnvIfNoCase Request_URI \
\.(?:gif|jpe?g|png)$ no-gzip dont-vary
Header append Vary User-Agent env=!dont-vary
</Location>
<LocationMatch /(trac|WebHelpers|RailsHelpers|files).*>
# HERE, WE TURN OFF PYLONS FOR THE OTHER LOCATIONS
SetHandler None
</LocationMatch>
<LocationMatch /project>
# TRAC IS RUNNING THROUGH FASTCGI
SetHandler fastcgi-script
Options +ExecCGI
</LocationMatch>
<Location "/project/pylonshq/login">
# TRAC's LOGIN FILE
AuthType Digest
AuthName "PylonsHQ"
AuthDigestFile /somewhere.htpasswd
Require valid-user
</Location>
<Location /svn>
DAV svn
SVNPath /home/svn/pylons
AuthType Digest
AuthName Pylons
AuthDigestFile /somewhere_else.htpasswd
<LimitExcept GET PROPFIND OPTIONS REPORT>
Require valid-user
</LimitExcept>
</Location>
</VirtualHost>
And the startup.py script that is inside /home/pylonshq/pylonshq:
from paste.deploy import loadapp
import os
import site
site.addsitedir('/home/pylonshq/lib/python2.4/site-packages/')
heredir = os.path.dirname(__file__)
app = loadapp("config:%s" % os.path.join(heredir, 'pylonshq.ini'))
So, things to note.
I'm not having Apache serve the projects public files (static media)
directly. This is because having Pylons serve static content is rather
powerful, and it can load static content from multiple paths at once,
capabilities HTTP servers like Apache and Lighttpd do not have. To
compensate for performance, I have mod_cache/mod_disk_cache caching the
content under the URL's (/img, /styles, etc) so that it significantly
lowers the load on the Python app and I get rather nice speed out of
it. (Note, the release version of Paste does not set Last-Modified
headers, upgrade to Paste==dev if you're using mod_cache so that the
Last-Modified headers are set)
I have the SetHandler for the root in a Location block. This is to
ensure that when Apache checks priorities on how to serve content under
various URL's, I can disable Pylons for the URL root's that I want Trac
to handle. You can add additional Location blocks for other URL spaces
that you'd like handled differently as well, and it makes it pretty
clean in my opinion.
The /home/pylonshq is a user by the name of 'pylonshq'. Sorry if that
caused some confusion. This user has a virtual Python setup, thus the
'import site, and site.addsitedir' lines in the startup.py. If you're
not using a virtual Python setup, those lines are not needed.
I'm using a virtual user to run the site so that I can test out updates
and such without worrying about whats system installed. If you have
multiple virtual hosted sites running with multiple virtual Python
package installs, you *should* use the PythonInterpreter option to have
each virtual host use a different Python interpreter. That way you can
avoid having Python path conflicts loading un-intended versions of a
module. (Note that this is also suggested for multiple Django sites as
Django though for a different reason).
I will be turning this into a doc, and likely the *recommended*
deployment method for Pylons (and probably Paste apps in general). I
also plan on updating the WSGI script so that the startup.py file is
not needed. Maybe a wsgi_paste script that lets you set the ini file in
the Apache config.
Any thoughts, questions?
Cheers,
Ben
ToddG
multiple apps? Is just another apache block/stanza with the appropriate
dir in PythonPath all that's needed?
[As an (old) Ian blog post discussed, this is something PHP makes super
easy and a common situation for many -- lots of scattered disparate
little applications rather than monolithic, uber-apps. But then there's
also the paste composite app thing... hmm. When to use which? Yes a
different topic, sorry!]
I think I'm veering into apache/modpy issues, not Pylons specifically.
Going further, am I correct in assuming if you have 3 apache modpy
children, the interp + app runs threaded in each? So in theory you're
getting a lot more request-handling from those 3 than from 3 modperl
processes which aren't running threaded?
I am probably going to be proxying to back-end modpy's like the common
modperl setup, so I'm trying to relate the different processes/threaded
issues ... thanks as usual.
Martin
1) Why did you use mod_python instead of SCGI? I thought SCGI was fast
and easy to deploy (what I've heard... no experience to back that up)
2) How come you chose apache? Most rails-people claim that Lighttpd
serves scripts/apps faster than apache.
I'm just curious, because I haven't deployed my app yet and so I'm
completely free in what I choose.Therefore it's interesting to hear
other people's reasons for their setup.
Cheers, Martin
ToddG
1) Many people like the convenience of one process to watch, i.e. the
main apache process [that monitors its own child processes]. Using SCGI
means you have externals processes to watch in addition to apache.
Though if you use a multi-apache setup for lightweight and modpython
apaches you have mulitple processes anyhow -- but many people are more
familiar with supporting good ol' 'httpd' as opposed to standalone
python processes.
2) People are already running other apps/sites largely dependent on
apache. Most of the Rails folk looked to lighttpd (and similar) because
Apache1/2 FastCGI support is apparently quite flakey, with mysterious
zombie processes and so on. Lighttpd offers better FCGI support so that
was a common solution.
Now that Mongrel exists (a standalone HTTP-talking ruby/C server),
people are migrating back to Apache (2.2.x) to take advantage of its
mod_proxy support for load-balancing -- to distribute load to multiple
Mongrel procs on the same or different servers. Also modruby (if still
in existence) is not often used, so that whole option (equiv of
modpython/modperl) is not there; Mongrel is [very] roughly the
equivalent of modpy/perl (without the apache part of course).
Ben Bangert
> Also have some questions?
>
> 1) Why did you use mod_python instead of SCGI? I thought SCGI was fast
> and easy to deploy (what I've heard... no experience to back that up)
mod_scgi and mod_fcgi don't get along. I'm using fcgi with Trac, as
running Trac through mod_python doesn't work (subversion binding issue
in 1.3.0 and Trac, which is still not fixed). Also, Apache can't check
FCGI or SCGI processes very well to find out when the process has
stopped responding (which has happened on occasion). Generally though,
I've heard of SCGI setups working quite well, and I believe Ian Bicking
uses that setup. I'd be interested in seeing his setup and
configuration. :)
> 2) How come you chose apache? Most rails-people claim that Lighttpd
> serves scripts/apps faster than apache.
lighttpd is still rather immature for a project, and has some issues. I
have no doubt it is faster, but since I have only one IP on my hosting
machine, this means I'd need to proxy into the lighttpd app from Apache
due to the other sites I'm serving up from Apache. This complicates the
configuration and a marginal speedup is not worth the hassle. Lighttpd
also doesn't have the caching abilities I noted.
Apache is an incredibly mature technology, and having the knowledge
that when Apache is up, my site is up is quite nice. As ToddG notes,
its a bit more reliable as well because the main Apache process watches
its children and if one of them dies, starts up a new one. It's a setup
that results in a shorter configuration, and less places for something
to go wrong, which means should something go wrong, its typically
faster to remedy.
There is the issue of Apache being a little heavier-weight as a result
of mod_python, but in my experience its pretty minimal, especially for
an easier setup. If you have to deal with sysadmin at a company,
they'll definitely feel more comfortable with Apache + mod_python for
similar reasons. Trying to deploy a multi-app setup (Apache + lighttpd
+ SCGI/FCGI process), across a cluster of half a dozen machines is
substantially more of a pain than just Apache.
Cheers,
Ben
Martin
Martin
Ian Bicking
> On Aug 29, 2006, at 2:23 PM, Martin wrote:
>
>> Also have some questions?
>>
>> 1) Why did you use mod_python instead of SCGI? I thought SCGI was fast
>> and easy to deploy (what I've heard... no experience to back that up)
>
> mod_scgi and mod_fcgi don't get along. I'm using fcgi with Trac, as
> running Trac through mod_python doesn't work (subversion binding issue
> in 1.3.0 and Trac, which is still not fixed). Also, Apache can't check
> FCGI or SCGI processes very well to find out when the process has
> stopped responding (which has happened on occasion). Generally though,
> I've heard of SCGI setups working quite well, and I believe Ian Bicking
> uses that setup. I'd be interested in seeing his setup and
> configuration. :)
Note that I primarily use the cgi2scgi script included with SCGI. This
way the server acts as a CGI script, but because cgi2scgi is a very
small bit of C code it has decent performance. This does cause some
problems, as Apache won't pass certain methods to CGI properly, and
certain headers (e.g., Authentication). It's still a bit slower than
mod_scgi or some other technique. This setup doesn't keep the server up
or responding.
--
Ian Bicking | ia...@colorstudy.com | http://blog.ianbicking.org