mod_python and paste apps

[Nicholas Piël]

Hi,

I wrote a small tutorial that explains how to connect your Pylons app
to Mod_Python. You can find the tutorial here: http://pylonshq.com/
project/pylonshq/wiki/ModPython

I made a small adjustment to the ModPythonGateway found over here:
http://projects.amor.org/misc/wiki/ModPythonGateway The resulting
'gateway' allows you to easily connect any paste app to mod_python.

Cheers,
Nicholas

[James Gardner]

Dear Nicholas,

Great job, thank you!

I've followed the instructions and noticed a few things I think might
need correcting:

Should the handler be ``PythonHandler mod_python.wsgi`` rather than
``PythonHandler modpython_gateway`` since that is what your instructions
recommend saving it as?

I found setup.py needed to somewhere on your PYTHONPATH rather than the
project directory, perhaps this could be explained?

If you use the old wsgi.py you posted there seems to be an extra `if``
on line 193 of wsgi.py. Should we now use modpython_gateway instead? In
which case probably best to remove the instructions to rename it to
wsgi.py which might be causing problem 1.?

What should you write if you want the application mounted at the URL
root rather than /imagesearch, I find that adding / as the SCRIPT_NAME
leads to infinite recursions. Any thoughts? How can you set a blank
SCRIPT_NAME?

Also, for those with a virtual Python install, if you add something
similar to this at the top of startup.py then mod_python will use the
libraries setup by easy_install in your virtual Python lib directory.
Might be handy for some people!

import site
site.addsitedir('/path/to/home/lib/python2.4/site-packages/')

Thanks again for your tutorial, really great stuff!

James

[Philip Jenvey]

On Jul 9, 2006, at 4:33 PM, James Gardner wrote:

>
> What should you write if you want the application mounted at the URL
> root rather than /imagesearch, I find that adding / as the SCRIPT_NAME
> leads to infinite recursions. Any thoughts? How can you set a blank
> SCRIPT_NAME?
>

The SCRIPT_NAME for an app deployed at the root should be '' (empty
string) instead of '/'.

I don't know if that's the cause of the recursion, though.

--
Philip Jenvey

[Jose Galvez]

Thought I would give this a try. I've got my website set up using a
virtual host, and when I followed the instructions in thw wiki, using ''
for the script_name I get:
1) infinite recursion unless I explicitly name the correct context (I
have a default context which get called for root calls
2) its having trouble getting files from my public folder, It can get
css files just fine, but gif in the folder can't be served.
3) I also set up an alias in apache that I use to serve lots of static
content, which can not be found at all now.

FYI the site works just fine with the paster http server as well as scgi_mod

Jose

[Nicholas Piël]

On Jul 10, 2006, at 1:33 AM, James Gardner wrote:

> I've followed the instructions and noticed a few things I think might
> need correcting:
>
> Should the handler be ``PythonHandler mod_python.wsgi`` rather than
> ``PythonHandler modpython_gateway`` since that is what your
> instructions
> recommend saving it as?

Yup "PythonHandler mod_python.wsgi" is correct! Halfway through the
writing of the tutorial i changed my mind not to hack up the wsgi
gateway. I therefor changed parts of it and this caused all the small
bugs. Sorry for that.

>
> I found setup.py needed to somewhere on your PYTHONPATH rather than
> the
> project directory, perhaps this could be explained?

Well, i did not notice any path problems but if you do you can add
the following mod_python directive to your .htaccess:

PythonPath "['/home/httpd/vhosts/nichol.as/httpdocs/imagesearch'] +
sys.path"

> If you use the old wsgi.py you posted there seems to PythonPath "['/
> home/httpd/vhosts/nichol.as/httpdocs/imagesearch'] + sys.path"be an

> extra `if``
> on line 193 of wsgi.py. Should we now use modpython_gateway
> instead? In
> which case probably best to remove the instructions to rename it to
> wsgi.py which might be causing problem 1.?

Yeah, i would recommend to take just this approach. I changed the
wiki to reflect this.

> What should you write if you want the application mounted at the URL
> root rather than /imagesearch, I find that adding / as the SCRIPT_NAME
> leads to infinite recursions. Any thoughts? How can you set a blank
> SCRIPT_NAME?

Well, i *needed* the SCRIPT_NAME since Apache wasn't able to find it
out on it's own and caused a recursion. If you do not want a
SCRIPT_NAME you can just remove that line from your .htaccess

Btw James, how did your Pylons talk on EuroPython go? I noticed that
Pylons wasn't mentioned in the webframeworkshootout. Thats a pity :/

Cheers,
Nicholas

[Nicholas Piël]

Hi Jose,

I think you are close to actually have it working but that the
caching of mod_python is causing the confusion. This can be caused
that some threads / processes already reflect the changes and others
still show the old config.

What you should do is STOP your httpd. (Really stop it don't just
restart it). I normally use "/sbin/services httpd stop" and keep
doing it until i see 'failed' ;) after which i restart it.

Hope that solves your problems!

[ja...@pythonweb.org]

Hi Nicholas,

Thanks for the update.

> Well, i *needed* the SCRIPT_NAME since Apache wasn't able to find it
> out on it's own and caused a recursion. If you do not want a
> SCRIPT_NAME you can just remove that line from your .htaccess

If I remove it I get a recursion too. Do you know a way of mounting the
application at the root URL without getting the recursion? It might be it
is a bug and I just have to work through the code which I'm happy to do
unless you know of a solution?

> Btw James, how did your Pylons talk on EuroPython go? I noticed that
> Pylons wasn't mentioned in the webframeworkshootout. Thats a pity :/

The talks went well thank you, although I think non-native English speakers
had a hard time understanding me so for future talks I'll prepare less
content and speak more slowly! Pylons is definitely considered up there
with Zope, Django, and TurboGears and the other frameworks recognise the
advantage of WSGI and I think will make more efforts to support it in
future so that's great. Yes, unfortunately I didn't present my 20 min wiki
but there was a lot of interest so I've promised to do a screencast and
email it to the EuroPython list. I'll cc it here too.

Cheers,

James

--------------------------------------------------------------------
mail2web - Check your email from the web at
http://mail2web.com/ .

[Nicholas Piël]

On Jul 10, 2006, at 11:20 AM, ja...@pythonweb.org wrote:

>> Well, i *needed* the SCRIPT_NAME since Apache wasn't able to find it
>> out on it's own and caused a recursion. If you do not want a
>> SCRIPT_NAME you can just remove that line from your .htaccess
>
> If I remove it I get a recursion too. Do you know a way of mounting
> the
> application at the root URL without getting the recursion? It might
> be it
> is a bug and I just have to work through the code which I'm happy
> to do
> unless you know of a solution?

Hmmm the following works for me:

SetHandler mod_python
PythonHandler mod_python.wsgi
PythonOption wsgi.application startup::app
PythonOption SCRIPT_NAME /imagesearch

I put this in a '.htaccess' in the same directory development.ini and
setup.py is in. About script_name, if i do not add a a leading slash
i get an error, when i add a slash to the end Pylons gives me an
error and when i omit it i get recursion.

Remember that when you do not have 'PythonDebug On' in your .htaccess
you need to stop and start your webserver if you really want to see
if your changes had any effect. This can be quite confusing.

>> Btw James, how did your Pylons talk on EuroPython go? I noticed that
>> Pylons wasn't mentioned in the webframeworkshootout. Thats a pity :/
>
> The talks went well thank you, although I think non-native English
> speakers
> had a hard time understanding me so for future talks I'll prepare less
> content and speak more slowly! Pylons is definitely considered up
> there
> with Zope, Django, and TurboGears and the other frameworks
> recognise the
> advantage of WSGI and I think will make more efforts to support it in
> future so that's great. Yes, unfortunately I didn't present my 20
> min wiki
> but there was a lot of interest so I've promised to do a screencast
> and
> email it to the EuroPython list. I'll cc it here too.

cool!

[Nicholas Piël]

On Jul 10, 2006, at 11:20 AM, ja...@pythonweb.org wrote:

>> Well, i *needed* the SCRIPT_NAME since Apache wasn't able to find it
>> out on it's own and caused a recursion. If you do not want a
>> SCRIPT_NAME you can just remove that line from your .htaccess
>
> If I remove it I get a recursion too. Do you know a way of mounting
> the
> application at the root URL without getting the recursion? It might
> be it
> is a bug and I just have to work through the code which I'm happy
> to do
> unless you know of a solution?

I think i misunderstood your first question. When you want to make
your controller available at the root url. Ie like "http://nichol.as"
instead of "http://nichol.as/imagesearch/flickr" you can use
mod_rewrite like this:

RewriteEngine On
RewriteRule ^[^(imagesearch)](.*)$ /imagesearch/$0
RewriteRule ^$ /imagesearch/flickr

And put in your httpd.conf or in a .htaccess in your web-root.

Good luck!
Nicholas

[Jose Galvez]

Dear Nicholas,
I don't think a rewrite rule will work every time.  For example in my situation, I'm using virtual hosts, so the pylons app really is the root for my server.  Currently when I run this with mod_SCGI I set up the mountpoint to be "/".  I'll try the setup again tonight so see if I can get it workwing
Jose

[timc3]

This is great, just what I am looking for as I am trying to get a
project that someone else has created working through mod_python and I
haven't had much time to look at Pylons yet.

One part I am stuck on is in the .ini file that I run from paster serve
mysite.ini says:

[app:main]
use = egg:mysite

And all the files seem to be under:
/usr/lib/python2.3/site-packages/mysite.

So what would my options be for:
PythonPath "['/home/httpd/vhosts/nichol.as/httpdocs/imagesearch'] +
sys.path"

PythonOption wsgi.application startup::app
PythonOption SCRIPT_NAME /imagesearch

Thanks.

Tim.

[Jose Galvez]

Someone please correct me if I'm wrong, but I thought that with mod_python you didn't need to run paster serve youinifile.ini.  At least that's what I've been trying to do, because I saw nothing in the tutorial about what to change in your ini file.  I was also assuming that the with mod_python the [app:main] section would be not needed.
Jose

[Jose Galvez]

Gave it one more try, and its still not working. I'm having real
problem with the recurrsion that's been mentioned in other treads. Also
I can't seem to be able to get at anything in my public folder other
then css files.
Jose

[timc3]

Yes I know that I don't need to run "paster serve inifile.ini" but I
want to convert an existing app from paster to mod_python (which I have
setup).

[Jose Galvez]

Dear timc3,
Sorry I wasn't trying to say that you were doing something wrong, I
wanted someone to reassure me that I wasn't doing something wrong, I
wanted to make sure that I wasn't supposed to run paster. again sorry
if thats not how it came off
Jose

[Jose Galvez]

finally got this to work! Thanks for the great wiki and all the great work
Jose

[Nicholas Piël]

Hi Tim,

Well, once someone has setup his app by doing:
python setup.py bdist_egg

And has handed you the egg then all you have to do (assuming you
already have mod_python and the wsgi_gateway) is this:

- Install the egg:
# easy_install helloworld-0.0.0dev-py2.4.egg

- Go to your web directory
# cd /home/httpd/vhosts/domainname.net/httpdocs/

- Create a config:
# paster make-config helloworld hello.ini

- Edit the config file
(change contact info etc... but it should even work when not
changing this file)

- Create a .htaccess containing:
SetHandler mod_python
PythonHandler mod_python.wsgi
PythonPath "['/home/httpd/vhosts/domainname.net/httpdocs/'] +

sys.path"
PythonOption wsgi.application startup::app

PythonOption SCRIPT_NAME /helloworld

- Create 'startup.py' containing:
from paste.deploy import loadapp
app = loadapp("config:/home/httpd/vhosts/domainname.net/httpdocs/
hello.ini")

Your app should now be visible under: http://domainname.net/helloworld

I hope this helps!

Cheers,
Nicholas

[Nicholas Piël]

Cool!

Don't forget to put your public site on the wiki ;)

( http://www.pylonshq.com/project/pylonshq/wiki/SitesUsingPylons )

[Damjan]

I have one comment about the part that says:
"""Make sure that you specify the correct SCRIPT_NAME which should be
the name of your Pylons Project and that your PythonPath points to the
correct directory."""

SCRIPT_NAME actually should be the location under which Apache
executes the WSGI application. So if the url of the root of your WSGI
application is http://myhost/whatever SCRIPT_NAME should be /whatever,
no matter what the pylons project is or the directory it's stored in.

Usually (at least in my tests) if you setup mod_python +
ModPythonGateway in a <Location> Apache directive, there's no need of
"PythonOption SCRIPT_NAME ..." because Apache itself will corectly
generate SCRIPT_NAME, and mod_python and ModPythonGateway will use
that. On the other hand, if ModPythonGateway is used in a <Directory>
directive or in .htaccess then SCRIPT_NAME is required because Apache
seems to not be able to generate the correct SCRIPT_NAME.

And then, there's another mod_python WSGI server implementation that
works better in the <Directory> case, and worse in the <Location> case,
but I've not had that much experience with it yet (and don't remember
the url).

+1 on saving ModPythonGateway as mod_python.wsgi
a WSGI server should really be a part of mod_python but the mod_python
developers seem to be stubborn about it (and incidentally the it's the
same with python's scgi module and developers).

[timc3]

Thanks for all the help.

I have got it work, although the stylesheets, images and javascripts in
the public directory are not showing at the moment.

[timc3]

Yes it looks like I am getting the same trouble with it not serving the
public folder. If I try an access a folder that is in public via my
webbrowser I get a pylons 404 error message.

[Jose Galvez]

Dear Tim,

I don't know if you access to the apache conf files, but I was having a
lot of trouble getting this to work as well, until I just put everything
in the apache conf rather then in htaccess files. I also put all the
mod_python related stuff in location tags. After I did that then it all
started to work fine. One thing that is different form my old config
which was an scgi config, is that before if I set up apache find my
pylons app with a path (say /pylons pointed to my pylons app) with then
the /pylons get added to the url, using the mod_python and adding
/pylons as a location, it does not get added to the path. Don't know if
any of this helps, but thats what was messing my install up. I really
never had any luck setting it up using the htaccess file, although I
really can't see a reason why that should be the case, I also was not
able to get mod_python to actually point the the root like I has
originally done with my scgi install.

Jose

[timc3]

Thanks for the help, I am now trying to get hellahella working with
mod_python.

I have gone through the same steps that I did before except putting the
configuration in to my virtualhosts file, but I still have problems
displaying the images etc..
This is my apache2 virtualhost file:

<VirtualHost 10.0.0.103>
ServerAdmin t...@timc3.com

DocumentRoot /www/hellahella/
<Location />
SetHandler mod_python
PythonHandler mod_python.wsgi
PythonPath "['/www/hellahella/'] +sys.path"
PythonOption wsgi.application startup::app
PythonOption SCRIPT_NAME /hellahella
</Location>

ErrorLog /var/log/apache2/error.log
LogLevel warn

CustomLog /var/log/apache2/access.log combined

</VirtualHost>

So if yours is working, could you post your configuration too?

[timc3]

Ok, I just realised I am being bloody daft. Another look at the URLS in
the files told me they are looking at:

http://10.0.0.3/stylesheets/hellahella.css

But of course with my rules added, I will serve that file at:

http://10.0.0.3/hella/stylesheets/hellahella.css.

What a muppet. Just going to redo my configuration files and then I
will post up the how to.

[timc3]

I put some directions of what I did here. All thats not working now is
simply the root directory but that is going to be easy to sort:

http://blog.timc3.com/2006/07/16/hellahella-on-apache-2/

[Ben Bangert]

Nice write-up Tim!

I'm a bit curious about your report on the performance of it running
with paster serve though. I run my websites with paster serve
(PylonsHQ, and others) and have benched it at well over 200+ req per
second. What were you running it on?

My home hellahella process runs for weeks at a time without any
performance variation over time as well. Was your ini file using:
[server:main]
use = egg:Paste#http

Or was it using egg:
egg:PasteScript#wsgiutils

I believe the wsgiutils one is slower and may have performance issues
over time, though I've never had a problem with either.

Again, thanks for the write-up!

Cheers,
Ben

[timc3]

I was running it on a Debian box running a 2.6 kernel. 667Mhz
processor and 512Mb Ram, which should be enough to run a webserver but
after a few hours of paster being open it was way unresponsive.

I could startup paster again and get some more information if you wish
but even on that spec machine I don't expect it to run like that (I
have developed some intensive applications on that machine that run
well). Besides I don't know much about Paste, but I am not willing to
start running that on my box when I have been using apache (albeit v1)
for many years.

The performance of mod_python with apache is now much more responsive
and much faster, but I am new to python development so maybe it says
more about my settings and system than paster.

>From memory, I wasn't using egg:PasteScript#wsgiutils