Re: If Not CGI...
Stephen Hansen
> Hi;
> I've caught a lot of flack (imagine that) about using CGI. I understand
> there are several other options, to wit: mod_python, fastcgi and wcgi. I've
> messed around with mod_python without luck. What are your suggestions?
Its a slightly complicated question.
First of all, it is not absolutely *wrong* to use CGI. Its just simply
decades out of date, slow, and has certain problems.
Primarily, that one must start Python for each and every request. That
adds up: that adds up a LOT over time.
The "solution" to all of these are various methods to run Python once,
and keep it running and loaded, and simply call into it (i.e., execute
some function you've defined) for those requests that need to be dynamic.
mod_python accomplishes this by embedding Python into Apache directly.
After you set it all up, there's various ways you can access it. The
simplest is the publisher handler. With it, you can 'call' a Python
function by linking to, say, /myform.py/view -- it'll call the 'view'
function in 'myform.py'.
With it, check out
http://www.modpython.org/live/current/doc-html/tutorial.html
"without luck" is a worthless statement to make, so I can't comment on it.
But with mod_python, you wouldn't just be running it and then running
your existing code as-is. The interface between the web server and your
code is different in it, so you'll have to reorganize some stuff.
FastCGI is a different kind of approach to the problem; it launches
Python alongside Apache, and that Python stays alive forever. It just
redirects requests to said process when they come in. I know very little
about this model, but believe its meant to sort of mimic a CGI
environment so you can sort of migrate to it easier, but I'm not
entirely sure. So I can't comment on this directly, but
http://docs.python.org/howto/webservers.html seems interesting (Though
it speaks of ALL of these options, so is a good read anyways)
Then there's mod_wsgi; this is like mod_python, but approaches the
problem differently, with a bit more formal structure with an eye for
interoperability. It implements the Web Server Gateway Interface
specification, and lets you easily load up "wsgi apps" and "wsgi
frameworks" which are a very, very nifty way to write modern web
applications.
But the thing is: a WSGI web application looks and is shaped nothing
like a CGI application. They're awesome. Btu different. Writing a WSGI
app from scratch without a framework is possible, but it seems like a
terribly painful thing to go about doing.
Instead, most people who are writing modern Python web applications, use
some sort of framework.
Django is one of the most popular. I like it. But I prefer pylons-- its
a little more low level, and that suits me. There's also TurboGears,
webpy, and on and on and on.
The cool thing about WSGI applications is that they are "stacks";
there's a lot of "middleware" that can sit between the server and your
end-application. You can add features and functionality and capabilities
to your app just by adding another piece of middleware.
For example, do you want to store state about what a user is doing? The
things in their shopping cart, for example? A really, really elegant way
is to use a "session" middleware -- Beaker for instance. It creates a
unique session key, and sets it on the user's cookies. In your web
application now, you can associate any kind of data you'd like with that
key. And you can even remember stuff about that user if they return to
your app later.
Now, you could do that all by hand-- but its painful, and doing it right
is not trivial.
My suggestion? I can't really give you one. You're in the middle of a
project. Doing it "right" from this point is basically a rewrite --
though it may not take as long as you suspect, if you use something like
Django which is -very- easy.
So maybe my suggestion is for now, to figure out what's wrong with
mod_python for you. It works just fine.
But then when this project is over, sit down and load up Django (or
another of the options: look around, find one that tickles you), and
spend a few days re-doing this project in that. Not for real, but for
practice, to figure out how you'd do it in a proper framework. See how
you like it. See how it works.
--
Stephen Hansen
... Also: Ixokai
... Mail: me+list/python (AT) ixokai (DOT) io
... Blog: http://meh.ixokai.io/
P.S. An added plus to the WSGI app's: they'll encourage (nay, force) you
to write better code. Including some encapsulation and classes and such.
Which is not to say that procedural programming is /wrong/, but, for a
lot of things, mixing in at least a little bit of OOP (even if you do
not buy the OOP koolaid) just for organizational purposes is utter win.
Tim Chase
The main reason is that CGI has the overhead of loading &
unloading the Python interpreter on every request. The other
methods load the Python interpreter once (or a small,
fixed-number of times), then handle lots of requests from that
process (or pool of processes), and then optionally unload if
server-load drops.
However CGI is old and fairly entrenched, so it's easy to find
with cheap hosting services -- what do they care if your site is
slow?
> I understand there are several other options, to wit:
> mod_python, fastcgi and wcgi. I've messed around with
> mod_python without luck. What are your suggestions?
Generally, one writes to a framework (Django[1], web.py[2],
TurboGears[3], CherryPy[4], etc) that either has a
preferred/suggested method of interface, or allows you to plug
into [m]any of the items you list. I know Django is happy with
mod_python and wsgi (and I suspect fastcgi, but I'll let you
google that). YMMV with the others. I've even seen an
abomination of a hack that ran Django under CGI (whooooie, is the
performance bad!). I think the general direction of the Python
web-world seems to be moving toward WSGI (and Graham Dumpleton's
work on mod_wsgi[5]; IIUC, he was heavily involved in the initial
mod_python) instead of mod_python.
Since you seem fairly adamant about *not* using a framework and
cobbling together the universe from the ground up, you might look
into Paul Boddie's "WebStack"[6] which abstracts away a number of
the main interfaces into a common interface. Kindly, his work
even allows you to plug into a CGI interface since that's what
you're familiar with, and then shift to a different interface.
-tkc
[5]
http://code.google.com/p/modwsgi/
[6]
http://www.boddie.org.uk/python/WebStack.html
John Nagle
> FastCGI is a different kind of approach to the problem; it launches
> Python alongside Apache, and that Python stays alive forever. It just
> redirects requests to said process when they come in. I know very little
> about this model, but believe its meant to sort of mimic a CGI
> environment so you can sort of migrate to it easier, but I'm not
> entirely sure. So I can't comment on this directly, but
> http://docs.python.org/howto/webservers.html seems interesting (Though
> it speaks of ALL of these options, so is a good read anyways)
> But the thing is: a WSGI web application looks and is shaped nothing
> like a CGI application. They're awesome. Btu different. Writing a WSGI
> app from scratch without a framework is possible, but it seems like a
> terribly painful thing to go about doing.
It's not that difficult. WSGI works as an interface to FCGI in
Apache. If you set up programs that way, they'll run either as
CGI programs (useful for debug) or WSGI programs.
WSGI/FCGI programs look a lot like CGI programs. The main
difference is that your transaction program is called as a subroutine,
and has to be reusable. Avoid global state and that's easy.
When you're called, you get various parameters, and at the
end, you return the HTTP content you want sent over the wire.
WSGI/FCGI scales well. The FCGI server will fire off extra
copies of the application as the load goes up, and will ask
unneeded copies to exit when the load goes down. If an FCGI
program crashes, the server just restarts a fresh copy.
So it's a robust mechanism.
Below is "Hello World" for FCGI/WSGI, without a "framework".
The biggest headache is getting the "mod_fcgi" module into Apache
(or running "lighthttpd"), getting it configured properly,
and getting it to find your FCGI program. (Note: if you put
an FCGI program in a directory that Apache considers valid
for CGI, Apache will run the program with CGI, not FCGI.
This works, but at CGI speeds, with a program reload every time.)
A FCGI/WSGI module for Python is here:
http://svn.saddi.com/py-lib/trunk/fcgi.py
John Nagle
#!/usr/local/bin/python2.6
import fcgi
#
# "Hello World" for FCGI/WSGI
#
def simpleApp(environ, start_response):
status = '200 OK'
headers = [('Content-type','text/plain')]
start_response(status, headers)
return ['Hello world from simple_application!\n']
#
# Main FCGI program
#
fcgi.WSGIServer(simpleApp).run()
Tim Chase
> I would like to explore rewriting the shopping cart in Django.
> The reality of the matter may make it difficult. Working
> literally from the time I awake to when I go to sleep and not
> having enough hours to complete everything I set for myself
> makes it difficult,
The reason for using a framework is because web development is
hard work -- the creators of these frameworks have gone before
you doing much of the hard work so you can concentrate on your
domain-specific tasks instead of the infrastructure.
When you want to drive across the country, do you begin by
procuring all the land in between and then paving your roads?
No...you leave that to people who do infrastructure full-time
(and with lots of experience in this area). You just get in your
car and drive to your destination (your domain-specific task) and
let the professionals deal with the infrastructure. That common
infrastructure investment is then shared between many users with
widely different destinations.
So you may find that, by switching to a web framework, you
actually end up *saving* time because you don't need to recreate
all the work of building a framework. Given that tens of
thousands of development hours (by some very smart & very
experienced people) have gone into some of the big-name
frameworks such as Django, those are tens-of-thousands of hours
that *you* don't have to spend.
-tkc