Attempting to get lighttpd, fastcgi & python working
venkatbo
As a precursor to getting tubogears/cherrypy/lighttpd to work, I'm
trying to get lighttpd, fastcgi & python working first, on a 2.4.21.x
i686 linux system. I tried following the steps in:
http://www.cleverdevil.org/computing/24/python-fastcgi-wsgi-and-lighttpd
Some of the lighttpd.conf settings are slightly different from those in
that article - fixing some erros in the original and accounting for
some changes in entry formats itself.
All components were built by 'root' and binaries run by 'root'. Python
files (.py(c)) were deployed to '..../site-packages' and run with
execute prop turned on.
I'm trying to run lighttpd on "dev-1686linux.my.org:8090" with the
fastcgi server setup to listen on "127.0.0.1:8091". The problem I'm
having is that though lighttpd comes up and is able to serve static
content, it fails to bring up the WSGIServer and serve dynamic content
(simple "Hello World") generated by the py script (test.pcgi).
The errors I get are:
....
[root@dev-i686linux myApp]# lighttpd -f ./lighttpd.conf
[root@dev-i686linux myApp]# Traceback (most recent call last):
File "/test/myApp/htdocs/test.fcgi", line 21, in ?
WSGIServer(myapp, bindAddress = ("127.0.0.1", 8091)).run()
File "/usr/lib/python2.4/site-packages/fcgi.py", line 1064, in run
sock = self._setupSocket()
Traceback (most recent call last):
File "/test/myApp/htdocs/test.fcgi", line 21, in ?
WSGIServer(myapp, bindAddress = ("127.0.0.1", 8091)).run()
File "/usr/lib/python2.4/site-packages/fcgi.py", line 1027, in
_setupSocket sock.bind(self._bindAddress)
File "/usr/lib/python2.4/site-packages/fcgi.py", line 1064, in run
sock = self._setupSocket()
File "/usr/lib/python2.4/site-packages/fcgi.py", line 1027, in
_setupSocket sock.bind(self._bindAddress)
File "<string>", line 1, in bind
socket.error: (98, 'Address already in use')
File "<string>", line 1, in bind
socket.error: (98, 'Address already in use')
...
It is unclear why it thinks the 127.0.0.1:8091 sock is already in use.
I ran this the first thing after a system reboot. No other apps are
using this port. I see this error no matter what port I choose, on the
server side. The browser sits there spinning and 'waiting' for the
response to url that incudes the .pcgi script.
The behviour is slightly different when I use UDSockets to listen on.
Again, I can see static content. When I try the py script, I see it
waiting indefinitely, but this time no errors are logged into the log
file. So no clue where it is hanging on the server side. The access
logs are not showing any "request" related entries even for the static
content that is being deliverd (albeit properly).
Can someone shed light on this. I thought if I can iron out this kink,
then I can get TurboGears (CherryPy ws) working with lighttpd and
fastcgi proxy-ing upfront.
Searched around quite a bit, but unable to find a clue. Would much
appreciate some pointers... Thanks a lot ...
/venkat
Following are the various config settings:
/test/myApp/lightpd.conf:
-------------------------
server.modules = (
"mod_rewrite",
"mod_access",
"mod_fastcgi",
"mod_accesslog" )
server.document-root = "/test/myApp/htdocs/"
server.errorlog = "/var/tmp/lighttpd.error.log"
index-file.names = ( "index.html", "index.htm" )
server.port = 8090
server.bind = "dev-1686linux.my.org"
server.error-handler-404 = "/test.fcgi"
fastcgi.server = ( ".fcgi" =>
(
(
"host" => "127.0.0.1",
"port" => 8091,
"bin-path" =>
"/test/myApp/htdocs/test.fcgi",
"min-procs" => 1
)
)
)
/test/myApp/htdocs/test.fcgi:
-----------------------------
#!/usr/bin/python2.4
# test.fcgi for myApp
from fcgi import WSGIServer
def myapp(environ, start_response):
print 'got request: %s' % environ
start_response('200 OK', [('Content-Type', 'text/plain')])
return ['Hello World!\n']
# use a tcp-socket based approach
WSGIServer(myapp, bindAddress = ("127.0.0.1", 8091)).run()
/test/myApp/fcgi.py:
--------------------
The WSCGIServer (fcgi.py) authored by Saddi is from:
http://svn.saddi.com/py-lib/trunk/fcgi.py
No mods here.
Jonathan LaCour
> trying to get lighttpd, fastcgi & python working first, on a 2.4.21.x
> i686 linux system. I tried following the steps in:
> http://www.cleverdevil.org/computing/24/python-fastcgi-wsgi-and-
> lighttpd
You will be better off getting a simple TurboGears application
working, as its a lot easier to setup and configure. Start off with
a quickstarted application, and then you can use the method described
on the TurboGears wiki for deploying under lighttpd using scgi:
http://www.turbogears.org/docs/deployment/lighttpd.html#scgi
Let me know if you have any problems, and I will walk you through it.
--
Jonathan LaCour
http://cleverdevil.org
Brian Beck
helpful, if that's possible. I already have a ton of stuff all over my
web server, so I'd like it to only talk to the TurboGears server if it
can't find what it's looking for, like the FastCGI approach.
Tim Jones
Tim
Damjan
> I ran this the first thing after a system reboot. No other apps are
> using this port. I see this error no matter what port I choose, on the
> server side. The browser sits there spinning and 'waiting' for the
> response to url that incudes the .pcgi script.
Is this your own server or a virtual server?
If it's a virtual server some of them can't bind to the 127.0.0.1
address.
Brian Beck
lighttpd and CherryPy and scgi_server and flup configurations I could
think of, I still couldn't get the setup I want to work. It's quite
frustrating. I tried everything from Jon LaCour's blog and the
FastTrack code with no luck.
Here's the setup I want.
I want to use TurboGears to host my home page. That's the root
directory of my web server, /.
I already have tons of crap already hosted on my web server, so I can't
just direct lighttpd at it with a simple regular expression or move it
all into a /static directory.
Since I want TurboGears to host from the root, I can't tell lighttpd to
turn to TurboGears using a regular expression either.
So, I need to use error-handler-404 (or whatever) to tell lighttpd
"when you don't find that file, run at this scgi/fcgi/py script" (or
something similar).
This also means I don't want to use a proxy by running another server
on my machine. I want to use a socket file.
Looking at lighttpd/error.log is rarely helpful. I've had everything
from permission errors (easy), connection refused, no fcgi handler
found, fcgi server disabled, blah blah blah. I don't even remember
which error messages go with which configuration, in my mind they're
all equally broken and frustrating.
Any ideas? I don't care if it uses FastCGI or SCGI or whatever, I just
want to solve the use case as described above.
--
Brian Beck
Adventurer of the First Order
Jonathan LaCour
> After a full weekend of tweaking and trying every combination of
> lighttpd and CherryPy and scgi_server and flup configurations I could
> think of, I still couldn't get the setup I want to work. It's quite
> frustrating. I tried everything from Jon LaCour's blog and the
> FastTrack code with no luck.
Sorry about that! Lets see if we can help you out here.
> I want to use TurboGears to host my home page. That's the root
> directory of my web server, /.
>
> I already have tons of crap already hosted on my web server, so I
> can't
> just direct lighttpd at it with a simple regular expression or move it
> all into a /static directory.
>
> Since I want TurboGears to host from the root, I can't tell
> lighttpd to
> turn to TurboGears using a regular expression either.
>
> So, I need to use error-handler-404 (or whatever) to tell lighttpd
> "when you don't find that file, run at this scgi/fcgi/py script" (or
> something similar).
So, you basically have a bunch of static files already on your web
server that you want to serve while still having the root page served
by TurboGears? I am sure that there is a way to make this happening
by using 404 error handler tweaks or some other hackery, but it would
certainly be a lot easier to do something else.
One option would be to put a static index.html at your root that
immediately redirects to a url where your TurboGears app is
published, so you go to "/" and it basically immediately redirects
you to "/blog" or whatever.
Another option would be to have your TurboGears app published at a
subdomain of your domain, so that your web server could redirect
traffic from "whatever.yourdomain.com" to your TurboGears
application, and all your static files would still work just fine.
Or... you could just register another domain ;)
If you still aren't happy with something like the above suggestions,
then you also might be able to implement some CherryPy filter based
on the static filter that will do exactly what you want, but that
imposes a lot of overhead and a lot of extra work.
> This also means I don't want to use a proxy by running another server
> on my machine. I want to use a socket file.
I am not sure where you got this. I don't know of any way to deploy
TurboGears applications that doesn't involve starting your
application process (which is a server). A socket file is just
another way for the web server to talk to your application process.
A regular socket over scgi isn't all that much different than a
socket file, and its not really all *that* much different from just
proxying to your CherryPy web server either (although I suspect lower
performance with a proxy).
> Looking at lighttpd/error.log is rarely helpful. I've had everything
> from permission errors (easy), connection refused, no fcgi handler
> found, fcgi server disabled, blah blah blah. I don't even remember
> which error messages go with which configuration, in my mind they're
> all equally broken and frustrating.
Well, I will agree with you there. The errors in that file are
indeed rarely useful. I wish I could help you more than the advice I
offer above!
> Any ideas? I don't care if it uses FastCGI or SCGI or whatever, I
> just
> want to solve the use case as described above.
It seems to me that you are making life a lot harder for yourself
than it needs to be by requiring that "/" be your TurboGears, and not
moving all your static content to a "/static/" directory or something
similar.
Best of luck!
Brian Beck
But... I'm getting the idea the idea that this is next to impossible.
You've even gone so far as to suggest registering a new domain :). I'm
quite forgiving of Python web framework issues regarding deployment...
but this is just getting silly.
> One option would be to put a static index.html at your root that
> immediately redirects to a url where your TurboGears app is
> published, so you go to "/" and it basically immediately redirects
> you to "/blog" or whatever.
This is the most promising option, and one I had considered. I'm being
incredibly stubborn, though. I want this to work not only for myself,
but out of principle! It should be possible with error-handler-404.
> I am not sure where you got this. I don't know of any way to deploy
> TurboGears applications that doesn't involve starting your
> application process (which is a server). A socket file is just
> another way for the web server to talk to your application process.
Right, I'm not trying to get around having another process running. I'm
trying to get around having some redundant ghost-server running on a
random port on my server. (Also, anyone who browses directly to the
proxy... any relative links that lead to static files wouldn't work
since it would not check lighttpd. Making that port not accessible...
more work.)
Really the socket file thing is orthogonal to the request routing.
I'll try the error-handler-404 approach with FastCGI from your site
again without using a socket file this time. Does it work with
TurboGears as presented, or do I have to make any changes? (I'm under
the impression that the recipe was for use with vanilla CherryPy.)
> It seems to me that you are making life a lot harder for yourself
> than it needs to be by requiring that "/" be your TurboGears, and not
> moving all your static content to a "/static/" directory or something
> similar.
I'm talking about thousands of files linked from all over the web (with
my permission)... that would break a lot of stuff. Wouldn't putting
some filter into CherryPy to look in a new directory when any incoming
file request is received completely defeat the purpose of using
lighttpd? I might as well just use the built-in server if that's the
case...
Thanks again. I'm ready to take an other shot at it!
Jonathan LaCour
> Thanks for the advice, Jon.
>
> But... I'm getting the idea the idea that this is next to impossible.
Okay, I went ahead and got something working for you this morning.
Its using error-handler-404 just like you want, and serving static
files out through lighttpd. Took about 15 minutes. How is that for
doing the impossible! ;)
I have created a simple TurboGears application with an scgi startup
script (the async one), and a lighttpd.conf file to get you going.
Here it is:
http://cleverdevil.org/werd.tar.gz
Let me know if you need any help adapting it to your needs. I just
did some basic testing, but it seemed to work for me. I am using a
regular socket, but it would be easy to make it use a socket file.
Brian Beck
two variables at once (lighttpd handler, socket file).
For some reason lighttpd doesn't seem to believe me that I want an
index file named just 'index'. So it goes to index.html right now if
it exists, and if it doesn't it just lists the files. :(
I've tried:
server.indexfiles = ( "index.html", "index" )
server.indexfiles = ( "index", "index.html" )
(I'm using 1.3, in 1.4 there is index-file.names... maybe that will
work.)
I also tried changing index in controllers.py to index_html, no luck.
(Is that even the right way to do that?)
Anyway, maybe it's time to upgrade lighttpd...
Thanks again,
Damjan
possible with WSGI middleware.
If I'm not wrong, there's a WSGI middleware that runs a request through
several WSGI applications until one of them retuns success.. and
there's another WSGI app that serves static files. I think those are
part of Paste.
Brian Beck
because that is attempted before error-handler-404 directive. :(
So, two more things I'm gonna try to tackle:
1) socket file
2) directory listing