cherrypy 3.0.1, lighttpd and fcgi through flup stop problem
asmodai
over the last few days I've been working on converting my own WSGI
stuff to
just use cherrypy. After many frustrations due to incomplete and
confusing
documentation of cherrypy I finally managed to pull everything
together on how
to use fcgi through flup with RoutesDispatcher. (Once I wrap my head
around
everything I hope to provide a few documentation patches.)
The only problem that I now have, after having patches my 3.0.1
install to not
die on the SIGTERM() argument issue (ticket #667) is that whenever I
stop
lighttpd it leaves a stray process around connected to the unix domain
socket.
I have to kill -9 it to even get rid of it.
It seems it has spawned 10 threads on this FreeBSD 6.2-STABLE machine
and
is waiting in a kse_release() loop (a wait state used inside libkse
threads
meaning a thread is waiting for something to do). When I stop lighttpd
it
sends a SIGTERM and subsequently 1 thread gets killed off, but the
remaining 9
happily stay after lighttpd has stopped. It seems as if the threads
are not
being stopped/the SIGTERM is not being propagated across all threads.
This is FreeBSD 6.2-RELEASE, Python 2.5 with threads, CherryPy 3.0.1 +
SIGTERM
fix, flup 0.5 (of a few days old), lighttpd 1.4.15.
Snippet from lighttpd.conf:
$HTTP["url"] =~ "" {
fastcgi.server = (
"/" => (
"kouyou-web.cgi" => (
"bin-path" => "/home/asmodai/www/kouyou-web.fcgi",
"socket" => "/tmp/labs.sock",
"check-local" => "disable",
"disable-time" => 1,
"min-procs" => 1,
"max-procs" => 1,
),
),
)
} # end of $HTTP["url"] =~ "^/"
kouyou-web.fcgi:
#!/usr/local/bin/python
import cherrypy
import os
from flup.server.fcgi import WSGIServer
from kouyou.controllers import *
current_dir = os.path.dirname(os.path.abspath(__file__))
dispatcher = cherrypy.dispatch.RoutesDispatcher()
dispatcher.connect('home', '', controller=HomeController())
config = {'/': {'request.dispatch': dispatcher,
'tools.staticdir.root': os.path.join(current_dir,
'static')},
'/css': {'tools.staticdir.on': True,
'tools.staticdir.dir': 'css'},
'/img': {'tools.staticdir.on': True,
'tools.staticdir.dir': 'img'}
}
app = cherrypy.tree.mount(None, config=config)
cherrypy.engine.start(blocking=False)
WSGIServer(app).run()
Any ideas, tips, hints, suggestions are more than welcome. But could
it be
cherrypy does not do the right thing?
Thanks in advance,
--
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
イェルーン ラウフロック ヴァン デル ウェルヴェン
http://www.in-nomine.org/ | http://www.rangaku.org/
How are the mighty fallen...
Robert Brewer
> over the last few days I've been working on converting my
> own WSGI stuff to just use cherrypy. After many frustrations
> due to incomplete and confusing documentation of cherrypy
> I finally managed to pull everything together on how to use
> fcgi through flup with RoutesDispatcher. (Once I wrap my head
> around everything I hope to provide a few documentation patches.)
Doc patches are always welcome. :) If you want a more informal place to
record your findings, http://tools.cherrypy.org is available for
immediate guest contributions anytime.
> The only problem that I now have, after having patches my 3.0.1
> install to not die on the SIGTERM() argument issue (ticket #667)
> is that whenever I stop lighttpd it leaves a stray process around
> connected to the unix domain socket.
> I have to kill -9 it to even get rid of it.
>
> It seems it has spawned 10 threads on this FreeBSD 6.2-STABLE
> machine and is waiting in a kse_release() loop (a wait state
> used inside libkse threads meaning a thread is waiting for
> something to do). When I stop lighttpd it sends a SIGTERM and
> subsequently 1 thread gets killed off, but the remaining 9
> happily stay after lighttpd has stopped. It seems as if the
> threads are not being stopped/the SIGTERM is not being
> propagated across all threads.
The problem is most likely that your WSGI server is starting its own
threads, but CP doesn't know how to tell it to shut down those threads.
Ideally, you'd write a very small adapter so that, when the SIGTERM is
handled, cherrypy.server (really a server manager) will send a stop
message to the flup.server.fcgi.WSGIServer. See
http://www.cherrypy.org/wiki/ServerAPI#HTTPServerAPI for a discussion of
what that adapter layer needs to look like. If that works well, I think
we'd be happy to include such an adapter in the distro to make it easier
for the next person.
Robert Brewer
System Architect
Amor Ministries
fuma...@amor.org
asmodai
On May 30, 11:50 pm, "Robert Brewer" <fuman...@amor.org> wrote:
> Doc patches are always welcome. :) If you want a more informal place to
> record your findings,http://tools.cherrypy.org is available for
> immediate guest contributions anytime.
Ok cool. Will keep that in mind.
[snip]
> See
> http://www.cherrypy.org/wiki/ServerAPI#HTTPServerAPI for a
> discussion of what that adapter layer needs to look like. If that
> works well, I think we'd be happy to include such an adapter in the
> distro to make it easier for the next person.
I will dig into that. I will need to fumble around on this before I
get it right.
I will update when I got things sorted out.
Thanks thus far,
--
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
イェルーン ラウフロック ヴァン デル ウェルヴェン
http://www.in-nomine.org/ | http://www.rangaku.org/
What is to be, will be. And what isn't to be sometimes happens...
asmodai
On May 30, 11:50 pm, "Robert Brewer" <fuman...@amor.org> wrote:
> The problem is most likely that your WSGI server is starting its own
> threads, but CP doesn't know how to tell it to shut down those threads.
> Ideally, you'd write a very small adapter so that, when the SIGTERM is
> handled, cherrypy.server (really a server manager) will send a stop
> message to the flup.server.fcgi.WSGIServer. See
> http://www.cherrypy.org/wiki/ServerAPI#HTTPServerAPI for a discussion of
> what that adapter layer needs to look like.
Reading that URL and looking at my setup, I do not think it matches.
Lighttpd is the one spawning the fcgi process which in turn starts up
cherrypy. Cherrypy is not the process in command and as such is not
the one in need to send the stop. Lighttpd sends flup a SIGTERM, flup
then starts to tear down the process and clean up the socket.
I suddenly realised the solution was fairly simple and I tested this
with the stop/start cycle of lighttpd as well as a restart:
import cherrypy
import os
from flup.server.fcgi import WSGIServer
from kouyou.controllers import *
current_dir = os.path.dirname(os.path.abspath(__file__))
dispatcher = cherrypy.dispatch.RoutesDispatcher()
dispatcher.connect('home', '', controller=HomeController())
config = {'/': {'request.dispatch': dispatcher,
'tools.staticdir.root': os.path.join(current_dir,
'static')},
'/css': {'tools.staticdir.on': True,
'tools.staticdir.dir': 'css'},
'/img': {'tools.staticdir.on': True,
'tools.staticdir.dir': 'img'}
}
app = cherrypy.tree.mount(None, config=config)
cherrypy.engine.start(blocking=False)
try:
try:
WSGIServer(app).run()
except SystemExit:
raise
except Exception, e:
print 'Content-Type: text/plain\r\n\r\n',
print 'Oops...'
print
print 'Kouyou detected an internal error:'
print
print e
print
import traceback
import StringIO
tb = StringIO.StringIO()
traceback.print_exc(file=tb)
print tb.getvalue()
finally:
cherrypy.engine.stop()
Comments/ideas/opinions?
--
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
イェルーン ラウフロック ヴァン デル ウェルヴェン
http://www.in-nomine.org/ | http://www.rangaku.org/
Ignorance is the opportunity to learn...
Robert Brewer
Aha. I hadn't correctly read your initial post; I didn't realize you
weren't using cherrypy.server.start in the first place.
So; your solution looks fine, except for all those print
statements...what's stdout at that point? Don't tell me that whole
script is run once per request!
asmodai
> So; your solution looks fine, except for all those print
> statements...what's stdout at that point? Don't tell me that whole
> script is run once per request!
It should be towards the webserver, which then serves it to the
client.
And for all I know it does not run that script once per request. The
fastcgi socket is created on webserver start:
[19:37] [asmodai@nexus] (506) {0} % sockstat | grep labs
www python 955 0 stream /tmp/labs.sock-0
www python 955 3 stream /tmp/labs.sock-0
Which equates to
PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU
COMMAND
955 www 10 20 0 12216K 9044K kserel 0 0:51 0.00%
python
So it spawns one process which has 10 threads and that's all handing
off incoming data for this host to the cherrypy code. So all in all
quite efficient from what I can tell.
--
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
イェルーン ラウフロック ヴァン デル ウェルヴェン
http://www.in-nomine.org/ | http://www.rangaku.org/
The Wheels of Life, they turn, ..., without me...