How to kill a background thread along with the server

[lhagan]

My bottle.py application runs a fairly lengthy background process
using threading. It works fine, but KeyboardInterrupt only shuts down
the server, not the background process. So in order to cancel
everything without waiting for the background process to complete, I
have to kill it manually.

The only way I can think of to resolve this is to modify bottle.py,
but there must be something else that I'm missing. Any thoughts?

[Sean M. Collins]

Catch the KeyboardInterrupt and send the kill signal to the thread.

--
Sean M. Collins
Core IT Pro, LLC
www.coreitpro.com

[Philip Horger]

@Sean Collins: Yeah, that's great. I know that bit. What I want to know (and presumably Ihagan as well, though presuming is generally a dangerous thing), is how to send the kill signal. I've got a main thread that desperately wants to tell Bottle to stop serving, let the run() function end, and allow the rest of the program to terminate. But there seems to be no provision for that in the bottle module, unless I'm missing something. And I hope I am, because it's trivial to fix it if someone points out the missing puzzle piece.

[Marcel Hellkamp]

Killing a running server is usually done with the SIGINT
(KeyboardInterrupt) signal. If your server is running in the main
thread, you can kill it by calling thread.interrupt_main() from a
different thread. If you run the server in a side-thread, you have to
tell the server somehow to stop on its own. This depends on the server
adapter used and may require you to start the server yourself instead of
using a server adapter.

[Philip Horger]

Makes sense. For anyone else interested in a solution, here's how I did it, though it's not very well generalized.

I have an HTTPServer class that inherits from a common server class I use all over my application. It contains a bottle.Bottle() as self.server, and None for self.wsgi (which is replaced later). I also created a Subclass of bottle's server adaptor for WSGI, which I called KillableWSGIRefServer. This overrides the server adaptor's run function to extract a callback from self.options (preventing the make_server() constructor from dealing with it) and applying it on the resulting server to report it to the HTTPServer instance, thus setting it's self.wsgi to the underlying WSGI server. In HTTPServer.close, I call self.server.close(), and then self.wsgi.shutdown(). The first one is not enough to stop the server, which was my problem earlier, but stopping the underlying server does the job nicely.

All I have to do to take advantage of this from there is to modify the bottle.run call in HTTPServer.run to contain the keyword arguments server=KillableWSGIRefServer, and setserver=self.setserver.

[lhagan]

Just getting back to this problem, but I have a solution!

For some background, I'm only using Bottle in development mode with
the built-in WSGIServer. I have a single file application very similar
to the Hello World example. Before doing anything else, it starts a
background thread that needs to continue running until the server is
killed. The issue is that the Bottle server gets the KeyboardInterrupt
first, so my application never has a chance to kill the background
process and it just goes on forever until I manually kill it.

The problem is actually a combination of a bug and a malfeature in
Python. Fortunately, I found a simple recipe on ActiveState by Allen
Downey that solves it. Unfortunately, this only works on UNIX-like
systems (so not on Windows). In case it helps someone in the future,
here is a working Bottle example:

----
#!/usr/bin/env python

from bottle import route, run
import threading, time, os, signal, sys, operator

# Workaround for missed SIGINT in multithreaded programs
# from Allen Downey (released under the PSF license)
# http://code.activestate.com/recipes/496735-workaround-for-missed-sigint-in-multithreaded-prog/
class MyThread(threading.Thread):
"""this is a wrapper for threading.Thread that improves
the syntax for creating and starting threads.
"""
def __init__(self, target, *args):
threading.Thread.__init__(self, target=target, args=args)
self.start()

class Watcher:
"""this class solves two problems with multithreaded
programs in Python, (1) a signal might be delivered
to any thread (which is just a malfeature) and (2) if
the thread that gets the signal is waiting, the signal
is ignored (which is a bug).

The watcher is a concurrent process (not thread) that
waits for a signal and the process that contains the
threads. See Appendix A of The Little Book of Semaphores.
http://greenteapress.com/semaphores/

I have only tested this on Linux. I would expect it to
work on the Macintosh and not work on Windows.
"""

def __init__(self):
""" Creates a child thread, which returns. The parent
thread waits for a KeyboardInterrupt and then kills
the child thread.
"""
self.child = os.fork()
if self.child == 0:
return
else:
self.watch()

def watch(self):
try:
os.wait()
except KeyboardInterrupt:
# I put the capital B in KeyBoardInterrupt so I can
# tell when the Watcher gets the SIGINT
print 'KeyBoardInterrupt'
self.kill()
sys.exit()

def kill(self):
try:
os.kill(self.child, signal.SIGKILL)
except OSError: pass

def background_process():
while 1:
print('background thread running')
time.sleep(1)

@route('/hello/:name')
def index(name='World'):
return '<b>Hello %s!</b>' % name

def main():
Watcher()
MyThread(background_process)

run(host='localhost', port=8080)

if __name__ == "__main__":
main()