Tornado + multiprocessing on OS X - OSError: [Errno 9] Bad file descriptor

[cleg]

Hello

I've user multiprocessing module to start my tornado server during
unit tests (and shutdown it after test run). But few days ago I've
started to recieve the following stacktraces:

Process Process-1:
Traceback (most recent call last):
File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/
python2.6/multiprocessing/process.py", line 231, in _bootstrap
self.run()
File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/
python2.6/multiprocessing/process.py", line 88, in run
self._target(*self._args, **self._kwargs)
File "/Users/clegger/Projects/zzz/zzz-server/src/broker.py", line
88, in main
http_server.listen(http_port)
File "/Users/clegger/Projects/zzz/zzz-server/eggs/tornado-1.1-
py2.6.egg/tornado/httpserver.py", line 155, in listen
self.start(1)
File "/Users/clegger/Projects/zzz/zzz-server/eggs/tornado-1.1-
py2.6.egg/tornado/httpserver.py", line 217, in start
ioloop.IOLoop.READ)
File "/Users/clegger/Projects/zzz/zzz-server/eggs/tornado-1.1-
py2.6.egg/tornado/ioloop.py", line 151, in add_handler
self._impl.register(fd, events | self.ERROR)
File "/Users/clegger/Projects/zzz/zzz-server/eggs/tornado-1.1-
py2.6.egg/tornado/ioloop.py", line 429, in register
self._control(fd, events, select.KQ_EV_ADD)
File "/Users/clegger/Projects/zzz/zzz-server/eggs/tornado-1.1-
py2.6.egg/tornado/ioloop.py", line 452, in _control
self._kqueue.control([kevent], 0)
OSError: [Errno 9] Bad file descriptor

Possibly, this happened after updating to 10.6.5, but I'm not sure.

What this could be?

[Ben Darnell]

I think that's what happens if you create an IOLoop before a fork and
then try to use it in the child process. If you're going to use fork,
you have to do it before anything creates the singleton IOLoop. For
this reason, you may want to run the server with subprocess instead of
multiprocessing to start it with a clean slate. Or look at
tornado.testing.AsyncHTTPTestCase and run the server in the same
process as the tests.

-Ben

[Paul Dmitryev]

Thank you for quick answer.

I've re-checked code, but forking goes before starting ioloop (ioloop
initialises in spawned process). And more of all - my code was working
a week ago.

But, I think using subprocess is a better solution, so I'll rewrite my
code instead of struggling with that weired bug.

Thank you for help.

Paul Dmitryev
Senior Developer | ☭Postindustria - Exploit Us
pavel.d...@postindustria.com | p: +380 63 231 84 24 | skype: paul.dmitryev