pymongo and gevent module in different threads

90 views
Skip to first unread message

Stephan

unread,
Nov 15, 2012, 8:22:51 AM11/15/12
to mongod...@googlegroups.com
Hello,

in my application I use the gevent event loop in a separate thread for a dedicated and optimized task. To run the gevent event loop in a another thread than the main thread, means to import gevent lazily.
The other parts of my application use the old threaded/synchronous approach. Unfortunately in the threaded/sync parts of the code I'm using pymongo as well.  However with newer versions of pymongo gevent is imported at module initialization time of pymongo. But this makes the lazy import in my other thread failing.


Here is a simplified example how to trigger the error. I executed with the following versions
* cpython2.7
* pymongo == 2.3
* gevent == 0.13.8


import pymongo
import threading

def error():
    import gevent
    gevent.sleep(1)

t = threading.Thread(target=error)
t.start()
t.join()


To make it running I have to
* import pymongo lazily everywhere
* ensure that the thread, running the gevent event loop, runs first to do the 'gevent import'
Or
* running the gevent-event-loop in the main thread

Both options are pretty hard to implement on my side, so I'm kindly asking if pymongo can defer the import of 'gevent' until it really uses its features ?
If this change is accepted, I can write a patch of my own and send a pull request here https://github.com/mongodb/mongo-python-driver

Important note to avoid confusion: I'm *not* using pymongo in the thread running gevent. So I do *not* need support for pymongo running with threads and gevent at the same time. In need pymongo in the threaded environment, just without the gevent import hassle

A. Jesse Jiryu Davis

unread,
Nov 15, 2012, 3:47:16 PM11/15/12
to mongod...@googlegroups.com
I understand your point. If you'd like to offer a pull request, please do!

A. Jesse Jiryu Davis

unread,
Nov 15, 2012, 3:48:23 PM11/15/12
to mongod...@googlegroups.com
Be aware that we're about to do a very large change, including renaming the connection.py and replica_set_connection.py files, so you might be better off waiting until that's done in the next week. Your choice.

Stephan

unread,
Nov 16, 2012, 1:30:41 AM11/16/12
to mongod...@googlegroups.com
Thanks for the warning. I agree, waiting for the big changes makes the merge easier.

A. Jesse Jiryu Davis

unread,
Nov 26, 2012, 11:32:21 AM11/26/12
to mongod...@googlegroups.com
Note to all, Stephan submitted a patch here and we're continuing the discussion on Github:

Reply all
Reply to author
Forward
0 new messages