pymongo: LoopExit: ('This operation would block forever', ...)

350 views
Skip to first unread message

michal....@intel.com

unread,
Dec 14, 2017, 5:45:07 PM12/14/17
to mongodb-user
Hi,

I'm using pymongo with django 1.10 and I started to encounter serious problem after switching from multiple MongoClients (created on demand) to single global object following this gist: https://gist.github.com/josephmosby/4497f8a4f675170180ab
The first approach was bad, because connections handles were not freed properly leading to application crash (too many files open error).
The second approach however works fine for the first HTTP request, but during the next request the first call on mongo collection (find_one) leads to the error below.
This is the same MongoClient object.
As I undestood, pymongo uses gevent as socket backend. Is it possible that somethings corrupts its state?
I tested using django built-in server (manage.py runserver)

Traceback:

Traceback (most recent call last):
  File "/home/user/pycharm_env/local/lib/python2.7/site-packages/django/core/handlers/exception.py", line 42, in inner
    response = get_response(request)
  File "/home/user/pycharm_env/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 249, in _legacy_get_response
    response = self._get_response(request)
  File "/home/user/pycharm_env/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 187, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "/home/user/pycharm_env/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 185, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)

    [secret code]

  File "/home/user/pycharm_env/local/lib/python2.7/site-packages/pymongo/collection.py", line 1224, in find_one
    for result in cursor.limit(-1):
  File "/home/user/pycharm_env/local/lib/python2.7/site-packages/pymongo/cursor.py", line 1176, in next
    if len(self.__data) or self._refresh():
  File "/home/user/pycharm_env/local/lib/python2.7/site-packages/pymongo/cursor.py", line 1087, in _refresh
    self.__send_message(q)
  File "/home/user/pycharm_env/local/lib/python2.7/site-packages/pymongo/cursor.py", line 924, in __send_message
    **kwargs)
  File "/home/user/pycharm_env/local/lib/python2.7/site-packages/pymongo/mongo_client.py", line 1026, in _send_message_with_response
    exhaust)
  File "/home/user/pycharm_env/local/lib/python2.7/site-packages/pymongo/mongo_client.py", line 1037, in _reset_on_error
    return func(*args, **kwargs)
  File "/home/user/pycharm_env/local/lib/python2.7/site-packages/pymongo/server.py", line 106, in send_message_with_response
    reply = sock_info.receive_message(request_id)
  File "/home/user/pycharm_env/local/lib/python2.7/site-packages/pymongo/pool.py", line 529, in receive_message
    self._raise_connection_failure(error)
  File "/home/user/pycharm_env/local/lib/python2.7/site-packages/pymongo/pool.py", line 649, in _raise_connection_failure
    raise error
LoopExit: ('This operation would block forever', <Hub at 0x7f42e4434a50 epoll pending=0 ref=0 fileno=39>)

Regards,
Michael

Bernie Hackett

unread,
Dec 15, 2017, 1:26:45 AM12/15/17
to mongodb-user
This seems to be an issue in Gevent, happening with a number of libraries. The speculation is not monkey patching early enough:

Reply all
Reply to author
Forward
0 new messages