Pymongo tailable curosr (await_data=true) blocks forever

[Umesh]

I'm using the MongoDB python driver 2.7.2. I use an infinite loop to poll new messages from a capped collection and process them as they come (with a tailable cursor and await data). It works for the most part, but in production, it seems that from time to time the call to cursor.next() blocks forever. This causes the application to hang forever until restart.

Why is this happening in the first place ? how to handle this situation ?

[Luke Lovett]

Cursors that have the "await data" flag set will block until there is more data available, instead of returning immediately with no data (https://docs.mongodb.com/manual/reference/method/cursor.addOption/#DBQuery.Option.awaitData). Calling cursor.next() will fetch more data if needed, which will block until there is something to return, or until the server kills the cursor (this will happen after 10 minutes, by default: https://docs.mongodb.com/manual/tutorial/iterate-a-cursor/#closure-of-inactive-cursors).

You might also find this example of using tailable cursors with pymongo useful: http://api.mongodb.com/python/3.3.0/examples/tailable.html. Keep in mind that this example is written for version 3.3 of PyMongo, which has a slightly different syntax than the old 2.x version of PyMongo, but the same concepts still apply.