Fixing Cursor TimeOut in Java Driver

[pouria....@gmail.com]

I am using MongoDB 2.6.3, sharded, and I am running my queries using Java Driver 2.11.3 .
On my application side, I have a query which runs a find() request against MongoDB (which returns a huge number of documents) and then using the returned cursor, it retrieves matching documents one-by-one and processes each separately (before moving to the next matching document in Cursor). As the processing takes some time and find() returns many documents, I get time-out exception on my cursor, which breaks the query in the middle and leaves me with incomplete results. I have seen several posts on cursor timeOut issue in MongoDB, but most of them were old (back from 2010) and they are not talking about the case when one is using the Java Driver.

Given the fact that I am using latest MongoDB and my client is Java Driver, and I am wondering what is the best/correct way to increase/disable timeOut for my curosor ?

 

[Rob Moore]

A couple of ideas.

1) Set your batch size to something very small (1 or 2). The time between the requests to the server (which is the only time that matters for a cursor timeout) is based on the time your application takes to process a batch of documents.  By default that is around 100 documents. By cutting the batch size down you decrease the time between the requests to the server. You should set the batch size before calling hasNext() or next() in the cursor.

2) Set the magical QUERYOPTION_NOTIMEOUT via the BDCursor.setOptions(int) method. This will stop the server timing out the cursor - ever. There is a risk that if your client application dies before the cursor is exhausted then the cursor can only be cleaned up by a restart if the mongod server. If you are not using a snapshot option then the cursor is just a little memory but you certainly don't want them piling up. You must set the options before calling hasNext() or next() on the cursor.

If you go with the second option you might want to save off the cursorid and server address so if there is a failure you can try and recreate the cursor to close it. If don't see a way to do it with the MongoDB Inc Java Driver but the Allanbank driver has it built in. You could even save the information from your iteration using the MongoDB Inc. driver and then construct the appropriate document with the Allanbank driver to then cleanup the cursor if there is a failure.

Rob.