php driver: Mongo cursor timing out on a small collection.

[Deep]

I am seeing a weird issue with MongoCursor timeout. It is not easily
reproducible with a simple test. The issue is that the MongoCursor is
timing out while iterating through a small collection with
a 'Uncaught exception 'MongoCursorTimeoutException' with message
'cursor timed out (timeout: 30000, time left: 0:0, status: 0)'
and the stack shows MongoCursor->rewind() causing this.

The code that causes this is:-
$query = array();
$queryFields = array();
$cursor = $collection->find($query, $queryFields);
foreach ($cursor as $a=>$b) { //Times out here
$p = $a["ip"] . ":" . b["port"];
$available[$p] = 0;
}

This piece of code is part of a small function which is called every
two minutes or so..The collection contains just 2 documents. Also, am
unable to reproduce this with a simple test but it happens in my prod
envt frequently atleast once 1 out of 10 calls.

I checked open bugs related to MongoCursor timeout but couldn't find
anything related to this issue.

Any idea as to why this might be happening? Is there any better way to
debug this so that I can find out the root cause?

Happening in php driver 1.0.8 and 1.1.4. Mongodb version 1.6.4.

[Kyle Banker]

You need to turn off cursor timeouts. See here:
http://groups.google.com/group/mongodb-user/browse_thread/thread/8bbdea444b47043f/0330855bf9fa67f2?lnk=gst&q=php+cursor+timeout#0330855bf9fa67f2

Also mentioned in the PHP driver docs:
http://www.php.net/manual/en/class.mongocursor.php

> --
> You received this message because you are subscribed to the Google Groups "mongodb-user" group.
> To post to this group, send email to mongod...@googlegroups.com.
> To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.
>
>

[Nat]

Iterating through small collection shouldn't cause cursor timeout? You
might do some write operation that locks mongod. Can you attach mongod
log file?

[Deepak Mallya]

I am not doing any explicit locking. The other thing is that I run map-reduce collections every hour but it is on a completely different collection. I don't see any errors in the mongod logs.

[Deepak Mallya]

Hi Kyle,

This is a very small collection with just 2 documents.  Is this a known issue with the MongoCursor timeout?

[Gates]

@Deepak:

It seems like you a few issues. Let's break these down.

#1: how does cursor timeout work?
------
The MongoDB PHP driver has two different timeouts. One is a
"connection" timeout, the other is for "cursors". The "cursor" timeout
indicates how long you will wait for getting the next element from a
find() command.

Docs are here:
http://www.php.net/manual/en/mongocursor.timeout.php

Basic syntax is the following:
$cursor = $collection->find($query)->timeout($milliseconds);

#2: why you're timing out
------
The error you're getting basically says that you're taking more than
30 seconds to load a document from your collection.

> Any idea as to why this might be happening?

A couple of things to look at.
1. Are you writing to this data from somewhere else? If you have too
many writes waiting to happen, then your read could be waiting on all
of the writes.
2. Are you dropping the collection at any point?
3. Are you doing lots of writes to _other_ collections? MongoDB only
has one write thread for the whole instance, you could be
inadvertently locking this data.

At the least, it would be nice to know what else is happening on the
server. Are you just doing this one call and the map-reduce? Usually
people doing map reduces are first writing lots of data.

- Gates

[Nat]

There might be warning that certain queries take longer. Can you send
me your mongod log? Let me sift through it.

[Deep]

Thanks to everyone for the answers.

We figured out the root cause of this issue and it seems to have been
due to our firewall settings between the client & the router. We were
using persistent connections which has a default keep-alive of 2 hours
and had a tcp connection timeout set to 30 mins at the firewall. After
capturing tcp dumps we figured out that the packets from the client
were not reaching the router as idle connections were dropped at the
firewall.

[Gaetan Voyer-Perrault]

@Deep: That makes sense. Thank you for closing this out.