How to do paging in Mongodb with node-mongodb-native cursor?

560 views
Skip to first unread message

Johnny Shi

unread,
Aug 19, 2011, 4:25:55 PM8/19/11
to node-mongodb-native
I am working on a REST API, I need to be able to page through MongoDB,
from what I understand cursor is the best way to do it. I can get the
cursor, but how to serialize it to send it to the client? and how
would the server de-serialize it when it comes back, and query with
it? Is this even possible?

collection.find({}, function(err, cursor) {
if (err) { console.log("Error in find: " +
err); return;}
cursor.each(function(err, item) {
if (err) { throw err;
}
if (!item) {
console.log("All done");
} else {

console.log(sys.inspect(item));
}
});
});

christkv

unread,
Aug 19, 2011, 4:35:31 PM8/19/11
to node-mongodb-native
You keep the query but also the limit and skip values, so when they
call the rest api for paging do this instead

collection.find(query, {skip:100, limit:100}).toArray/each/stream

don't attempt to serialize the cursor as it won't work.

in you rest api do

/api/users?limit=100&skip=100

and translate that to the find query above how you see fit.

Johnny Shi

unread,
Aug 19, 2011, 4:39:07 PM8/19/11
to node-mongodb-native
From what I heard,

collection.find(query, {skip:100, limit:100}).toArray/each/stream

is not fast, because MongoDB will walk the first 100 docs, is that
true with nodejs-mongodb-native? If so, would Range query be the best
way to do paging then?

Regards,

Johnny

christkv

unread,
Aug 19, 2011, 4:59:34 PM8/19/11
to node-mongodb-native
That I don't know :)

The driver does not walk the 100 documents, as the Query Wire Command
actually contains fields for limit and skip. I would be surprised if
the db did that. If you try ranged queries and have time to check if
it is slower please post it here as I'm curious if this is the case
myself.

christkv

unread,
Aug 19, 2011, 5:02:07 PM8/19/11
to node-mongodb-native
One more thing if you do need very very high throughput on a rest api
I would consider something like varnish to sit before it caching :).
It will make your life a lot easier if you expect a lot of traffic as
it can handle the http caching and is insanely fast. Where I work we
use it infront of our internal rest api with great success.
Reply all
Reply to author
Forward
0 new messages