skip speed

[Joost]

Hi all,

Is there a way to speedup queries using 'skip'? I'm using skip as
described on http://www.mongodb.org/display/DOCS/Advanced+Queries#AdvancedQueries-{{skip%28%29}}
but when I get into skip(1000000) (so skip 1mio documents) it gets
extremely slow.

Is there another way to walk through all documents in a collection? I
need to stream them to XML and want to do this in batches.

Cheers,

Joost

[Eliot Horowitz]

skip is always going to be slow.
one option is to store the last record you looked at use $gt

you can't just keep the cursor open? internally queries use a cursor,
so you can leave a cursor open and just stream in batches from it

> --
> You received this message because you are subscribed to the Google Groups "mongodb-user" group.
> To post to this group, send email to mongod...@googlegroups.com.
> To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.
>
>

[Joost]

Thanks. I will look into this cursor thing. I'm checking MongoDB in
combination with Sphinx (www.sphinxsearch.com) in Ruby. However
indexing a MongoDB collection using Sphinx (via XML) is hard. Probably
need to create a custom xml exporter written in C or something.

On Feb 2, 7:04 pm, Eliot Horowitz <eliothorow...@gmail.com> wrote:
> skipis always going to be slow.

> one option is to store the last record you looked at use $gt
>
> you can't just keep the cursor open?  internally queries use a cursor,
> so you can leave a cursor open and just stream in batches from it
>
> On Tue, Feb 2, 2010 at 12:59 PM, Joost <jo...@joopp.com> wrote:
> > Hi all,
>
> > Is there a way to speedup queries using 'skip'? I'm usingskipas

> > described onhttp://www.mongodb.org/display/DOCS/Advanced+Queries#AdvancedQueries-{{skip%28%29}}
> > but when I get intoskip(1000000) (soskip1mio documents) it gets