Java driver batch size
Allen
trying to modify some of my queries to make a single database
roundtrip. These queries return just under 60,000 documents, so I set
my cursor's batch size to 60000, thinking that would do the trick.
However, the number of roundtrips did not change: each time, the
driver calls Result._advance() three times. The first time it's
called, _numFetched is usually ~8000. Why is the batch size seemingly
ignored? I tried disabling the sort on my cursor (it uses an index),
but that didn't make any difference.
Scott Hernandez
round-trips. It really depends on how large the 60K docs are.
The server will limit the response (batch) to the min(batchSize,
maxResponseBytes).
> --
> You received this message because you are subscribed to the Google Groups "mongodb-user" group.
> To post to this group, send email to mongod...@googlegroups.com.
> To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.
>
Allen
that the server is placing on the batch size? min(batchSize,
maxResponseBytes) explains why three roundtrips are occurring: 60K
docs is ~9MB in my collection. I was just curious as to how the
server/client would perform if I could retrieve all of those documents
in one roundtrip...
Scott Hernandez
> So, there is no way for the Java driver to override this constraint
> that the server is placing on the batch size? min(batchSize,
> maxResponseBytes) explains why three roundtrips are occurring: 60K
> docs is ~9MB in my collection. I was just curious as to how the
> server/client would perform if I could retrieve all of those documents
> in one roundtrip...
That is a server limit and nothing you can change with the batchsize
from the client, unfortunately.
Unless your network latency is very high then a few batches shouldn't
make too big a deal, nor would it reduce the total time to process all
the docs. How long does it take to process all those docs?
If it is very slow it could be that the issue isn't with the
client/server interaction but with finding or retrieving the
documents. Can you compare the performance from the mongo javascript
shell with java
to see if one is very different from the other?
Allen
about what setting the batch size actually does. We were profiling
some of our queries, and noticed that there were more calls to
Result._advance() than we expected, given the 60K batch size. You've
explained why, so I'm content :-). However, when I have some time,
I'll do a bit of comparison between the shell and our app. Thanks for
your help!
On Jan 26, 3:52 pm, Scott Hernandez <scotthernan...@gmail.com> wrote: