Re: sharded pipeline failed on shard work arounds?

[Kamlesh]

Yes, that is correct. Our typical query will be returning 100,000's of docs quite frequently.

Any ways to do this? 

On Tuesday, October 2, 2012 7:09:40 PM UTC-4, Brandon Black wrote:

Hi Kamlesh,

In a sharded environment, your mongos will run the aggregation pipeline on each shard up to the first $group or $sort. It sounds like you're hitting the 16MB limit on each mongod's aggregation response to the mongos.

Is that correct?

On Monday, October 1, 2012 10:26:27 AM UTC-7, Kamlesh wrote:

Hi,
    So I have a sharded collection with 5M+ documents, and need to do some queries against it.
The queries are reaching the 16MB limit on a single shard itself, well before any $skip or $limit pipelines can be applied. This is even occurs after limiting results to minimum fields required.
Are there any other solutions to be able to query these amounts of documents?

Thanks!

[William Zola]

Hi Kamlesh!

As I mentioned in the other thread you opened, your other choices with MongoDB are using the built-in Map/Reduce command or the Hadoop adaptor.

 -William