I have an aggregation that uses limit at the end to keep the results under the 16MB limit. However, when I run it in a sharded environment, it fails. The $limit comes after a $sort - and I guess that mongos has to get the entire sorted shard results back and interleave with the other shards in order to produce the full sorted list that gets limited.
Is there anyway around this? Can I output the results of aggregation to a temporary collection?
Is this no way to better limit the amount of documents you're including by having better constraints in a $match - With out seeing your actual query, I believe that would be the first place I would look.
On Thursday, October 4, 2012 10:20:24 AM UTC-7, Mark Hansen wrote:
> I have an aggregation that uses limit at the end to keep the results under > the 16MB limit. However, when I run it in a sharded environment, it > fails. The $limit comes after a $sort - and I guess that mongos has to get > the entire sorted shard results back and interleave with the other shards > in order to produce the full sorted list that gets limited.
> Is there anyway around this? Can I output the results of aggregation to a > temporary collection?
I also assume that you've tried the aggregation without the sort? In general, can you try your query w/ and w/o each successive step in your aggregation pipeline to see if the exception when and where that exception is throw?
On Thursday, October 4, 2012 1:20:24 PM UTC-4, Mark Hansen wrote:
> I have an aggregation that uses limit at the end to keep the results under > the 16MB limit. However, when I run it in a sharded environment, it > fails. The $limit comes after a $sort - and I guess that mongos has to get > the entire sorted shard results back and interleave with the other shards > in order to produce the full sorted list that gets limited.
> Is there anyway around this? Can I output the results of aggregation to a > temporary collection?