|Aggregation Framework generally faster than MapReduce?||Mark Hansen||10/4/12 9:24 AM|
Has anybody tried a comparison of the same aggregation function using the AF vs. MR? If so, what (if any) performance improvements did you achieve? We have a MR library that we are thinking of converting over to AF and wondering if it will be worth the effort in terms of improved performance.
|Re: Aggregation Framework generally faster than MapReduce?||Mark Hansen||10/4/12 3:02 PM|
Thanks Sean. Did a quick test today on one of our long running MR transformations. The equivalent AF ran 7 times faster. So, I think we'll be porting a lot of code this weekend ... ;-)
On Thursday, October 4, 2012 1:04:12 PM UTC-4, Sean Reilly wrote:
TL;DR: Oh heck yes! It will almost certainly be worth it.
|Re: Aggregation Framework generally faster than MapReduce?||Sam Martin||10/4/12 3:15 PM|
Only downside is to output result to another collection requires the client to fetch the data and save it. with MR you can do it directly to another collection on the server... but i think this feature is coming for AF.
|Re: Aggregation Framework generally faster than MapReduce?||Mark Hansen||10/4/12 6:30 PM|
Yes, and some of our result sets are > 16MB, so we cannot us AF in those situations.