Re: Handling Array of Large size in Map/Reduce Mongo

[gregor]

Can you share the exact error message you are receiving?

On Tuesday, August 28, 2012 10:08:00 AM UTC+1, Maruthi Shanmugam wrote:

Hi 

We want to handle array of huge sizes which is generated as a result of Map/Reduce. 

The challenge we are facing is , when the array.length is more than 13K , the map/reduce job fails  stating that array is too large.

Can you please let us know ,how to overcome this challenge. 

Regards

Maruthi S

[Maruthi Shanmugam]

com.mongodb.CommandResult$CommandFailure: command failed [command failed [mapreduce] { "serverUsed" : "XXXX" , "assertion" : "value too large to reduce" , "assertionCode" : 13070 , "errmsg" : "db assertion failure" , "ok" : 0.0}

[Scott Hernandez]

Is there any related log entries on the server at this time?

Each document is limited to much less than 16MB (8MB if I remember
correctly) so you can't have an array, or any value in the doc, which
will exceed this for any given emitted key (the group by part on the
emit call). Do you expect that you are trying to do this by creating a
large array? If so, can you create multiple emitted documents during
the map phase to break things up?

> --
> You received this message because you are subscribed to the Google
> Groups "mongodb-user" group.
> To post to this group, send email to mongod...@googlegroups.com
> To unsubscribe from this group, send email to
> mongodb-user...@googlegroups.com
> See also the IRC channel -- freenode.net#mongodb