MongoDB 2.4 Map-Reduce returns no results unless jsmode:true

[John Cole]

Trying out 2.4 before moving it to production, I've run across an issue with map-reduce.

The output of my MR replaces a collection, if I run it with jsmode:false, the collection is empty.  With jsmode:true, output is normal.

Using 2.2, jsmode:false works fine.

Here is my map reduce command.

db.runCommand({ mapreduce: "agents", 

 map : function Map() {

var data = { count: 1, systems : {}};

data.systems[this.sys] = 1;

emit(this.email, data); 

},

 reduce : function Reduce(key, values) {

var reduced = {count: 0, systems: {}}; // initialize a doc (same format as emitted value)

values.forEach(function(val) {

for(var key in val.systems) {

if (reduced.systems[key]) {

reduced.systems[key] += val.systems[key];

} else {

reduced.systems[key] = val.systems[key];

}

}

reduced.count += val.count; 

});

return reduced;

},

 finalize : function Finalize(key, reduced) {

/*  

// Make final updates or calculations

reduced.avgAge = reduced.age / reduced.count;

*/

reduced.syscount = 0;

reduced.maxcount = 0;

for(var sys in reduced.systems)

{

reduced.syscount++;

if (reduced.systems[sys] > reduced.maxcount) {

reduced.maxcount = reduced.systems[sys];

}

}

return reduced;

},

 query : { "sys" : { "$ne" : "stakeholders" }, "status" : "Active" },

 out : { replace: "jwc_test"  }, 

 jsMode : true 

 });

[Ben Becker]

Hi John,

To verify, was this encountered with the released v2.4.0 binary, or v2.4.0-rc2 (or earlier)?  When I run the included script on a sample dataset, the output collection is populated, but I also see duplicate fields for each finalized document.

It appears that your finalize function is updating properties of the 'reduced' argument.  This behavior was changed recently; the arguments were previously considered read-only, but they are now mutable.  The Spidermonkey implementation did not honor the read-only flag, but the behavior is slightly different.  See SERVER-8624 for some details on this change.

For an immediate solution, I would suggest modifying your finalize function so that only primitive values are copied from the object.  This can be done 'manually' for each element you want to copy (e.g. reducedResults.maxcount = reduced.system[sys]), or you can perform a deep copy.  I've filed SERVER-9066 to fix the duplicate field name issue.

     Best Regards,

     Ben Becker

[John Cole]

Ben,

  Yes, that was on the release version of 2.4.0 but I didn't get any returned values when the jsMode was false.  Tried again with 2.4.1 and I'm seeing the duplicate values when jsMode is false, but not when jsMode is true.  I would expect both to operate the same, regardless of an issue in the finalizer.

  As to the finalizer function, how would one perform an average or any other post processing if you can't modify the the reduced value now?  Is there an example on the proper way to do this now?

Thanks,

John

[Ben Becker]

Hi John,

Apologies for the delay.  I found and fixed the bug that was causing duplicate field names.  Please see https://jira.mongodb.org/browse/SERVER-9066 for details.  This is currently slated for backport to v2.4.2.

Once released, could you give that a try, and report back if you experience any issues?

     Thanks,

     Ben

[John Cole]

Ben,

  I've tried 2.4.2 rc0 and I'm still seeing issues.  Does rc0 have the fix for SERVER-9066?

Thanks

John

[Ben Becker]

Hi John,

Version 2.4.2-rc0 should indeed fix the problem.  I just ran your test script with jsmode=true and jsmode=false, and both produced the correct results in v2.4.2-rc0.  What kind of issues are you seeing now?

Regards,

Ben