findAndModify throws exception

[RW Novak]

Using: db version v1.8.3, pdfile version 4.5

When findAndModify is called with an update doc that contains an _id
field, upsert is true, and new is true, an exception is thrown if the
query does not match an existing doc.
For example, in the shell:

> db.coll1.find();
{ "_id" : ObjectId("4e901e87d6d170982fc87ee1"), "name" : "Object 1",
"someProp" : "Property Value for 1" }
{ "_id" : ObjectId("4e901f30d6d170982fc87ee2"), "name" : "Object 2",
"someProp" : "Property Value for 2" }

> db.coll1.findAndModify({ "query" : { "name" : "asdf" }, "update" : { "name" : "Object 3", "someProp" : "3" }, "upsert" : true, "new" : true } );
{
"_id" : ObjectId("4ea84bbc66ac51d939de73b3"),
"name" : "Object 3",
"someProp" : "3"
}

> db.coll1.find();
{ "_id" : ObjectId("4e901e87d6d170982fc87ee1"), "name" : "Object 1",
"someProp" : "Property Value for 1" }
{ "_id" : ObjectId("4e901f30d6d170982fc87ee2"), "name" : "Object 2",
"someProp" : "Property Value for 2" }
{ "_id" : ObjectId("4ea84bbc66ac51d939de73b3"), "name" : "Object 3",
"someProp" : "3" }

> db.coll1.findAndModify({ "query" : { "name" : "asdf" }, "update" : { "_id" : 4, "name" : "Object 4", "someProp" : "4" }, "upsert" : true, "new" : true } );
Wed Oct 26 14:05:30 uncaught exception: findAndModifyFailed failed:
"exception: assertion db/../util/../util/../db/../bson/
bsonobjbuilder.h:119"

> db.coll1.find();
{ "_id" : ObjectId("4e901e87d6d170982fc87ee1"), "name" : "Object 1",
"someProp" : "Property Value for 1" }
{ "_id" : ObjectId("4e901f30d6d170982fc87ee2"), "name" : "Object 2",
"someProp" : "Property Value for 2" }
{ "_id" : ObjectId("4ea84bbc66ac51d939de73b3"), "name" : "Object 3",
"someProp" : "3" }
{ "_id" : 4, "name" : "Object 4", "someProp" : "4" }


The server log says:

Wed Oct 26 14:05:30 [conn52] test Assertion failure !e.eoo() db/../
util/../util/../db/../bson/bsonobjbuilder.h 119
0x10003913e 0x100040056 0x100329b8f 0x100312bf2 0x1003136cf
0x1001e55ab 0x1001e9654 0x1002c4d9e 0x1002c948d 0x1003e86dd
0x1007e4689 0x7fff868a8fd6 0x7fff868a8e89
0 mongod 0x000000010003913e
_ZN5mongo12sayDbContextEPKc + 174
1 mongod 0x0000000100040056
_ZN5mongo8assertedEPKcS1_j + 342
2 mongod 0x0000000100329b8f
_ZN5mongo16CmdFindAndModify3runERKSsRNS_7BSONObjERSsRNS_14BSONObjBuilderEb
+ 6975
3 mongod 0x0000000100312bf2
_ZN5mongo11execCommandEPNS_7CommandERNS_6ClientEiPKcRNS_7BSONObjERNS_14BSONObjBuilderEb
+ 3090
4 mongod 0x00000001003136cf
_ZN5mongo12_runCommandsEPKcRNS_7BSONObjERNS_10BufBuilderERNS_14BSONObjBuilderEbi
+ 703
5 mongod 0x00000001001e55ab
_ZN5mongo11runCommandsEPKcRNS_7BSONObjERNS_5CurOpERNS_10BufBuilderERNS_14BSONObjBuilderEbi
+ 59
6 mongod 0x00000001001e9654
_ZN5mongo8runQueryERNS_7MessageERNS_12QueryMessageERNS_5CurOpES1_ +
12244
7 mongod 0x00000001002c4d9e
_ZN5mongoL13receivedQueryERNS_6ClientERNS_10DbResponseERNS_7MessageE +
334
8 mongod 0x00000001002c948d
_ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_8SockAddrE
+ 1309
9 mongod 0x00000001003e86dd
_ZN5mongo10connThreadEPNS_13MessagingPortE + 653
10 libboost_thread-mt.dylib 0x00000001007e4689
thread_proxy + 137
11 libSystem.B.dylib 0x00007fff868a8fd6
_pthread_start + 331
12 libSystem.B.dylib 0x00007fff868a8e89
thread_start + 13

Is this a known issue?

[Marc]

This appears to be a bug. Thank you for bringing it to our
attention! I have created Jira case SERVER-4157: findAndModify throws
exception when _id is in the update document and "new":true
https://jira.mongodb.org/browse/SERVER-4157

As a work-around, adding an _id value to your query document prevents
the exception from being thrown.

> db.coll1.findAndModify({ "query" : { "_id" : 0, "name" : "asdf" }, "update" : { "_id" : 4, "name" : "Object 4", "someProp" : "4" }, "upsert" : true, "new" : true } );

Thank you again for alerting us to this.

Sincerely,
Marc

[RW Novak]

Marc, Thanks for the reply. Curious behavior with the workaround
though. The exception is no longer thrown (yea!), but the
document returned is empty. I was expecting the new, inserted
document.

There are no errors in the server log and all the upserts are
persisted, as expected.

> db.coll1.findAndModify({ "query" : { "_id" : "asdf" }, "update" : { "name" : "Object 6", "someProp" : "6" }, "upsert" : true, "new" : true } );
{
"_id" : ObjectId("4ea9309266ac51d939de73b7"),
"name" : "Object 6",
"someProp" : "6"
}

> db.coll1.findAndModify({ "query" : { "_id" : "asdf" }, "update" : { "_id" : 7, "name" : "Object 7", "someProp" : "7" }, "upsert" : true, "new" : true } );
{ }

> db.coll1.find();
{ "_id" : ObjectId("4e901e87d6d170982fc87ee1"), "name" : "Object 1",
"someProp" : "Property Value for 1" }
{ "_id" : ObjectId("4e901f30d6d170982fc87ee2"), "name" : "Object 2",
"someProp" : "Property Value for 2" }
{ "_id" : ObjectId("4ea84bbc66ac51d939de73b3"), "name" : "Object 3",
"someProp" : "3" }

{ "_id" : 4, "name" : "Object 4", "someProp" : "4" }

{ "_id" : 5, "name" : "Object 5", "someProp" : "5" }
{ "_id" : ObjectId("4ea9306066ac51d939de73b6"), "name" : "Object 5",
"someProp" : "5" }
{ "_id" : ObjectId("4ea9309266ac51d939de73b7"), "name" : "Object 6",
"someProp" : "6" }
{ "_id" : 7, "name" : "Object 7", "someProp" : "7" }

[Marc]

Another good catch!

From my own experimentation, an empty document will be returned if the
_id value in the query document does not match the _id value in the
update document.

The following returns the expected result:
> db.test.findAndModify({ "query" : { _id:2}, "update" : { _id:2, "name" : "Object 2" }, "upsert" : true, "new" : true } );
{ "_id" : 2, "name" : "Object 2" }

I have created an additional Jira ticket:
SERVER-4163
findAndModify returns an empty document with "new" : true flag if the
_id value in the query document does not match the _id value in the
update document.
https://jira.mongodb.org/browse/SERVER-4163

Thanks again for alerting us to these issues!