How come $addToSet and $push behave differently with change streams?

129 views

Skip to first unread message

Timo Kilpilehto

unread,

Mar 12, 2018, 5:25:43 PM3/12/18

to mongodb-user

Hi,

I've been exploring the change streams feature of 3.6 and as I was experimenting with it's behaviour regarding embedded documents. I discovered that using $addToSet always reports the entire embedded collection as changed in the updatedFields i.e. the entire collection array is there. However, using simple $push gives finer detail of the change as it reports the particular index as changed. I'm wondering if this is by design and perhaps why so or maybe it's a bug or what is the gotcha that I'm not getting?

Sample code:



db.createCollection("sample");
db.sample.insert({
  embedded: [{
    name: "initial"
  }]
});


printjson(db.sample.findOne({}));


watcher=db.sample.watch();


db.sample.update({}, {
  $addToSet: {
    embedded: {
      name: "added to set"
    }
  }
});


printjson(watcher.next());
// {
//   "_id" : {
//     "_data" : BinData(0,"glqmxgkAAAABRmRfaWQAZFqmxbuuM8l1+kCAmQBaEASPFrTqbjBBSJoRiNbRVccuBA==")
//   },
//   "operationType" : "update",
//   "ns" : {
//     "db" : "test",
//     "coll" : "sample"
//   },
//   "documentKey" : {
//     "_id" : ObjectId("5aa6c5bbae33c975fa408099")
//   },
//   "updateDescription" : {
//     "updatedFields" : {
//       "embedded" : [
//         {
//           "name" : "initial"
//         },
//         {
//           "name" : "added to set"
//         }
//       ]
//     },
//     "removedFields" : [ ]
//   }
// }


db.sample.update({}, {
  $push: {
    embedded: {
      name: "this was pushed"
    }
  }
});


printjson(watcher.next());
// {
//   "_id" : {
//     "_data" : BinData(0,"glqmxkwAAAABRmRfaWQAZFqmxbuuM8l1+kCAmQBaEASPFrTqbjBBSJoRiNbRVccuBA==")
//   },
//   "operationType" : "update",
//   "ns" : {
//     "db" : "test",
//     "coll" : "sample"
//   },
//   "documentKey" : {
//     "_id" : ObjectId("5aa6c5bbae33c975fa408099")
//   },
//   "updateDescription" : {
//     "updatedFields" : {
//       "embedded.2" : {
//         "name" : "this was pushed"
//       }
//     },
//     "removedFields" : [ ]
//   }
// }


watcher.close();

Wan Bachtiar

unread,

Mar 13, 2018, 9:41:54 PM3/13/18

to mongodb-user

I’m wondering if this is by design and perhaps why so or maybe it’s a bug or what is the gotcha that I’m not getting?

Hi Timo,

The updateDescription field returned as part of the Change Stream update event describes fields that were updated or removed by the update operation. i.e. delta or potentially affected.

Now let’s take a look at the update operators to understand the reason: $push by default just append the value to the end of the array. This is what you have observed, where it returns only the last array element index with the new value. An append does not need to see the other values in the array, it just append to the end of the array. This operation has no potential to affect the existing values in the array.

However, if you also specify $push modifiers such as $position where the operation needs to see the other values, the whole array would be returned. This is because the other values, are shifted along the array indexes. See also update/push_node.cpp#L193-226
For example:

db.sample.update({}, 
                 {"$push":{"embedded":{"$position":0, 
                                       "$each":[ {"name": "prepend"} ]
                                      }
                          }
})

The whole array would be returned in updateDescription.updatedFields:

...
  "updateDescription": {
    "updatedFields": {
      "embedded": [
        {"name": "prepend"},
        {"name": "initial"}
      ]
    }
...

This is similar to $addToSet update operator. The operator ensures that there are no duplicate items added to the array/set by running a deduplication. It goes through any existing value in the array to make sure the newly added value does not already exist. You may also noticed that if the new value already existed in the array there are no update operation, therefore no change stream event. See also update/addtoset_node.cpp#L106-141