Update command with upsert creates duplicate documents on Concurrent requests

[narendra chava]

 I'm using C# mongodb driver

TestCase: 1

        [Fact]
        public void Should_Return_Success_Status_Code_For_All_Concurrent_Requests_DbCommand1()
        {
            //Arrange
            Parallel.ForEach(Enumerable.Range(1, 5), i =>
            {
                string cmd = "{ \"update\" : \"brands\", \"ordered\" : true, \"updates\" : [{ \"q\" : { \"id\" : \"brandid8cdae601-4ec3-411b-9793-62c69a8a0431\", \"fromDate\" : { \"$lte\" : \"2018-05-31T21:52:00+00:00\" }, \"toDate\" : { \"$gt\" : \"2018-05-31T21:52:00+00:00\" } }, \"u\" : { \"created\" : \"2018-05-31T21:53:00+00:00\", \"lastUpdated\" : \"2018-05-31T21:53:00+00:00\", \"status\" : \"Active\", \"fromDate\" : \"2018-05-31T21:52:00+00:00\", \"toDate\" : \"9999-12-31T23:59:00+00:00\", \"id\" : \"brandid8cdae601-4ec3-411b-9793-62c69a8a0431\", \"name\" : \"BrandName40e26234-43d6-499e-93a8-d34a54275e7f\" }, \"upsert\" : true }] }";
                Db.RunCommand(new JsonCommand<BsonDocument>(cmd));
            });

            //Assert
            string cmdDoc = $@"{{find: '{DocumentCollection.Brands}',filter:{{id:'brandid8cdae601-4ec3-411b-9793-62c69a8a0431'}}, }}";
            var result = Db.RunCommand(new JsonCommand<BsonDocument>(cmdDoc));
            var brandDocuments = result["cursor"]["firstBatch"].AsBsonArray;

            brandDocuments.Count.Should().Be(1);
        }

TestCase: 2

        [Fact]
        public void Should_Return_Success_Status_Code_For_All_Concurrent_Requests_DbCommand()
        {
            string cmd1 = "{ \"update\" : \"brands\", \"ordered\" : true, \"updates\" : [{ \"q\" : { \"id\" : \"brandid8cdae601-4ec3-411b-9793-62c69a8a0431\", \"fromDate\" : { \"$lte\" : \"2018-05-31T21:52:00+00:00\" }, \"toDate\" : { \"$gt\" : \"2018-05-31T21:52:00+00:00\" } }, \"u\" : { \"created\" : \"2018-05-31T21:53:00+00:00\", \"lastUpdated\" : \"2018-05-31T21:53:00+00:00\", \"status\" : \"Active\", \"fromDate\" : \"2018-05-31T21:52:00+00:00\", \"toDate\" : \"9999-12-31T23:59:00+00:00\", \"id\" : \"brandid8cdae601-4ec3-411b-9793-62c69a8a0431\", \"name\" : \"BrandName40e26234-43d6-499e-93a8-d34a54275e7f\" }, \"upsert\" : true }] }";
            Db.RunCommand(new JsonCommand<BsonDocument>(cmd1));

            //Arrange
            Parallel.ForEach(Enumerable.Range(1, 5), i =>
            {
                string cmd = "{ \"update\" : \"brands\", \"ordered\" : true, \"updates\" : [{ \"q\" : { \"id\" : \"brandid8cdae601-4ec3-411b-9793-62c69a8a0431\", \"fromDate\" : { \"$lte\" : \"2018-05-31T21:52:00+00:00\" }, \"toDate\" : { \"$gt\" : \"2018-05-31T21:52:00+00:00\" } }, \"u\" : { \"created\" : \"2018-05-31T21:53:00+00:00\", \"lastUpdated\" : \"2018-05-31T21:53:00+00:00\", \"status\" : \"Active\", \"fromDate\" : \"2018-05-31T21:52:00+00:00\", \"toDate\" : \"9999-12-31T23:59:00+00:00\", \"id\" : \"brandid8cdae601-4ec3-411b-9793-62c69a8a0431\", \"name\" : \"BrandName40e26234-43d6-499e-93a8-d34a54275e7f\" }, \"upsert\" : true }] }";
                Db.RunCommand(new JsonCommand<BsonDocument>(cmd));
            });

            //Assert
            string cmdDoc = $@"{{find: '{DocumentCollection.Brands}',filter:{{id:'brandid8cdae601-4ec3-411b-9793-62c69a8a0431'}}, }}";
            var result = Db.RunCommand(new JsonCommand<BsonDocument>(cmdDoc));
            var brandDocuments = result["cursor"]["firstBatch"].AsBsonArray;

            brandDocuments.Count.Should().Be(1);
        }

Testcase 1 creates duplicate documents whereas the Testcase 2 doesn't.

the only difference between test case 1 & 2 is , In Testcase 2 I make sure that the document already exists in the database before concurrent requests

Is this supposed to be the expected behavior ?

[Wan Bachtiar]

Is this supposed to be the expected behavior ?

Hi Nahendra,

Yes this is an expected behaviour for update when upsert is true.

Likely what happened is:

• Two (or more) updates come in with upsert:true for a document that doesn’t yet exist.
• Neither update finds the document so both threads try to insert it.
• If both updates contain the same value for a field that has a unique index this will trigger a duplicate key violation.
• If there are no unique index constraints you’ll end up with two identical documents (except for _id).

See also update: use unique indexes for more information.

There is also an open ticket related to this behaviour SERVER-14322. Feel free to add yourself as a watcher or upvote to receive updates on the ticket.

Regards,
Wan.

​