Concurrency race condition with $push

[Reid Morrison]

We are re-writing existing code that calls MongoDB directly through the Ruby driver to use Mongoid instead. We have one race condition where several servers may be processing the same identity at the same time. Each server after loading the Identity document needs to add a new first_name and last_name to an embedded list if it is not already there, along with a counter.

The challenge is that if 2 servers load the same document, they both see that the name is not already in the list, they will both perform a $push to add the name atomically with a count of 1 to the same identity document in Mongo. This results in the name being added twice to the list. To prevent this condition we added the $not to the where clause to ensure that the name had not already been added in the milliseconds since the document was retrieved, as follows:

identities.update(
            {'_id' => identity['_id'], 'names' => {'$not' => {'$elemMatch' => {'first_name'=>first_name, 'last_name' => last_name}}}},
            {'$push' => {'names' => {'first_name'=>first_name, 'last_name' => last_name, 'count'=>1}}},
            {:safe => true}
          )}
If the update fails to update any documents, it then does a $inc on the document since the name is now there.

Any suggestions on how to make this work in Mongoid?

Thank you
Reid

[Reid Morrison]

Below is an example of what this document looks like:

{
    "_id" : ObjectId("4e3af068f41fd52120000001"),
    "names" : [
      {
        "first_name" : "JOE",
        "last_name" : "BLOGGS",
        "count" : 12
       },
      {
        "first_name" : "JOE",
        "last_name" : "BLOG",
        "count" : 1
       }
    ]
}

[raygao-gmail]

I had that question a few days ago. My solution was to move the domain logic from the DB into the application model. It's probably better for you to create an enhanced update function. And, you can add this hook into the model via callbacks.

http://rubydoc.info/github/mongoid/mongoid/master/Mongoid/Callbacks

before_update
your custom code ()
end

Since your web app is probably clustered, that race conditioned is no longer in the DBs, .i.e. in the app controller, you can use cookies / session variables to identify user who needs to save document.

-Ray

[Reid Morrison]

The problem is occurring in Resque workers where there is no session or cluster. The current Mongo solution works very well, it first tries to add the additional name, which works 99.9% of the time. In the 0.1% of the time the where clause prevents the update, the code just calls $inc. Switching this to some kind of application or database level locking mechanism would be very inefficient, since additional network calls would have to be made before and after any change was made to any document, when in only 0.1% of the cases would it be necessary.

Using Mongoid::Document#remove_change(:field_name) I can keep the Mongo update code and still modify the value in the Mongoid model without Mongoid also trying to make the change. So far this appears to be working very well for any Mongoid field.