Re: MongoDB and inconsistent data

[Bytor99999]

By default the WriteConcern of Mongo is set to None, meaning your update is a send and forget and Mongo can do the save after you left, but it also means a read can come in before the write has occurred, and therefore not see the change. You can set the write concern globally in configuration or per write operation.

http://www.littlelostmanuals.com/2011/11/overview-of-basic-mongodb-java-write.html

btw. Are you at USC? I am in Seal Beach and an alumni.

Mark

On Wednesday, October 24, 2012 7:57:12 AM UTC-7, Sumita barahmand wrote:

Hi,

I am trying to implement some of the most popular social networking actions i.e. add a friend, delete a friend, accept friendship, view profile and etc using MongoDB.

To handle my queries I created a users collection where every user has an id, username, address and some other attributes. 

In addition, every user document has an array type attribute which keeps a track of the id for all of the user's friends.

For example:

user:{"_id":1, "username":"sumita", "friends":[2,3,4]}

user:{"_id":2, "username":"miki", "friends":[1]}

user:{"_id":3, "username":"nisha", "friends":[1]}

user:{"_id":4, "username":"lucky", "friends":[1]}

I have a multi-threaded benchmark for which every thread tries to emulate a user issuing a social networking action. For example thread 1 can emulate user with _id=1 deleting a friend from her friend list say friend with _id=2. And thread 2 can emulate a user i.e. user with _id=2 retrieving her list of friends. The first action will modify the documents to look as follows (as the friendship relationship is symmetric):

user:{"_id":1, "username":"sumita", "friends":[3,4]}

user:{"_id":2, "username":"miki", "friends":[]}

user:{"_id":3, "username":"nisha", "friends":[1]}

user:{"_id":4, "username":"lucky", "friends":[1]}

I have realized that with MongoDB when I increase the level of concurrency (increase the number of threads) or the percentage of update involved actions, there are cases in which the read observes inconsistent data. For example in the previous example, thread 2 may try accessing the list of friend ids for user with _id=2 and get [1] as the result instead of [] which is wrong.

I have read that MongoDB has one global collection lock so to my understanding  the read can only access a record once the updates on that record have completed. And this means that I should not see any inconsistent data.

Any kind of help or suggestions is appreciated.

Sumita Barahmand

http://www-scf.usc.edu/~barahman/

http://dblab.usc.edu/users/bg/

[Sumita barahmand]

Hi Mark,

Thanks for the prompt response. I have always been using the Normal writeConcern. Probably I should try the safe writeConcern. I will try it and let you know if I succeeded.

Yes I go to USC. I'm a 4th year PhD student in the database lab:) 

Sumita

[Bytor99999]

Cool. I think Normal is the default, which is send and forget. Meaning on the client side it sends the update to Mongo, then goes its merry business. Something like safe, would be that you get a reply back, so you know something happened and you can look at that response to see what happened, errors or number of documents updated etc.

So a PhD in databases? Sounds, well, um, great for you. You will have a great future in that line. I am actually a former recovering Oracle DBA(2000), and have worked with databases for over 20 years. Actually every job and application had databases. I still have dreams of Third Normal form and Star Schema modeling.  I graduated USC in 1990 with a degree in Communications. Fight On!

Mark

[Sam Millman]

The recommended default is for full WriteConcern (safe writes basically), there has been talk on here of implementing it by default but the answer that went around is that there are many who still rely (new and old) on this feature so to do that would break a lot of apps out there.

--
You received this message because you are subscribed to the Google
Groups "mongodb-user" group.
To post to this group, send email to mongod...@googlegroups.com
To unsubscribe from this group, send email to
mongodb-user...@googlegroups.com
See also the IRC channel -- freenode.net#mongodb

[Bytor99999]

Well that setting can be set at the database level. But more importantly on each operation. So in an update() call you can pass in the WriteConcern just for that update(). Because some use cases might need a higher WriteConcern where another less. But it would cause those other apps to change. But I think it gives you a lot more flexibility and seems more correct way to do it.

Good Luck

Mark