Causal consistency with Mongo 4.2.1 and GO driver 1.1.2

[Kostya Vasilyev]

( sorry for the cross-post to mongodb-user, this forum looks like the better place )

Hello,

I'm seeing some violations of "read what you wrote" with Mongo 4.2.1 and GO driver 1.1.2

This is on my development machine (Ubuntu), there is no cluster, just a single database, and it's quite small (hundreds of documents if that).

It has plenty of memory (32 gigabytes) and a fast disk (NVME). The file system is ext4 (I know XFS is preferred for performance, but there is barely any load here).

What happens is: one GO coroutine issues an update of a small document, setting a integer (Number) to a new value.

A second or so later, another thread does a read and does not see the write. Sometimes.

Repeating the read a few seconds later will then properly read back the new value.

Both threads (GO coroutines) use the same Mongo connection.

I tried to configure a read and a write concern like below, even though it's my understanding that "majority" is the default for both. It did not help.

readConcern := readconcern.Majority()

writeConcern := &writeconcern.WriteConcern{}
writeConcern = writeConcern.WithOptions(
   writeconcern.WMajority())

mongoConn, mongoErr := mongo.Connect(mongoCtx, &options.ClientOptions{
   Hosts:        []string{"localhost"},
   ReadConcern:  readConcern,
   WriteConcern: writeConcern})

From reading the docs*, I'm not clear if casual consistency should be automatically enabled for Mongo 3.6 or newer, or if it's something that needs to be enabled by the application or the driver.

If the former, why would it not be enabled?

If the latter, how do I enable casual consistency with the official GO driver?

[*] https://docs.mongodb.com/manual/core/read-isolation-consistency-recency/#sessions

[Kostya Vasilyev]

Sanity check....

I added code to read back the just updated data item on the same coroutine, in the same function which calls the update - and to check if the intended change made it through.

In the cases where my larger failure (between two coroutines) occurs, I also do get the debug printout that the update did not make it, the lines beginning with "***":

// Update the folder
_ = model.UpdateFolder(dbFolder,
   bson.M{
      "seed_update":  dbFolder.SeedUpdate,
      "unread_count": dbFolder.UnreadCount,
   })

// DEBUG check
sr := model.CollFolder.FindOne(context.Background(), bson.M{"_id":dbFolder.Id}, &options.FindOneOptions{})
srFolder := mail.SyncDbFolder{}
sr.Decode(&srFolder)
if srFolder.SeedUpdate != dbFolder.SeedUpdate {
   fmt.Printf("*****\n*****\n***** SeedUpdate does not match")
}

model.UpdateFolder:

res, err := model.CollFolder.UpdateOne(context.Background(),
   bson.M{"_id": folder.Id},
   bson.M{
      "$set": values,
   },
)

fmt.Printf("Updated folder: %d\n", res.ModifiedCount)

^^ this prints "Updated folder: 1" not 0

and finally the folder struct is defined as (some fields omitted):

type SyncDbFolder struct {
   Id        primitive.ObjectID `bson:"_id"`

   SeedValidity string `bson:"seed_validity"`
   SeedCreate   uint64 `bson:"seed_create"`
   SeedUpdate   uint64 `bson:"seed_update"`
   SeedDelete   uint64 `bson:"seed_delete"`

   UnreadCount uint64 `bson:"unread_count"`
}

Other times, when the large failure does not occur, the "***" printout does not appear either, so the update is written properly.

I've tried Mongo 4.2.1 on Ubuntu and Mongo 4.2.1 and 4.0 on Fedora (using el8 / el7 packages). The issue is present in all of them.

Is this starting to look like a bug in either the driver or maybe Mongo itself (if that's even possible)?

-- K

[Bradley Weston]

Someone correct me if I’m wrong but I think the consistency and write concern are not equal.

Your write can be guaranteed with a write concern but that doesn’t mean that the write has been successfully done yet.

Therefore the read can be stale, MongoDB is an eventually consistent database not like an ACID compliment one.

--
You received this message because you are subscribed to the Google Groups "mongodb-go-driver" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-go-dri...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-go-driver/a92a17d0-cd85-45df-884a-fc8e32fdaee0%40googlegroups.com.

[Kostya Vasilyev]

I'm not a Mongo expert by any means, but....

https://docs.mongodb.com/manual/core/causal-consistency-read-write-concerns/

Says that with read and write concerns both set to "majority" - one gets "Read own writes"

Maybe that page applies to replicated setups only, the discussion on much of the page does, but what about a single database scenario?

Or am I reading too much into what "read own writes" means?

But here the docs are more clear:

https://docs.mongodb.com/manual/core/read-isolation-consistency-recency/#sessions

---

To provide causal consistency, MongoDB 3.6 enables causal consistency in client sessions. A causally consistent session denotes that the associated sequence of read operations with "majority" read concern and write operations with "majority" write concern have a causal relationship that is reflected by their ordering. Applications must ensure that only one thread at a time executes these operations in a client session.

---

This to me fairly cleary says that in a single thread, a read will see any writes performed earlier by same thread.

And I'm seeing failures in my "sanity check" code which does a read after a write on the same coroutine.

Now coroutines are not threads, in theory the coroutine could switch threads just between the write and the read, but somehow it seems unlikely, given how I'm able to reproduce the failure time and again.

-- K

To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-...@googlegroups.com.

[Kostya Vasilyev]

Oh and another "sanity check" this time purely mental...

The whole database is ~5 megabytes. The system has an Core i7 9700 with 32 GB of RAM and an NVME drive.

Should then "eventual" - even if that's what I'm seeing - be basically a matter of milliseconds?

And yet, after a failed write, sometimes I still see the old value on reads after 10 seconds or more.

Doesn't make sense.

-- K

[Divjot Arora]

Hi Kostya,

It's very strange that writes aren't being picked up for 10 seconds. Could you provide some self-contained example code that we could run to try to reproduce the issue? Also, is the mongod process running with any specific configuration or just "mongod"? 

-- Divjot

[Kostya Vasilyev]

On Tuesday, November 5, 2019 at 12:41:23 AM UTC+3, Divjot Arora wrote:

Hi Kostya,

It's very strange that writes aren't being picked up for 10 seconds.

I'll clarify:

Sometimes changes will be picked up if I retry the read 2-3 seconds later.

Sometimes they're not there still after 10-15 seconds and I start a new testing round. I don't know if these do make it eventually or not.

 

Could you provide some self-contained example code that we could run to try to reproduce the issue?

Working on it, no luck so far.

 

Also, is the mongod process running with any specific configuration or just "mongod"? 

Default options and settings, I'm using mongo-org-server / -shell from the official Mongo package repositories and didn't make any config changes.

-- K

[Kostya Vasilyev]

Seems I can make the issue go away by using readConcern = Local

Is this an appropriate fix if I'm interested in SQL-like behavior, where a read that starts after a write has completed is guaranteed to see the results of that write?

-- K