Write-concern vs. Journaling

[pouria....@gmail.com]

Refering to MongoDB's documentation,

About Journaling, I see:
- Mongodb uses Write-ahead logging (Journaling) to guarantee durability of a write operation.
- Journaling is enabled by default.

About Write Concern I see:
- Write concern describes the guarantee that MongoDB provides when reporting on the success of a write operation.
- Default write concern is "Acknowledged".
- Acknowledged write concern does not confirm that the write operation has persisted to the disk system.

Now considering a use case, where I have Sharded MongoDB (with the default journaling/write-concern described above), with NO replica, and I am doing updates via Java Driver (from my client):

If by default, journaling is On, then Server only gets back to me, once the update logs are on disk, which means data is persistent. So how come Acknowledged mode does NOT guarantee the durability ?

Any explanation that helps me resolve the confusion would be great !

[toji....@10gen.com]

With Acknowledged, mongod confirms it has received your data and changed in-memory view of data. It does not acknowledge persistence to disk. Journaling set to true takes it one step further, acknowledging write after data committed to journal. 

Now journaling is enabled by default. That means you have a journal for crash recovery. Your writes still have to use it, if you want, by using the correct write concern.

Toji

[pouria....@gmail.com]

Thanks for reply.
So basically what you are saying is even when my client is inserting with "acknowledged" write-concern, MongoDB is still doing the Journaling (doing write-ahead-logging, once in-memory component of BTree gets changed), but the server does NOT wait so logs gets persistent on disk, and it immediately gets back to the client after applying in-memory changes. Please correct me if I am missing any point here.

Pouria

[toji....@10gen.com]

Yes. 

Then journals gets written to disk in batch called group commits on a frequency based on commitIntervalMs parameter. 

So, with Acknowledged, if server crashes before the write is committed to journals on disk, you could lose that write. Hence, Acknowledged mode does not guarantee durability.

Toji 

[pouria....@gmail.com]

Thanks for Clarification !

Just as a side question:
Is it possible to turn-off/on Journaling selectively ?
Specifically, what I mean is if I can import and load my data with Journaling Off (for it makes my loading much faster), but once I start using the collection through my application client to make updates, I turn Journaling On and use Journaled write-concern in my client, as I do care about durability at that time ?

Thanks,

[Asya Kamsky]

Changing journal setting (on vs off) requires a restart of mongod process, so it's not a practical option unless you are loading data into mongod for the first time ever.

Asya

--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
 
For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user...@googlegroups.com.
To post to this group, send email to mongod...@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-user.

To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/fbe069bf-5dc7-47ae-918e-72520412536e%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[pouria....@gmail.com]

I see, Thanks !
I can restart mongod processes.
All I wanna do is to load into an empty collection, with journaling Off (as it improves the performance of my bulk load) and then restarting the processes with journaling On, and doing updates with "Journaled" write-concern so the server can guarantee durability of the updates.

Pouria

[Asya Kamsky]

That's a common technique when first loading a large dataset into an
empty instance.

Same for replication - it's faster to load the data into a standalone
node and then convert it to a replica set than to make it a replica
and then load documents (that would have to be replicated one by one
basically).

Asya

> https://groups.google.com/d/msgid/mongodb-user/2f862f1d-fa78-4c45-b067-6d23702f102d%40googlegroups.com.