MongoDB with --nojournal

[Amir Saman Memaripour]

Hello everyone,

I want to have MongoDB without journaling for a research project. Running mongod with  --journal, the storage engine will write-back the data to the underlying files on disk using msync every 200ms. As there is no journal file in --nojournal mode, I was wondering if MongoDB uses the same approach and call msync periodically or it just writes back data to disk for every write operation? In case the write-back is still periodic in --nojournal mode, what's the best way to make MongoDB do that per write?

Thanks

[Asya Kamsky]

What exactly is the nature of the research project?

I'm not sure where you found the information you list but it's not correct.  The journal is flushed by default every 100ms if the journal is on the same volume as the data files, or every 30ms if it's on a separate volume.  The data files are flushed every 60 seconds by default, and that's done asynchronously.

If you run mongod with --nojournal there will be no difference to how it flushes the data files - still every 60 seconds.  There will not be anything flushed more often than that.   The data is written to the memory mapped files in both cases, and they are flushed once a minute in both cases.

You might want to watch one of these two presentations which have a lot of details on MongoDB internals:

http://www.mongodb.com/presentations/mongosf-2012-journaling-and-storage-engine

http://www.mongodb.com/presentations/storage-engine-internals

Asya

--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
 
For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user...@googlegroups.com.
To post to this group, send email to mongod...@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/821f6626-ce65-45d1-b341-ac97322ce412%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Amir Saman Memaripour]

Thanks for the reply. Is there any option that I can use to force MongoDB msync the memory mapped region to the corresponding file on every write operation? My other question is about the protected and copy-on-write view. Is there any way to force MongoDB only write on the protected-view?

Thanks

[Asya Kamsky]

It might be better if you describe exactly what you are trying to achieve in terms of requirements.

For example, writeConcern "fsync" will cause data files to be flushed on every write - normally it's the journal that would be flushed, not the data files, but if you turn off journal then it'll fsync the data files.  

What I'm trying to understand is why do you specifically want the data files fsync'ed rather than the journal (the former adds a significant amount of latency while the latter adds relatively speaking a lot less latency).

You also have to elaborate on your question about "protected and copy-on-write view" and if "there any way to force MongoDB only write on the protected-view" - trust me, it's better to explain what you are trying to achieve at the end, rather than asking very specific pointed questions, since there may be a different way to get what you need than a way you may be visualizing.   When the journaling is disabled, there is no private view, the files are mapped only once.

Asya

To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/395272d1-2e33-4661-81dd-413e4329b8fa%40googlegroups.com.

[Amir Saman Memaripour]

Thanks Asya, I think you addressed the solution for the issue. I am going to use "fsync" as the WriteConcern and see if it solves my problem.

What I want to do is using new memory technologies as the storage media for MongoDB. Currently, I'm emulating them using DRAM, so I have no intention for postponing write operations (as the latency of writing to the memory-mapped view and real data file is the same). I have another reason for turning off journaling. One reason is that I don't care about consistency at this point. Considering that, I have another question. Having mongod running with --nojournal option, what will be the normal behavior on write operations? Do we have the same procedure of creating WriteIntents or the data will be directly written to the memory-mapped region?

Thanks

[Asya Kamsky]

I guess I still don't quite understand what you mean by saying you "don't care about consistency" at this point - I hope you realize that this means if your mongod process crashes suddenly, your data files are not guaranteed to be consistent and the safest thing to do would be to start from scratch.

And I'm not sure what you are trying to test for - without journaling the data is written directly to the memory mapped files, is that what you meant?   Since there are multiple places to write and multiple pages that would have to be flushed to disk, it's possible some of the written pages will get flushed and others won't before the process crashes (for example).

Asya

To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/6296363b-45be-42ec-89dc-0a4e12926bf6%40googlegroups.com.

[Amir Saman Memaripour]

As it's an experiment and I just want to get numbers, I am not concerned about losing data. I just want to make sure that after writing some data while having WriteConcern=FSYNC and --nojournal, the data will be synchronized to the underlying data file.

Thanks

[Asya Kamsky]

Well, I can guarantee you that this is what happens when you have write concern fsync and nojournal.

We even have tests for that :)

Asya

To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/fec2bc28-1f8e-4913-bf6c-c5594bb4b6eb%40googlegroups.com.

[Amir Saman Memaripour]

Thanks, you saved me lots of time :)

[Amir Saman Memaripour]

One other question about running mongodb with --nojournal and having a client connected to it with WriteConcern = FSYNC. As every write operation (INSERT or UPDATE) might lead to more than one region of memory begin updated, I was wondering whether it will lead to a single msync or multiple msyncs.

For example, assume that I am going to insert a new record. As there is no more room for the record, mongod needs to create a new extent and update list of deleted records. Besides that, there might be another memory region for corresponding namespace that gets updated as a result of this operation. Will it result to multiple msyncs or just a single msync?

Thanks

[Felipe Albrecht]

May you point out where are these tests or how are they performed?

To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/CAOe6dJD%3Ds4s04O%3DOfHZwe%3DQg%2BYnj1meBopRfqrMjJw6sVCcP_Q%40mail.gmail.com.