Mongodb Performance

[Amit Patel]

Hello Experts,

I am new to mongodb , I want to ask some questions related to mongodb performance.

1:- Below URL is mentioning that mongodb automatically uses the free memory of server and keeps the active working set (Data and index) in memory then what is the use of touch command ? i mean if server automatically kepp data in memory then why we need to execute touch command to load data in memory.



http://docs.mongodb.org/manual/faq/fundamentals/

2:- how to improve the performance of write queries since write query will directly get execute on disk drive then there would be so many Disk I/O on the server and it will decrease the performance of the server.

3:-  is there any tool or script which can easily show all the slow queries?

4:- is there there any tool which can check and show all the collection which are not properly indexed and tell us what index we should create?

Please let me know if you need any other details.

Thanks.
Amit 

[Adam C]

if server automatically kepp data in memory then why we need to execute touch command to load data in memory.

The server will keep data in memory after it has been used, but it does not pre-fetching of data from disk.  That is what the touch command is for, to remove the first use penalty for that data.  Imagine you are starting mongod immediately after a reboot of a server for example, if touch was not available you would have to manually access all of the data in order to load it into memory.

how to improve the performance of write queries since write query will directly get execute on disk drive then there would be so many Disk I/O on the server and it will decrease the performance of the server.

That is not how writes work, it is more complex than that.  A combination of immediate syncing to the journal (every 100ms) and then a sync to the data files themselves (every 60 seconds) is used.  Yes, you still have to persist data to disk, but all operations are done in memory first for speed and then persisted separately.

is there any tool or script which can easily show all the slow queries? 

First, all slow queries are logged in the log files.  There are many utilities to help you parse these messages and the profiler should you chose to use it.  I have used Dex and mtools, I am sure there are others.

is there there any tool which can check and show all the collection which are not properly indexed and tell us what index we should create?

There are some ways to use the tools I listed to determine what indexes you might need, but it is different for each workload and data set, you really need to analyze your usage and make your own decisions - there is no simple set of instructions to suit everyone.

Adam 

[Amit Patel]

Hello Adam,

Thanks for the help.

I really appreciate your quick response and apologies for responding to you very late.

You said in your reply that "all operations are done in memory first for speed and then persisted separately" So my question is, if all the mongo write operation happens in memory and i have enough RAM in my server then  we will never face  too many disk I/O or CPU high load related issue on the server even when there are so many write operation are getting execute on the server because all the write operation are happening in memory. am i right?

Another one thing, do i need to do any extra configuration in mongo to do write operation in memory or mongo will automatically always do write operation in memory?

And if the write operation are happening in memory then how the data will get persist on the disk? do i need to do some extra configuration to persist data on disk or mongo will automatically persist data on disk separately?

And i am trying to understand how the write queries get execute in memory, does mongo put whole table in memory and then write queries get execute in memory as we do in mongo shell?

And my last question is, can we setup a architecture like this  in mongo, where all the write queries will get execute on mongodb master server and all the read queries will get execute on slave servers as we do in MySQL, I know mongo db support replication but this url "http://docs.mongodb.org/manual/core/read-preference/#replica-set-read-preference"  says that,   secondaries replicate operations from the primary with some delay so  Ensure that your application can tolerate stale data if you choose to use a non-primary mode" And i have a application which   require fully up-to-date data, so is it not possible for me to use this type of architecture?

Please tell me if you need any details.

Thanks.
Amit

--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
 
For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user...@googlegroups.com.
To post to this group, send email to mongod...@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-user.

To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/a8cdfc22-2067-4e63-b3cc-fa38aeac4891%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Asya Kamsky]

Amit,

First, since you already found the MongoDB docs you should read the section on storage: http://docs.mongodb.org/manual/faq/storage/

Since the data has to be flushed to disk, you absolutely can run out of IOPs.

Second, while you can configure reads to go to secondaries, it is not a good idea, nor is it necessary since MongoDB works differently than MySQL so you won't necessarily get the same performance improvement by reading from secondaries (plus the secondaries also have to make all the same writes as the primaries).

Asya

To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/CANyEVrNT68vzAcfEG%3D0MyLyGChmPec3_D9m_ig6dde2ijt75Rg%40mail.gmail.com.

[Amit Patel]

Hey Asya and Adam,

Thanks for reply.

I said in the beginning that i am new to mongodb.

That'swhy i asked some questions in very simple language.

i read this url http://docs.mongodb.org/manual/faq/storage/ but i didn't get my answer.

So my questions are still same, could you please give m answer of below questions.

do i need to do any extra configuration in mongo to do write operation in memory or mongo will automatically always do write operation in memory?

And if the write operation are happening in memory then how the data will get persist on the disk? do i need to do some extra configuration to persist data on disk or mongo will automatically persist data on disk separately?

And i am trying to understand how the write queries get execute in memory, does mongo put whole table in memory and then write queries get execute in memory as we do in mongo shell?

And if we can not split the read and write queries between the master and slave then what is the use of replication except backup and without replication how can i improve the performance of read queries?

Let me know if you need any further detail from my end.

Thanks.
Amit.

[Asya Kamsky]

Amit,

I'm sorry your questions were not answered by the FAQ section.

MongoDB writes data first in RAM (via memory mapped files) both into data files and journal and they are then fsync'ed to disk.  You do not to do anything - that's just how it works.

Replica sets are not for scaling - sharding is for scaling, replica sets are for High Availability, disaster recovery, maintenance without down time and to have extra copies for backups, ETL/atypical long running queries, analytics, etc.   See more here:   http://askasya.com/post/canreplicashelpscaling

Asya

To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/b79a57da-6a7d-4bc7-98c6-275fcf29add7%40googlegroups.com.