Limit MongoDB RAM use

[Cédric Réali]

Hello, 

I have read in various places that it was impossible to limit the RAM used by MongoDB:
Virtual memory size and resident size Will APPEAR to be very large for the mongod process. This is benign: virtual memory space larger than Will Be just the size of the datafiles open and mapped; Will resident size vary DEPENDING on the Amount of memory not used by other processes on the machine.

Off, I just saw that it seems possible to do via "ulimit" (http://docs.mongodb.org/manual/reference/ulimit/).

Does one of you have already used this method to limit the use of RAM MongoDB? Does this can cause errors?

Thank you in advance.

Cordially.

NB: MongoDB is installed on 3 VM 8GB of memory and takes 98% sometimes ...

[anil]

Hi Cedric,

MongoDB uses all the free memory on a machine as its cache. Usually the monitoring tools will show MongoDB using lot of memory, but it's usage is really dynamic controlled by the underlying operating systems memory manager. If there is need to memory by any other component, the OS will take the memory from MongoDB instance and assign to other processes as needed. A lot of the resident memory is mostly used for caching the memory pages pages by the OS and is dynamic based on the usage.

Ideally it should work fine but we don't explicitly test MongoDB instances by limiting memory using ulimit. Please let us know how it goes for you!

Additionally I would also mention that limiting the amount of RAM to be used may affect your performance based on the index / data size. Since you are limiting the RAM to smaller size, what used to be served from cached data earlier may end up being paged-in / served from disk in limited RAM scenario.

Let me know if you have any additional questions.

-Anil

[William Zola]

Hi Cedric!

A couple of points:

1) It's normal and expected for MongoDB RAM use to grow to the amount of RAM installed on the machine.  You can manage this somewhat by changing the amount of RAM allocated to a virtual machine.

2) DO NOT try to limit MongoDB RAM use by setting 'ulimit'.  Doing so will cause MongoDB to crash, and can cause potential data loss.

3) If you're running MongoDB in production, you should not have any other RAM-hungry process running on that machine.

4) In general, you should take one of two strategies when limiting RAM usage for MongoDB

• Limit RAM usage by sizing the amount of RAM allocated to a VM
• If you're not using a VM, use 'cgroups' to limit the amount of RAM available to MongoDB.  For a user-friendly interface to 'cgroup', I'd recommend Docker

Note that it's probably not advisable to use Docker on top of a VM in production.

 -William 

[Cédric Ré]

Thank you for your answers.

I use 3 VM with 8GB of RAM on each. MongoDB is the only process on each VM (VM dedicated to MongoDB) 

The problem is that in MongoDB uses between 95 and 98%.

The monitoring (Nagios) send Warning when MongoDB use 98% of the RAM....So an email per hour ...

If it's impossible to limit, it's not a problem, but in this case I do not understand why MongoDB provides ulimit ...

Cordially.

[William Zola]

Hi Cedric!

The fact that MongoDB is using more than 95% of RAM on these VMs indicates that it is working properly and making maximum use of the memory on the VM.  

If you're getting alerts from Nagios, the problem is with your Nagios configuration, and not with MongoDB.

Note that 'ulimit' is provided by the operating system, and not by MongoDB.  Because MongoDB performs disk I/O and memory management completely differently than any other database out there, the 'ulimit' command does not work with MongoDB the way it does with other programs.  

I hope this helps!

 -William 

[Cédric Ré]

Thanks for your reply.

I have done well to ask for information before trying;). 

Cordially.

Le mercredi 30 avril 2014 10:11:59 UTC+2, Cédric Réali a écrit :

[Kishore Kumar]

Hi,

Can you please let me know how I can find the memory consuming queries. My swap memory always reaching to 100%.

[Asya Kamsky]

Please don’t reply to old threads and instead start a new thread and describe the problem you’re having in as much detail as you can (both about MongoDB version, configuration and OS, etc).

Asya

--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
 
For other MongoDB technical support options, see: https://docs.mongodb.com/manual/support/
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+unsubscribe@googlegroups.com.
To post to this group, send email to mongod...@googlegroups.com.
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/46daa004-2ad7-493d-8bce-96c280cf96c2%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Asya Kamsky
Lead Product Manager
MongoDB
Download MongoDB - mongodb.org/downloads
We're Hiring! - https://www.mongodb.com/careers