How to prevent 2 separate databases in the same MongoDB instance competing for RAM?

[dhsieh]

Is there a way in MongoDB to cap the RAM usage at per database in the
same MongoDB instance so that different apps can co-exist using the
same MongoDB instance? One our bigger challenges today is to maximize
the MongoDB usage so that it offers the maximum ROI. This can only be
achieved to share across multiple apps (thus different databases).

[dhsieh]

There are other discssion threads for this topic. Specifically one for
Windows in http://captaincodeman.com/2011/02/27/limit-mongodb-memory-use-windows/.
However, I am looking for Linux solution.

[dhsieh]

Even though in http://captaincodeman.com/2011/02/27/limit-mongodb-memory-use-windows/
proposed using Windows
“Windows System Resource Manager”(WSRM)to cap CPU and Memory available
to mongod process so that it can make room for other non-MongoDB apps,
this is not the same for a dedicated MongoDB instance but capping RAM
at per MongoODB database level.

[Tim Rieder]

Read up on `ulimit` You can find all the details you need here:

http://docs.oracle.com/cd/E19082-01/819-2239/ulimit-1/index.html

--
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To post to this group, send email to mongod...@googlegroups.com.
To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.

[Adam C]

While you can limit individual processes and other such methods, as you have found, it is not possible to limit per-database in MongoDB.  The amount of physical RAM (resident memory) used by the process is actually managed by the OS and is based on the usage of the memory mapped data by the database.

If you wish to have multiple users with different data sets running on a host with no contention, then the sum of the size of the working data set for each user/db would have to be smaller than the total physical RAM available to MongoDB, otherwise you will see contention in the form of increased faults to disk (you will see some anyway as old data is paged out etc.).  There is no way around that (beyond partitioning the instances using virtualization) at the moment, nor can I think of an obvious way to do it.

Adam

[dhsieh]

That's what I suspect to be the case. I suppose the only solution
would be to run each app's MongoDB in its own VM and this is not an
idea solution. In my opinion, MongoDB should really explore other
options such as:

(1) Database-level resource consumption (RAM and/or CPU) capping
(2) Collection and/or Document-level cache priority

[Adam C]

You can always request these as features in Jira, though the specific behavior you are talking about would be a very complex change to make and a significant departure from the current way mongo operates.  The SERVER project would be where you would ask for and vote for such features:

https://jira.mongodb.org/browse/SERVER

If enough people want them and it is not completely impossible in the code, then it will get onto the list, though the effort/reward payoff is also a consideration.

Adam