Mongo Enterprise In-memory storage engine queries?

[Ketan Sawhney]

When we use in-memory feature of mongodb enterprise edition, as I could gather from documentation, you could specify the memory for the DB when it starts. Now in the case of database growing, do we need to also keep on growing RAM for the server, or we have an option for hybrid approach in which it would keep last used mongo objects in memory and rest stay on hard disk. Or we can specify which objects we would like to store in-memory and rest could stay on disk.

Our current setup includes:

3 server with 2 in-memory and 1 WiredTiger storage engine (All running in a replica set).

Thanks.

[Stephen Steneker]

On Wednesday, 6 December 2017 08:07:02 UTC+11, Ketan Sawhney wrote:

When we use in-memory feature of mongodb enterprise edition, as I could gather from documentation, you could specify the memory for the DB when it starts. Now in the case of database growing, do we need to also keep on growing RAM for the server, or we have an option for hybrid approach in which it would keep last used mongo objects in memory and rest stay on hard disk.

Hi Ketan,

As noted in the documentation, the In-Memory Storage Engine does not maintain any on-disk data or indexes in order to enable more predictable latency for database operations. The In-Memory storage engine requires all data (including indexes and oplog) to fit into the specified inMemorySizeGB setting, so if your requirements grow over time you will need to scale this allocation as well.

If you have a more typical use case where your data set may be much larger than available RAM, you should instead use the default WiredTiger storage engine. The operational behavior is similar to what you have described: WiredTiger allocates an internal cache for most recently used data and indexes, and loads additional data/indexes from disk as required. In addition to WiredTiger's internal cache, MongoDB also benefits from filesystem cache (see: To what size should I set the WiredTiger internal cache?).

Or we can specify which objects we would like to store in-memory and rest could stay on disk. 

As at MongoDB 3.6, there is no mechanism to explicitly pin collections or indexes in memory. Relevant feature requests to watch/upvote are WT-2452: Implement a mechanism that gives priority in cache to some tables and SERVER-23321: Allow pinning specific WT collections in memory.

Regards, Stennie