mongod runs out of memory, what WiredTiger cache size to use

[Jabba Laci]

Hi,

I have a Digital Ocean VPS with 512 MB RAM, with MongoDB 3.2.4 preinstalled. I ran into a memory issue: when I wanted to import a dump with mongorestore, the mongo server crashed. First I thought it was a problem with mongorestore so I posted an issue to them: https://github.com/mongodb/mongo-tools/issues/60 . We figured out that it was the mongod server that ran out of memory.

When I tried it, the machine didn't have any swap. After adding a swap file, the import was successful. During the import the memory usage was 90% and 1 GB of my 2 GB large swap file was used. Now, as I use the database, the memory and swap usage is about the same.

The mongorestore guys suggested to reduce the wiredtiger cache size, which defaults to 1GB. I would like to ask help about it. What cache size is suggested? What swap size should I use? Then, how to modify the mongo config file?

Thanks,

Laszlo

[Tim Hawkins]

What is the size of the data you are working with, and what is the working set size?

--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
 
For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user...@googlegroups.com.
To post to this group, send email to mongod...@googlegroups.com.
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/65c969c9-2d36-4a5a-b0de-1c8f748943e3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jabba Laci]

> db.stats()
{
        "db" : "mydb",
        "collections" : 6,
        "objects" : 123259,
        "avgObjSize" : 3004.9865405365936,
        "dataSize" : 370391636,
        "storageSize" : 148717568,
        "numExtents" : 0,
        "indexes" : 15,
        "indexSize" : 13631488,
        "ok" : 1
}

From RockMongo (more readable):

Size             154.83m
Objects         123259
Data Size         353.23m
Storage Size     141.83m
Indexes         15
Index Size         13m

[Kevin Adistambha]

Hi Laszlo,

I have a Digital Ocean VPS with 512 MB RAM

The WiredTiger default cache size is set to the larger value of:

• 60% of RAM minus 1 GB, or
• 1 GB

By using only 0.5 GB of RAM, you have a severely under-provisioned server. In this case, you will run into memory issues, OOM killer (which you have already seen), and performance issues.

There are a couple of suggestions that you may be able to implement:

1. Use the command line option --wiredTigerEngineConfigString="cache_size=xxxM" to start mongod. I would recommend changing the xxx to no more than 256 given your memory size, and you may find a lower value more appropriate depending on your application and server use case.
2. Switch to the MMAPv1 storage engine, which does not have a minimum setting.

Having said that, to solve the memory issue, ideally you need to provision a larger memory for your server.

Best regards,
Kevin

​

[Antonio Malcolm]

On Friday, April 1, 2016 at 1:22:42 AM UTC-7, Kevin Adistambha wrote:

Hi Laszlo,

I have a Digital Ocean VPS with 512 MB RAM

The WiredTiger default cache size is set to the larger value of:

• 60% of RAM minus 1 GB, or
• 1 GB

By using only 0.5 GB of RAM, you have a severely under-provisioned server.

​

Based on what? Are you stating those with smaller requirements for their websites or applications shouldn't use MongoDB?
I think it's a bad assumption, that everyone needs GBs of cache, and forcing them to configure a size, in MB, via the command line, rather than providing a means of doing so, via the configuration file.

[Antonio Malcolm]

On Friday, April 1, 2016 at 1:22:42 AM UTC-7, Kevin Adistambha wrote:

Also, the command arg,--wiredTigerEngineConfigString, is not listed on the MongoDB documentation pages, or in the options listed with mongod --help.
Is there some place we can reference command args which might not be in either of those two places?