Setting stack-size ulimit for mongodb to reduce connection overhead

[Ati]

The default setting for ulimit -s is 8192. MongoDB does not specify any default value for ulimit -s (stack-size). However this case study suggests to set stack-size to 1024. How does the default stack-size 8291 impacts mongodb performance. Does changing this value to 1024 help to reduce connection per overhead in order to improve performance?

Thanks

Ati

[William Berkeley]

Hi Ati. The default size of 8192 determines part of the overhead for each connection to mongod: each connection can have an overhead of 8192KB for the stack size, using up some RAM.

As the case study discussed, Server Density saw queued queries because of contention for RAM. This RAM contention was caused, in part, by the number of connections to mongod processes (and this number was large, ~1500/mongod). Each connection had an ~10MB RAM overhead because of the stack size ulimit, and with the number of connections they had this used too much RAM. Part of their solution was to reduce the ulimit for the stack size to 1024. This isn't free- when you reduce the ulimit, you will cause problems whenever a connection thread runs into the new, lower ulimit. This was a very specific fix for the very specific situation that Server Density found themselves in. Unless you find yourself suffering from a similar performance bottleneck, have thought about if decreasing the stack size ulimit makes sense, and have tested it for adverse effects, I wouldn't recommend changing the ulimit. Don't prematurely optimize.

-Will

[Asya Kamsky]

Everyone,  hold on a second, before we get carried away down the wrong path...   The page you linked was talking about MongoDB 1.8.

If you look at this comment https://blog.serverdensity.com/mongodb-connection-overhead/#comment-1252746511 it correctly points out that the limit is current 1MB.

It's actually the *lower* of 1MB or your ulimit -s setting, so unless you lower the limit below 1MB it will be irrelevant.

You can see the note in 2.0 release notes (that was almost three years ago):  http://docs.mongodb.org/manual/release-notes/2.0/#default-stack-size

The default stack size is lesser of the system setting or 1MB.

Asya

--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
 
For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user...@googlegroups.com.
To post to this group, send email to mongod...@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/6a3837fc-12cc-4dd9-9fa1-a286ec6f37f0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[William Berkeley]

Thanks for the correction, Asya.

-Will

[Ati]

Thanks Asya.