MongoDB eating a lot of memory!

[simon66]

So I just switched over to MongoDB from MySQL, and so far is great! Really fast! But it looks like that I'll be switching over to MySQL again.

There is something wrong with my mongod program. It's eating about 8GB of ram and I don't know what to do.

Here is a picture

http://postimage.org/image/7rxazp49b/full/

if you notice the mongod window in the bottom right.

Any help?

[Nat]

It's normal since MongoDB uses memory mapped file to access database files. OS will manage its memory usage. If you have plenty of memory and you access most of the data in your data files, it's possible that mongodb will consume memory as much as your database size.

http://www.mongodb.org/display/DOCS/Caching

[Varela]

MongoDB is document based and doesn't have strict schema like MySQL, so it should keep schema in each document rather than have one schema definition per table. That's why size of you database should be large than same MySQL database, larger database size should use more memory.

[Sam Millman]

"keep schema in each document rather than have one schema definition per table"

Mongodb does not house schema definition at all.

As Nat said the reason why MongoDB uses a lot of memory is because it uses memory mapped files.

On 30 November 2011 10:28, Varela <zmiev...@gmail.com> wrote:

MongoDB is document based and doesn't have strict schema like MySQL, so it should keep schema in each document rather than have one schema definition per table. That's why size of you database should be large than same MySQL database, larger database size should use more memory.

--
You received this message because you are subscribed to the Google Groups "mongodb-user" group.

To view this discussion on the web visit https://groups.google.com/d/msg/mongodb-user/-/VHsHUgR9_ZMJ.

To post to this group, send email to mongod...@googlegroups.com.
To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.

[Varela]

By schema I mean name and type of the fields

[Sam Millman]

I am not sure what you mean, maybe you can elaborate?

If you mean that a field name and type (if there is one, 90% of all fields don't have a stored type) will consume some memory then yes that is correct but that is not the reason why memory is eaten by mongo, this is more to do with the smallest allocation possible in comparison with the size and length of your field names/values.

MongoDB in general uses more memory due to memory mapped files, also judging by that screen shot I would say he has not got a lot of BSON types stored and also his field names are relatively small.

On 30 November 2011 11:17, Varela <zmiev...@gmail.com> wrote:

By schema I mean name and type of the fields

--

You received this message because you are subscribed to the Google Groups "mongodb-user" group.

To view this discussion on the web visit https://groups.google.com/d/msg/mongodb-user/-/3mduEb6i6DgJ.

[Varela]

Completelly agree with you - that memory mapped files is the reason. But I have a suspiction that MongoDB requires at least in 2 times more disc space for the same data than relational databases. Since MongoDB put to memory projection of DB files there can be IO performance problems with replacing memory pages while MySQL will be able just keep all required data in memory. I'm going to do research on this.

[Sam Millman]

Again I'm not sure what you mean by problems.

I mean hot sets are kept in memory using the LRU algorithm, the files are just memory mapped to pages.

On 30 November 2011 13:04, Varela <zmiev...@gmail.com> wrote:

Completelly agree with you - that memory mapped files is the reason. But I have a suspiction that MongoDB requires at least in 2 times more disc space for the same data than relational databases. Since MongoDB put to memory projection of DB files there can be IO performance problems with replacing memory pages while MySQL will be able just keep all required data in memory. I'm going to do research on this.

--

You received this message because you are subscribed to the Google Groups "mongodb-user" group.

To view this discussion on the web visit https://groups.google.com/d/msg/mongodb-user/-/KemhE8Hx4sIJ.

[Sam Millman]

There can be IO problems with mapping yes, but I am tbh I am not sure what you mean by "problems" so I can't really give a answer.

[Scott Hernandez]

You may find it helpful to read these. And if there are things that
are unclear, let us know.

http://www.mongodb.org/display/DOCS/Excessive+Disk+Space
http://www.mongodb.org/display/DOCS/Checking+Server+Memory+Usage
http://www.mongodb.org/display/DOCS/Caching