how does the oplog handle large documents?

[Søren Bramer Schmidt]

Mongodb has a limitation of 16 mb per document. The oplog is a capped collection storing the modification to a document and some meta data. So if a 16 mb document is inserted the oplog has to store 16 mb + meta data. How is this handled?

[Asya Kamsky]

Great question!   It turns out that while the maximum document size is 16MB, MongoDB internally has a different limit.

Straight from the code:

    const int BSONObjMaxUserSize = 16 * 1024 * 1024;

    /*
       Sometimes we need objects slightly larger - an object in the replication local.oplog
       is slightly larger than a user object for example.
    */
    const int BSONObjMaxInternalSize = BSONObjMaxUserSize + ( 16 * 1024 );

Reference:   https://github.com/mongodb/mongo/blob/master/src/mongo/bson/util/builder.h#L50

Asya

On Sat, Nov 23, 2013 at 7:38 AM, Søren Bramer Schmidt <s...@trustpilot.com> wrote:

Mongodb has a limitation of 16 mb per document. The oplog is a capped collection storing the modification to a document and some meta data. So if a 16 mb document is inserted the oplog has to store 16 mb + meta data. How is this handled?

--
--
You received this message because you are subscribed to the Google
Groups "mongodb-user" group.
To post to this group, send email to mongod...@googlegroups.com
To unsubscribe from this group, send email to
mongodb-user...@googlegroups.com
See also the IRC channel -- freenode.net#mongodb
 
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Søren Bramer Schmidt]

Thank you Asya. Much appreciated. You just prompted me to dig into the code base which is surprisingly well documented[.

You received this message because you are subscribed to a topic in the Google Groups "mongodb-user" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/mongodb-user/LRHrPr4CfoA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to mongodb-user...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--

Søren Bramer Schmidt

Trustpilot Dhaka