pre-splitting a large amount of chunks

[Alon Horev]

Hi All,

Is there any downside for pre-splitting 2 million chunks?

    Thanks, Alon Horev 

[Rob Moore]

Yes -

The mongos needs to load the 2 million chunks from the config server for every "change" to the chunks.   There is a Jira ticket to change that.  https://jira.mongodb.org/browse/SERVER-1326

Even then 2 million chunks is a lot of memory to use on every shard server and mongos.

Could you increase the chunk size to reduce the required number of chunks?  Since you are pre-splitting you are obviously trying to avoid moves so that shouldn't be that much penalty for a larger chunk size.

Rob.

[Alon Horev]

looks like there's a duplicate that's already been solved: https://jira.mongodb.org/browse/SERVER-5475

And about the chunk size: I implemented my own shard balancing algorithm by adding/updating the tag ranges programmatically. The reason for that is I having other considerations except storage size.. like putting the data close to the application. now if the application moves to a different data center I'd like to move it's data along with it (if the shard key was mutable I'd be able to change the 'data_center' field but because it's immutable I must use more finely grained information).

--
You received this message because you are subscribed to the Google
Groups "mongodb-user" group.
To post to this group, send email to mongod...@googlegroups.com
To unsubscribe from this group, send email to
mongodb-user...@googlegroups.com
See also the IRC channel -- freenode.net#mongodb