MongoDB Chunks Creation Slow

[Weishan Ang]

Hi all,

We are using sh.splitAt to pre-create the chunks and it's taking quite a long time. Based on config.changelog, it takes about 3s to create a single chunk. We have disabled the balancer while the splitAt script is running. It is taking more than 2 days now.

Are there any ways to speed up the splitat command?

MongoDB Version: 3.2.9

ChunkSize: 256MB

No. Of Chunks to create: ~260k

Thanks!

Regards,

Wei Shan

[Kevin Adistambha]

Hi We Shan,

We are using sh.splitAt to pre-create the chunks and it’s taking quite a long time.

By “pre-creating the chunks”, do you mean you are performing sh.splitAt() on an empty collection?

Based on config.changelog, it takes about 3s to create a single chunk. We have disabled the balancer while the splitAt script is running. It is taking more than 2 days now.

MongoDB Version: 3.2.9

ChunkSize: 256MB
No. Of Chunks to create: ~260k

Every splitAt() command will require a write in the config servers. That is, the command does not run in a single machine, but it involves all config servers acknowledging that the chunk split has happened. So if you have three config servers, the write has to be acknowledged and verified three times, and also subject to how powerful the config servers are and also the network that connects all the pieces together. Creating such a large number of chunks will therefore take some time.

Also, please note that the latest MongoDB is currently 3.2.10. I would recommend you to upgrade for bugfixes and improvements.

Best regards,
Kevin

​

[Weishan Ang]

Hi Kevin,

Thanks for the response! I have another question regarding the splitAt() command.

We noticed that every splitAt() command has an open connections to the shard servers. Why is that the case? 

I'm referring to pre-creating chunks which means I have an empty collection. Our understanding is that it should only communicate with the config server. shard servers should not be aware of any chunk movement.