when to start sharding

34 views

Skip to first unread message

Kumar shiva

unread,

Oct 26, 2016, 2:10:27 AM10/26/16

to mongodb-user

HI,
I am DBA for smart-city project.
we are anticipating huge sensor data over a period of time.
Should we start sharding from the beginning. ( with no data , sharding won't make any difference)

Is it good idea to start setting up shard only when there is reasonable data in the database.
In such case, we will have to ask for downtime & start the testing all over again. :-(

thanks and regards

Shiva

Kevin Adistambha

unread,

Nov 3, 2016, 3:24:27 AM11/3/16

to mongodb-user

Hi Shiva

we are anticipating huge sensor data over a period of time.
Should we start sharding from the beginning. ( with no data , sharding won’t make any difference)

Is it good idea to start setting up shard only when there is reasonable data in the database.

If you know that you will need sharding in the near future, it’s probably best to prepare for it. However, there are concepts related to sharding that you would need to review:

In sharding, choosing the right shard key is very important, since you cannot change the shard key later unless you dump the existing data, recreate the collection with a new shard key, and load the data back into the new collection. For more information, please see:

Please note that if you decide to shard at a later date, there are some restrictions regarding sharding an existing collection. See Sharding Existing Collection Data Size.

Also, you would need to know about MongoDB Replica Set, since typically each shard in a sharded cluster consist of a replica set. In MongoDB 3.2, the sharded clusters Config Servers which hold the cluster’s metadata can also be deployed as a replica set (which is highly recommended).

I would recommend you to perform a thorough testing with your use case and expected load.