Scaling to 1000k RPM

[Naveen Kumar]

Hi,

We are using MongoDB for persistent storage for the last 2 years. It is working vey well at 400k RPM but our data and traffic are growing rapidly. We are expecting more than 600k rpm in next 2 months. We should be ready with a design which can handle 1000k rpm.

Our current MongoDB cluster status is

• Version: 2.6
• Total No of DBs: 200
• Main collections: Users and Devices
• Number of docs: 120M devices and 120M users
• Average doc size: 5KB
• 3 Shards, 3 mongos and 3 configs servers.
• Each shard has 1 primary, 1 secondary and 1 arbiter.
• Shard key on Users is _id_hashed and Devices is unique_id(random string generated by us)
• reads: 300k per minute
• writes: 100k per minute

Our goal is scaling to 1000k RPM. Will this design work at high scale if we add more shards to cluster? Do we need to make any changes to this? what are the advantages of moving to mongo 3.2? 

Thanks,

Naveen Kumar

[Kevin Adistambha]

Hi Naveen,

We are using MongoDB for persistent storage for the last 2 years. It is working vey well at 400k RPM but our data and traffic are growing rapidly. We are expecting more than 600k rpm in next 2 months. We should be ready with a design which can handle 1000k rpm.

Our goal is scaling to 1000k RPM. Will this design work at high scale if we add more shards to cluster?

Adding more shards will definitely provide better scaling. If you think that your current setup is about to hit a limitation of some kind (query performance, working set issues, etc.) then it is better to add new shard as early as possible. Adding a new shard when the system is at capacity will put even more pressure to the whole cluster, since the cluster must serve queries and also process chunk migrations to the new shard. Please see Considerations for more information.

However, only your own testing with your specific use case and hardware can reveal if your deployment will be able to scale with your predicted workload.

Regarding your current setup:

Each shard has 1 primary, 1 secondary and 1 arbiter.

I would recommended to have three data-bearing nodes to provide much better fault tolerance and durability. You may be able to find more information at Operation Checklist, specifically the Replication and Sharding sections.

what are the advantages of moving to mongo 3.2?

MongoDB 2.6 will be reaching its end of life at the end of October 2016. Therefore I would recommend to upgrade to the 3.2 series as soon as practical. There are many advantages of MongoDB 3.2 compared to previous versions, such as:

• the WiredTiger storage engine that provides document-level locking and better concurrency performance compared to MMAPv1 (which is the only storage engine available to MongoDB 2.6)
• Document validation
• Replication protocol version 1
• Config servers as a replica set
• readConcern setting
• Partial indexes

Please see the MongoDB 3.2 Release Notes for more information regarding major improvements in 3.2.

Best regards,
Kevin

​