MongoDb huge cluster vs many cluster

[Roman Skvazh]

What is the best architecture approach for big heavy load project? Mix of queues-like structures, many updates for over 100 millions objects, append-only inserts etc.

1. one huge cluster (many shards with replica sets) with non-sharded collections in many databases (global lock optimization, balance with movePrimary) and sharded collections with tag-aware balancing and one mongos on each application server.
2. many N small independent clusters (1-3 shards with replica sets) and N mongos on each application server.

Thank you.

[Asya Kamsky]

Interesting question.

Generally, it depends on which of these will be simpler for you operationally.   From the point of view of application of course a single cluster may seem simpler (a single connection URI) but it still has to know which DB to connect to for which part, so having different URIs is not really a big deal here.

Operationally, some things will be simpler when you have a single cluster, and others will be simpler when you have multiple smaller clusters.   It's hard to say with certainty which will be better for you.   In many ways multiple systems are easier to tune because you can adjust hardware based on requirements of the specific part of the application that's using it, but you have to track a lot more things.   Sharding a large system allows you to add shards and using tags you can "allocate" specific shards to certain workloads...

I think a single larger cluster while simpler conceptually will probably be slightly more difficult to get right, but once you do, you'll be all set.  Replica sets are easier to manage and keep clean separation, but you will constantly be tweaking them and always need to monitor them separately...

Asya

--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
 
For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user...@googlegroups.com.
To post to this group, send email to mongod...@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/5a3e3a0e-b196-43a1-969a-2d6ff84f1169%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[s.molinari]

Hehehe...ka..ching! Another "it depends". LOL! I love it Asya. It is cool knowing MongoDB can fit so many use cases.

We are going with smaller instances. This decision is simply from a business standpoint, as our system will have databases (or rather the data in them) owned by our customers. Our situation could be different than yours though, If you are the owner of the data, then I'd say a single instance is the better choice too. I've even had discussions with people, who say a single database is the better choice for our multi-tenant environment (not with Mongo in particular, but in general). This would be impractical for us with Mongo though, as I have learned. One of the worlds largest multi-tenant environments (it might be the largest) is on only one database, believe it or not. Salesforce.com. (it is on Oracle though, yuck.....). 

Scott