Is this setup good enough for a startup?

[Octavian Covalschi]

Hello,

I'm looking for community's _opinion_ for a potential setup for a SAAS startup that's using mongodb as it's main storage...

There will be multiple databases (each user will have a database). 

Initially there will be only 1 dedicated server (with about 20GB of RAM), 2 x HDD (RAID 1).

3 node replica set - 1 data, 1 arbiter and 1 hidden data (to take regular backups to an external machine).

I don't know how many users I want to handle yet with this setup, but I'm planning to implement some constraints (e.g. N req/sec or N req/day)

What do you think? Of course I'm aware of the chance that entire server will crash, but besides that, should I expect decent performance, durability as long working set fits in RAM? And of course I'll do my testing too, but I wonder if I'm missing any detail that is better to take care if it from the beginning than deal with it later.

I already have a small website with not a lot of traffic that runs on 1 node mongodb with regular backups and I have never had problems(it's been running for 1.5 years now), but this time i will expect to be under much havier load.

Thank you in advance.

PS: Sorry, it turned out to be longer than I hoped...

[Russell Smith]

Is this all on a single server?

Russ
Rainforest | +1-650-919-3216 | rainforestqa.com

> --
> You received this message because you are subscribed to the Google
> Groups "mongodb-user" group.
> To post to this group, send email to mongod...@googlegroups.com
> To unsubscribe from this group, send email to
> mongodb-user...@googlegroups.com
> See also the IRC channel -- freenode.net#mongodb

[Max Schireson]

Any idea how much activity per user/db? Are you thinking 10s of users/databases, thousands, or millions?

One thing to think about is each database has its own data files. There is a smallfiles option where they start at 16mb; that will add up with many databases.

-- Max

--

[Octavian Covalschi]

Thanks for trying to help me.

I'm trying to implement an API driven service, with mongodb being main storage engine. So besides mongo I'll have couple of more components and yes, all this on one server, at 1st....

My initial thought is to implement a proof of concept of what I have in mind, but in the same time to be ready to scale everything without downtime, that's why I'm thinking to set up replica set from the beginning...  with the hope that this setup will be enough for a while.

One more additional server will not help me, since if one of the servers goes down, rest of it won't work anyway...  

As for the databases, I was hoping a few hundreds will be enough to proof the concept and thanks for reminding about smallfiles

To use an external mongodb provider will add overall network latency, which I'm trying to minimize as well, although indeed it may be an option too... 

Thanks again.

[Max Schireson]

I think a few hundred databases could work if the load on each is low.

If you add a second server, you could have a primary on server a, a
secondary on server b, and an arbiter in the cloud on a small server.
That would let you remain up if you have a hardware failure with 2
machines plus a little.

Another thing to think about is the transition to sharding. One way
people can get in trouble is to wait until the system is slammed to
add shards. Then the system doesn't have enough capacity to perform
chunk migrations, so no data gets onto the new servers and the
existing servers remain overloaded. If you think you might have to add
shards later, I'd think about starting out sharded.

One thing I've seen is letting a single physical machine act as a
primary for one shard and a secondary for another. That way your
overall hardware requirement doesn't grow, but you could add capacity
quickly by bringing in additional hardware and moving from having
multiple shards on a box to one. It does add complexity to your system
though (need to set up sharding , config servers, mongos etc) but puts
you on a potentially smoother growth path for later. (The config
servers and mongos don't need a lot of hardware.)

Really whether its worth the complexity depends on how
predictable/controllable you expect the load on the system to be.

I hope this is helpful,

-- Max

On Fri, Oct 26, 2012 at 6:34 PM, Octavian Covalschi

[Octavian Covalschi]

Thank you, that's very insightful, although I have never used sharding before this would be one more reason to learn it.