Mongodb setup in Production and Disaster recovery site

[codevaly]

Hi

I am planing to setup a 3 shard mongodb cluster in our production environment. Each shard going to have one primary and two secondary nodes. 

Could someone please explain to me what are the best way to setup our disaster recovery site? Normally the DR site is a cold site and no one have access to it. Once the production is not accessible, we are switching to the DR site and the data had to be update from the all component perspective.

Any thoughts?

Thanks

Roshan.

[Asya Kamsky]

The point of a three member replica set is that if one goes down, you
will have failover to second node - the third which is the DR node
will never become a primary because if two other nodes go down, then
there won't be a majority left to elect it a new primary...

How exactly are you thinking of DR here in this scenario?

Asya

> --
> You received this message because you are subscribed to the Google Groups
> "mongodb-user"
> group.
>
> For other MongoDB technical support options, see:
> http://www.mongodb.org/about/support/.
> ---
> You received this message because you are subscribed to the Google Groups
> "mongodb-user" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to mongodb-user...@googlegroups.com.
> To post to this group, send email to mongod...@googlegroups.com.
> Visit this group at http://groups.google.com/group/mongodb-user.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/mongodb-user/ceb471a6-5b4f-495d-9872-e91ed516eabb%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[codevaly]

Thanks for your reply.

My initial plan was to build a shard with 3 node in PROD and 3 in DR (with priority 0, so they will not promote primary in case). Also from the 3 servers in DR actually one is running and other 2 is in stop state.

I need actually to reconfigure the DR (by starting the other 2 stopped nodes), when we are in DR (prod is completely inaccessible). So, is this a best way to implement PROD-DR scenario or any other ways you are suggesting?

Appreciate any comments from you because I am totally lost with this architecture. Another problem is how to handle the DR-PROD scenario once the PROD is back again from the failure.

Thanks

Roshan.  

[Asya Kamsky]

A better way to configure this would be to have two nodes in your
primary center and one in DR center.

Now if you lose the primary center you can bring up two new nodes that
with DR node will make up a new replica set (if that's what you want
to do).

If you want to have failover without manual reconfiguration, then you
need more nodes.

Asya

> https://groups.google.com/d/msgid/mongodb-user/aa145a20-a18d-45fd-9c10-30e268982933%40googlegroups.com.

[codevaly]

Thanks for the reply. But how I sync the data inserted to the DR to PROD when I switch on to PROD back?

/Roshan. 

[s.molinari]

If you want to have failover without manual reconfiguration, then you
need more nodes.  

I thought, since you don't want the DR data center to become primary in any automated election, then you need to set up all DR nodes as "non-voting". Because of this, I was under the impression getting the DR datacenter up and running with a primary would always entail a manual interaction. 

And I would have thought, if the PROD system comes back online, you'd have to sync them back up to the DR cluster, then manually reset all the nodes back to the old cluster PROD/ DR set-up. 

In other words, I'd also love to understand how this could be done with automation.:-)

Scott

[codevaly]

Thanks for the reply.

Yes, i also thought that. So when we switch on to DR, I need to reconfigure the replica set and the shard manually with the DR nodes. Once we switch back to PROD, again I need to manually reconfigure the cluster with PROD and DR servers. How about the data inserted in DR to get sync back to PROD after we switch back to prod again? Is it automatically or any special way?

The reason why I am asking this is all the nodes in my DR site are non-voting members, so no master will elect from DR. 

Thanks

Roshan.