MongoDB backup and restore of clusters

[ksac...@gmail.com]

I have read the mongodb docs that talk about mongodump, mongorestore, replicas, etc. Our database is a 3 node cluster. I tested few things on how to how to bring in a fresh node if a node fails, and all that works for me.

My problem is how to do I backup the MongodDB data files without shutting the mongodb cluster that can be used later to re-generate the whole cluster. The question arises in my mind because if a node fails, adding a new node is easy(start with blank location of /data/db and things fall in place). If I had a single node and no replica set, I can simply do mondodump and mongorestore and that should work is what i understand. However the mongodb docs say that mongodump cannot be used to restore replica sets.

So say if I have a script or something that takes mongodb dumps from a primary, how do I use this mondodb dump to generate the whole cluster if I have to? Basically, I am trying to do periodic backups and want those backups to recover the whole cluster in case all 3 nodes die. What is the best practice for it? I could not really understand this from the docs

[Rhys Campbell]

It sounds like you only have a single replica set? i.e. No sharding? If this is the case then then you have fewer problems than you thought. You can use mongodump with the oplog flag to get a consistent backup of a single replicaset. LVM Snapshots are another option...

https://docs.mongodb.com/manual/tutorial/backup-with-filesystem-snapshots/

With multiple shards this becomes an issue as it will be almost impossible to get all shards to produce cluster-wide consistent backups without stopping all writes.

If you do have more than one shard there is the commercial backup offering in MongoDB Enterprise.

There is also this option from percona...

https://github.com/Percona-Lab/mongodb_consistent_backup

[ksac...@gmail.com]

Thanks Rhys. We do not have shards and we have a 3 node cluster. Are you saying do mongodump on each nodes of a cluster with opflog flag? Or let me outline the steps below and tell me if I am right

1. Pick any mongo node in the cluster(assume they are all in sync)

2. Run mongodump on it

3. Keep the files safe from the mongodump.

So above are only partial steps, say i pick any node at random in a cluster, and use mongodump with oplog flag and then how do i build a full cluster from this dump? Like say if I have output of mongodump with oplog(assuming i just pick any node periodically and run the mongodump), what is the recommended step to create a cluster from it?

[Rhys Campbell]

You only need to mongodump form one node in a replicaset since they all contain the same data.

[ksac...@gmail.com]

And then to restore it, I will just create a fresh replica set and run mongorestore? When I do that, if all nodes are not added in replica set, it complaints. Is there a way to use the mongorestore and use the mongodump generated from replica set to be used for restore only one one node even if other 2 absent?

[Rhys Campbell]

It complains? What does it say? I regularly test backups taken from replica sets to standalone mongod hosts without any issues.