Backup in sharded environment

[hmeiser]

Hi,

is there a recommended backup/restore procedure to be able to recover
a failed node?

As far as I understand in a sharded environment my backup of a node
gets useless after autosharding moved the documents from that shard to
another?

ciao.hans.

[Kristina Chodorow]

Moving chunks from one shard to another is just a bunch of insert/remove operations, so as long as the backup isn't too old, it should be able to catch up to the current state (just like a backup in a non-sharded environment).  If it is too out of date, you could take a more recent snapshot or let it resync from scratch (if your not in a hurry).

MongoDB shouldn't move chunks around too much.  We let people set chunk size to 1 so they can "see" the sharding, but in a real system, you won't have tons of migrating all the time (hopefully).

--
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To post to this group, send email to mongod...@googlegroups.com.
To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.

[hmeiser]

Ok, but this will work only if not all nodes of a shard died.
If we have 2 shards and with 2 nodes each (4 nodes). Then if both
nodes of a shard
die we can not use our backup to restore them because some documents
may be
moved to the dead nodes and will be lost after backup.
In this case we have to restore the complete set of nodes with the
backup?

ciao.hans.

On 6 Sep., 16:14, Kristina Chodorow <krist...@10gen.com> wrote:
> Moving chunks from one shard to another is just a bunch of insert/remove
> operations, so as long as the backup isn't too old, it should be able to
> catch up to the current state (just like a backup in a non-sharded
> environment).  If it is too out of date, you could take a more recent
> snapshot or let it resync from scratch (if your not in a hurry).
>
> MongoDB shouldn't move chunks around too much.  We let people set chunk size
> to 1 so they can "see" the sharding, but in a real system, you won't have
> tons of migrating all the time (hopefully).
>

> On Mon, Sep 6, 2010 at 5:27 AM, hmeiser <drehtam...@googlemail.com> wrote:
> > Hi,
>
> > is there a recommended backup/restore procedure to be able to recover
> > a failed node?
>
> > As far as I understand in a sharded environment my backup of a node
> > gets useless after autosharding moved the documents from that shard to
> > another?
>
> > ciao.hans.
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "mongodb-user" group.
> > To post to this group, send email to mongod...@googlegroups.com.
> > To unsubscribe from this group, send email to

> > mongodb-user...@googlegroups.com<mongodb-user%2Bunsu...@googlegroups.com>
> > .

[Kristina Chodorow]

The point of a replica set is to try to keep at least one member up at all times: you should try to minimize the risk of all of them crashing (put different members in different data centers, if possible).  If the worst happens and all of the members of a replica set crash, you can lose data.  You can try running repair on the crashed nodes, but repair removes corrupt records. 

To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.

[Markus Gattol]

hmeiser> If we have 2 shards and with 2 nodes each (4 nodes). Then if
hmeiser> both nodes of a shard die ...

[skipping a lot of lines ...]

This is why a minimum of 3 shared-nothing physical machines is
recommended. Basically

num-shards * 3 = minimum-number-of-nodes i.e.
2 * 3 = 6 in your case

The only reason to have less than 3 members (one primary and two
secondaries) in a replica set is if you had very
strict/odd/unusual/unhealthy/<you name it> constrains on disk space, but
then again, less than 3 members is not recommended (see "Saving Disk
Space" in the link below)

http://www.markus-gattol.name/ws/mongodb.html#do_i_need_an_arbiter