how to restore mongodump from mongo shard and replicated clusters

[Martinus Martinus]

Hi,

I have my cluster configuration like this :

shard 1 replica set :

- cluster1:10001         /data/cluster1/set1     master

- cluster1:10002         /data/cluster1/set2

- cluster1:10003         /data/cluster1/set3

shard 2 replica set :

- cluster2:10004         /data/cluster2/set1     master

- cluster2:10005         /data/cluster2/set2

- cluster2:10006         /data/cluster2/set3

config servers :

- cluster1:20001        /data/config1

- cluster1:20002        /data/config2

- cluster2:20003        /data/config3

mongos servers :

- cluster1:27017

And below is the step I used to backup my file :

1. I go into the mongos server and do :

    mongos> use config

    mongos> db.settings.update( { _id: "balancer" }, { $set : { stopped: true } } , true );

2. Then I use mongodump to dump the config database on mongos server :

    mongodump --host cluster1 --port 27017 --db config

 

3. Then I go into each cluster and do the mongodump :

    cluster1 :

    mongodump --host cluster1 --port 10001 --out /backup/mongodump-set1 --oplog

    

    cluster2 :

    mongodump --host cluster1 --port 10004 --out /backup/mongodump-set2 --oplog

So basically until here I have all the backup and I turn back on the balancer :

   use config

   db.settings.update( { _id : "balancer" }, { $set : { stopped : false }}, true);

Then I tried to delete the existing collection and tried to restore using the backup, so far so good for the other mongod, but when I tried to do this on config mongod server in cluster2 I got something like this :

bin/mongorestore --host cluster2 --port 20003 --db config /mongodb/mongodb-linux-x86_64-2.2.0/dump/config

terminate called after throwing an instance of 'std::bad_alloc'

  what():  std::bad_alloc

Could someone tell me what to do to make all the data can be restored and run again normally?

Thanks.

[Andre de Frere]

Hi Martinus,

I think your questions on backup and restore have been answered in a later post.  Are you still having issues with the below?

Regards,

André

[Martinus Martinus]

Hi Andre,

Yeah, I successfully made and restore my backup. But, I'm wondering why the database size on the cluster where I restore my backup is smaller than the original cluster? 

From original clusters :

admin (empty)

config (empty)

animal (empty)

animal2 (empty)

animal3 3.9521484384313226GB

animal4 0.203125GB

animal5 0.203125GB

animal6 0.203125GB

animal7 0.203125GB

test (empty)

animal8 19.8984375GB

animal9 0.203125GB

From new restore clusters :

config 0.046875GB

animal3 0.953125GB

animal4 0.078125GB

animal5 0.078125GB

animal6 0.078125GB

animal7 0.078125GB

animal8 2.90625GB

animal9 0.203125GB

Thanks.

Best regards,

Martinus

--
You received this message because you are subscribed to the Google
Groups "mongodb-user" group.
To post to this group, send email to mongod...@googlegroups.com
To unsubscribe from this group, send email to
mongodb-user...@googlegroups.com
See also the IRC channel -- freenode.net#mongodb

[Andre de Frere]

Hi Martinus,

The size difference could be because when the data is imported into the database it will not include any empty extents that your old database may have.  MongoDB will keep a list of deleted blocks from within the data files where deleted documents and collections used to live.  These blocks will be reused when new data comes along.  When you mongodump the database, you will not dump the empty blocks.

You can check and make sure that the number of records match in both databases, but a smaller data file size  should not necessarily be a cause for concern.

Regards,

André

[Martinus Martinus]

Hi Andre,

Thanks for your complete explanation. And will it be better to do the mongodump from secondary node on each shard while the secondary node is still running and also didn't get locked (since we can't do mongodump when it's still on lock mode)?

Thanks again.

Best regards,

Martinus

--

[Andre de Frere]

Hi Martinus,

Dumping from Secondaries is generally a better idea, as it reduces the load on a Primary.

Regards,

André

[Martinus Martinus]

HI Andre,

I found out one of my collection have different count() value returned in the backup, while in the original node is less than that. Is this mean not all the data in the master node being copied by the secondary node during the backup?

Thanks.

Best regards,

Martinus

Regards,

André

--

[Martinus Martinus]

Hi,

And I see one of the document inside that collection already been updated at November 2nd 2012, but in the backup there is only document updated at October 31th 2012.

Thanks.

Best regards,

Martinus

[Andre de Frere]

Hi Martinus,

Potentially the secondary was behind the Primary in replication when the backup was taken, which would be seen in replication lag for that node.

Regards,

André