Issues with mongo dump and restore

[Om]

Hi,

I took a dump of the database using mongodump. Then I deleted ALL the
databases in mongo. Then I tried to use mongorestore to restore the
database from the dump. But I get following error:


Error creating index StorageProperty: 10185 can't find a shard to put
new db on
To resume index restoration, run mongorestore on filedump/
system.indexes.bson manually.
Aborted

Why did this happen? Are there any limitations to restore when it
comes to sharding?

[Eliot Horowitz]

Looks like you tried to restore via mongos?
Did you add the shards first?
Have you looked at:
http://www.mongodb.org/display/DOCS/Backing+Up+Sharded+Cluster

> --
> You received this message because you are subscribed to the Google Groups "mongodb-user" group.
> To post to this group, send email to mongod...@googlegroups.com.
> To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.
>

[Om]

I'm trying to get a mongodump and then restore from that dump but it
is not working.

I took a dump of the data using mongodump. I wiped out all the
databases and then ran mongorestore. However, I get following error
message:


Fri Feb 3 10:10:06 { key: { _id: 1 }, ns:
"bstoragesite.StorageProperty", name: "_id_" }
Error creating index bstoragesite.StorageProperty: 10185 can't find a

shard to put new db on

To resume index restoration, run mongorestore on filedump/bstoragesite/
system.indexes.bson manually.

Am I doing something wrong?

[Om]

Hi Eliot,

so just mongodump and then restoring data won't work? It seems the
procedure is a lot more complex in case of shards.

[Scott Hernandez]

It can be more complicated to restore, yes.

http://www.mongodb.org/display/DOCS/Backing+Up+Sharded+Cluster

[Om]

Scott,

I was hoping that I can just restore through mongorestore directly
(without restoring each shard as mentioned in that link). This seems
like it will require manual intervention since shards are distributed
across different machines. I was hoping that I can restore through
mongos but that doesn't seem to be the case. Is that right?

[Scott Hernandez]

You can, but you must restore the config first, then restart
everything (so it picks up the config metadata) and restore the rest
of the databases.

If you are just restoring the databases, or collections, then you
should restore the config db at all.

The important thing to do first is make sure the sharded cluster is
all setup and configured before restoring your dbs/collections.

[Om]

Scott,

I thought since i'm backing up all the databases (which includes the
config db stored by mongo) .. mongo can recreate the shards as well
from this config db backup. Is that not true?

[Scott Hernandez]

At the moment the config db is restored but no special processing is
done to reload that config data before restoring the rest of the dbs.
It will work, but isn't as easy as it should be right now.

Basically mongorestore is not sharding aware and needs to be. See this
issue for the coming changes:
https://jira.mongodb.org/browse/SERVER-3853

[Om]

Scott,

so can you please tell me the protocol for backup and restore in case
of sharded setup? The link lists that I need to shut down all the
mongod, config svrs and mongos before I restore. Is that true?

[Scott Hernandez]

It doesn't say to shutdown anything. The short of it is, as I said,
restore the config db, flushRouterConfig all the mongos instance (or
restart them), then restore the other sharded dbs. The rest of it is
optimizations and checks so you don't stomp on existing data. You must
clear the existing dbs before you restore, in case that wasn't clear.
The process I list above is for a new sharded cluster without any user
data.

[Om]

On the link: http://www.mongodb.org/display/DOCS/Backing+Up+Sharded+Cluster.
Go to "Restoring the entire cluster". The first sentence reads "Stop
all processes. No server should be running mongod, mongos, or a config
server mongod processes."

That's why I thought I need to stop them all. This is my use case:
I've a sharded system with user data. I've done a mongodump of this
data. Now I need to restore from this dump. Dump contains both user
data as well as mongo-generated data. How can I restore it? How can I
restore just the config servers from this dump? I can dump the
"config" data in the datafile for config servers. Will that be enough?
How do I perform flushRouterConfig? How do I restore other shards (by
dumping the data in their data directories??)? Can this shard restore
be done through mongos?

I'm sorry if I'm asking too many questions but I'm having a lot of
issues with this process. Thanks a lot for your help and time.