Remove Shard forever "draining", no chunks left and 3 DBs

[h0lyRS]

Hey guys.

I've been reading some references regarding shard removal, but there
is a situation I'm not able to solve. Hope someone can shed a light.

There is a setup with 3 config servers, 2 "mongos" processes, and one
replica set. Then, this replica set is added to the mongos as a
shard. Time went by and the servers that were hosting this replica
set are to be decomissioned, so I made a DB backup using mongodump and
try to disconnect the shard from the mongos process via removeshard
command.

It ends up like this:

mongos> db.runCommand( { removeshard : "shard0000" } );
{
"msg" : "draining ongoing",
"state" : "ongoing",
"remaining" : {
"chunks" : NumberLong(0),
"dbs" : NumberLong(3)
},
"ok" : 1
}

Seeing that this goes on forever, I then try to add a shard using new
servers (new replicaset), so maybe the old shard would "drain" data
over the new one. It doesn't happen, so now I got 2 shards, one is
draining, the other is just... there.

I'm not sure what to do next. Please help?

[Adam C]

Hi there,

As the removeshard page (http://www.mongodb.org/display/DOCS/
removeshard+command) indicates, you have to manually move the
databases once all the chunks are done draining off.

If you need to figure out which database the removeshard output refers
to, you can use the printShardingStatus command. It will tell you what
is the "primary" shard for each non-partitioned database.

If any of the DBs have the drained shard listed as the primary, you
will need to issue a move primary command (again listed on the page
linked).

Finally, re-run removeshard and you should be done.

Adam.

[h0lyRS]

Hi Adam,
Thanks for the reply!
I'm not sure if you're a regular user or a MongoDB developer.
If you allow me to make a suggestion, I think the help page could be
improved a bit.  For example...I'm still a bit confused because of
this snippet found on the page:
"Important : you should only issue the movePrimary command after
draining has completed - in general you should never use movePrimary
if you still have undrained sharded collection data on the primary
shard."
Since the draining has not been reported as completed, I thought this
was not my case, but you're telling me that in fact I should be
looking at the chunks data, not the status.It is also not very clear
what is the "shard0001" that is given in the example, since in my case
there was not another shard initially (I just wanted to kind of
"disconnect" my replica set from the sharding). Should I always create
another shard when draining?  Am I not able to go back to the
unsharded model once I add a shard?
A direct link to the movePrimary command could also help
someone:http://wiki.mongodb.org/display/DOCS/movePrimary+Command
Thanks again!

[Adam C]

Agreed on the documentation, it could be clearer, and I have assigned
an issue to myself to clean that up.

As for converting back to an unsharded collection as mentioned on the
Configuring Sharding page, there is currently no way to un-shard a
collection. See:

http://www.mongodb.org/display/DOCS/Configuring+Sharding

From that page:

"Running the "shardcollection" command will mark the collection as
sharded with a specific key. Once called, there is currently no way to
disable sharding or change the shard key, even if all the data is
still contained within the same shard. It is assumed that the data may
already be spread around the shards. If you need to "unshard" a
collection, drop it (of course making a backup of data if needed), and
recreate the collection (loading the backup data)."

So, once you have completed your work you will need to backup, drop,
recreate the collection for it to be properly unsharded.

Adam