ERROR: couldn't unset sharding :( { errmsg: "not master", ok: 0.0 }

[Erez Zarum]

We have a MongoDB Cluster with 3 shard servers in a ReplicaSet
configuration as follow:

host1:27019 - config

ReplicaSet - shard0000
host1:27018 - shard0000 (1)
host4:27018 - shard0000 (2)
host4:27020 - shard0000 (3) - arbiterOnly

ReplicaSet - shard0001
host2:27018 - shard0001 (1)
host5:27018 - shard0001 (2)
host5:27020 - shard0001 (3) - arbiterOnly

ReplicaSet - shard0002
host3:27018 - shard0002 (1)
host6:27018 - shard0002 (2)
host6:27020 - shard0002 (3) - arbiterOnly

Every application server runs a mongos as well, around 14 mongos running.
I went ahead and change a primary node in the replicasets by issuing
rs.stepDown(120); on the primary node on each shard(replicaset).
all went alright, no issues here.
Then looking at the mongos logs i have received this: ERROR: couldn't
unset sharding :( { errmsg: "not master", ok: 0.0 }

I have found out this problem seems to come from mongos, so i have
restarted mongos and all is alright.

Everytime a new node in the ReplicaSet gets elected to be a primary i
will need to restart mongos?

[Nat]

- which version of mongodb are you using?
- after you see the error message in the log, did mongos stop working? If you make a call via mongos, does it error out?
- what is your entry in config db?

--
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To post to this group, send email to mongod...@googlegroups.com.
To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.

[Erez Zarum]

Everyone has the same version, 1.6.5
I can connect to mongo through mongos but i can't do anything.

By entry, which entry do you mean?

Thanks.

[Eliot Horowitz]

Did you keep getting those errors or just once?
You should get it once per connection, ~20-100 times depending on your app.

[Erez Zarum]

I didn't really follow how many times i have seen those errors.
So if it was only once it is because of the window time when a new
primary is being elected?

[Eliot Horowitz]

Correct.
So some during transition are normal.

[Reinaldo]

Actually I've seen this happened in our cluster multiple times where
the only solution is to restart all mongos.

My setup is 2 replica sets (3 machines each) with 3 mongoc and mongos
running on all Tomcat Servers.

We can reproduce by stepping down a master on one of the replica sets,
it does not go away after a few calls, or a minute, the only way to
fix it so far is to restart the mongos

Is there a bug associated with this ? we are running 1.6.5 and 1.7.4
and we have seen this issue on both.
Reinaldo

On Jan 18, 7:07 am, Eliot Horowitz <eliothorow...@gmail.com> wrote:
> Correct.
> So some during transition are normal.
>

> On Tue, Jan 18, 2011 at 9:46 AM, Erez Zarum <e...@icinga.org.il> wrote:
> > I didn't really follow how many times i have seen those errors.
> > So if it was only once it is because of the window time when a new primary
> > is being elected?
>
> > On 01/18/2011 03:30 PM, Eliot Horowitz wrote:
>
> >> Did you keep getting those errors or just once?
> >> You should get it once per connection, ~20-100 times depending on your
> >> app.
>

> >> On Tue, Jan 18, 2011 at 4:07 AM, Erez Zarum<e...@icinga.org.il>  wrote:
>
> >>> We have a MongoDB Cluster with 3 shard servers in a ReplicaSet
> >>> configuration
> >>> as follow:
>
> >>> host1:27019 - config
>
> >>> ReplicaSet - shard0000
> >>> host1:27018 - shard0000 (1)
> >>> host4:27018 - shard0000 (2)
> >>> host4:27020 - shard0000 (3) - arbiterOnly
>
> >>> ReplicaSet - shard0001
> >>> host2:27018 - shard0001 (1)
> >>> host5:27018 - shard0001 (2)
> >>> host5:27020 - shard0001 (3) - arbiterOnly
>
> >>> ReplicaSet - shard0002
> >>> host3:27018 - shard0002 (1)
> >>> host6:27018 - shard0002 (2)
> >>> host6:27020 - shard0002 (3) - arbiterOnly
>
> >>> Every application server runs amongosas well, around 14mongosrunning.
> >>> I went ahead and change a primary node in the replicasets by issuing
> >>> rs.stepDown(120); on the primary node on each shard(replicaset).
> >>> all went alright, no issues here.

> >>> Then looking at themongoslogs i have received this: ERROR:  couldn't
> >>> unset
> >>> sharding :( { errmsg: "notmaster", ok: 0.0 }

>
> >>> I have found out this problem seems to come frommongos, so i have
> >>> restarted

> >>>mongosand all is alright.

[Eliot Horowitz]

Can you send the exact error message?

I think this is the issue: https://jira.mongodb.org/browse/SERVER-2523

Just pushed a fix.