Error: write with bad shard config and no server id!

Flaszter

unread,

Sep 19, 2011, 6:58:54 AM9/19/11

to mongodb-user

Hi!

-------------------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------------------------------
ENVIRONMENT(MongoDB 2.0)

We have a mongodb sharding environment with 3 replica set where one
replica set has:
1 primary
2 secondary
2 Arbiter

Our environment uses 6 servers (Windows server 2008) with the
following config:

1st server:
1st replica set primary
3rd replica set secondary
2nd replica set arbiter
+ 1 config server

2nd server:
2nd replica set primary
1st replica set secondary
3rd replica set arbiter
+ 1 config server

3rd server:
3rd replica set primary
2nd replica set secondary
1st replica set arbiter
+ 1 config server

4th server:
1st replica set secondary
3rd replica set secondary
2nd replica set arbiter

5th server:
2nd replica set secondary
1st replica set arbiter
3rd replica set arbiter

6th server (application server host a web site)
1 mongos instance (routing server)

-------------------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------------------------------

PROBLEM

We make write load on the servers and after little time we get a lot
of error from this from the C# driver:

Safemode detected an error 'write with bad shard config and no server
id!'
{ "err" : "write with bad shard config and no server id!", "code" :
10422, "n" : 0, "lastOp" : NumberLong("5652597361524867121"),
"connectionId" : 1779, "ok" : 1.0 }

We checked also the routing server's config and we found the following
messages:

StorageTest.lastKnownTransactionId could not initialize cursor across
all shards because : ns: StorageTest.lastKnownTransactionId
ClusteredCursor::query

You need to know the StorageTest is our database and the
lastKnowTransactionId is a collection which has a shard key : _id.

Have you got any idea why we get this error?
We can only solved this error via routing server restart.

Thanks,
Flaszter

Greg Studer

unread,

Sep 20, 2011, 2:19:44 PM9/20/11

to mongod...@googlegroups.com

If this is repeatable, is it possible to run the load test with the latest 1.8 nightly mongos, high verbosity ( log level 2? ) on the mongos and post the mongos logs (just the single mongos downgraded, everything else the same)? I'm guessing now the issue has to do with the shard version getting reset, but hard to say what the cause is. Are migrations occurring during this test, or replica set failovers?

> StorageTest.

lastKnownTransactionId could not initialize cursor across
all shards because : ns: StorageTest.lastKnownTransactionId
ClusteredCursor::query

This error is just telling you that the query can't proceed b/c of the connection issue above.

Running:

mongo <mongos>
> use admin
> db.runCommand({ flushRouterConfig : 1 })

may be a workaround for this state that doesn't require a mongos reboot.

Flaszter

unread,

Sep 21, 2011, 11:32:59 AM9/21/11

to mongodb-user

Hi Greg!

I downloaded the latest 1.8.3 nightly build (mongodb-win32-
x86_64-1.8.3)and I created a new mongos instance with this.
All mongod instance is uses 2.0 (mongodb-win32-x86_64-2.0.0) mongodb
release.

Right now i don't get the mentioned exception.
How did you know?
What's wrong with the 2.0... Any Idea?

Thanks

Greg Studer

unread,

Sep 28, 2011, 7:38:32 PM9/28/11

to mongodb-user

Sorry for the delay, somehow lost this thread earlier and was
traveling.

I don't know if this is a 2.0 issue in particular, but we added more
messaging in 1.8.4, hence the nightly. If you grep your logs for :
"resetting shard version" there may be more info there. Think a
symptom of this issue we may have fixed recently.

Flaszter

unread,

Oct 4, 2011, 1:25:24 PM10/4/11

to mongodb-user

Hi,

I have tested the execution of load testing on two environment.
1.8.3:
There was no error. Everything is fine.

2.0 nightly build:
Got the same exception.
I have investigated on all logs (config servers, routing server,
mongod servers), and found another interesting info:"warning: bad
serverID set in setShardVersion and none in info: EOO".

Another problem is we have to use the 2.0 mongodb because it allow us
to use w="majority".

Thanks

Greg Studer

unread,

Oct 5, 2011, 10:51:37 AM10/5/11

to mongodb-user

ah, this is a known issue, there are fixes in place for 2.0.1. Think
the 2.0 nightly includes them now.

rusan

unread,

Oct 27, 2011, 1:38:38 AM10/27/11

to mongodb-user

On 5 окт, 18:51, Greg Studer <g...@10gen.com> wrote:
> ah, this is a known issue, there are fixes in place for 2.0.1. Think
> the 2.0 nightly includes them now.

We use 2.0.1 and have same in logs:

Thu Oct 27 05:36:41 [initandlisten] connection accepted from
10.20.65.42:6876 #31801
Thu Oct 27 05:36:41 [conn31801] warning: bad serverID set in