Upgrading to 2.0 with Replica Sets

[rasitha]

Hi,

I was running v1.8.2 with a 3-node replica set with one arbiter and
trying to upgrade to 2.0. I stopped my primary (which made the second
host the primary) and then copied 2.0 files.

When I start that instance again, it's correctly connecting to the
other two nodes but runs into the following error:

Thu Sep 22 09:45:17 [rsStart] replSet error self not present in the
repl set configuration:
Thu Sep 22 09:45:17 [rsStart] { _id: "A", version: 5, members:
[ { _id: 0, host: "WMONGO1:10000" }, { _id: 1, host:
"SMONGO1:10000" }, { _id: 2, host: "HMONGO1:10000", arbiterOnly:
true } ] }
Thu Sep 22 09:45:17 [rsStart] replSet info Couldn't load config yet.
Sleeping 20sec and will try again.
Thu Sep 22 09:45:37 [rsStart] trying to contact localhost:10000
Thu Sep 22 09:45:37 [rsStart] trying to contact HMONGO1:10000
Thu Sep 22 09:45:37 [rsStart] trying to contact SMONGO1:10000

Nothing in the configuration nor the host names have changed.

If I put back the old EXEs, it works again and becomes the secondary.

Did anyone run into a similar issue?

Thanks!

[rasitha]

Tried these steps and that didn't work either. Still giving me the
same error.

1. Connected to the new primary and removed the host that I'm
upgrading from the replica set configuration.

2. Removed all local data files from the upgrading instance erasing
all replication information and started db as a fresh instance with
replica sets enabled & having the same name as others.

3. Connected to the current primary and added the new instance:
rs.add("HOST:PORT"). That logged a "DBClientCursor::init call()
failed" message but the replica set configuration was given to the
node that I'm trying to upgrade.

But that node is still failing with the same error:
"replSet error self not present in the repl set configuration:" but
the configuration looks just like it should look.

Did the replica set configuration structure change in 2.0 where it
doesn't understand the 1.8.2 structure anymore? Is that why it cannot
find itself in the configuration?

I'm trying to avoid taking down all the instances and hoping I can
upgrade node by node.

Any help would be appreciated.

Thanks!

[rasitha]

There seems to be an issue with 2.0 with Replica Sets running on
windows.

I'm setting up a totally new instance and still running into the same
error: "replSetInitiate exception: can't find self in the replset
config my port: 10001"

If I use "localhost:10001" in my rs.initiate() call and use a single
member in the replica set, that works. But as soon as I use the host
name (even with a single member) it doesn't work and throws above
error.

Per the first entry of the log file (when mongod is started), it shows
me the port I'm using and the host name. And those are identical to
what I'm using in the rs.initiate(c) command. But somehow mongo thinks
those are two separate values.

Fri Sep 23 15:33:52 [initandlisten] MongoDB starting : pid=1928
port=10001 dbpath=F:\mongodb\MONGODEVREPLSETB\data 64-bit
host=MONGODEV1


Does anyone know anything I could be missing? I tried using all
lowercase as well as using the IP address they all give the same
error.

Thanks,
Ras

[Kristina Chodorow]

Hostname resolution shouldn't have changed between 1.8 and 2.0, but it
sounds like something did. If you do rs.initiate() (nothing in the
parens) it should output what it thinks its hostname is, something
like:

{
"info2" : "no configuration explicitly specified -- making
one",
"me" : "myHostName:10001",
"info" : "Config now saved locally. Should come online in
about a minute.",
"ok" : 1
}

Does that work for you? What does it say the hostname is?

[rasitha]

Here's what I get:

> rs.initiate()

{
"info2" : "no configuration explicitly specified -- making
one",

"me" : "MONGODEV1:10001",
"errmsg" : "couldn't initiate : can't find self in the replset
config my
port: 10001",
"ok" : 0
}

Looks like it has something to do with authentication. In my config
file I commented out keyFile= option and then it worked.


> rs.initiate()

{
"info2" : "no configuration explicitly specified -- making
one",

"me" : "MONGODEV1:10001",

"info" : "Config now saved locally. Should come online in
about a minut
e.",
"ok" : 1
}

thoughts?

[Kristina Chodorow]

That sounds like a bug, can you paste the whole log file from starting
up with the keyFile option (when it wouldn't let you intiate())?

[rasitha]

Here's a fresh copy of the log: (I removed all data files and started
a fresh copy and then issued the rs.initiate() command )

Fri Sep 23 16:38:57 [initandlisten] MongoDB starting : pid=1472

port=10001 dbpath=F:\mongodb\MONGODEVREPLSETB\data 64-bit
host=MONGODEV1

Fri Sep 23 16:38:57 [initandlisten] db version v2.0.0, pdfile version
4.5
Fri Sep 23 16:38:57 [initandlisten] git version:
695c67dff0ffc361b8568a13366f027caa406222
Fri Sep 23 16:38:57 [initandlisten] build info: windows (6, 1, 7601,
2, 'Service Pack 1') BOOST_LIB_VERSION=1_42
Fri Sep 23 16:38:57 [initandlisten] options: { config: "F:\mongodb
\MONGODEVREPLSETB\conf\mongodb.conf", dbpath: "F:\mongodb
\MONGODEVREPLSETB\data", keyFile: "F:\mongodb\MONGODEVREPLSETB\conf
\key", logappend: "true", logpath: "F:\logs\MONGODEVREPLSETB\log.log",
port: 10001, replSet: "B", rest: "true", service: true }
Fri Sep 23 16:38:57 [initandlisten] journal dir=F:/mongodb/
MONGODEVREPLSETB/data/journal
Fri Sep 23 16:38:57 [initandlisten] recover : no journal files
present, no recovery needed
Fri Sep 23 16:38:57 [initandlisten] waiting for connections on port
10001
Fri Sep 23 16:38:57 [websvr] admin web console waiting for connections
on port 11001
Fri Sep 23 16:38:57 [initandlisten] connection accepted from
127.0.0.1:50081 #1
Fri Sep 23 16:38:57 [conn1] note: no users configured in
admin.system.users, allowing localhost access
Fri Sep 23 16:38:57 [conn1] end connection 127.0.0.1:50081
Fri Sep 23 16:38:57 [initandlisten] connection accepted from
127.0.0.1:50082 #2
Fri Sep 23 16:38:57 [conn2] authenticate: { authenticate: 1, nonce:
"6472c37b3dae1818", user: "__system", key:
"906a1df0cfed8f42e17be58837808a5f" }
Fri Sep 23 16:38:57 [rsStart] replSet can't get local.system.replset
config from self or any seed (EMPTYCONFIG)
Fri Sep 23 16:38:57 [rsStart] replSet info you may need to run
replSetInitiate -- rs.initiate() in the shell -- if that is not
already done
Fri Sep 23 16:39:07 [rsStart] replSet can't get local.system.replset
config from self or any seed (EMPTYCONFIG)
Fri Sep 23 16:39:15 [initandlisten] connection accepted from
127.0.0.1:50083 #3
Fri Sep 23 16:39:17 [rsStart] replSet can't get local.system.replset
config from self or any seed (EMPTYCONFIG)
Fri Sep 23 16:39:19 [conn3] replSet replSetInitiate admin command
received from client
Fri Sep 23 16:39:19 [conn3] replSet info initiate : no configuration
specified. Using a default configuration for the set
Fri Sep 23 16:39:19 [conn3] replSet created this configuration for
initiation : { _id: "B", members: [ { _id: 0, host:
"MONGODEV1:10001" } ] }
Fri Sep 23 16:39:19 [conn3] replSet replSetInitiate config object
parses ok, 1 members specified
Fri Sep 23 16:39:19 [initandlisten] connection accepted from
192.168.3.153:50084 #4
Fri Sep 23 16:39:19 [conn3] replSet replSetInitiate exception: can't

find self in the replset config my port: 10001

Fri Sep 23 16:39:19 [conn4] end connection 192.168.3.153:50084

[Kristina Chodorow]

Okay, I think I know what's causing this. To check, can you try (in
the shell you ran rs.initiate() in) run:

> use admin
> db.runCommand({_isSelf : 1})

and paste the output? Hopefully it'll be something about needing to
log in.

[rasitha]

Here's the output: (since isSelf was successful I also ran
rs.initiate() command and that failed again)

connecting to: localhost:10001/admin
> use admin
switched to db admin
> db.runCommand({_isSelf:1})
{ "id" : ObjectId("4e7d92712b6a3b5a8b3b8654"), "ok" : 1 }

> rs.initiate()
{
"info2" : "no configuration explicitly specified -- making
one",
"me" : "MONGODEV1:10001",
"errmsg" : "couldn't initiate : can't find self in the replset
config my
port: 10001",
"ok" : 0
}
>

Thanks!

[rasitha]

Any update on this? If this is a bug, I can create a JIRA issue and
hopefully a fix can be included in the next release.

Please let me know.

Thanks!

> ...
>
> read more »

[Kristina Chodorow]

This is a bug. Unfortunately, the _isSelf command was disappointingly
successful, so I'm not sure what's going on yet. I've filed a bug at
https://jira.mongodb.org/browse/SERVER-3953.

> > > > > > > > > > When I start that instance again, it's...
>
> read more »