Failure Migrating Seconary to 3.2 / WiredTiger

[Andrew Smith]

Hi

I have a 3.0 cluster on the pre-wired tiger storage engine.

I'm trying to upgrade from 3.0 to 3.2 and upgrade to wired tiger at the same time.

Following theses guides: https://docs.mongodb.com/manual/release-notes/3.2-upgrade/#upgrade-a-replica-set-to-3-2 https://docs.mongodb.com/manual/tutorial/change-replica-set-wiredtiger/

I upgraded my binaries to to 3.2 on my secondary and restarted it.

The initial sync started well but after 22gb (of 200gb) was synced it failed. Now when I resync (by doing cfg = rs.conf(), rs.reconfig(cfg) I get the following error in my secondary logs:

2016-08-17T08:24:39.885+0000 I REPL     [ReplicationExecutor] New replica set config in use: { _id: "rs0", version: 1432509, members: [ { _id: 0, host: "mongodba.infra.svc.cluster.local:27017", arbiterOnly: false, buildIndexes: true, hidden: false, priority: 1.0, tags: {}, slaveDelay: 0, votes: 1 }, { _id: 1, host: "mongodbb.infra.svc.cluster.local:27017", arbiterOnly: false, buildIndexes: true, hidden: false, priority: 0.5, tags: {}, slaveDelay: 0, votes: 1 }, { _id: 2, host: "mongodbarba.infra.svc.cluster.local:27017", arbiterOnly: true, buildIndexes: true, hidden: false, priority: 0.4, tags: {}, slaveDelay: 0, votes: 1 } ], settings: { chainingAllowed: true, heartbeatIntervalMillis: 2000, heartbeatTimeoutSecs: 10, electionTimeoutMillis: 10000, getLastErrorModes: {}, getLastErrorDefaults: { w: 1, wtimeout: 0 } } }

2016-08-17T08:24:39.885+0000 I REPL     [ReplicationExecutor] This node is mongodbb.infra.svc.cluster.local:27017 in the config

2016-08-17T08:24:40.292+0000 I REPL     [ReplicationExecutor] syncing from: mongodba.infra.svc.cluster.local:27017

2016-08-17T08:24:40.799+0000 I REPL     [SyncSourceFeedback] setting syncSourceFeedback to mongodba.infra.svc.cluster.local:27017

2016-08-17T08:24:41.084+0000 I REPL     [SyncSourceFeedback] SyncSourceFeedback error sending update, response: { ok: 0.0, errmsg: "Unexpected field durableOpTime in UpdateInfoArgs", code: 2 }

2016-08-17T08:24:41.084+0000 I REPL     [SyncSourceFeedback] SyncSourceFeedback falling back to old style UpdatePosition command

Is there anything I'm doing wrong here? Any pointers greatly appreciated!

Thanks

[Kevin Adistambha]

Hi Andrew,

I upgraded my binaries to to 3.2 on my secondary and restarted it.

The initial sync started well but after 22gb (of 200gb) was synced it failed. Now when I resync (by doing cfg = rs.conf(), rs.reconfig(cfg) I get the following error in my secondary logs:

2016-08-17T08:24:41.084+0000 I REPL [SyncSourceFeedback] SyncSourceFeedback error sending update, response: { ok: 0.0, errmsg: “Unexpected field durableOpTime in UpdateInfoArgs”, code: 2 }

2016-08-17T08:24:41.084+0000 I REPL [SyncSourceFeedback] SyncSourceFeedback falling back to old style UpdatePosition command

The message you saw is not an error message. Since currently you have a mixed MongoDB versions in your replica set, this message informs you that there is an extra field sent by the 3.2 Secondary that the 3.0 Primary does not recognize, and it is ignoring this extra field.

Regarding your initial sync issue, could you post some more details:

• Your OS name and version, along with your hardware specifications.
• What error message you are seeing that may be correlated to the sync failure.
• What are the output of the rs.status() and rs.printSlaveReplicationInfo() command.
• What are the exact sequence of commands you executed to perform the upgrade (from the terminal and the mongo shell)

Please note that for production deployments, it may be less disruptive if you upgrade the binary version first before changing the storage engine.

Best regards,
Kevin

​