rs.sync failing because of matching opTime

[davidmo]

hi all. i have a 3 member replicate cluster. the primary and one secondary are 3.4.7. the other secondary is 3.4.17 (which i just found out).

the second secondary syncs to the first secondary. when i try to run rs.sync to get it to sync to the primary is get the following:

2018-10-06T15:27:04.973-0400 I NETWORK  [conn1218] received client metadata from 127.0.0.1:54114 conn1218: { application: { name: "MongoDB Shell" }, driver: { name: "MongoDB Internal Client", version: "3.4.17" }, os: { type: "Linux", name: "CentOS Linux release 7.5.1804 (Core) ", architecture: "x86_64", version: "Kernel 3.10.0-862.11.6.el7.x86_64" } }

2018-10-06T15:27:30.236-0400 I REPL     [replication-2] Choosing new sync source because the user has requested to use mdb06:27017 as a sync source

2018-10-06T15:27:30.236-0400 I REPL     [replication-2] Canceling oplog query due to OplogQueryMetadata. We have to choose a new sync

source. Current source: mdb07:27017, OpTime { ts: Timestamp 1538854050000|1, t: 18 }, its sync source index:0

2018-10-06T15:27:30.236-0400 W REPL     [rsBackgroundSync] Fetcher stopped querying remote oplog with error: InvalidSyncSource: sync source mdb07:27017 (config version: 5; last applied optime: { ts: Timestamp 1538854050000|1, t: 18 }; sync source index: 0; primary index: 0) is no longer valid

2018-10-06T15:27:30.236-0400 I REPL     [rsBackgroundSync] choosing sync source candidate by request: mdb06:27017

2018-10-06T15:27:30.236-0400 I ASIO     [NetworkInterfaceASIO-RS-0] Connecting to mdb06:27017

2018-10-06T15:27:30.238-0400 I REPL     [SyncSourceFeedback] SyncSourceFeedback error sending update to mdb07:27017: InvalidSyncSource: Sync source was cleared. Was mdb07:27017

2018-10-06T15:27:30.245-0400 I ASIO     [NetworkInterfaceASIO-RS-0] Successfully connected to mdb06:27017, took 9ms (1 connections now open to mdb06:27017)

2018-10-06T15:27:30.251-0400 I ASIO     [NetworkInterfaceASIO-RS-0] Connecting to mdb06:27017

2018-10-06T15:27:30.254-0400 W REPL     [rsBackgroundSync] Fetcher stopped querying remote oplog with error: InvalidSyncSource: Sync source's last applied OpTime { ts: Timestamp 1538854050000|1, t: 18 } is not greater than our last fetched OpTime { ts: Timestamp 1538854050000|1, t: 18 }. Choosing new sync source.

2018-10-06T15:27:30.254-0400 I REPL     [rsBackgroundSync] could not find member to sync from

2018-10-06T15:27:30.258-0400 I ASIO     [NetworkInterfaceASIO-RS-0] Successfully connected to mdb06:27017, took 7ms (2 connections now open to mdb06:27017)

2018-10-06T15:27:35.257-0400 I REPL     [SyncSourceFeedback] SyncSourceFeedback error sending update to mdb06:27017: InvalidSyncSource: Sync source was cleared. Was mdb06:27017

2018-10-06T15:27:41.256-0400 I REPL     [rsBackgroundSync] sync source candidate: mdb07:27017

 

the rs.sync fails and it continues to sync to the other secondary.

any body know what is going on ? will an initial sync fix this ? can something else fix it ? i checked change list for 3.4.7 - 3.4.17 but nothing jumped out.

thanks in advance.

david