When I rs.add('new-server'), the new server is trying to sync off PRIMARY?

52 views
Skip to first unread message

Davis Ford

unread,
Jan 18, 2017, 10:34:23 PM1/18/17
to mongodb-user
I've been upgrading my repl set the last couple days; the plan is to spawn new 3.0 instances (we were on 2.6.5) -- let them sync to a secondary, and then start taking the older nodes out.

Now, I have one 2.6.5 instance left, and I'm spawning a 3.0 AMI image that is ready to go, but the last two times I have tried:

> rs.add('new-instance-addr')

...when I run rs.status() it indicates the new instance which is in state STARTUP2 is syncing to the PRIMARY?  The first time it happened, I did not catch it but it did not end well for our production deployment until I pre-maturely killed the instance that was syncing to the PRIMARY.  Tonight I spawned a new instance, and immediately checked it and it again chose to sync from the PRIMARY, so I killed it.

AFAICT there's no way to indicate a sync preference when you add a member to a replica set.  But I cannot have a new member sync off the PRIMARY.  

What am I missing?  Prior to this, I spawned a number of servers and added them to the repl set and they all synced off one of the secondaries.

Thanks in advance

Davis Ford

unread,
Jan 18, 2017, 10:40:31 PM1/18/17
to mongodb-user
Answering my own question.  Not sure if this information is still applicable, but I'm going to try it: http://stackoverflow.com/a/28486223/843567

Davis Ford

unread,
Jan 18, 2017, 10:58:23 PM1/18/17
to mongodb-user
...and Nope...did not work.

1. PRIMARY> rs.add("ip-10-0-0-113:27017")               // <- on primary 
2. STARTUP2> rs.syncFrom("ip-10-0-0-133:27017")   // <- on new host trying to add to replica set

The command said OK:

repl-emagine:STARTUP2> rs.syncFrom('ip-10-0-0-133:27017')
{
 
"syncFromRequested" : "ip-10-0-0-133:27017",
 
"prevSyncTarget" : "ip-10-0-0-19:27017",
 
"ok" : 1
}


...but it still said it was syncing from the PRIMARY (here as ip-10-0-0-19:27017 <= PRIMARY)

{
 
"_id" : 8,
 
"name" : "ip-10-0-0-113:27017",
 
"health" : 1,
 
"state" : 5,
 
"stateStr" : "STARTUP2",
 
"uptime" : 391,
 
"optime" : Timestamp(0, 0),
 
"optimeDate" : ISODate("1970-01-01T00:00:00Z"),
 
"syncingTo" : "ip-10-0-0-19:27017",
 
"configVersion" : 42,
 
"self" : true
}


what am I missing here?  I did try the rs.syncFrom() command several times and kept refreshing the rs.status() but to no avail.  Nothing changed.  I cannot have a new server bootstrap it's initial sync from the PRIMARY.  How can I work around this?

The last two servers I spun up and added to the replica set sync'd happily from another secondary.

Davis Ford

unread,
Jan 18, 2017, 11:13:48 PM1/18/17
to mongodb-user
Answering my own question again.  It was covered in the docs:

Sync Logic
Only modify the default sync logic as needed, and always exercise caution. rs.syncFrom() will not affect an in-progress initial sync operation. To affect the sync target for the initial sync, run rs.syncFrom() operation before initial sync.
If you run rs.syncFrom() during initial sync, MongoDB produces no error messages, but the sync target will not change until after the initial sync operation

Reply all
Reply to author
Forward
0 new messages