rs.stepDown() Issues

186 views
Skip to first unread message

A. Jalil @AJ

unread,
Sep 24, 2015, 7:37:51 PM9/24/15
to mongodb-user
Greetings -

I thought I was done with the first Replica Set RS0 migration up until I tried to do the primary and things went south :(
Please note, both SECONDARY members migrated successfully and their stat looks good.. Now, I need to do the PRIMARY..

rs0:PRIMARY> rs.stepDown()
2015-09-24T15:51:40.390-0500 DBClientCursor::init call() failed
2015-09-24T15:51:40.391-0500 Error: error doing query: failed at src/mongo/shell/query.js:81
2015-09-24T15:51:40.395-0500 trying reconnect to 127.0.0.1:27017 (127.0.0.1) failed
2015-09-24T15:51:40.395-0500 reconnect 127.0.0.1:27017 (127.0.0.1) ok

I'd appreciate your input.. Thx.

@AJ

Stephen Steneker

unread,
Sep 24, 2015, 8:23:46 PM9/24/15
to mongodb-user
Hi AJ,


This is actually expected behaviour:
  https://docs.mongodb.org/v3.0/reference/method/rs.stepDown/#behavior

The rs.stepDown() disconnects current clients (including the shell connection you are running this command from) to ensure clients have an accurate view of the replica set. If you run another command in the mongo shell it should automatically reconnect and discover the current replica set configuration.

Regards,
Stephen

Harish CS

unread,
Sep 24, 2015, 8:37:40 PM9/24/15
to mongodb-user
Upon successful stepdown, rs.stepDown() forces all clients currently connected to the database to disconnect. This helps ensure that the clients maintain an accurate view of the replica set.

Regards
HC

A. Jalil @AJ

unread,
Sep 24, 2015, 11:51:02 PM9/24/15
to mongodb-user
Hi Stephen,

You said:  " If you run another command in the mongo shell, it should automatically reconnect.."

I am not sure what you mean ? are you saying to run rs.stepDown() from mongS or form config server ? what command are you referring to ?

Please give me an example.. Thanks.
@AJ

Message has been deleted

A. Jalil @AJ

unread,
Sep 25, 2015, 12:00:22 AM9/25/15
to mongodb-user
oops is there a way to edit my stats, I left some original hostnames & I need to remove them for security purpose please.. Thanks.

Stephen Steneker

unread,
Sep 25, 2015, 12:01:38 AM9/25/15
to mongodb-user
On Friday, 25 September 2015 13:51:02 UTC+10, A. Jalil @AJ wrote:
You said:  " If you run another command in the mongo shell, it should automatically reconnect.."

I am not sure what you mean ? are you saying to run rs.stepDown() from mongS or form config server ? what command are you referring to ?

Hi AJ,

You can actually see this happen in the output you posted:

rs0:PRIMARY> rs.stepDown()
2015-09-24T15:51:40.390-0500 DBClientCursor::init call() failed
2015-09-24T15:51:40.391-0500 Error: error doing query: failed at src/mongo/shell/query.js:81
2015-09-24T15:51:40.395-0500 trying reconnect to 127.0.0.1:27017 (127.0.0.1) failed
2015-09-24T15:51:40.395-0500 reconnect 127.0.0.1:27017 (127.0.0.1) ok
) ok

The rs.stepDown() disconnected your active session and there was an error on the next query. The mongo shell then tried to reconnect (and succeeded).

In general, if your mongo shell connection is dropped the next command you run will automatically try to reconnect.

Regards,
Stephen

MoroccoIT

unread,
Sep 25, 2015, 12:07:46 AM9/25/15
to mongod...@googlegroups.com
I am able to reconnect, but still getting the same error.. the only change I see is the prompt gets changed from rs0:PRIMARY to rs.SECONDARY but when I do rs.status() again the primary server remains the same..

rs0:PRIMARY> rs.stepDown()
2015-09-24T22:44:50.611-0500 DBClientCursor::init call() failed
2015-09-24T22:44:50.612-0500 Error: error doing query: failed at src/mongo/shell/query.js:81
2015-09-24T22:44:50.613-0500 trying reconnect to 127.0.0.1:27017 (127.0.0.1) failed
2015-09-24T22:44:50.613-0500 reconnect 127.0.0.1:27017 (127.0.0.1) ok
rs0:SECONDARY>

A. Jalil @AJ

unread,
Sep 25, 2015, 12:08:05 AM9/25/15
to mongodb-user
I am able to reconnect, but still getting the same error.. the only change I see is the prompt gets changed from rs0:PRIMARY to rs.SECONDARY but when I do rs.status() again the primary server remains the same..

rs0:PRIMARY> rs.stepDown()
2015-09-24T22:44:50.611-0500 DBClientCursor::init call() failed
2015-09-24T22:44:50.612-0500 Error: error doing query: failed at src/mongo/shell/query.js:81
2015-09-24T22:44:50.613-0500 trying reconnect to 127.0.0.1:27017 (127.0.0.1) failed
2015-09-24T22:44:50.613-0500 reconnect 127.0.0.1:27017 (127.0.0.1) ok
rs0:SECONDARY>

A. Jalil @AJ

unread,
Sep 25, 2015, 12:12:28 AM9/25/15
to mongodb-user
Just ran stats again, I need to make [ server-1.com ] a secondary so I can shut it down and zip the dbpath file & transfer it to the new server.. 


rs0:PRIMARY> rs.status()
{
        "set" : "rs0",
        "date" : ISODate("2015-09-25T03:36:59Z"),
        "myState" : 1,
        "members" : [
                {
                        "_id" : 3,
                        "name" : "server-1.com:27017",
                        "health" : 1,
                        "state" : 1,
                        "stateStr" : "PRIMARY",
                        "uptime" : 5552361,
                        "optime" : Timestamp(1443123243, 2),
                        "optimeDate" : ISODate("2015-09-24T19:34:03Z"),
                        "electionTime" : Timestamp(1443132999, 1),
                        "electionDate" : ISODate("2015-09-24T22:16:39Z"),
                        "self" : true
                },
                {
                        "_id" : 4,
                        "name" : "server-2.com:27017",
                        "health" : 1,
                        "state" : 2,
                        "stateStr" : "SECONDARY",
                        "uptime" : 342499,
                        "optime" : Timestamp(1443123243, 2),
                        "optimeDate" : ISODate("2015-09-24T19:34:03Z"),
                        "lastHeartbeat" : ISODate("2015-09-25T03:36:59Z"),
                        "lastHeartbeatRecv" : ISODate("2015-09-25T03:36:58Z"),
                        "pingMs" : 1,
                        "syncingTo" : "hostname"
                },
                {
                        "_id" : 5,
                        "name" : "server-3.com:27017",
                        "health" : 1,
                        "state" : 2,
                        "stateStr" : "SECONDARY",
                        "uptime" : 105553,
                        "optime" : Timestamp(1443123243, 2),
                        "optimeDate" : ISODate("2015-09-24T19:34:03Z"),
                        "lastHeartbeat" : ISODate("2015-09-25T03:36:58Z"),
                        "lastHeartbeatRecv" : ISODate("2015-09-25T03:36:59Z"),
                        "pingMs" : 1,
                        "syncingTo" : "hostname"
                },
                {
                        "_id" : 6,
                        "name" : "server-4.com:27017",
                        "health" : 1,
                        "state" : 2,
                        "stateStr" : "SECONDARY",
                        "uptime" : 178014,
                        "optime" : Timestamp(1443123243, 2),
                        "optimeDate" : ISODate("2015-09-24T19:34:03Z"),
                        "lastHeartbeat" : ISODate("2015-09-25T03:36:58Z"),
                        "lastHeartbeatRecv" : ISODate("2015-09-25T03:36:58Z"),
                        "pingMs" : 1,
                        "syncingTo" : "hostname"
                },
                {
                        "_id" : 7,
                        "name" : "server-5.com:27017",
                        "health" : 1,
                        "state" : 2,
                        "stateStr" : "SECONDARY",
                        "uptime" : 39170,
                        "optime" : Timestamp(1443123243, 2),
                        "optimeDate" : ISODate("2015-09-24T19:34:03Z"),
                        "lastHeartbeat" : ISODate("2015-09-25T03:36:58Z"),
                        "lastHeartbeatRecv" : ISODate("2015-09-25T03:36:58Z"),
                        "pingMs" : 1,
                        "syncingTo" : "hostname"
                }
        ],
        "ok" : 1
}

Stephen Steneker

unread,
Sep 25, 2015, 12:16:28 AM9/25/15
to mongodb-user

Hi AJ,

The disconnection on rs.stepDown() is expected (not an error).

Your prompt on rs0 changed from "primary" to “secondary”. Was this member re-elected as a primary afterward?

You can make the current host ineligible to become primary for a period of time using the rs.freeze() command:
http://docs.mongodb.org/v3.0/reference/method/rs.freeze/

You can also influence which member is elected primary by adjusting the priority values:
http://docs.mongodb.org/manual/tutorial/adjust-replica-set-member-priority/

Regards,
Stephen

A. Jalil @AJ

unread,
Sep 25, 2015, 12:24:32 AM9/25/15
to mongodb-user
I think the error message is misleading. server-1.com has always been PRIMARY and still PRIMARY regardless after I ran stepDown few times.
I know I am done with server-2.com and server-3.com & are already been copied to AWS. But I need to make server-1.com a SECONDARY in order to shut it down and migrate it..  

A. Jalil @AJ

unread,
Sep 25, 2015, 12:28:03 AM9/25/15
to mongodb-user
Does it matter if I use config or admin DB before run stepDown ?

A. Jalil @AJ

unread,
Sep 25, 2015, 12:30:03 AM9/25/15
to mongodb-user
Ok, it works now :)  I had to use config DB before I can run stepDown.. All good.. Thanks !

A. Jalil @AJ

unread,
Sep 25, 2015, 12:35:32 AM9/25/15
to mongodb-user
Very strange. the PRIMARY node got changed to SECONDARY for about 2 minutes, and then it reset itself to PRIMARY gain. How do I force it to stay SECONDARY ?

A. Jalil @AJ

unread,
Sep 25, 2015, 12:42:45 AM9/25/15
to mongodb-user
I was reading the mongo Doc, I see I can freeze it after the stepDown exp: rs.stepDown(120)  to freeze it for 120 seconds.I will freeze for 1hr until I copy dbpath, I think that should work..
Reply all
Reply to author
Forward
0 new messages