Received: by 10.236.76.4 with SMTP id a4mr30889142yhe.2.1335813498806; Mon, 30 Apr 2012 12:18:18 -0700 (PDT) X-BeenThere: mongodb-user@googlegroups.com Received: by 10.236.140.106 with SMTP id d70ls5351223yhj.6.gmail; Mon, 30 Apr 2012 12:18:10 -0700 (PDT) Received: by 10.236.190.130 with SMTP id e2mr85080yhn.20.1335813490027; Mon, 30 Apr 2012 12:18:10 -0700 (PDT) Date: Mon, 30 Apr 2012 12:18:09 -0700 (PDT) From: idris To: mongodb-user@googlegroups.com Message-ID: <31952500.1659.1335813489236.JavaMail.geo-discussion-forums@ynjj38> In-Reply-To: <11559380.708.1335798003486.JavaMail.geo-discussion-forums@yny11> References: <33475618.387.1335367025334.JavaMail.geo-discussion-forums@vbbdh3> <15243932.450.1335370536984.JavaMail.geo-discussion-forums@ynje10> <1612989.20.1335376811323.JavaMail.geo-discussion-forums@vbsf10> <10530436.959.1335380628916.JavaMail.geo-discussion-forums@yny11> <27742326.703.1335390045149.JavaMail.geo-discussion-forums@vbjb10> <10875417.2809.1335393338220.JavaMail.geo-discussion-forums@vbqq1> <20905042.1672.1335666830683.JavaMail.geo-discussion-forums@vbbfr18> <11294108.1008.1335751733517.JavaMail.geo-discussion-forums@ynbq3> <11559380.708.1335798003486.JavaMail.geo-discussion-forums@yny11> Subject: Re: [mongodb-user] Re: Why does rs.reconfig always take down primary briefly? MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_1657_1054469.1335813489229" ------=_Part_1657_1054469.1335813489229 Content-Type: multipart/alternative; boundary="----=_Part_1658_11587786.1335813489229" ------=_Part_1658_11587786.1335813489229 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Unfortunately, I don't have those from this past time. I can tell you that before the reconfig, everything was normal (_id=1 was primary, no lag on any secondaries). After the reconfig, the host that had been primary was now secondary, and there was no primary. This lasted a couple of minutes, until the primary was finally elected, resulting in a normal status again (_id=1 primary, no lag on secondaries). If I remember correctly, while there was no primary, rs.status() reported that the other nodes were unreachable. You can see that in the primary's logs I posted before: [rsHealthPoll] replSet info 10.0.51.3:27017 is down (or slow to respond): socket exception [rsHealthPoll] replSet info 10.0.51.1:27017 is down (or slow to respond): socket exception [rsHealthPoll] replSet info 10.0.51.20:27017 is down (or slow to respond): socket exception [rsHealthPoll] replSet info 10.0.51.21:27017 is down (or slow to respond): socket exception [rsHealthPoll] replSet info 10.0.51.4:27017 is down (or slow to respond): socket exception [rsHealthPoll] replSet info mongo901:27017 is down (or slow to respond): socket exception On Monday, April 30, 2012 10:00:03 AM UTC-5, Kyle Banker wrote: > > Can you please post rs.status() before and after as well? > > On Sunday, April 29, 2012 10:08:53 PM UTC-4, idris wrote: >> >> rs.conf() before: >> >>> { >>> >>> "_id" : "myrepl", >>> >>> "version" : 19, >>> >>> "members" : [ >>> >>> { >>> >>> "_id" : 0, >>> >>> "host" : "10.0.51.1:27017", >>> >>> "priority" : 10, >>> >>> "tags" : { >>> >>> "dc" : "va" >>> >>> } >>> >>> }, >>> >>> { >>> >>> "_id" : 1, >>> >>> "host" : "10.0.51.2:27017", >>> >>> "priority" : 20, >>> >>> "tags" : { >>> >>> "dc" : "va" >>> >>> } >>> >>> }, >>> >>> { >>> >>> "_id" : 2, >>> >>> "host" : "10.0.51.3:27017", >>> >>> "tags" : { >>> >>> "dc" : "va" >>> >>> } >>> >>> }, >>> >>> { >>> >>> "_id" : 3, >>> >>> "host" : "10.0.51.20:27017", >>> >>> "priority" : 0, >>> >>> "hidden" : true, >>> >>> "tags" : { >>> >>> "dc" : "va" >>> >>> } >>> >>> }, >>> >>> { >>> >>> "_id" : 4, >>> >>> "host" : "10.0.51.21:27017", >>> >>> "arbiterOnly" : true >>> >>> }, >>> >>> { >>> >>> "_id" : 5, >>> >>> "host" : "10.0.51.4:27017", >>> >>> "tags" : { >>> >>> "dc" : "va" >>> >>> } >>> >>> }, >>> >>> { >>> >>> "_id" : 6, >>> >>> "host" : "mongo901:27017", >>> >>> "votes" : 0, >>> >>> "priority" : 0, >>> >>> "hidden" : true, >>> >>> "tags" : { >>> >>> "dc" : "ca" >>> >>> } >>> >>> } >>> >>> ] >>> >>> } >>> >>> >>> >> rs.conf() after: >> >>> { >>> >>> "_id" : "myrepl", >>> >>> "version" : 20, >>> >>> "members" : [ >>> >>> { >>> >>> "_id" : 0, >>> >>> "host" : "10.0.51.1:27017", >>> >>> "priority" : 10, >>> >>> "tags" : { >>> >>> "dc" : "va" >>> >>> } >>> >>> }, >>> >>> { >>> >>> "_id" : 1, >>> >>> "host" : "10.0.51.2:27017", >>> >>> "priority" : 20, >>> >>> "tags" : { >>> >>> "dc" : "va" >>> >>> } >>> >>> }, >>> >>> { >>> >>> "_id" : 2, >>> >>> "host" : "10.0.51.3:27017", >>> >>> "tags" : { >>> >>> "dc" : "va" >>> >>> } >>> >>> }, >>> >>> { >>> >>> "_id" : 3, >>> >>> "host" : "10.0.51.20:27017", >>> >>> "votes" : 0, >>> >>> "priority" : 0, >>> >>> "hidden" : true, >>> >>> "tags" : { >>> >>> "dc" : "va" >>> >>> } >>> >>> }, >>> >>> { >>> >>> "_id" : 4, >>> >>> "host" : "10.0.51.21:27017", >>> >>> "arbiterOnly" : true >>> >>> }, >>> >>> { >>> >>> "_id" : 5, >>> >>> "host" : "10.0.51.4:27017", >>> >>> "tags" : { >>> >>> "dc" : "va" >>> >>> } >>> >>> }, >>> >>> { >>> >>> "_id" : 6, >>> >>> "host" : "mongo901:27017", >>> >>> "votes" : 0, >>> >>> "priority" : 0, >>> >>> "hidden" : true, >>> >>> "tags" : { >>> >>> "dc" : "ca" >>> >>> } >>> >>> } >>> >>> ] >>> >>> } >>> >>> >>> >> I don't see anything useful in the logs regarding up/down state before >> the reconfig, but I know that all nodes were up and reachable from all >> other nodes. This was not a one-time event, the same thing happens every >> time I do a reconfig. >> >> >> On Saturday, April 28, 2012 11:14:31 PM UTC-5, Scott Hernandez wrote: >>> >>> Can you post the config -- rs.conf() -- before and after as well as >>> the status -- rs.status()? >>> >>> Also, please post the logs from before the save showing that state of >>> the world in terms of who was up/down via the replica status messages. >>> >>> On Sat, Apr 28, 2012 at 7:33 PM, idris <> wrote: >>> > *bump* Is this worthy of a bug report in Jira? >>> > >>> > On Wednesday, April 25, 2012 6:35:38 PM UTC-4, idris wrote: >>> >> >>> >> By the way, the docs >>> >> ( >>> http://www.mongodb.org/display/DOCS/Reconfiguring+when+Members+are+Up) >>> also >>> >> say that the primary steps down and closes sockets from clients upon >>> a >>> >> reconfig, but only "in certain circumstances". It would be helpful to >>> know >>> >> which circumstances will cause a stepdown. >>> >> >>> >>> In certain circumstances, the primary steps down (perhaps >>> transiently) on >>> >>> a reconfiguration. On a step-down, the primary closes sockets from >>> clients >>> >>> to assure the clients know quickly that the server is no longer >>> primary. >>> >>> Thus, your shell session may experience a disconnect on a reconfig >>> command. >>> >> >>> >> >>> >> On Wednesday, April 25, 2012 5:40:45 PM UTC-4, idris wrote: >>> >>> >>> >>> Running 7 nodes total: >>> >>> 1 primary, 3 secondaries, 1 arbiter, 1 backup (hidden, priority:0), >>> 1 >>> >>> off-site secondary (hidden, priority:0, votes:0) >>> >>> >>> >>> Here are the logs of the primary and a secondary when I ran the >>> reconfig >>> >>> command. In this reconfig, I set had votes:0 on the backup node. >>> >>> Primary Log: http://pastebin.com/3a0fPGzA >>> >>> Secondary Log: http://pastebin.com/SGxeJqVK >>> >>> >>> >>> I stripped out all of the noise (connection accepted, end >>> connection) >>> >>> from those logs as well. >>> >>> >>> >>> Primary definitely relinquishes it's primary state... >>> >>>> >>> >>>> Wed Apr 25 17:20:42 [conn417285] replSet replSetReconfig config >>> object >>> >>>> parses ok, 7 members specified >>> >>>> Wed Apr 25 17:20:42 [conn417285] replSet replSetReconfig [2] >>> >>>> Wed Apr 25 17:20:42 [conn417285] replSet info saving a newer config >>> >>>> version to local.system.replset >>> >>>> Wed Apr 25 17:20:42 [conn417285] replSet saveConfigLocally done >>> >>>> Wed Apr 25 17:20:42 [conn417285] replSet relinquishing primary >>> state >>> >>>> Wed Apr 25 17:20:42 [conn417285] replSet SECONDARY >>> >>>> Wed Apr 25 17:20:42 [conn417285] replSet closing client sockets >>> after >>> >>>> reqlinquishing primary >>> >>>> Wed Apr 25 17:20:42 [conn417285] replSet PRIMARY >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> On Wednesday, April 25, 2012 3:03:48 PM UTC-4, Kyle Banker wrote: >>> >>>> >>> >>>> How many total nodes were you running when you removed the >>> slavedelay >>> >>>> node? >>> >>>> >>> >>>> On Wednesday, April 25, 2012 2:00:11 PM UTC-4, idris wrote: >>> >>>>> >>> >>>>> Running version 2.0.4. I'll try to dig up some logs. Is there >>> any >>> >>>>> documentation on which configuration changes will and will not >>> force an >>> >>>>> election or a stepDown? >>> >>>>> >>> >>>>> On Wednesday, April 25, 2012 12:15:36 PM UTC-4, Kyle Banker wrote: >>> >>>>>> >>> >>>>>> Which version of MongoDB are you running? These was an issue in >>> v1.8 >>> >>>>>> but should not be an issue post 2.0. >>> >>>>>> >>> >>>>>> On Wednesday, April 25, 2012 11:17:05 AM UTC-4, idris wrote: >>> >>>>>>> >>> >>>>>>> For example, I just had a slaveDelay member and I removed the >>> >>>>>>> slaveDelay, then saved the config using rs.reconfig(), and the >>> primary went >>> >>>>>>> down briefly, and apparently dropped all connections to clients. >>> Why does >>> >>>>>>> it need to do this? Shouldn't there be an option for "don't go >>> down upon >>> >>>>>>> reconfig" or something? >>> >>>>>>> >>> >>>>>>> -Idris >>> > >>> > -- >>> > You received this message because you are subscribed to the Google >>> Groups >>> > "mongodb-user" group. >>> > To view this discussion on the web visit >>> > https://groups.google.com/d/msg/mongodb-user/-/8pRjE2tPn7MJ. >>> > >>> > To post to this group, send email to mongodb-user@googlegroups.com. >>> > To unsubscribe from this group, send email to >>> > mongodb-user+unsubscribe@googlegroups.com. >>> > For more options, visit this group at >>> > http://groups.google.com/group/mongodb-user?hl=en. >>> >> >> On Saturday, April 28, 2012 11:14:31 PM UTC-5, Scott Hernandez wrote: >>> >>> Can you post the config -- rs.conf() -- before and after as well as >>> the status -- rs.status()? >>> >>> Also, please post the logs from before the save showing that state of >>> the world in terms of who was up/down via the replica status messages. >>> >>> On Sat, Apr 28, 2012 at 7:33 PM, idris <> wrote: >>> > *bump* Is this worthy of a bug report in Jira? >>> > >>> > On Wednesday, April 25, 2012 6:35:38 PM UTC-4, idris wrote: >>> >> >>> >> By the way, the docs >>> >> ( >>> http://www.mongodb.org/display/DOCS/Reconfiguring+when+Members+are+Up) >>> also >>> >> say that the primary steps down and closes sockets from clients upon >>> a >>> >> reconfig, but only "in certain circumstances". It would be helpful to >>> know >>> >> which circumstances will cause a stepdown. >>> >> >>> >>> In certain circumstances, the primary steps down (perhaps >>> transiently) on >>> >>> a reconfiguration. On a step-down, the primary closes sockets from >>> clients >>> >>> to assure the clients know quickly that the server is no longer >>> primary. >>> >>> Thus, your shell session may experience a disconnect on a reconfig >>> command. >>> >> >>> >> >>> >> On Wednesday, April 25, 2012 5:40:45 PM UTC-4, idris wrote: >>> >>> >>> >>> Running 7 nodes total: >>> >>> 1 primary, 3 secondaries, 1 arbiter, 1 backup (hidden, priority:0), >>> 1 >>> >>> off-site secondary (hidden, priority:0, votes:0) >>> >>> >>> >>> Here are the logs of the primary and a secondary when I ran the >>> reconfig >>> >>> command. In this reconfig, I set had votes:0 on the backup node. >>> >>> Primary Log: http://pastebin.com/3a0fPGzA >>> >>> Secondary Log: http://pastebin.com/SGxeJqVK >>> >>> >>> >>> I stripped out all of the noise (connection accepted, end >>> connection) >>> >>> from those logs as well. >>> >>> >>> >>> Primary definitely relinquishes it's primary state... >>> >>>> >>> >>>> Wed Apr 25 17:20:42 [conn417285] replSet replSetReconfig config >>> object >>> >>>> parses ok, 7 members specified >>> >>>> Wed Apr 25 17:20:42 [conn417285] replSet replSetReconfig [2] >>> >>>> Wed Apr 25 17:20:42 [conn417285] replSet info saving a newer config >>> >>>> version to local.system.replset >>> >>>> Wed Apr 25 17:20:42 [conn417285] replSet saveConfigLocally done >>> >>>> Wed Apr 25 17:20:42 [conn417285] replSet relinquishing primary >>> state >>> >>>> Wed Apr 25 17:20:42 [conn417285] replSet SECONDARY >>> >>>> Wed Apr 25 17:20:42 [conn417285] replSet closing client sockets >>> after >>> >>>> reqlinquishing primary >>> >>>> Wed Apr 25 17:20:42 [conn417285] replSet PRIMARY >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> On Wednesday, April 25, 2012 3:03:48 PM UTC-4, Kyle Banker wrote: >>> >>>> >>> >>>> How many total nodes were you running when you removed the >>> slavedelay >>> >>>> node? >>> >>>> >>> >>>> On Wednesday, April 25, 2012 2:00:11 PM UTC-4, idris wrote: >>> >>>>> >>> >>>>> Running version 2.0.4. I'll try to dig up some logs. Is there >>> any >>> >>>>> documentation on which configuration changes will and will not >>> force an >>> >>>>> election or a stepDown? >>> >>>>> >>> >>>>> On Wednesday, April 25, 2012 12:15:36 PM UTC-4, Kyle Banker wrote: >>> >>>>>> >>> >>>>>> Which version of MongoDB are you running? These was an issue in >>> v1.8 >>> >>>>>> but should not be an issue post 2.0. >>> >>>>>> >>> >>>>>> On Wednesday, April 25, 2012 11:17:05 AM UTC-4, idris wrote: >>> >>>>>>> >>> >>>>>>> For example, I just had a slaveDelay member and I removed the >>> >>>>>>> slaveDelay, then saved the config using rs.reconfig(), and the >>> primary went >>> >>>>>>> down briefly, and apparently dropped all connections to clients. >>> Why does >>> >>>>>>> it need to do this? Shouldn't there be an option for "don't go >>> down upon >>> >>>>>>> reconfig" or something? >>> >>>>>>> >>> >>>>>>> -Idris >>> > >>> > -- >>> > You received this message because you are subscribed to the Google >>> Groups >>> > "mongodb-user" group. >>> > To view this discussion on the web visit >>> > https://groups.google.com/d/msg/mongodb-user/-/8pRjE2tPn7MJ. >>> > >>> > To post to this group, send email to mongodb-user@googlegroups.com. >>> > To unsubscribe from this group, send email to >>> > mongodb-user+unsubscribe@googlegroups.com. >>> > For more options, visit this group at >>> > http://groups.google.com/group/mongodb-user?hl=en. >>> >> >> On Saturday, April 28, 2012 11:14:31 PM UTC-5, Scott Hernandez wrote: >>> >>> Can you post the config -- rs.conf() -- before and after as well as >>> the status -- rs.status()? >>> >>> Also, please post the logs from before the save showing that state of >>> the world in terms of who was up/down via the replica status messages. >>> >>> On Sat, Apr 28, 2012 at 7:33 PM, idris <> wrote: >>> > *bump* Is this worthy of a bug report in Jira? >>> > >>> > On Wednesday, April 25, 2012 6:35:38 PM UTC-4, idris wrote: >>> >> >>> >> By the way, the docs >>> >> ( >>> http://www.mongodb.org/display/DOCS/Reconfiguring+when+Members+are+Up) >>> also >>> >> say that the primary steps down and closes sockets from clients upon >>> a >>> >> reconfig, but only "in certain circumstances". It would be helpful to >>> know >>> >> which circumstances will cause a stepdown. >>> >> >>> >>> In certain circumstances, the primary steps down (perhaps >>> transiently) on >>> >>> a reconfiguration. On a step-down, the primary closes sockets from >>> clients >>> >>> to assure the clients know quickly that the server is no longer >>> primary. >>> >>> Thus, your shell session may experience a disconnect on a reconfig >>> command. >>> >> >>> >> >>> >> On Wednesday, April 25, 2012 5:40:45 PM UTC-4, idris wrote: >>> >>> >>> >>> Running 7 nodes total: >>> >>> 1 primary, 3 secondaries, 1 arbiter, 1 backup (hidden, priority:0), >>> 1 >>> >>> off-site secondary (hidden, priority:0, votes:0) >>> >>> >>> >>> Here are the logs of the primary and a secondary when I ran the >>> reconfig >>> >>> command. In this reconfig, I set had votes:0 on the backup node. >>> >>> Primary Log: http://pastebin.com/3a0fPGzA >>> >>> Secondary Log: http://pastebin.com/SGxeJqVK >>> >>> >>> >>> I stripped out all of the noise (connection accepted, end >>> connection) >>> >>> from those logs as well. >>> >>> >>> >>> Primary definitely relinquishes it's primary state... >>> >>>> >>> >>>> Wed Apr 25 17:20:42 [conn417285] replSet replSetReconfig config >>> object >>> >>>> parses ok, 7 members specified >>> >>>> Wed Apr 25 17:20:42 [conn417285] replSet replSetReconfig [2] >>> >>>> Wed Apr 25 17:20:42 [conn417285] replSet info saving a newer config >>> >>>> version to local.system.replset >>> >>>> Wed Apr 25 17:20:42 [conn417285] replSet saveConfigLocally done >>> >>>> Wed Apr 25 17:20:42 [conn417285] replSet relinquishing primary >>> state >>> >>>> Wed Apr 25 17:20:42 [conn417285] replSet SECONDARY >>> >>>> Wed Apr 25 17:20:42 [conn417285] replSet closing client sockets >>> after >>> >>>> reqlinquishing primary >>> >>>> Wed Apr 25 17:20:42 [conn417285] replSet PRIMARY >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> On Wednesday, April 25, 2012 3:03:48 PM UTC-4, Kyle Banker wrote: >>> >>>> >>> >>>> How many total nodes were you running when you removed the >>> slavedelay >>> >>>> node? >>> >>>> >>> >>>> On Wednesday, April 25, 2012 2:00:11 PM UTC-4, idris wrote: >>> >>>>> >>> >>>>> Running version 2.0.4. I'll try to dig up some logs. Is there >>> any >>> >>>>> documentation on which configuration changes will and will not >>> force an >>> >>>>> election or a stepDown? >>> >>>>> >>> >>>>> On Wednesday, April 25, 2012 12:15:36 PM UTC-4, Kyle Banker wrote: >>> >>>>>> >>> >>>>>> Which version of MongoDB are you running? These was an issue in >>> v1.8 >>> >>>>>> but should not be an issue post 2.0. >>> >>>>>> >>> >>>>>> On Wednesday, April 25, 2012 11:17:05 AM UTC-4, idris wrote: >>> >>>>>>> >>> >>>>>>> For example, I just had a slaveDelay member and I removed the >>> >>>>>>> slaveDelay, then saved the config using rs.reconfig(), and the >>> primary went >>> >>>>>>> down briefly, and apparently dropped all connections to clients. >>> Why does >>> >>>>>>> it need to do this? Shouldn't there be an option for "don't go >>> down upon >>> >>>>>>> reconfig" or something? >>> >>>>>>> >>> >>>>>>> -Idris >>> > >>> > -- >>> > You received this message because you are subscribed to the Google >>> Groups >>> > "mongodb-user" group. >>> > To view this discussion on the web visit >>> > https://groups.google.com/d/msg/mongodb-user/-/8pRjE2tPn7MJ. >>> > >>> > To post to this group, send email to mongodb-user@googlegroups.com. >>> > To unsubscribe from this group, send email to >>> > mongodb-user+unsubscribe@googlegroups.com. >>> > For more options, visit this group at >>> > http://groups.google.com/group/mongodb-user?hl=en. >>> >> ------=_Part_1658_11587786.1335813489229 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable Unfortunately, I don't have those from this past time.  I can tell you= that before the reconfig, everything was normal (_id=3D1 was primary, no l= ag on any secondaries).  After the reconfig, the host that had been pr= imary was now secondary, and there was no primary.  This lasted a coup= le of minutes, until the primary was finally elected, resulting in a normal= status again (_id=3D1 primary, no lag on secondaries).

= If I remember correctly, while there was no primary, rs.status() reported t= hat the other nodes were unreachable.  You can see that in the primary's logs I posted before: =
[rsHealthPoll] replSet info 10.0.51.3:27017 is down (or slow to respond): = socket exception
[rsHealthPoll] replSet info 10.0.51.1:27017 is down= (or slow to respond): socket exception
[rsHealthPoll] replSet info= 10.0.51.20:27017 is down (or slow to respond): socket exception
[r= sHealthPoll] replSet info 10.0.51.21:27017 is down (or slow to respond): so= cket exception
[rsHealthPoll] replSet info 10.0.51.4:27017 is down = (or slow to respond): socket exception
[rsHealthPoll] replSet info = mongo901:27017 is down (or slow to respond): socket exception



On Monday, April 30, 2012 10:00:0= 3 AM UTC-5, Kyle Banker wrote:
= Can you please post rs.status() before and after as well?

On Sunday,= April 29, 2012 10:08:53 PM UTC-4, idris wrote:
rs.conf() before:
{
=09"_id" := "myrepl",
=09= "version" : 19,
=09"members" : [
=09=09{
=09=09=09"_id" : 0,
=09=09=09"host" : "10.0.51.1:27017",
=09=09=09"priority" := 10,
=09=09=09= "tags" : {
=09=09=09=09"dc" : "va"
=09=09=09}
=09=09},
=09=09{
=09=09=09"_id" : 1,
=09=09=09"host" : "10.0.51.2:27017",
=09=09=09"prio= rity" : 20,
= =09=09=09"tags" : {
=09=09=09=09"dc" : "va"
= =09=09=09}
=09=09},
=09=09{
=09=09=09"_id" : 2,
=09=09=09"host" = : "10.0.51.3:27017= ",
=09=09=09"tags" : {
=09=09=09=09"dc" : "va"
=09=09=09}
=09=09},
=09=09{
=09=09=09"_id" : 3,
=09=09=09"host" : "10.0.51.20:27017",
=09=09=09"pr= iority" : 0,
= =09=09=09"hidden" : true,
=09=09=09"tags" : {
=09=09=09=09"dc" : "va"
=09=09=09}
=09=09},
=09=09{
=09=09=09"= _id" : 4,
=09= =09=09"host" : "10.0.51.21:27017",
=09=09=09"arbiterOnly" : true
=09=09},
=09=09{
=09=09=09"_id" : 5,
=09=09=09"host" : = "10.0.51.4:27017",=
=09=09=09"tags" : {
= =09=09=09=09"dc" : "va"
=09=09=09}
=09=09},
=09=09{
=09=09=09"_id" : 6,
=09=09=09"host" : "mongo901:270= 17",
=09=09=09= "votes" : 0,
=09=09=09"priority" : 0,
=09=09=09"hidden" : true,
=09=09=09"tags" : {
=09=09=09=09= "dc" : "ca"
= =09=09=09}
=09=09}
=09]
}

=

rs.conf() after:
{
=09"_id" : "myrepl",
=09"version" : 20,
=09"members" : [
=09=09{
=09=09=09"_id" : 0,
=09=09=09"host" = : "10.0.51.1:27017= ",
=09=09=09"priority" : 10,
=09=09=09"tags" : {
=09=09=09=09"dc" : "va"
=09=09=09}
=09=09},
=09=09{
=09=09=09"_id" : 1,
=09=09=09"host" : "10.0.51.2:2= 7017",
=09= =09=09"priority" : 20,
=09=09=09"tags" : {
=09=09=09=09"dc" : "va"
=09=09=09}
=09=09},
=09=09{
=09=09=09"_id" = : 2,
=09=09=09= "host" : "10.0.= 51.3:27017",
=09=09=09"tags" : {
=09=09=09=09"dc" : "va"
=09=09=09}
=09=09},
=09=09{
=09=09=09"_id" : 3,
=09=09=09"host"= : "10.0.51.20:27017<= /a>",
=09=09= =09"votes" : 0,
=09=09=09"priority" : 0,
=09=09=09"hidden" : true,
=09=09=09"tags" : {
=09=09=09=09"dc" : "va"
=09=09=09}
=09=09},
=09=09{
=09=09=09"_id" : 4,
=09=09=09"host" : "10.0.51.21:27017",
=09=09=09"arbiterOnly" : true<= /blockquote>
=09=09}= ,
=09=09{
=09=09=09"_id" : 5,
=09=09=09"host" : "10.0.51.4:27017",
=09=09=09"tags" : {
=09=09=09=09"dc" : "va"
=09=09=09}
<= blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-l= eft-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;pa= dding-left:1ex">=09=09},
=09=09{
=09=09=09"_id" : = 6,
=09=09=09"host" : "mongo901:27017",
=09=09=09"votes" : 0,
=09=09=09"priority" : 0,
=09=09=09"hidden"= : true,
=09= =09=09"tags" : {
=09=09=09=09"dc" : "ca"
=09=09=09}
= =09=09}
=09]
= }


I don't see a= nything useful in the logs regarding up/down state before the reconfig, but= I know that all nodes were up and reachable from all other nodes.  Th= is was not a one-time event, the same thing happens every time I do a recon= fig.


On Saturday, April 28, 2012 11:14:31 PM UTC-5,= Scott Hernandez wrote:
Can you post= the config -- rs.conf() -- before and after as well as
the status -- rs.status()?

Also, please post the logs from before the save showing that state of
the world in terms of who was up/down via the replica status messages.

On Sat, Apr 28, 2012 at 7:33 PM, idris <> wrote:
> *bump*  Is this worthy of a bug report in Jira?
>
> On Wednesday, April 25, 2012 6:35:38 PM UTC-4, idris wrote:
>>
>> By the way, the docs
>> (http://www.mongodb.org/display/= DOCS/Reconfiguring+when+Members+are+Up) also
>> say that the primary steps down and closes sockets from client= s upon a
>> reconfig, but only "in certain circumstances". It would be hel= pful to know
>> which circumstances will cause a stepdown.
>>
>>> In certain circumstances, the primary steps down (perhaps = transiently) on
>>> a reconfiguration. On a step-down, the primary closes sock= ets from clients
>>> to assure the clients know quickly that the server is no l= onger primary.
>>> Thus, your shell session may experience a disconnect on a = reconfig command.
>>
>>
>> On Wednesday, April 25, 2012 5:40:45 PM UTC-4, idris wrote:
>>>
>>> Running 7 nodes total:
>>> 1 primary, 3 secondaries, 1 arbiter, 1 backup (hidden, pri= ority:0), 1
>>> off-site secondary (hidden, priority:0, votes:0)
>>>
>>> Here are the logs of the primary and a secondary when I ra= n the reconfig
>>> command. In this reconfig, I set had votes:0 on the backup= node.
>>> Primary Log: http://pastebin.com/3a0fPGzA
>>> Secondary Log: http://pastebin.com/SGxeJqVK
>>>
>>> I stripped out all of the noise (connection accepted, end = connection)
>>> from those logs as well.
>>>
>>> Primary definitely relinquishes it's primary state...
>>>>
>>>> Wed Apr 25 17:20:42 [conn417285] replSet replSetReconf= ig config object
>>>> parses ok, 7 members specified
>>>> Wed Apr 25 17:20:42 [conn417285] replSet replSetReconf= ig [2]
>>>> Wed Apr 25 17:20:42 [conn417285] replSet info saving a= newer config
>>>> version to local.system.replset
>>>> Wed Apr 25 17:20:42 [conn417285] replSet saveConfigLoc= ally done
>>>> Wed Apr 25 17:20:42 [conn417285] replSet relinquishing= primary state
>>>> Wed Apr 25 17:20:42 [conn417285] replSet SECONDARY
>>>> Wed Apr 25 17:20:42 [conn417285] replSet closing clien= t sockets after
>>>> reqlinquishing primary
>>>> Wed Apr 25 17:20:42 [conn417285] replSet PRIMARY
>>>
>>>
>>>
>>>
>>> On Wednesday, April 25, 2012 3:03:48 PM UTC-4, Kyle Banker= wrote:
>>>>
>>>> How many total nodes were you running when you removed= the slavedelay
>>>> node?
>>>>
>>>> On Wednesday, April 25, 2012 2:00:11 PM UTC-4, idris w= rote:
>>>>>
>>>>> Running version 2.0.4.  I'll try to dig up so= me logs.  Is there any
>>>>> documentation on which configuration changes will = and will not force an
>>>>> election or a stepDown?
>>>>>
>>>>> On Wednesday, April 25, 2012 12:15:36 PM UTC-4, Ky= le Banker wrote:
>>>>>>
>>>>>> Which version of MongoDB are you running? Thes= e was an issue in v1.8
>>>>>> but should not be an issue post 2.0.
>>>>>>
>>>>>> On Wednesday, April 25, 2012 11:17:05 AM UTC-4= , idris wrote:
>>>>>>>
>>>>>>> For example, I just had a slaveDelay membe= r and I removed the
>>>>>>> slaveDelay, then saved the config using rs= .reconfig(), and the primary went
>>>>>>> down briefly, and apparently dropped all c= onnections to clients.  Why does
>>>>>>> it need to do this?  Shouldn't there = be an option for "don't go down upon
>>>>>>> reconfig" or something?
>>>>>>>
>>>>>>> -Idris
>
> --
> You received this message because you are subscribed to the Google= Groups
> "mongodb-user" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/mongodb-user/-= /8pRjE2tPn7MJ.
>
> To post to this group, send email to mongodb-user@googlegroups.com.
> To unsubscribe from this group, send email to
> mongodb-user+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/mongodb-user?hl=3Den.

On Saturday, April 28, 2012 11:14:31 PM UTC-5, S= cott Hernandez wrote:
Can you post t= he config -- rs.conf() -- before and after as well as
the status -- rs.status()?

Also, please post the logs from before the save showing that state of
the world in terms of who was up/down via the replica status messages.

On Sat, Apr 28, 2012 at 7:33 PM, idris <> wrote:
> *bump*  Is this worthy of a bug report in Jira?
>
> On Wednesday, April 25, 2012 6:35:38 PM UTC-4, idris wrote:
>>
>> By the way, the docs
>> (http://www.mongodb.org/display/= DOCS/Reconfiguring+when+Members+are+Up) also
>> say that the primary steps down and closes sockets from client= s upon a
>> reconfig, but only "in certain circumstances". It would be hel= pful to know
>> which circumstances will cause a stepdown.
>>
>>> In certain circumstances, the primary steps down (perhaps = transiently) on
>>> a reconfiguration. On a step-down, the primary closes sock= ets from clients
>>> to assure the clients know quickly that the server is no l= onger primary.
>>> Thus, your shell session may experience a disconnect on a = reconfig command.
>>
>>
>> On Wednesday, April 25, 2012 5:40:45 PM UTC-4, idris wrote:
>>>
>>> Running 7 nodes total:
>>> 1 primary, 3 secondaries, 1 arbiter, 1 backup (hidden, pri= ority:0), 1
>>> off-site secondary (hidden, priority:0, votes:0)
>>>
>>> Here are the logs of the primary and a secondary when I ra= n the reconfig
>>> command. In this reconfig, I set had votes:0 on the backup= node.
>>> Primary Log: http://pastebin.com/3a0fPGzA
>>> Secondary Log: http://pastebin.com/SGxeJqVK
>>>
>>> I stripped out all of the noise (connection accepted, end = connection)
>>> from those logs as well.
>>>
>>> Primary definitely relinquishes it's primary state...
>>>>
>>>> Wed Apr 25 17:20:42 [conn417285] replSet replSetReconf= ig config object
>>>> parses ok, 7 members specified
>>>> Wed Apr 25 17:20:42 [conn417285] replSet replSetReconf= ig [2]
>>>> Wed Apr 25 17:20:42 [conn417285] replSet info saving a= newer config
>>>> version to local.system.replset
>>>> Wed Apr 25 17:20:42 [conn417285] replSet saveConfigLoc= ally done
>>>> Wed Apr 25 17:20:42 [conn417285] replSet relinquishing= primary state
>>>> Wed Apr 25 17:20:42 [conn417285] replSet SECONDARY
>>>> Wed Apr 25 17:20:42 [conn417285] replSet closing clien= t sockets after
>>>> reqlinquishing primary
>>>> Wed Apr 25 17:20:42 [conn417285] replSet PRIMARY
>>>
>>>
>>>
>>>
>>> On Wednesday, April 25, 2012 3:03:48 PM UTC-4, Kyle Banker= wrote:
>>>>
>>>> How many total nodes were you running when you removed= the slavedelay
>>>> node?
>>>>
>>>> On Wednesday, April 25, 2012 2:00:11 PM UTC-4, idris w= rote:
>>>>>
>>>>> Running version 2.0.4.  I'll try to dig up so= me logs.  Is there any
>>>>> documentation on which configuration changes will = and will not force an
>>>>> election or a stepDown?
>>>>>
>>>>> On Wednesday, April 25, 2012 12:15:36 PM UTC-4, Ky= le Banker wrote:
>>>>>>
>>>>>> Which version of MongoDB are you running? Thes= e was an issue in v1.8
>>>>>> but should not be an issue post 2.0.
>>>>>>
>>>>>> On Wednesday, April 25, 2012 11:17:05 AM UTC-4= , idris wrote:
>>>>>>>
>>>>>>> For example, I just had a slaveDelay membe= r and I removed the
>>>>>>> slaveDelay, then saved the config using rs= .reconfig(), and the primary went
>>>>>>> down briefly, and apparently dropped all c= onnections to clients.  Why does
>>>>>>> it need to do this?  Shouldn't there = be an option for "don't go down upon
>>>>>>> reconfig" or something?
>>>>>>>
>>>>>>> -Idris
>
> --
> You received this message because you are subscribed to the Google= Groups
> "mongodb-user" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/mongodb-user/-= /8pRjE2tPn7MJ.
>
> To post to this group, send email to mongodb-user@googlegroups.com.
> To unsubscribe from this group, send email to
> mongodb-user+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/mongodb-user?hl=3Den.

On Saturday, April 28, 2012 11:14:31 PM UTC-5, Scott H= ernandez wrote:
Can you post the con= fig -- rs.conf() -- before and after as well as
the status -- rs.status()?

Also, please post the logs from before the save showing that state of
the world in terms of who was up/down via the replica status messages.

On Sat, Apr 28, 2012 at 7:33 PM, idris <> wrote:
> *bump*  Is this worthy of a bug report in Jira?
>
> On Wednesday, April 25, 2012 6:35:38 PM UTC-4, idris wrote:
>>
>> By the way, the docs
>> (http://www.mongodb.org/display/= DOCS/Reconfiguring+when+Members+are+Up) also
>> say that the primary steps down and closes sockets from client= s upon a
>> reconfig, but only "in certain circumstances". It would be hel= pful to know
>> which circumstances will cause a stepdown.
>>
>>> In certain circumstances, the primary steps down (perhaps = transiently) on
>>> a reconfiguration. On a step-down, the primary closes sock= ets from clients
>>> to assure the clients know quickly that the server is no l= onger primary.
>>> Thus, your shell session may experience a disconnect on a = reconfig command.
>>
>>
>> On Wednesday, April 25, 2012 5:40:45 PM UTC-4, idris wrote:
>>>
>>> Running 7 nodes total:
>>> 1 primary, 3 secondaries, 1 arbiter, 1 backup (hidden, pri= ority:0), 1
>>> off-site secondary (hidden, priority:0, votes:0)
>>>
>>> Here are the logs of the primary and a secondary when I ra= n the reconfig
>>> command. In this reconfig, I set had votes:0 on the backup= node.
>>> Primary Log: http://pastebin.com/3a0fPGzA
>>> Secondary Log: http://pastebin.com/SGxeJqVK
>>>
>>> I stripped out all of the noise (connection accepted, end = connection)
>>> from those logs as well.
>>>
>>> Primary definitely relinquishes it's primary state...
>>>>
>>>> Wed Apr 25 17:20:42 [conn417285] replSet replSetReconf= ig config object
>>>> parses ok, 7 members specified
>>>> Wed Apr 25 17:20:42 [conn417285] replSet replSetReconf= ig [2]
>>>> Wed Apr 25 17:20:42 [conn417285] replSet info saving a= newer config
>>>> version to local.system.replset
>>>> Wed Apr 25 17:20:42 [conn417285] replSet saveConfigLoc= ally done
>>>> Wed Apr 25 17:20:42 [conn417285] replSet relinquishing= primary state
>>>> Wed Apr 25 17:20:42 [conn417285] replSet SECONDARY
>>>> Wed Apr 25 17:20:42 [conn417285] replSet closing clien= t sockets after
>>>> reqlinquishing primary
>>>> Wed Apr 25 17:20:42 [conn417285] replSet PRIMARY
>>>
>>>
>>>
>>>
>>> On Wednesday, April 25, 2012 3:03:48 PM UTC-4, Kyle Banker= wrote:
>>>>
>>>> How many total nodes were you running when you removed= the slavedelay
>>>> node?
>>>>
>>>> On Wednesday, April 25, 2012 2:00:11 PM UTC-4, idris w= rote:
>>>>>
>>>>> Running version 2.0.4.  I'll try to dig up so= me logs.  Is there any
>>>>> documentation on which configuration changes will = and will not force an
>>>>> election or a stepDown?
>>>>>
>>>>> On Wednesday, April 25, 2012 12:15:36 PM UTC-4, Ky= le Banker wrote:
>>>>>>
>>>>>> Which version of MongoDB are you running? Thes= e was an issue in v1.8
>>>>>> but should not be an issue post 2.0.
>>>>>>
>>>>>> On Wednesday, April 25, 2012 11:17:05 AM UTC-4= , idris wrote:
>>>>>>>
>>>>>>> For example, I just had a slaveDelay membe= r and I removed the
>>>>>>> slaveDelay, then saved the config using rs= .reconfig(), and the primary went
>>>>>>> down briefly, and apparently dropped all c= onnections to clients.  Why does
>>>>>>> it need to do this?  Shouldn't there = be an option for "don't go down upon
>>>>>>> reconfig" or something?
>>>>>>>
>>>>>>> -Idris
>
> --
> You received this message because you are subscribed to the Google= Groups
> "mongodb-user" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/mongodb-user/-= /8pRjE2tPn7MJ.
>
> To post to this group, send email to mongodb-user@googlegroups.com.
> To unsubscribe from this group, send email to
> mongodb-user+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/mongodb-user?hl=3Den.
------=_Part_1658_11587786.1335813489229-- ------=_Part_1657_1054469.1335813489229--