Re: Secondary node showing "(not reachable/healthy)" in rs.status() when doing long compact()

1,669 views
Skip to first unread message

Aleksei T

unread,
Jul 25, 2013, 12:42:42 AM7/25/13
to mongod...@googlegroups.com
Yes we are experiencing something similar after a compact operation on a secondary node - now it cannot catch up (optimeDate is stuck at the same timestamp for hours) with "not reachable/healthy" status.  Any help figuring this out would be appreciated.

On Monday, May 27, 2013 7:50:53 PM UTC-7, Tsz Ming Wong wrote:
Hello,

I remember sometimes ago, when doing a long compact() process on secondary,, it will show as "RECOVERING" in primary's rs.status(), but recently we are seeing error

{
    "_id": 8,
    "name": "my-secondary-node:27017",
    "health": 0,
    "state": 8,
    "stateStr": "(not reachable/healthy)",
    "uptime": 0,
    "optime": Timestamp(1369707789000,
    5),
    "optimeDate": ISODate("2013-05-28T02:23:09Z"),
    "lastHeartbeat": ISODate("2013-05-28T02:23:11Z"),
    "pingMs": 0,
    "errmsg": "DBClientBase::findN: transport error: my-secondary-node:27017 ns: admin.$cmd query: { replSetHeartbeat: \"Mongo\", v: 57, pv: 1, checkEmpty: false, from: \"my-primary-node:27017\", $auth: {} }"
},


However, when I connect to the secondary on primary via shell, it is showing "RECOVERING"

 mongo --host my-secondary-node
MongoDB shell version: 2.2.4
connecting to: my-secondary-node:27017/test
Mongo:RECOVERING> 

Anyone encountered this before?


Asya Kamsky

unread,
Jul 27, 2013, 8:44:13 PM7/27/13
to mongod...@googlegroups.com
Sounds like in your case the oplog wasn't large enough to contain all the change operations that happened when the secondary was compacting.  That's a guess, since we don't actually have information needed to diagnose exactly what happened and why.
--
--
You received this message because you are subscribed to the Google
Groups "mongodb-user" group.
To post to this group, send email to mongod...@googlegroups.com
To unsubscribe from this group, send email to
mongodb-user...@googlegroups.com
See also the IRC channel -- freenode.net#mongodb
 
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply all
Reply to author
Forward
0 new messages