Channel not found from the primary node

[Zhao Hongwei]

Hello, 

We use driver

"org.reactivemongo" %% "play2-reactivemongo" % "0.12.6-play26"

config mongo uri as:

mongodb.uri = "mongodb://abc:123@prod-mongo-master:27017,prod-mongo-slave:27017/attr?rm.nbChannelsPerNode=100"

and try to use actor to save docs into mongodb, like that:

class MongoSinkActor(mongoService: MongoService) extends Actor with ActorLogging {


  override def receive: Receive = {
    case RequestMongoMessage(js) =>
      log.debug(s"get request message $js")
      mongoService.saveRequest(js).map {
        case Left(mes) => log.error(s"handle request got error [$js]  [$mes]")
        case Right(_) =>
      }
  }
}

but there is some error occurs after application starts a while:

20:54:40.518 [error] r.api.Failover2 - [Supervisor-1/Connection-2] Got an error, no more attempts to do. Completing with a failure...

reactivemongo.core.actors.Exceptions$ChannelNotFound: MongoError['Channel not found from the primary node: 'prod-mongo-master:27017' { authenticated:0, connected:100, channels:100 } (Supervisor-1/Connection-2)']

Caused by: reactivemongo.core.actors.Exceptions$InternalState: null

        at reactivemongo.ConnectAll$IsMaster(239, {{NodeSet None Node[prod-mongo-master:27017: Primary (100/100 available connections), latency=9223372036854775807], auth=Set() | Node[prod-mongo-slave:27017: Secondary (100/100 available connections), latency=0], auth=Set(Authenticated(attr,clink)) }})(<time:1504011274429>)

        at reactivemongo.IsMaster(239, {{NodeSet None Node[prod-mongo-master:27017: Primary (100/100 available connections), latency=9223372036854775807], auth=Set() | Node[prod-mongo-slave:27017: Secondary (100/100 available connections), latency=1], auth=Set(Authenticated(attr,clink)) }}(<time:1504011274429>)

        at reactivemongo.ConnectAll({{NodeSet None Node[prod-mongo-master:27017: Primary (100/100 available connections), latency=9223372036854775807], auth=Set() | Node[prod-mongo-slave:27017: Secondary (100/100 available connections), latency=1], auth=Set(Authenticated(attr,clink)) }})(<time:1504011274428>)

        at reactivemongo.RefreshAll({{NodeSet None Node[prod-mongo-master:27017: Primary (100/100 available connections), latency=9223372036854775807], auth=Set() | Node[prod-mongo-slave:27017: Secondary (100/100 available connections), latency=1], auth=Set(Authenticated(attr,clink)) }})(<time:1504011274428>)

        at reactivemongo.ConnectAll$IsMaster(200, {{NodeSet None Node[prod-mongo-master:27017: Primary (100/100 available connections), latency=9223372036854775807], auth=Set() | Node[prod-mongo-slave:27017: Secondary (100/100 available connections), latency=1], auth=Set(Authenticated(attr,clink)) }})(<time:1504011268507>)

        at reactivemongo.IsMaster(200, {{NodeSet None Node[prod-mongo-master:27017: Unknown (100/100 available connections), latency=9223372036854775807], auth=Set() | Node[prod-mongo-slave:27017: Secondary (100/100 available connections), latency=1], auth=Set(Authenticated(attr,clink)) }}(<time:1504011268507>)

        at reactivemongo.ConnectAll$IsMaster(237, {{NodeSet None Node[prod-mongo-master:27017: Unknown (100/100 available connections), latency=9223372036854775807], auth=Set() | Node[prod-mongo-slave:27017: Secondary (100/100 available connections), latency=1], auth=Set(Authenticated(attr,clink)) }})(<time:1504011264429>)

        at reactivemongo.IsMaster(237, {{NodeSet None Node[prod-mongo-master:27017: Unknown (100/100 available connections), latency=9223372036854775807], auth=Set() | Node[prod-mongo-slave:27017: Secondary (100/100 available connections), latency=1], auth=Set(Authenticated(attr,clink)) }}(<time:1504011264429>)

        at reactivemongo.RefreshAll({{NodeSet None Node[prod-mongo-master:27017: Unknown (100/100 available connections), latency=9223372036854775807], auth=Set() | Node[prod-mongo-slave:27017: Secondary (100/100 available connections), latency=1], auth=Set(Authenticated(attr,clink)) }})(<time:1504011264428>)

        at reactivemongo.ConnectAll({{NodeSet None Node[prod-mongo-master:27017: Unknown (100/100 available connections), latency=9223372036854775807], auth=Set() | Node[prod-mongo-slave:27017: Secondary (100/100 available connections), latency=1], auth=Set(Authenticated(attr,clink)) }})(<time:1504011264428>)

We are due production release and this issue is very critical.

[Cédric Chantepie]

As said, there could be various/external causes. First enable logging and check it (see Troubleshooting). Then try to reproduce the possible issue as an isolated test case.