[0.12.5] MongoDBSystem history Thead-Safety problem

[Roland Bär]

Hi

We use ReactiveMongo 0.12.5 and we had two 100% CPU Usage problems in the last 2 weeks. A thread dump contains 12 threads like this:

reactivemongo-akka.actor.default-dispatcher-5304 tid=210835 [RUNNABLE]

reactivemongo.core.actors.MongoDBSystem$$anonfun$reactivemongo$core$actors$MongoDBSystem$$processing$1.$anonfun$applyOrElse$29(MongoDBSystem$$anonfun$reactivemongo$core$actors$MongoDBSystem$$processing$1) MongoDBSystem.scala:553

reactivemongo.core.actors.MongoDBSystem$$anonfun$reactivemongo$core$actors$MongoDBSystem$$processing$1$$Lambda$1428.apply$mcV$sp()

akka.actor.Scheduler$$anon$4.run() Scheduler.scala:140

akka.dispatch.TaskInvocation.run() AbstractDispatcher.scala:38

akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec() ForkJoinExecutorConfigurator.scala:43

akka.dispatch.forkjoin.ForkJoinTask.doExec() ForkJoinTask.java:260

akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinTask) ForkJoinPool.java:1339

akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool$WorkQueue) ForkJoinPool.java:1979

akka.dispatch.forkjoin.ForkJoinWorkerThread.run() ForkJoinWorkerThread.java:107

MongoDBSystem.scala on line 553 looks like this:

while (history.size > historyMax) { // compensate EvictionQueue safety

    history.poll() // line 553

}

history is a shaded.google.common.collect.EvictingQueue. This Queue is not thread safe so I guess all threads end up in a endless loop. 

More context from a head dump: 

• history had 176 entries
  
• MongoDBSystem Actor mailbox had 12000 Entries
  
• MongoDBSystem Actor not working on messages because the 12 threads mentioned above used all threads
  

Is this a ReactiveMongo bug or some sort of 'wrong API usage' from our side?

[Cédric Chantepie]

You can try 0.12.8-SNAPSHOT

[Roland Bär]

Thanks for your response. I've deployed 0.12.8-SNAPSHOT to 2 applications. I'll keep an eye on both applications over next week and report back on Fr 1. Dec.

On Wednesday, November 22, 2017 at 7:25:22 PM UTC+1, Cédric Chantepie wrote:

You can try 0.12.8-SNAPSHOT

[Roland Bär]

No problems this week so I can confirm the bug is fixed.