ActorInitializationException: You cannot create an instance of [StandardDBSystem] explicitly ...

[Matej Krchniak]

Hi,

we are currently migrating to reactivemongo 0.12.0 and encountered this error when creating database:

[info]   akka.actor.ActorInitializationException: You cannot create an instance of [reactivemongo.core.actors.StandardDBSystem] explicitly using the constructor (new). You have to use one of the 'actorOf' factory methods to create a new actor. See the documentation.
[info]   at reactivemongo.api.MongoConnection.database(api.scala)
[info]   at akka.actor.ActorInitializationException$.apply(Actor.scala:167)
[info]   at akka.actor.Actor$class.$init$(Actor.scala:423)
[info]   at reactivemongo.core.actors.StandardDBSystem.<init>(actors.scala:1240)
[info]   at reactivemongo.api.MongoDriver.dbsystem$lzycompute$1(api.scala:960)
[info]   at reactivemongo.api.MongoDriver.reactivemongo$api$MongoDriver$$dbsystem$1(api.scala:955)
[info]   at reactivemongo.api.MongoDriver$$anonfun$connection$2$$anonfun$apply$29.apply(api.scala:976)
[info]   at reactivemongo.api.MongoDriver$$anonfun$connection$2$$anonfun$apply$29.apply(api.scala:976)
[info]   at reactivemongo.api.MongoConnection$$anonfun$probe$1$$anonfun$apply$22.apply(api.scala:503)
[info]   at reactivemongo.api.MongoConnection$$anonfun$probe$1$$anonfun$apply$22.apply(api.scala:496)

We are constructing database access this way:

class MongoDatabase(uri: String) extends ExecutionContextProvider {
  private val parsedUri = MongoConnection.parseURI(uri).getOrElse(sys.error(s"Database not configured!"))
  private val driver: MongoDriver = new MongoDriver()
  private val connection = driver.connection(parsedUri)

  def database: Future[DefaultDB] = connection.database(parsedUri.db.getOrElse(sys.error(s"Database not specified in uri`")))

  override implicit val executionContext: ExecutionContext = connection.actorSystem.dispatcher

  def close(): Unit = driver.close(10.seconds)
}

Problem manifests only in tests where multiple tests are running in parallel (each module has it's own database and driver). Stacktrace points to this code in driver:

Await.result(connection.mapTo[MongoConnection].map { c =>
  // TODO: Review
  c.history = () => dbsystem.internalState
  c
}, Duration.Inf)

Here is what happens when it fails:

1. system.actorOf(Props(dbsystem), nm) is called, actor reference is returned. But as akka documentation states "Actors are automatically started asynchronously when created.", actor may not be created when reference is returned.
2. AddConnection is sent do supervisor
3. MongoConnection wrapper is created and returned
4. c.history is initialized
   
5. MongoConnection.database is called, which in turn calls waitIsAvailable, which will call probe, which will initialize dbsystem outside of actorOf because dbsystem actor is still not created
   

This code actually violates some akka principles, because it is:

• sharing actor internal state
• it probably won't work as expected when actor is restarted, because same instance will be returned

You can find more information here: https://groups.google.com/forum/#!topic/akka-user/L3wV_9hm-hQ

[Cédric Chantepie]

Are you initializing multiple `MongoConnection` ?

[Matej Krchniak]

As you can see in MongoDatabase, only one `MongoConnection` is created. But each sbt module has it's own embedded mongo database with own `MongoDriver` and `MongoConnection` is tests. So from `MongoDriver` perspective only one `MongoConnection` is created.

Dňa streda, 8. marca 2017 9:14:29 UTC+1 Cédric Chantepie napísal(-a):

[Cédric Chantepie]

Only one driver must be alive per app, and generally also a single MongoConnection (pool), otherwise you will starve Akka ressources.

[Matej Krchniak]

I understand. We followed documentation, so we have only one instance of driver and connection per app. Problem occurs when two apps are running test concurrently in the same JVM. Then first app starves cpu resources for second app which fails, because test are quick and try to access database, but akka had no change to create dbsystem actor. So our workaround is to run tests serially.

My only worry is that same can happen in production, when app or user is too quick and tries to access database and akka is slow due to high cpu load and had no change to create dbsystem actor yet.

[Cédric Chantepie]

There is no magic solution but scaling appropriatly the pool size and options.