Problem with configuration for router with remote routee; routee creates actor pool with BalancingDispatcher

[Boris Capitanu]

Hello,

Here is the situation:  I have two actor systems "systemA" and "systemB" started in different processes.

Actor system A starts before B, and doesn't do anything (B will deploy to A).

Actor system B starts and creates a router actor with a remotely-deployed routee, as follows:

val router = systemB.actorOf(FromConfig.props(Props[Receiver]), "receiver")

with configuration:

  actor.deployment {

    /receiver {

      router = round-robin-pool

      nr-of-instances = 1

      target.nodes = [ "akka.tcp://systemA@localhost:2553" ]

    }

(I'm only testing with 1 instance now, but there will typically be more in production)

The Receiver actor, in part:

class Receiver extends Actor {

  val workerPool = context.actorOf(FromConfig.props(Props[Worker]), name = "workerPool")

  ...

}

what I'd like to use in the configuration for "workerPool" is this:

  actor.deployment {

    /receiver  ... as above ...

    "/receiver/*/workerPool" {

      router = random-pool

      nr-of-instances = 4

      pool-dispatcher {

        type = BalancingDispatcher

      }

    }

  }

But this isn't working.  Again, the idea is that systemB will deploy the "Receiver" actor to systemA (as part of the round-robin-pool), and in turn the receiver actor creates a worker pool of "Worker" actors that use a balancing dispatcher.

The error I see coming out of the process running "systemB" is:

16:29:44.477 [ERROR] - configuration problem while creating [akka://systemA/remote/akka.tcp/systemB@localhost:61012/user/receiver/c1/workerPool] with router dispatcher [akka.actor.default-dispatcher] and mailbox [akka.actor.default-mailbox] and routee dispatcher [akka.actor.default-dispatcher] and mailbox [akka.actor.default-mailbox]

akka.actor.ActorInitializationException: exception during creation

at akka.actor.ActorInitializationException$.apply(Actor.scala:164)

at akka.actor.ActorCell.create(ActorCell.scala:596)

Caused by: akka.ConfigurationException: configuration problem while creating [akka://systemA/remote/akka.tcp/systemB@localhost:61012/user/receiver/c1/workerPool] with router dispatcher [akka.actor.default-dispatcher] and mailbox [akka.actor.default-mailbox] and routee dispatcher [akka.actor.default-dispatcher] and mailbox [akka.actor.default-mailbox]

at akka.actor.LocalActorRefProvider.actorOf(ActorRefProvider.scala:752)

at akka.remote.RemoteActorRefProvider.actorOf(RemoteActorRefProvider.scala:297)

Caused by: com.typesafe.config.ConfigException$BadPath: path parameter: Invalid path 'akka.actor.deployment./akka.tcp/systemB@localhost:61012/user/receiver/c1/workerPool.pool-dispatcher': Token not allowed in path expression: '@' (Reserved character '@' is not allowed outside quotes) (you can double-quote this token if you really want it here)

at com.typesafe.config.impl.Parser.parsePathExpression(Parser.java:1057)

at com.typesafe.config.impl.Parser.parsePath(Parser.java:1097)

I also tried to configure the worker pool in code, as follows (based on the example in the docs):

Note: for this I have removed the "/receiver/*/workerPool" configuration entry since I was defining it in code.

  val workerPool = context.actorOf(

    RandomPool(

      nrOfInstances = 4,

      routerDispatcher = "router-dispatcher"

    ).props(Props[Worker].withDispatcher("my-dispatcher")),

    name = "workerPool"

  )

with "my-dispatcher" defined as:

my-dispatcher {

  type = BalancingDispatcher

}

but that also didn't work, with error:

17:10:43.257 [ERROR] - Dispatcher [my-dispatcher] not configured for router of akka://systemA/remote/akka.tcp/systemB@localhost:61412/user/receiver/c1/workerPool

akka.actor.ActorInitializationException: exception during creation

at akka.actor.ActorInitializationException$.apply(Actor.scala:164)

at akka.actor.ActorCell.create(ActorCell.scala:596)

Caused by: akka.ConfigurationException: Dispatcher [my-dispatcher] not configured for router of akka://systemA/remote/akka.tcp/systemB@localhost:61412/user/receiver/c1/workerPool

at akka.actor.LocalActorRefProvider.actorOf(ActorRefProvider.scala:735)

at akka.remote.RemoteActorRefProvider.actorOf(RemoteActorRefProvider.scala:297)

Have I uncovered a bug with the configuration, or is this expected behavior?

Thank you,

-Boris

[Boris Capitanu]

In the mean time I found that I was looking at a slightly older version of the configuration (for 2.3-M1), and found the "BalancingPool" in the 2.3.0 docs.

However, using the following config:

  actor.deployment {

    /receiver {

      router = round-robin-pool

      nr-of-instances = 1

      target.nodes = [ "akka.tcp://systemA@localhost:2553" ]

    }

    "/receiver/*/workerPool" {

      router = balancing-pool

      nr-of-instances = 4

    }

  }

and the following actor creation code in the "Receiver":

  val workerPool = context.actorOf(FromConfig.props(Props[Worker]), "workerPool")

I still get the first exception from my previous post:

 

path parameter: Invalid path 'BalancingPool-/akka.tcp/systemB@localhost:61671/user/receiver/c1/workerPool': Token not allowed in path expression: '@' (Reserved character '@' is not allowed outside quotes) (you can double-quote this token if you really want it here)

Same error if I try to define it in code, rather than in config (having removed the configuration entry for "/receiver/*/workerPool" shown above):

  val workerPool = context.actorOf(

    BalancingPool(4).props(Props[Worker]),

    name = "workerPool"

  ) 

The error:

path parameter: Invalid path 'BalancingPool-/akka.tcp/systemB@localhost:61705/user/receiver/c1/workerPool': Token not allowed in path expression: '@' (Reserved character '@' is not allowed outside quotes) (you can double-quote this token if you really want it here)

-Boris 

[Akka Team]

Hi Boris,

That path looks weird:
BalancingPool-/akka.tcp/systemB@localhost:61705/user/receiver/c1/workerPool

Maybe an URI encoding missing somewhere. Do you have a minimized code that reproduces the problem?

-Endre

--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+...@googlegroups.com.
To post to this group, send email to akka...@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

--

Akka Team

Typesafe - The software stack for applications that scale

Blog: letitcrash.com
Twitter: @akkateam

[Boris Capitanu]

Hi Endre,

 

Do you have a minimized code that reproduces the problem?

https://github.com/borice/balancing-dispatcher-test

What you want to look at is common.conf and Receiver.scala

Thanks for looking into this. It's mainly an experiment... I was evaluating whether the BalancingPool could be used as a worker pool... and it seems great in terms of the way it works by sharing the same mailbox, however one problem I see is if one of the workers gets stuck on some work that produces an infinite loop, even if I have a Timeout be triggered in the "Receiver" to indicate the particular piece of work has been taking longer than allowed, I don't think I have a way to "kill and restart" that balancing pool actor (without killing everyone else). 

Do you see a way to do that, without adding another level of indirection?  

Thanks,

Boris

[Boris Capitanu]

Hi Endre,

Have you had a chance to look at this?  Do you want me to create a ticket?

Thanks,
Boris

[Martynas Mickevičius]

Hi Boris,

I dug a little bit deeper and it seems to be a bug in a Balanced Router code. I have created an issue for this. You can track it at: https://www.assembla.com/spaces/akka/tickets/3981-balancingpool-uses-disallowed-characters-for-the-dispatcher-config-path#/activity/ticket:

Martynas