Work distribution in cluster using routers

[Jem Mawson]

Hi. I'm working through Akka in Action, section 14.3.2 - "Work Distribution using Routers".

I have a simple work distribution example project and it is exhibiting two unwanted behaviours. Can someone please help me understand where I've gone wrong?

1. Despite using a router to distribute work to the workers, all work continues to happen on the same node (even when max workers is reached for that node).
2. Despite stopping the workers, the system is unable to process work once the max number of workers is hit.

I hope someone can help me as I can't see what I've done wrong.

Thanks

Jem

[Martynas Mickevičius]

    1. Despite using a router to distribute work to the workers, all work continues to happen on the same node (even when max workers is reached for that node).

 

This can be caused by the known regression in akka 2.3.1. In that case the second node is unable to join the cluster, so all work happens on the first node. I have tried your example code with akka 2.3.0 and workers are being distributed between two or more nodes.

    2. Despite stopping the workers, the system is unable to process work once the max number of workers is hit.

I hope someone can help me as I can't see what I've done wrong.

I see that you are stopping workers with a PoisonPill message. This stops the worker and its mailbox. In this case router is not going to restart workers. However if you use Kill message to kill worker, then ActorKilledException is thrown in the worker which kills the worker but also notifies the router. Router will act (escalate, restart, ...) to this exception according to its supervision strategy.

Martynas Mickevičius

Typesafe – Reactive Apps on the JVM

[Jem]

Thank you Martynas for your help.

I've dropped down to 2.3.0 and modified the Worker so that it stays alive. (commit). Retesting, I get a different result to you. The app honours the ClusterRouterPoolSetting totalInstances, but it ignores maxInstancesPerNode and allowLocalRoutees.

With Akka 2.3.0 and this code inside a Cluster Singleton:

    context.actorOf(ClusterRouterPool(

      RoundRobinPool(10),

      ClusterRouterPoolSettings(

        totalInstances = 30,

        maxInstancesPerNode = 10,

        allowLocalRoutees = false,

        useRole = None)

    ).props(Props[Worker]), name = "worker-router")

I see that 30 workers are created on that node and nothing on any other.

Is creating the router inside a singleton somehow locking the routees down to that node? Why do you and I see different results for the same source?

Cheers

Jem

--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to a topic in the Google Groups "Akka User List" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/akka-user/KMBzbh8lXZg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to akka-user+...@googlegroups.com.
To post to this group, send email to akka...@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[Martynas Mickevičius]

So I forked your example project and added routee number tracking between nodes. You can find it here: https://github.com/2m/scratch-akka-cluster-singleton/tree/distributed

When you start the first seed node with the sbt "run-main sample.cluster.simple.SimpleClusterApp 2551" you will notice singleton printing the message [info] Worker router has no routees, which means that the allowLocalRoutees setting is honored.

When you start another seed node you will start seeing [info] Node with port Some(2552) has 10 routees which says that maxInstancesPerNode setting is also being honored.

If you start three more simple nodes you will start seeing:

[info] Node with port Some(45419) has 10 routees.

[info] Node with port Some(2552) has 10 routees.

[info] Node with port Some(52133) has 10 routees.

which tells us that totalInstances setting is also being honored.

Can you run this example and see if you see what I see? :)

[Jem]

Your example worked just fine. I foolishly logged the result of the work as it arrived back at the controller and inferred incorrectly that where the log message appeared is where the work occurred. A simple and obvious-in-hindsight error.

Thanks for your time!

--

[Martynas Mickevičius]

Great! Glad it cleared things up!

You received this message because you are subscribed to the Google Groups "Akka User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+...@googlegroups.com.

To post to this group, send email to akka...@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

--