Akka cluster and scheduled tasks

[Brian Wawok]

So let's say I have an akka cluster of 5 nodes. 4 are "worker nodes" and 1 is the "master" node, the master implemented via an akka singleton ( http://doc.akka.io/docs/akka/snapshot/contrib/cluster-singleton.html ).

Then I use a worker registration type pattern to have the workers tell the master when they are free and hand out work(ala http://letitcrash.com/post/29044669086/balancing-workload-across-nodes-with-akka-2 ). 

This all works good. If the master node dies, a new master is elected. Cool!

Now what I need to fit on top of this, is scheduled tasks. I want the master to schedule a unit of work every 30 seconds, and give it to one of his workers. How do I do this?

On a single machine you just do a scheduled task.. but if I do that on all nodes of the cluster, I would get the task happening 5 times. Possible ideas:

a) Schedule task on every machine. If your machine does not have the master node running on it, skip it. So it only runs once. (would need to see if I can figure out which node a singleton proxy points to, for the implementation of this)

b) Schedule task on every machine. When the schedule hits, send a message to master to say HEY DO THE TASK NOW... master would need to keep track of when he last did it, and if he already did it in the last 30 seconds skip it (so say he gets 5 messages to do work for task A, he would do the first task and delete the next 4).

c) ?????

Part of the trick is as nodes crash and the master singleton moves around, the tasks need to keep happening. I am ok with a task being skipped once in a while (like during a master switch), but would want to generally prevent them from being run multiple times.

[Martynas Mickevičius]

Hi Brian.

what about scheduling a task on a master (cluster singleton) and then delegating the work to workers using Cluster Aware Round Robin Router?

--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+...@googlegroups.com.
To post to this group, send email to akka...@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

--

Martynas Mickevičius

Typesafe – Reactive Apps on the JVM

[Brian Wawok]

So to do the cluster singleton, I fire one up on each node (but only 1 really runs). So I schedule this task on all nodes, and it will only run on the active one?  If there is a failover, then it will start happening on the new master node?

[jxtps]

+1, I have effectively the same situation - I need "singleton scheduled tasks" more so that "singleton actors" (the billing code should only run once per day).

What are the best practices for this scenario?

I guess I could schedule the task in the start callback of the Singleton, and stop it in the stop callback. Then when the cluster leader changes the schedule would be first stopped, then restarted when the Singleton starts up again on the new leader. Maybe that's a workable solution?

If the leader is terminated, do the ActorSystem.scheduler.schedule() tasks that were initiated on it disappear? They aren't replicated across the cluster or something crazy like that, right? (Since they're called on the ActorSystem it looks ambiguous)

[Andrew James Ramirez]

Hi, I am just new to akka and in programming, but someone told me that ActorSystem.scheduler is not accurate because it depends on the thread workload.

And if so, then a scheduled task for 5 minutes maybe delayed up to X minutes depending on the load of the application? Is that true? 

For me the only inaccuracy I could think of is the akka.scheduler.tick-duration implementation which I think would just be about 1 second maximum inaccuracy (if you use the default property). 

[√iktor Ҡlang]

What the Akka Scheduler (well, really the implementation: L*A*R*S) is that it -defers- the execution of a task until -after- a duration has passed. (note the difference between *at* a specific time and *after* a specific duration)

this can be witnessed by the lack of a method on Scheduler that takes a Date/Timestamp where to execute.

--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+...@googlegroups.com.
To post to this group, send email to akka...@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

--

Cheers,

√