Scale out AKKA application

[Piotr Dytkowski]

Hi all,

I recently wrote an application in AKKA and it works well. Now it's time to think about scaling it and eliminating the single point of failure.

Application is pretty simple. It consumes and stores two types of data: M and F in actors (MActor and FActor). It consumes also D (in DActor) which results in looking up appropriate M and F and producing DR (DResponse).

M and F are stored in Maps (inside the actors, maps are of type <Key, ActorRef>).

1) My current strategy is to deploy application on two servers every message is processed by both servers. It's not critical to synchronize state between different nodes, but would be nice to have if cheap. 

Problem is when I scale out every additional node increases amount of output messages and adds redundant processing.

2) I read about akka cluster (http://doc.akka.io/docs/akka/2.0.1/cluster/cluster.html) but it says "Currently the only possible partition points are routed actors." and partition points in my case are single actors.

3) I read about akka Cluster Singleton (http://doc.akka.io/docs/akka/snapshot/contrib/cluster-singleton.html) but it's not a recommended approach. There are moments when there is no actor in the cluster and afaik state wouldn't be restored.

4) I was thinking if the design of the system is wrong and maybe MActor and FActor should be routers instead and then Distributed Router could be used to scale out the system but this actor has some logic inside regarding the routing itself and expiry of the entries (actors). And then again how would be the state handed over when one of the nodes fail and actors have to be recreated on the other node?

To sum it up. There are some clustering strategies available. But how they deal with the state in case of node failure. Is any of them appropriate for a given use case?

I hope my explanations are clear. Shout if more info is needed.

Thanks for help!

[Björn Antonsson]

Hi,

You haven't really described why you want to scale out.

Is it size? The M and F datasets don't fit in memory/

Is it speed of updating? Processing M and F takes time.

Is it query speed? You want to be able to handle more queries faster.

Is it resilience? You want to have redundant servers in case of failure.

What you should do and how your application should work is quite different depending on why you want to scale out.

B/

--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+...@googlegroups.com.
To post to this group, send email to akka...@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

-- 

Björn Antonsson

Typesafe – Reactive Apps on the JVM

twitter: @bantonsson

[Piotr Dytkowski]

Hi Björn,

Sorry for not stating this clear,

Main purpose is resilience (others are not a concern atm).

I have a few servers where I can deploy it and and application should be available when one of the servers goes down.

Thanks

Piotr

[Björn Antonsson]

Hi Piotr,

So if it is resilience, then duplicating the insertion and round robin the queries will get you a long way. What you need to decide is what happens when you bring up a new server while others have been running for a while.

Will the new server try to sync the M and F datasets? Is it ok if there are missed M and/or F updates? And so on.

B/

--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+...@googlegroups.com.
To post to this group, send email to akka...@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[Piotr Dytkowski]

That solves the problem, perfect.

Thanks a lot.