Newbie Question: akka cluster and auto rebalancing

[Qing Zheng]

Hi,

  I am new to akka. I used Hazelcast before. In hazelcast cluster the framework takes care of cluster auto re balancing, auto store n copies (n is configurable) of back up data on different nodes, Hazelcast executor service allows me to send a task to the "keyOwner" which allows me to have distributed data grid and computing grid. I am looking for similar out of shelf functionalities in akka, but could not find it. Looks like I have implement all of these by myself. Any pointer will be highly appreciated.

  Thanks

  Qing

[Konrad Malawski]

Welcome to Akka.

Firstly, Hazlecast and Akka are not really the same beast. Akka is not "just a cache", it's active entities doing things - i.e. Actors, similar to like Erlang works with processes.

So it may help to change the mindset a bit, since it's not a cache. It can however model very similar (though strictly more powerful) things.

For example for rebalancing and persistence etc you should have a look at:

- cluster sharding (which combined with akka persistence gives you a solid foundation for super scalable event sourcing)

- distributed data (which is picking the opposite trade off, being very fast in memory replicated data).

Happy reading!

-- 
Konrad `ktoso` Malawski
Akka @ Lightbend

--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+...@googlegroups.com.
To post to this group, send email to akka...@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[Qing Zheng]

Hi Konrad,

  Thanks for the pointers.My data is in tree like structure.Potentially the there can have millions of tree node objects. Akka distributed data does not fit, I cannot afford to have tree data distributed across all the cluster nodes. Assuming one cluster node can handle 1 million tree node objects. when my tree reaches to 2 million tree node objects, I can just start another instance of cluster node and each cluster node can handle 1 million tree node objects and can provide backup of 1 million tree node objects for the other cluster node (assuming n+1 redundancy). I can send a task with hazelcast executor service to the "keyOwner" and the hazelcast framework takes care of which cluster node should handle the task. Cluster re balancing for new cluster node or existing cluster node failure are all handled by hazelcast framework. Looks like akka cluster sharding is the solution. My understanding by reading the document is that I have to implement cluster re-balancing all by myself with akka for a new cluster node joining a cluster or an existing cluster node that is no long available. And how to save backup data on different cluster node (for n+1 redundancy).

  If you can enlighten me what is correct akka way to handle this, it will be greatly appreciated.

  Thanks

  Qing

[Patrik Nordwall]

Cluster Sharding takes care of rebalancing automatically, and if the default strategy is not enough you can plug in your own.

As for redundancy/durability you have to save the state of the entity actors somewhere. Many use Akka Persistence for that but you can use something else.

/Patrik

To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscribe@googlegroups.com.

To post to this group, send email to akka...@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

--

Patrik Nordwall
Akka Tech Lead
Lightbend -  Reactive apps on the JVM
Twitter: @patriknw