akka cluster seed nodes on ec2

[Scott Clasen]

Anyone have good tricks for standing up an akka cluster on ec2 WRT seed nodes?  

Would like to be able to auto scale it so not having to manually join a node is necessary.

Could use elastic IPs would rather not. Could maybe use a tcb elb? not sure that would work, would rather not.

Guess I can use ZK which I already have stood up...what weird things can happen if a node attempts to join a cluster via a seed that has been partitioned away from the cluster?

[Ryan Tanner]

We have Vagrant auto-update the hosts file over ssh.  Kinda clunky but it works for small clusters.

[Patrik Nordwall]

Hi Scott,

Using the AWS API together with Cluster(system).joinSeedNodes should be possible. The first seed node must be marked somehow.

From docs:

You may also use Cluster(system).joinSeedNodes, which is attractive when dynamically discovering other nodes at startup by using some external tool or API. When using joinSeedNodes you should not include the node itself except for the node that is supposed to be the first seed node, and that should be placed first in parameter to joinSeedNodes.

Regards,

Patrik

--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ: http://akka.io/faq/
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+...@googlegroups.com.
To post to this group, send email to akka...@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/groups/opt_out.

--

Patrik Nordwall
Typesafe -  Reactive apps on the JVM
Twitter: @patriknw

[Timothy Perrett]

Scott, 

What did you ever end up doing about this? As yet, I have not seen any decent story for auto-scalling akka-cluster on AWS; it strikes me that over a long enough period all the seed nodes could be reaped by the ASG, but that there would be enough folks wanting to do this that there would be a decent solution for it.

Cheers

Tim

[Roland Kuhn]

Hi Tim,

you could have one small group which is always on and acts as entry point (seed nodes) while the rest are auto-scaling. Would that solve the issue?

Regards,

Roland

Dr. Roland Kuhn
Akka Tech Lead
Typesafe – Reactive apps on the JVM.
twitter: @rolandkuhn

[Timothy Perrett]

Hey Roland - doesnt "always on" sound like "will never fail"... ? I wouldn't be comfortable manually managing that cluster, so it would need an ASG, which in turn will remove the oldest node when shrinking, so over time you'd end up loosing all your original seed nodes if your cluster is expanding and contracting enough. 

[Patrik Nordwall]

Hi Tim,

You can use whatever nodes as seed nodes, except for when you start up a fresh cluster from scratch. When you start a node you can use the AWS API to discover other EC2 instances and use all or a few as seed nodes.

The special case is when starting a new cluster, and then I imagine that you can mark one node as special using AWS metadata, or a special argument in the start script. This special node must put itself first in the list of seed nodes. In all other cases a node should not include itself as seed node.

The reason for the special first seed node is to avoid creating several separate clusters when starting from an empty cluster.

I would be interested in understanding why this would not be feasible solution.

Cheers,

Patrik

[James Bellenger]

Hi Tim,

We have a similar-ish setup over here. We ended up registering all nodes in zookeeper and doing discovery through that. This works well for initial cluster startup as well as nodes joining an existing cluster. It also works well for hybrid environments that are not all on aws.

[Akka Team]

Hi Tim,

what I meant was to query the AWS API for the current members of that ASG and use the response to build the seed node list. Whether or not that list contains the same nodes as some other node has seen previously is not relevant (seed nodes are only used by the joining node, they are not special otherwise).

Regards,

Roland

On Tue, Feb 11, 2014 at 7:20 PM, Timothy Perrett <tim...@getintheloop.eu> wrote:

--

Akka Team

Typesafe - The software stack for applications that scale

Blog: letitcrash.com
Twitter: @akkateam