akka cluster manual join - Ignoring received gossip intended for someone else

[akhil]

Hi,

I am trying to join 3 akka nodes with each other using the Cluster join api.

The code snippet used on all 2 nodes is:

    Cluster(system).join(new Address("akka.tcp", "mysystem", "127.0.0.1", 2552))

    Cluster(system).join(new Address("akka.tcp", "mysystem", "127.0.0.1", 2553))

    Cluster(system).join(new Address("akka.tcp", "mysystem", "127.0.0.1", 2554))

application.conf snippet:

 cluster {      

      roles = ["worker"]

      auto-down = on

      auto-join = off

      seed-node-timeout = 30s

      failure-detector {

        threshold = 18.0

        acceptable-heartbeat-pause = 25s

      }

    }

First I bring up all 3 nodes cluster works fine and then restart node 3.

I get the following in console

Node1:

[info] 2014-01-07T22:23:11.801+0000, level=INFO, thread=mysystem-akka.actor.default-dispatcher-14, logger=Cluster(akka://mysystem), Cluster Node [akka.tcp://mysy...@127.0.0.1:2552] - Existing member [UniqueAddress(akka.tcp://mysy...@127.0.0.1:2554,1469131007)] is trying to join, ignoring

Node3:

[info] 2014-01-07T22:24:24.873+0000, level=INFO, thread=mysystem-akka.actor.default-dispatcher-5, logger=Cluster(akka://mysystem), Cluster Node [akka.tcp://mysy...@127.0.0.1:2554] - Ignoring received gossip intended for someone else, from [akka.tcp://mysy...@127.0.0.1:2552] to [UniqueAddress(akka.tcp://mysy...@127.0.0.1:2554,-1075005655)]

How do I resolve this?

I am trying this to cluster akka on AWS autoscaling.

Akhil

[Patrik Nordwall]

Hi Akhil,

You are trying to join node3 before it (same host:port) has been removed from the cluster. After a while (28 seconds according to your config) the failure detector will notice the shutdown node, and auto-down and removal will follow.

I recommend that you use Cluster(system).joinSeedNodes(...) for programatic joining, because it has built in retry mechanism.

What version of akka are you using, by the way? 2.2.3 or 2.3-M2 are the recommended versions at the moment.

Cheers,

Patrik

--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ: http://akka.io/faq/
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+...@googlegroups.com.
To post to this group, send email to akka...@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/groups/opt_out.

--

Patrik Nordwall
Typesafe -  Reactive apps on the JVM
Twitter: @patriknw

[Akhil Kodali]

Using joinSeedNodes seems to solve the problem and I am using akka 2.2.3

How would joinSeedNodes behave in case of a rolling nodes.

i.e.

Initially cluster is composed of Nodes [N1,N2,N3]  this would eventually change to [N4,N5,N6] one at a time.

joinSeedNodes [N2,N3,N4] on N4

joinSeedNodes [N3,N4,N5] on N5

joinSeedNodes [N4,N5,N6] on N6

----
Regards,
Akhil Kodali

You received this message because you are subscribed to a topic in the Google Groups "Akka User List" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/akka-user/Y0LfoDqRMyA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to akka-user+...@googlegroups.com.

[Patrik Nordwall]

The documentation describes it as:

You may also use Cluster(system).joinSeedNodes, which is attractive when dynamically discovering other nodes at startup by using some external tool or API. When using joinSeedNodes you should not include the node itself except for the node that is supposed to be the first seed node, and that should be placed first in parameter to joinSeedNodes.

To avoid forming separated islands when starting from an empty cluster one node must be marked as special, know as the first seed node in the documentation.

You can use all known (discovered) nodes as seed nodes candidates. It doesn't have to be limited to a few (3).

Regards,

Patrik