Unable to get two node cluster to join in hazelcast cluster on AWS using docker container.

[bi...@paysense.in]

Hi,

I am trying to get two nodes join cluster using discovery-strategies as mentioned on https://github.com/hazelcast/hazelcast-aws as preferred approach and I am not able to do it. When I bring up the docker container, I have checked that all my firewall is disabled and when I try to telnet from one node to other node on 5701(hazelcast port), I am able to do it. Am I missing something here or should I need to some change in container settings?

I am able to get two nodes join cluster using https://github.com/hazelcast/hazelcast-aws#configuring-with-awsconfig-deprecated using docker container but this is deprecated and hence wanted to suggested approach to give it a try. I have attached my hazelcast.xml. Any help will be really helpful. 

Thanks,

Bipul.

[Rahul]

Did you try this: https://github.com/hazelcast/hazelcast-docker

[bi...@paysense.in]

Hi, I am using orientdb docker image (https://hub.docker.com/_/orientdb/) which uses hazelcast and attached is the hazelcast.xml which is provided as configuration to docker container.

[baris]

Hi,

Is there any specific reason to use directly Docker on EC2 host rather than AWS ECS?

Could you please also share the logs?

Moreover, is it possible to check below items on your environment?

1] Please run docker containers with `--network=host` networking on EC2 hosts.

2] Please add below interface binding to `hazelcast.xml` configuration file.

```xml

<interfaces enabled="true">

        <interface>10.0.*.*</interface>

</interfaces>

```

Please note that `10.0.*.*` depends on your subnet or CIDR block.

please let us know, if further help or support is required. thanks in advance.

[bi...@paysense.in]

Thank you for replying. Please find my inlined relies below:

Is there any specific reason to use directly Docker on EC2 host rather than AWS ECS?

There is no specific reason to use docker on EC2 rather than AWS ECS. Did not explore ECS solution. 

Logs from two node is following:

1] Please run docker containers with `--network=host` networking on EC2 hosts.

This is already being done. 

2] Please add below interface binding to `hazelcast.xml` configuration file.

This is already added in my configuration file. For other config parameters, I have attached my configuration file in my first post of this thread. 

Thanks in advance,

Bipul.

[baris....@gmail.com]

Hi,

I know you shared the config, however, from screenshot instance binding seems to docker ip, not AWS host private ip in your subnet.

regards

[bi...@paysense.in]

Hi Baris,

My AWS private subnet is 172.31.0.0/16 and hence container ip is 172.31.4.86 as of my understanding. 

I get the same log + TCPIPJoiner log when I use awsconfig plugin to join two nodes in the cluster and it runs fine. 

Thanks,

Bipul.

[baris....@gmail.com]

Hi,

Please make sure that 2.0.2-SNAPSHOT is accessible with cp or CLASSPATH, there should be similar log entries when you start hazelcast instance

WARNING: Describe instances will be queried with iam-role assigned to EC2 instance, please make sure given iam-role have ec2:DescribeInstances policy attached.

or you can check here

please let us know, if further help or support is required. thanks in advance.

[murali parepally]

Hi,
How should i check CIDR block/subnet  in linux ?

[Josef Cacek]

Hi,

you can use `ip addr` command to list network interfaces with their
details. E.g. one of the entries in my case is:

2: enp0s31f6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
fq_codel state UP group default qlen 1000
link/ether c8:5b:76:a5:51:f8 brd ff:ff:ff:ff:ff:ff
inet 192.168.1.13/24 brd 192.168.1.255 scope global noprefixroute enp0s31f6
valid_lft forever preferred_lft forever
inet6 fe80::8f49:b6ff:3fdf:e493/64 scope link noprefixroute
valid_lft forever preferred_lft forever

Then from the item "inet 192.168.1.13/24" I know the subnet is "192.168.1.*".

Also, have you seen my responses to your question on Gitter?
https://gitter.im/hazelcast/hazelcast?at=5e9728bd74bfed5a1b389cca

-- Josef

> --
> You received this message because you are subscribed to the Google Groups "Hazelcast" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to hazelcast+...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/hazelcast/d6dd57e1-967c-456e-a450-a49636d68833%40googlegroups.com.

--
This message contains confidential information and is intended only for the
individuals named. If you are not the named addressee you should not
disseminate, distribute or copy this e-mail. Please notify the sender
immediately by e-mail if you have received this e-mail by mistake and
delete this e-mail from your system. E-mail transmission cannot be
guaranteed to be secure or error-free as information could be intercepted,
corrupted, lost, destroyed, arrive late or incomplete, or contain viruses.
The sender therefore does not accept liability for any errors or omissions
in the contents of this message, which arise as a result of e-mail
transmission. If verification is required, please request a hard-copy
version. -Hazelcast

[murali parepally]

Hi Josef,

thanks for quick reply. I have seen your response in gitter but when i am trying it i am getting the message that while running on host network ports wont be published. It would be great if you share the steps  to resolve my issue posted in gitter  .PFB for quick view abt issue.

what I did is I am running my tomcat in docker container in machine 1 and took that image to run in machine2. I found that container ip is same in both machines.

Regards,

Murali

To view this discussion on the web visit https://groups.google.com/d/msgid/hazelcast/CAN5h8r6Ciz8fRcWG6S71ZsTuwRDiJxze1tsNq_V5SQzkBoHcsw%40mail.gmail.com.

[Josef Cacek]

Let's continue the discussion on Gitter as it's more interactive.
Thanks,
-- Josef

> To view this discussion on the web visit https://groups.google.com/d/msgid/hazelcast/CAE2j3SOm1k%3DwC74S61v54dqTNJVZ2jXOmr_Y2Owwnt9vWoMiMA%40mail.gmail.com.