hazelcast with payara in docker in AWS

[Hans Pikkemaat]

Hi,

I'm using the latest version of payara.

I'm running payara inside a docker container in AWS in an EC2 instance.

I want to use Hazelcast so I want both instances to be able to find each other.

This is what I did:

- configure AWS security group in which the instances are running to accept connections on port 4900

- enable hazelcast

     - set-hazelcast-configuration --enabled true

- restart payara

- find ip addresses of instances using aws query

- configure tcpip 

     - set-hazelcast-configuration --dynamic --clustermode tcpip --tcpipmembers <ip1>:4900,<ip2>:4900

I can see in the logs that hazelcast is restarting but get this Exception:

java.lang.IllegalArgumentException: Target is this node! -> [172.18.0.5]:4900

It simply doesn't work.

What am I doing wrong?

regards, Hans

[Rudy De Busscher]

Hi,

Can it be possible that Hazelcast is starting on an 'internal' IP address of the container (and not the one you supplied in set-hazelcast-configuration)

You might get more feedback on a Hazelcast forum about this issue (What configuration must be used to join 2 Hazelcast instances in Docker on EC2 using tcpip mode)

PS, no need to enable hazelcast as it is active by default and it is not recommended to restart Payara within Docker Container (perform any config in the build image step)

Regards

rudy

[Rudy De Busscher]

Hi Hans,

I'm not familiar enough with AWS to help you on you issue as it seems to work perfectly within 'plain docker' as you mention.

Community version is using Hazelcast 4.x and thus should support the Hazelcast-aws library. Some additional links that might give you more info

Info on how to configure Hazelcast in AWS : https://hazelcast.com/blog/how-to-set-up-hazelcast-imdg-on-aws-ecs/

Use custom hazelcast-config.xml in Payara : https://docs.payara.fish/community/docs/documentation/payara-server/hazelcast/configuration.html#setting-hazelcast-configuration-file

Easiest way to find out if custom config file is used is to use invalid content in the xml file and see if this results in an error at startup.

The file must be in the <PAYARA_HOME>/glassfish/domains/<domain>/config directory and only the filename can be specified in the Payara config. 

See https://github.com/payara/Payara/blob/67ce52a3c3fdb51e3ddfecce2681975dc75671c5/nucleus/payara-modules/hazelcast-bootstrap/src/main/java/fish/payara/nucleus/hazelcast/HazelcastCore.java#L312

Regards

Rudy

-- received outside of Forum

Hi,

I found a lot of information about configuring hazelcast but the big question is if I can simply use a hazelcast configuration file (as described on the hazelcast website) and configure that in payara.

I tried that but did not get any feedback in the logs about that configuration being loaded and also 'get-hazelback-configuration' doesn't show information from that configuration file. So how can I confirm that payara has loaded the hazelback-config.xml file that I supply? 

I also found a hazelcast-aws library which I loaded using 'add-library' but did not see any changes. Do you know if that is necessary?

Could you let me know how to enable debug logging in payara for hazelback? Maybe I can find out more about what is happening.

regards,

Hans

PS: I was able to setup two payara DAS servers in docker (in the same network) with tcpip mode. But was unable to set it up like this in AWS. See here: https://github.com/thehpi/payara-server-hazelcast