Error during joining cluster

[Daniele Rossi]

Hi,

I installed VerneMQ 1.12.3 on three Debian Bullseye machines mqtt1, mqtt2, mqtt3.

I'm trying to join mqtt2 in cluster with mqtt1 but I'm receiving the following error:

===============================================================

root@mqtt2:~# vmq-admin cluster join discovery-node=mqtt1.mydomain.com

Couldn't join cluster due to not_reachable

===============================================================

Machines are in the same network and there is no firewall in between them, and mqtt2 can correctly reach mqtt1:

===============================================================
root@mqtt2:~# ping -c 3 mqtt1.mydomain.com

PING mqtt1.mydomain.com (192.168.110.61) 56(84) bytes of data.
64 bytes from mqtt1.mydomain.com (192.168.110.61): icmp_seq=1 ttl=64 time=0.394 ms
64 bytes from mqtt1.mydomain.com (192.168.110.61): icmp_seq=2 ttl=64 time=0.410 ms
64 bytes from mqtt1.mydomain.com (192.168.110.61): icmp_seq=3 ttl=64 time=0.501 ms

--- mqtt1.mydomain.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2056ms
rtt min/avg/max/mdev = 0.394/0.435/0.501/0.047 ms

===============================================================

Could you help me please?

Thank you very much!
Bye

[Tom Allen]

Check listener.vmq.clustering in vernemq.conf?  I believe the default port is 18883, you could try "telnet mqtt1.mydomain.com 18883" to make sure it's listening on that interface.

[André Fatton]

Hi Daniele,

you need to use the VerneMQ node name in the cluster join command. That is, not the nodes' domain name, but the nodename as you have configured it in the vernemq.conf file.

I hope this helps,

André

[Daniele Rossi]

Hi Tom,

Check listener.vmq.clustering in vernemq.conf?  I believe the default port is 18883, you could try "telnet mqtt1.mydomain.com 18883" to make sure it's listening on that interface.

This is my listener.vmq.clustering setting on mqtt1.mydomain.com:

===========================================================
listener.vmq.clustering = 192.168.110.61:44053
===========================================================

mqtt2 can telnet mqtt1 correctly:

===========================================================
root@mqtt2:~# telnet mqtt1.mydomain.com 44053
Trying 192.168.110.61...
Connected to mqtt1.mydomain.com.
Escape character is '^]'.
===========================================================

Thank you!

[Daniele Rossi]

Hi André,

you need to use the VerneMQ node name in the cluster join command. That is, not the nodes' domain name, but the nodename as you have configured it in the vernemq.conf file.

 

nodename on mqtt1 is the following:

===============================================================
nodename = Ver...@mqtt1.mydomain.com
===============================================================

Thank you!

[shacky]

Any hint to understand how to move further?

Thank you very much!

--
You received this message because you are subscribed to the Google Groups "vernemq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vernemq-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/vernemq-users/dec1d49c-7384-41c1-87c7-ad371d637600n%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[André Fatton]

Hi Daniele,

apologies for not responding quicker. As there are no literal commands and errors given, I might misunderstand what you are asking.

But please try the following: just give all your nodes a name using nodename=a...@127.0.0.1 / b...@127.0.0.1 / c...@127.0.0.1, in their respective vernemq.conf files.

The names will be stored by the port mapper daemon to match a name to a port (you can use "epmd -names" to check that).

The use a join command like: "sudo vmq-admin cluster join discovery-node=a...@127.0.0.1", issued on Node b, for example.

This will allow you to cluster using the underlying Erlang distribution mechanisms.

Note that there's an additional point: The clustered nodes will contact each other after initial clustering and then set up additional mutual TCP connections to exchange MQTT messages. To do this, they will get the IP:Port configured as the `vmq` listener in the remote vernemq.conf file over an RPC call, and then try to set up a connection to that remote IP:Port. The `vmq` listener needs to be given as an IP, not a hostname.

I hope this helps,

André

[Daniele Rossi]

Hi André, sorry for my late answer.

But please try the following: just give all your nodes a name using nodename=a...@127.0.0.1 / b...@127.0.0.1 / c...@127.0.0.1, in their respective vernemq.conf files.

The names will be stored by the port mapper daemon to match a name to a port (you can use "epmd -names" to check that).

I tried to configure the nodename as following: mq...@127.0.0.1 on node 1, mq...@127.0.0.1 on node 2 and mq...@127.0.0.1 on node 3.

Then I tried to join the cluster from node2 to node1 but it is not working:

root@mqtt2:~# vmq-admin cluster join discovery-node=mq...@127.0.0.1

Couldn't join cluster due to not_reachable

Please note that mqtt1 (node 1), mqtt2 (node 2) and mqtt3 (node 3) are on different virtual machines in the same network, so my question is "how the joiner can reach the other node if its name is @127.0.0.1? In the upper command I don't specify the address anywhere.

[André Fatton]

Apologies, Daniele, I seem to have been too tired to understand this was not a local dev cluster with nodes on the same machine.

Have you tried to configure nodenames with their respective IPs, though?

If needed we can do a quick call one of these days, there's some possibly confusing technicalities to this (EPMD and how it separates names from hosts).

Keep us posted, in any case.

André

[Daniele Rossi]

Hi André, thank you for your answer.

Apologies, Daniele, I seem to have been too tired to understand this was not a local dev cluster with nodes on the same machine.

Yes, it's not a local dev cluster: nodes are on separate machine.

 

Have you tried to configure nodenames with their respective IPs, though?

I will try and let you know.

 

If needed we can do a quick call one of these days, there's some possibly confusing technicalities to this (EPMD and how it separates names from hosts).

Yes, I think this would be useful!

I will write you a private message to let us organise a call.

[Daniele Rossi]

Hi André,

I really don't know why, but I reconfigured all three server from scratch and now they joined the cluster!

Thank you!