Lost messages in TCP and GOSSIP-Router configuration

Frederic

unread,

Sep 9, 2022, 12:48:16 PM9/9/22

to jgroups-dev

Hello,

I'm using jgroups v5.2.6 to communicate between several java clients app and one single java server app installed on two different private networks. Due to firewalls, I run a gossip router on the same server where the server app is installed.

My problem is that after a while, when several communicatiosn have been established, I lost some messages and it is sometimes imposisble to communicate between client apps and server app.

I join the logs over several days and my configuration.

During several years I was using the version jgroups-4.0.10.Final without any problem.

Do you think it is a question of configuration ?

Server app or client app are restarted and seems to generate the views that are merged but some client seems disapeared from the views.

Thanks for help.

Fred

NetworkJgroupsMessagingAnalysis.txt

tcpgossip.xml

Bela Ban

unread,

Sep 10, 2022, 5:50:25 AM9/10/22

to jgrou...@googlegroups.com

First off, I see duplicate members in a view: ][viewAccepted] ** view:
[SiNOP-SERVICE_CANOPP01|3] (4) [SiNOP-SERVICE_CANOPP01,
SiNOP-ONE_CANOPP01, SiNOP-ONE_CANOPP01, SiNOP-ONE_CANOPP11]

SiNOP-ONE_CANOPP01 occurs twice. This is not necessarily an issue, as
SiNOP-ONE_CANOPP01 is only a logical name, and the underlying UUID is
probably different. But it points to a problem in how you name your
instances.

Looking at your logs, there are tons of merge view. This points to an
issue in members communicating. I think the problem is that you have a
firewall, but members communicate *directly* with each other, as the
transport is TCP!

You don't use the GossipRouter for communication between members, but
merely for discovery (TCPGOSSIP). If you want communication through the
GossipRouter, then use TUNNEL(gossip_router_hosts="..."):PING

Also:
* RED should be above the transport (TUNNEL), *not* above the discovery
protocol
* FD_SOCK2 won't work here, either, as it uses direct TCP connections
between members. These will be blocked unless you list the port range
for FD_SOCK2 in your firewall. I suggest comment FD_SOCK2 for now, to
see if the new config works.

Cheers,

> --
> You received this message because you are subscribed to the Google
> Groups "jgroups-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to jgroups-dev...@googlegroups.com
> <mailto:jgroups-dev...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jgroups-dev/abe0c2a4-a717-415a-abb7-404f28621818n%40googlegroups.com
> <https://groups.google.com/d/msgid/jgroups-dev/abe0c2a4-a717-415a-abb7-404f28621818n%40googlegroups.com?utm_medium=email&utm_source=footer>.

--
Bela Ban | http://www.jgroups.org

Frederic

unread,

Sep 10, 2022, 6:11:06 AM9/10/22

to jgroups-dev

Hi Bela,

Thanks for your quick reply and your answers.

I had a doubt about TCP or Tunnel, since I've got the ton of Merge views also.

I got duplicates members when I restarted the same client app. It's like old members were not cleaned from the list.

I'm going to try the Tunnel configuration as you proposed.

Thanks,

Fred

Bela Ban

unread,

Sep 11, 2022, 8:22:03 AM9/11/22

to jgrou...@googlegroups.com

On 10.09.22 12:11, Frederic wrote:
> Hi Bela,
>
> Thanks for your quick reply and your answers.
>
> I had a doubt about TCP or Tunnel, since I've got the ton of Merge views
> also.
> I got duplicates members when I restarted the same client app. It's like
> old members were not cleaned from the list.
> I'm going to try the Tunnel configuration as you proposed.

+1. I suggest post the changed config, so I can review it for correctness.

> <https://groups.google.com/d/msgid/jgroups-dev/abe0c2a4-a717-415a-abb7-404f28621818n%40googlegroups.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/jgroups-dev/abe0c2a4-a717-415a-abb7-404f28621818n%40googlegroups.com?utm_medium=email&utm_source=footer>>.
>
>
> --
> Bela Ban | http://www.jgroups.org <http://www.jgroups.org>

>
> --
> You received this message because you are subscribed to the Google
> Groups "jgroups-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to jgroups-dev...@googlegroups.com
> <mailto:jgroups-dev...@googlegroups.com>.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/jgroups-dev/dbf70291-5ee7-45e1-9a62-beb5a9eb7cb0n%40googlegroups.com
> <https://groups.google.com/d/msgid/jgroups-dev/dbf70291-5ee7-45e1-9a62-beb5a9eb7cb0n%40googlegroups.com?utm_medium=email&utm_source=footer>.

Frederic

unread,

Sep 12, 2022, 7:21:08 AM9/12/22

to jgroups-dev

Hi Bela,

I will try the following configuration :

<jgroups:config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:jgroups="jgroups"
xsi:schemaLocation="jgroups http://www.jgroups.org/schema/jgroups.xsd">

<jgroups:TUNNEL jgroups:gossip_router_hosts="${jgroups.tunnel.gossip_router_hosts:localhost[12000]}"/>
<jgroups:PING/>
<jgroups:MERGE3/>
<jgroups:FD_ALL3/>
<jgroups:VERIFY_SUSPECT/>
<jgroups:pbcast.NAKACK2 jgroups:use_mcast_xmit="false"/>
<jgroups:UNICAST3/>
<jgroups:pbcast.STABLE/>
<jgroups:pbcast.GMS/>
<jgroups:UFC/>
<jgroups:MFC/>
<jgroups:FRAG2/>
<jgroups:pbcast.STATE_TRANSFER/>
</jgroups:config>

I got it from https://github.com/belaban/JGroups/blob/jgroups-5.2.6.Final/conf/tunnel.xml

Do you have some suggestions ?

Thanks.

Fred

Bela Ban

unread,

Sep 12, 2022, 8:44:21 AM9/12/22

to jgrou...@googlegroups.com

That looks fine, let me know whether oit fixes your issue

> <https://groups.google.com/d/msgid/jgroups-dev/dbf70291-5ee7-45e1-9a62-beb5a9eb7cb0n%40googlegroups.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/jgroups-dev/dbf70291-5ee7-45e1-9a62-beb5a9eb7cb0n%40googlegroups.com?utm_medium=email&utm_source=footer>>.
>
>
> --

> Bela Ban | http://www.jgroups.org <http://www.jgroups.org>
>
> --
> You received this message because you are subscribed to the Google
> Groups "jgroups-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to jgroups-dev...@googlegroups.com
> <mailto:jgroups-dev...@googlegroups.com>.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/jgroups-dev/eb152b16-ed95-4b6e-b8c8-aabbff0ec94bn%40googlegroups.com
> <https://groups.google.com/d/msgid/jgroups-dev/eb152b16-ed95-4b6e-b8c8-aabbff0ec94bn%40googlegroups.com?utm_medium=email&utm_source=footer>.

Frederic

unread,

Sep 14, 2022, 8:54:31 AM9/14/22

to jgroups-dev

Hi Bela,

I still have lot of MergeView, but not all clients were rebooted with the new configuration.

I will have a better status tomorrow.

Fred

Bela Ban

unread,

Sep 14, 2022, 9:10:00 AM9/14/22

to jgrou...@googlegroups.com

Yes, if not all clients are running the same config, then you'll still
have issues...

> https://github.com/belaban/JGroups/blob/jgroups-5.2.6.Final/conf/tunnel.xml <https://github.com/belaban/JGroups/blob/jgroups-5.2.6.Final/conf/tunnel.xml>

> <https://groups.google.com/d/msgid/jgroups-dev/eb152b16-ed95-4b6e-b8c8-aabbff0ec94bn%40googlegroups.com?utm_medium=email&utm_source=footer <https://groups.google.com/d/msgid/jgroups-dev/eb152b16-ed95-4b6e-b8c8-aabbff0ec94bn%40googlegroups.com?utm_medium=email&utm_source=footer>>.
>
> --
> Bela Ban | http://www.jgroups.org <http://www.jgroups.org>

>
> --
> You received this message because you are subscribed to the Google
> Groups "jgroups-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to jgroups-dev...@googlegroups.com
> <mailto:jgroups-dev...@googlegroups.com>.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/jgroups-dev/17e068a7-f384-4b73-bcc5-eefb3e247830n%40googlegroups.com <https://groups.google.com/d/msgid/jgroups-dev/17e068a7-f384-4b73-bcc5-eefb3e247830n%40googlegroups.com?utm_medium=email&utm_source=footer>.

Frederic

unread,

Sep 14, 2022, 1:13:27 PM9/14/22

to jgroups-dev

Hi Bela,

Here is the complete picture of the architecture :

- in one network side : one server with 6 gossip router running on different port number + one server app

1st cluster :
cluster_name = CLUSTER_DIMT_14
initialHosts=${jgroups.tunnel.gossip_router_hosts:CANOPP01[15502]}
channel.getName() = SiNOP-SERVICE_CANOPP01
<TUNNEL gossip_router_hosts="${jgroups.tunnel.gossip_router_hosts:CANOPP01[15502]}"/>
2nd cluster :
cluster_name = CLUSTER_AITSAT_CE_14
initialHosts=${jgroups.tunnel.gossip_router_hosts:CANOPP01[15505]}
channel.getName() = SiNOP-SERVICE_CANOPP01
<TUNNEL gossip_router_hosts="${jgroups.tunnel.gossip_router_hosts:CANOPP01[15505]}"/>
3rd ... etc.

- in another network side separeted by a firewall : several app clients on several computers

The communication is established between all of them (client-to-server, server-to-client, client-to-client) thanks to tunnel configuration + gossip router

Each Gossip Router has its own logs. We have into the logs on a specific router logs related to other clusters. example :

logs of gossip router : CLUSTER_DIMT_14
2022-09-14 18:35:02 DEBUG set property org.jgroups.stack....@6ce1f601.mcast_addr to default value /224.0.75.75
2022-09-14 18:35:03 DEBUG added SiNOP-ONE_PC213404 (10.148.5.138:52559) to group SiNOPCluster_CLUSTER_DIMT_14
2022-09-14 18:35:05 DEBUG added SiNOP-ONE_PC215441 (10.148.5.55:60065) to group SiNOPCluster_CLUSTER_DIMT_14
2022-09-14 18:35:05 DEBUG added SiNOP-ONE_PC215403 (10.148.0.95:59492) to group SiNOPCluster_CLUSTER_DIMT_14
2022-09-14 18:35:06 DEBUG added SiNOP-ONE_PC215432 (10.148.4.153:49316) to group SiNOPCluster_CLUSTER_DIMT_14
2022-09-14 18:35:07 DEBUG added SiNOP-ONE_PC717406 (10.148.1.180:57009) to group SiNOPCluster_CLUSTER_DIMT_14
2022-09-14 18:35:08 DEBUG added SiNOP-ONE_PC215436 (10.148.5.214:58520) to group SiNOPCluster_CLUSTER_DIMT_14
2022-09-14 18:35:11 DEBUG added SiNOP-ONE_PC711354 (10.148.5.160:54062) to group SiNOPCluster_CLUSTER_DIMT_14
2022-09-14 18:35:11 DEBUG added SiNOP-ONE_PC215435 (10.148.0.181:57357) to group SiNOPCluster_CLUSTER_DIMT_14
2022-09-14 18:35:17 DEBUG added SiNOP-ONE_PC215407 (10.148.1.35:63280) to group SiNOPCluster_CLUSTER_DIMT_14
2022-09-14 18:35:19 DEBUG added SiNOP-ONE_PC215404 (10.148.6.108:57663) to group SiNOPCluster_CLUSTER_DIMT_14
2022-09-14 18:36:06 DEBUG added SiNOP-SERVICE_CANOPP01 (10.149.135.6:54353) to group SiNOPCluster_CLUSTER_DIMT_14
2022-09-14 18:36:23 WARN dest 589353e1-264a-47be-8683-9b51470568f2 in cluster SiNOPCluster_CLUSTER_DIMT_14 not found
2022-09-14 18:36:50 WARN dest 589353e1-264a-47be-8683-9b51470568f2 in cluster SiNOPCluster_CLUSTER_DIMT_14 not found
2022-09-14 18:36:51 WARN dest 589353e1-264a-47be-8683-9b51470568f2 in cluster SiNOPCluster_CLUSTER_DIMT_14 not found
2022-09-14 18:37:03 WARN dest 589353e1-264a-47be-8683-9b51470568f2 in cluster SiNOPCluster_CLUSTER_DIMT_14 not found
2022-09-14 18:37:05 WARN dest 589353e1-264a-47be-8683-9b51470568f2 in cluster SiNOPCluster_CLUSTER_DIMT_14 not found
2022-09-14 18:37:06 WARN dest 589353e1-264a-47be-8683-9b51470568f2 in cluster SiNOPCluster_CLUSTER_DIMT_14 not found
2022-09-14 18:37:08 WARN dest 589353e1-264a-47be-8683-9b51470568f2 in cluster SiNOPCluster_CLUSTER_DIMT_14 not found
2022-09-14 18:37:22 WARN dest 589353e1-264a-47be-8683-9b51470568f2 in cluster SiNOPCluster_CLUSTER_DIMT_14 not found
2022-09-14 18:37:23 WARN dest 589353e1-264a-47be-8683-9b51470568f2 in cluster SiNOPCluster_CLUSTER_DIMT_14 not found
2022-09-14 18:37:23 WARN dest 589353e1-264a-47be-8683-9b51470568f2 in cluster SiNOPCluster_CLUSTER_DIMT_14 not found
2022-09-14 18:37:34 WARN dest 589353e1-264a-47be-8683-9b51470568f2 in cluster SiNOPCluster_CLUSTER_DIMT_14 not found
2022-09-14 18:37:37 WARN dest 589353e1-264a-47be-8683-9b51470568f2 in cluster SiNOPCluster_CLUSTER_DIMT_14 not found
2022-09-14 18:41:38 WARN dest 589353e1-264a-47be-8683-9b51470568f2 in cluster SiNOPCluster_CLUSTER_DIMT_14 not found
2022-09-14 18:41:48 DEBUG connection to 10.148.5.55:64001 closed
2022-09-14 18:41:48 DEBUG removed SiNOP-ONE_PC215441 (10.148.5.55:60065) from group SiNOPCluster_CLUSTER_DIMT_14
2022-09-14 18:41:49 ERROR failed sending unicast message to 10.148.5.55:64001: java.net.SocketTimeoutException: connect timed out
java.net.SocketTimeoutException: connect timed out
at java.net.PlainSocketImpl.waitForConnect(Native Method) ~[?:?]
at java.net.PlainSocketImpl.socketConnect(Unknown Source) ~[?:?]
at java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) ~[?:?]
at java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) ~[?:?]
at java.net.AbstractPlainSocketImpl.connect(Unknown Source) ~[?:?]
at java.net.SocksSocketImpl.connect(Unknown Source) ~[?:?]
at java.net.Socket.connect(Unknown Source) ~[?:?]
at org.jgroups.util.Util.connect(Util.java:419) ~[jgroups-5.2.6.Final.jar:5.2.6.Final]
at org.jgroups.blocks.cs.TcpConnection.connect(TcpConnection.java:97) ~[jgroups-5.2.6.Final.jar:5.2.6.Final]
at org.jgroups.blocks.cs.TcpConnection.connect(TcpConnection.java:89) ~[jgroups-5.2.6.Final.jar:5.2.6.Final]
at org.jgroups.blocks.cs.BaseServer.getConnection(BaseServer.java:317) ~[jgroups-5.2.6.Final.jar:5.2.6.Final]
at org.jgroups.blocks.cs.BaseServer.send(BaseServer.java:230) ~[jgroups-5.2.6.Final.jar:5.2.6.Final]
at org.jgroups.stack.GossipRouter.sendToMember(GossipRouter.java:710) ~[jgroups-5.2.6.Final.jar:5.2.6.Final]
at org.jgroups.stack.GossipRouter.route(GossipRouter.java:603) ~[jgroups-5.2.6.Final.jar:5.2.6.Final]
at org.jgroups.stack.GossipRouter.receive(GossipRouter.java:343) ~[jgroups-5.2.6.Final.jar:5.2.6.Final]
at org.jgroups.blocks.cs.BaseServer.receive(BaseServer.java:204) ~[jgroups-5.2.6.Final.jar:5.2.6.Final]
at org.jgroups.blocks.cs.TcpConnection$Receiver.run(TcpConnection.java:307) ~[jgroups-5.2.6.Final.jar:5.2.6.Final]
at java.lang.Thread.run(Unknown Source) ~[?:?]
2022-09-14 18:41:49 WARN dest 6c87e136-b899-4928-9b3c-798b648ee8c1 in cluster SiNOPCluster_CLUSTER_DIMT_14 not found
2022-09-14 18:41:49 WARN dest 6c87e136-b899-4928-9b3c-798b648ee8c1 in cluster SiNOPCluster_CLUSTER_DIMT_14 not found
2022-09-14 18:41:50 WARN dest 6c87e136-b899-4928-9b3c-798b648ee8c1 in cluster SiNOPCluster_CLUSTER_DIMT_14 not found
2022-09-14 18:41:50 WARN dest 589353e1-264a-47be-8683-9b51470568f2 in cluster SiNOPCluster_CLUSTER_DIMT_14 not found
2022-09-14 18:41:53 WARN dest 6c87e136-b899-4928-9b3c-798b648ee8c1 in cluster SiNOPCluster_CLUSTER_DIMT_14 not found
2022-09-14 18:41:56 WARN dest 6c87e136-b899-4928-9b3c-798b648ee8c1 in cluster SiNOPCluster_CLUSTER_DIMT_14 not found
2022-09-14 18:48:40 DEBUG connection to 10.148.1.35:51042 closed
2022-09-14 18:48:40 DEBUG removed SiNOP-ONE_PC215407 (10.148.1.35:63280) from group SiNOPCluster_CLUSTER_DIMT_14
2022-09-14 18:48:40 WARN dest d1dfcc45-82c2-4199-9dcf-19eac3207a45 in cluster SiNOPCluster_CLUSTER_DIMT_14 not found

logs of gossip router : CLUSTER_AITSAT_CE_14
2022-09-14 18:34:57 DEBUG set property org.jgroups.stack....@65f87a2c.mcast_addr to default value /224.0.75.75
2022-09-14 18:34:57 DEBUG added SiNOP-ONE_PC711354 (10.148.5.160:54062) to group SiNOPCluster_CLUSTER_DIMT_14
2022-09-14 18:34:58 DEBUG added SiNOP-ONE_PC215404 (10.148.6.108:57663) to group SiNOPCluster_CLUSTER_DIMT_14
2022-09-14 18:35:03 DEBUG added SiNOP-ONE_PC213404 (10.148.5.138:52559) to group SiNOPCluster_CLUSTER_DIMT_14
2022-09-14 18:35:05 DEBUG added SiNOP-ONE_PC215441 (10.148.5.55:60065) to group SiNOPCluster_CLUSTER_DIMT_14
2022-09-14 18:35:05 DEBUG added SiNOP-ONE_PC215403 (10.148.0.95:59492) to group SiNOPCluster_CLUSTER_DIMT_14
2022-09-14 18:35:06 DEBUG added SiNOP-ONE_PC215387 (10.148.58.47:62880) to group SiNOPCluster_CLUSTER_AIT_CUO_14
2022-09-14 18:35:06 DEBUG added SiNOP-ONE_PC215432 (10.148.4.153:49316) to group SiNOPCluster_CLUSTER_DIMT_14
2022-09-14 18:35:07 DEBUG added SiNOP-ONE_PC717406 (10.148.1.180:57009) to group SiNOPCluster_CLUSTER_DIMT_14
2022-09-14 18:35:08 DEBUG added SiNOP-ONE_PC720186 (10.148.51.20:49902) to group SiNOPCluster_CLUSTER_AIT_CUO_14
2022-09-14 18:35:08 DEBUG added SiNOP-ONE_PC215436 (10.148.5.214:58520) to group SiNOPCluster_CLUSTER_DIMT_14
2022-09-14 18:35:11 DEBUG added SiNOP-ONE_PC215435 (10.148.0.181:57357) to group SiNOPCluster_CLUSTER_DIMT_14
2022-09-14 18:35:12 DEBUG added SiNOP-ONE_PC210516 (10.148.6.99:54550) to group SiNOPCluster_CLUSTER_AITSAT_CE_14
2022-09-14 18:35:14 DEBUG added SiNOP-ONE_PC211139 (10.148.3.122:49984) to group SiNOPCluster_CLUSTER_AITSAT_CE_14
2022-09-14 18:35:15 DEBUG added SiNOP-ONE_PC215324 (10.148.2.76:54570) to group SiNOPCluster_CLUSTER_AITSAT_CE_14
2022-09-14 18:35:15 DEBUG added SiNOP-ONE_PC215475 (10.148.58.108:50129) to group SiNOPCluster_CLUSTER_AIT_CUO_14
2022-09-14 18:35:17 DEBUG added SiNOP-ONE_PC215407 (10.148.1.35:63280) to group SiNOPCluster_CLUSTER_DIMT_14
2022-09-14 18:35:19 DEBUG added SiNOP-ONE_PC212327 (10.148.2.36:59226) to group SiNOPCluster_CLUSTER_AITSAT_CE_14
2022-09-14 18:35:55 DEBUG added SiNOP-SERVICE_CANOPP01 (10.149.135.6:54350) to group SiNOPCluster_CLUSTER_AIT_CUO_14
2022-09-14 18:36:05 DEBUG added SiNOP-SERVICE_CANOPP01 (10.149.135.6:54352) to group SiNOPCluster_CLUSTER_AITSAT_CE_14
2022-09-14 18:36:06 DEBUG added SiNOP-SERVICE_CANOPP01 (10.149.135.6:54353) to group SiNOPCluster_CLUSTER_DIMT_14
2022-09-14 18:36:08 DEBUG added SiNOP-SERVICE_CANOPP01 (10.149.135.6:54361) to group SiNOPCluster_CLUSTER_TEST_14
2022-09-14 18:36:34 WARN dest 589353e1-264a-47be-8683-9b51470568f2 in cluster SiNOPCluster_CLUSTER_DIMT_14 not found
2022-09-14 18:36:35 WARN dest 589353e1-264a-47be-8683-9b51470568f2 in cluster SiNOPCluster_CLUSTER_DIMT_14 not found
2022-09-14 18:36:35 WARN dest 589353e1-264a-47be-8683-9b51470568f2 in cluster SiNOPCluster_CLUSTER_DIMT_14 not found
2022-09-14 18:36:38 WARN dest 589353e1-264a-47be-8683-9b51470568f2 in cluster SiNOPCluster_CLUSTER_DIMT_14 not found
2022-09-14 18:36:38 WARN dest 589353e1-264a-47be-8683-9b51470568f2 in cluster SiNOPCluster_CLUSTER_DIMT_14 not found
2022-09-14 18:36:48 WARN dest 589353e1-264a-47be-8683-9b51470568f2 in cluster SiNOPCluster_CLUSTER_DIMT_14 not found
2022-09-14 18:36:53 WARN dest 589353e1-264a-47be-8683-9b51470568f2 in cluster SiNOPCluster_CLUSTER_DIMT_14 not found
2022-09-14 18:36:53 WARN dest 589353e1-264a-47be-8683-9b51470568f2 in cluster SiNOPCluster_CLUSTER_DIMT_14 not found
2022-09-14 18:37:00 WARN dest 4fbafdf4-1215-4e05-9082-8fae98351a83 in cluster SiNOPCluster_CLUSTER_AITSAT_CE_14 not found
2022-09-14 18:37:00 WARN dest 4fbafdf4-1215-4e05-9082-8fae98351a83 in cluster SiNOPCluster_CLUSTER_AITSAT_CE_14 not found
2022-09-14 18:37:00 WARN dest ad0ee75f-7df0-480f-b3a1-dc44932e5334 in cluster SiNOPCluster_CLUSTER_AITSAT_CE_14 not found
2022-09-14 18:37:01 WARN dest 4fbafdf4-1215-4e05-9082-8fae98351a83 in cluster SiNOPCluster_CLUSTER_AITSAT_CE_14 not found
2022-09-14 18:37:01 WARN dest ad0ee75f-7df0-480f-b3a1-dc44932e5334 in cluster SiNOPCluster_CLUSTER_AITSAT_CE_14 not found

On that two first clusters, all app were restarted with the same tunnel configuration.

By analyzing these logs, It seems that one of the gossip router are receiving messages that he should not receive. Do you think that this configuration doesn't work properly and generate lot of MergeView issues, because they are located on the same host and they have the same channle name defined ?

Here it is the code to initialize the communication from the java app :

channel = new JChannel(gossipRouterConfigFilename); // tunnel.xml

src_name = this.sourceCmpName + "_" + host_name;
LOGGER.info("NETWORKBUS START : src_name = " + src_name);
channel.setName(src_name);
LOGGER.info("NETWORKBUS START : channel.getName() = " + channel.getName());

channel.setReceiver(this);

channel.setDiscardOwnMessages(true);

LOGGER.info("NETWORKBUS START : SiNOPCluster connect ... ");
channel.connect("SiNOPCluster_" + cluster_name);

String channelState = channel.getState();
LOGGER.info("NETWORKBUS START : " + channel.getClusterName() + " connected | State = " + channelState);
LOGGER.info("NETWORKBUS START : channel.getAddressAsString() = " + channel.getAddress());

Thanks in adavance for your help.

Fred

Frederic

unread,

Sep 15, 2022, 4:58:03 AM9/15/22

to jgroups-dev

Hi Bela,

I did a new test on a single workstation. I ran only one tunnel/gossip-router one one port number. I ran 4 diferent clustersand 3 clients.

The logs of the router seems to be good :

2022-09-15 09:33:23 DEBUG set property org.jgroups.stack....@65f87a2c.mcast_addr to default value /224.0.75.75
2022-09-15 09:34:03 DEBUG added PROD_CLUSTER_AITSAT_CE_SiNOP-SERVICE_DVOSDBN011 (10.149.40.125:53248) to group SiNOPCluster_PROD_CLUSTER_AITSAT_CE
2022-09-15 09:34:14 DEBUG added PROD_CLUSTER_DIMT_SiNOP-SERVICE_DVOSDBN011 (10.149.40.125:61165) to group SiNOPCluster_PROD_CLUSTER_DIMT
2022-09-15 09:34:22 DEBUG added PROD_CLUSTER_MGS_SiNOP-SERVICE_DVOSDBN011 (10.149.40.125:57504) to group SiNOPCluster_PROD_CLUSTER_MGS
2022-09-15 09:34:51 DEBUG added PROD_CLUSTER_DIMT_SiNOP-ONE_DVOSDBN011 (10.149.40.125:58804) to group SiNOPCluster_PROD_CLUSTER_DIMT
2022-09-15 09:35:04 DEBUG added PROD_CLUSTER_AITSAT_CE_SiNOP-ONE_DVOSDBN011 (10.149.40.125:58520) to group SiNOPCluster_PROD_CLUSTER_AITSAT_CE

2022-09-15 09:35:21 DEBUG added PROD_CLUSTER_MGS_SiNOP-ONE_DVOSDBN011 (10.149.40.125:52595) to group SiNOPCluster_PROD_CLUSTER_MGS

But communication between " PROD_CLUSTER_DIMT_SiNOP-ONE_DVOSDBN011" and "PROD_CLUSTER_DIMT_SiNOP-SERVICE_DVOSDBN011" is not established however they are connected to the same group " SiNOPCluster_PROD_CLUSTER_DIMT"

In one hand, the client logs " PROD_CLUSTER_DIMT_SiNOP-ONE_DVOSDBN011" :

initialHosts=${jgroups.tunnel.gossip_router_hosts:DVOSDBN011[15502]}
15-09-2022 09:34:40.088 [JavaFX-Launcher] INFO [c.t.s.s.s.i.f.NetworkBus][start] NETWORKBUS START : gossipRouterConfigFilename = tunnel_PROD_CLUSTER_DIMT.xml
15-09-2022 09:34:40.088 [JavaFX-Launcher] INFO [c.t.s.s.s.i.f.NetworkBus][start] NETWORKBUS START : create JChannel ...
15-09-2022 09:34:41.093 [JavaFX-Launcher] INFO [c.t.s.s.s.i.f.NetworkBus][start] NETWORKBUS START : src_name = PROD_CLUSTER_DIMT_SiNOP-ONE_DVOSDBN011
15-09-2022 09:34:41.094 [JavaFX-Launcher] INFO [c.t.s.s.s.i.f.NetworkBus][start] NETWORKBUS START : channel.getName() = PROD_CLUSTER_DIMT_SiNOP-ONE_DVOSDBN011
15-09-2022 09:34:41.094 [JavaFX-Launcher] INFO [c.t.s.s.s.i.f.NetworkBus][start] NETWORKBUS START : SiNOPCluster connect ...
-------------------------------------------------------------------
GMS: address=PROD_CLUSTER_DIMT_SiNOP-ONE_DVOSDBN011, cluster=SiNOPCluster_PROD_CLUSTER_DIMT, physical address=10.149.40.125:58804
-------------------------------------------------------------------
15-09-2022 09:34:53.175 [JavaFX-Launcher] INFO [c.t.s.s.s.i.f.NetworkBus][viewAccepted] ** view: [PROD_CLUSTER_DIMT_SiNOP-ONE_DVOSDBN011|0] (1) [PROD_CLUSTER_DIMT_SiNOP-ONE_DVOSDBN011]
15-09-2022 09:34:53.175 [JavaFX-Launcher] DEBUG [c.t.s.s.s.i.f.NetworkBus][viewAccepted] SiNOP-SERVICE found ? false
15-09-2022 09:34:53.187 [JavaFX-Launcher] INFO [c.t.s.s.s.i.f.NetworkBus][start] NETWORKBUS START : SiNOPCluster_PROD_CLUSTER_DIMT connected | State = CONNECTED
15-09-2022 09:34:53.189 [JavaFX-Launcher] INFO [c.t.s.s.s.i.f.NetworkBus][start] NETWORKBUS START : channel.getAddressAsString() = PROD_CLUSTER_DIMT_SiNOP-ONE_DVOSDBN011
...
15-09-2022 09:35:49.815 [jgroups-8,SiNOPCluster_PROD_CLUSTER_DIMT,PROD_CLUSTER_DIMT_SiNOP-ONE_DVOSDBN011] INFO [c.t.s.s.s.i.f.NetworkBus][viewAccepted] ** view: MergeView::[PROD_CLUSTER_DIMT_SiNOP-ONE_DVOSDBN011|1] (1) [PROD_CLUSTER_DIMT_SiNOP-ONE_DVOSDBN011], 1 subgroups: [PROD_CLUSTER_DIMT_SiNOP-ONE_DVOSDBN011|0] (1) [PROD_CLUSTER_DIMT_SiNOP-ONE_DVOSDBN011]
15-09-2022 09:35:49.816 [jgroups-8,SiNOPCluster_PROD_CLUSTER_DIMT,PROD_CLUSTER_DIMT_SiNOP-ONE_DVOSDBN011] DEBUG [c.t.s.s.s.i.f.NetworkBus][viewAccepted] SiNOP-SERVICE found ? false
15-09-2022 09:35:49.816 [jgroups-8,SiNOPCluster_PROD_CLUSTER_DIMT,PROD_CLUSTER_DIMT_SiNOP-ONE_DVOSDBN011] DEBUG [c.t.s.s.s.i.f.NetworkBus][viewAccepted] PROD_CLUSTER_DIMT_SiNOP-ONE_DVOSDBN011 is member of the groups belonging to the view ? true
15-09-2022 09:35:49.816 [jgroups-8,SiNOPCluster_PROD_CLUSTER_DIMT,PROD_CLUSTER_DIMT_SiNOP-ONE_DVOSDBN011] DEBUG [c.t.s.s.s.i.f.NetworkBus][viewAccepted] Source PROD_CLUSTER_DIMT_SiNOP-ONE_DVOSDBN011 is unique member ? true (nb_members=1)
15-09-2022 09:35:49.816 [jgroups-8,SiNOPCluster_PROD_CLUSTER_DIMT,PROD_CLUSTER_DIMT_SiNOP-ONE_DVOSDBN011] DEBUG [c.t.s.s.s.i.f.NetworkBus][viewAccepted] PROD_CLUSTER_DIMT_SiNOP-ONE_DVOSDBN011 is the coordinator of this view ? true

15-09-2022 10:35:56.810 [jgroups-100,SiNOPCluster_PROD_CLUSTER_DIMT,PROD_CLUSTER_DIMT_SiNOP-ONE_DVOSDBN011] INFO [c.t.s.s.s.i.f.NetworkBus][viewAccepted] ** view: MergeView::[PROD_CLUSTER_DIMT_SiNOP-ONE_DVOSDBN011|14] (1) [PROD_CLUSTER_DIMT_SiNOP-ONE_DVOSDBN011], 1 subgroups: [PROD_CLUSTER_DIMT_SiNOP-ONE_DVOSDBN011|13] (1) [PROD_CLUSTER_DIMT_SiNOP-ONE_DVOSDBN011]
15-09-2022 10:35:56.811 [jgroups-100,SiNOPCluster_PROD_CLUSTER_DIMT,PROD_CLUSTER_DIMT_SiNOP-ONE_DVOSDBN011] DEBUG [c.t.s.s.s.i.f.NetworkBus][viewAccepted] SiNOP-SERVICE found ? false
15-09-2022 10:35:56.811 [jgroups-100,SiNOPCluster_PROD_CLUSTER_DIMT,PROD_CLUSTER_DIMT_SiNOP-ONE_DVOSDBN011] DEBUG [c.t.s.s.s.i.f.NetworkBus][viewAccepted] PROD_CLUSTER_DIMT_SiNOP-ONE_DVOSDBN011 is member of the groups belonging to the view ? true
15-09-2022 10:35:56.811 [jgroups-100,SiNOPCluster_PROD_CLUSTER_DIMT,PROD_CLUSTER_DIMT_SiNOP-ONE_DVOSDBN011] DEBUG [c.t.s.s.s.i.f.NetworkBus][viewAccepted] Source PROD_CLUSTER_DIMT_SiNOP-ONE_DVOSDBN011 is unique member ? true (nb_members=1)
15-09-2022 10:35:56.811 [jgroups-100,SiNOPCluster_PROD_CLUSTER_DIMT,PROD_CLUSTER_DIMT_SiNOP-ONE_DVOSDBN011] DEBUG [c.t.s.s.s.i.f.NetworkBus][viewAccepted] PROD_CLUSTER_DIMT_SiNOP-ONE_DVOSDBN011 is the coordinator of this view ? true

15-09-2022 10:38:57.293 [jgroups-105,SiNOPCluster_PROD_CLUSTER_DIMT,PROD_CLUSTER_DIMT_SiNOP-ONE_DVOSDBN011] INFO [c.t.s.s.s.i.f.NetworkBus][viewAccepted] ** view: MergeView::[PROD_CLUSTER_DIMT_SiNOP-ONE_DVOSDBN011|15] (1) [PROD_CLUSTER_DIMT_SiNOP-ONE_DVOSDBN011], 1 subgroups: [PROD_CLUSTER_DIMT_SiNOP-ONE_DVOSDBN011|14] (1) [PROD_CLUSTER_DIMT_SiNOP-ONE_DVOSDBN011]
15-09-2022 10:38:57.293 [jgroups-105,SiNOPCluster_PROD_CLUSTER_DIMT,PROD_CLUSTER_DIMT_SiNOP-ONE_DVOSDBN011] DEBUG [c.t.s.s.s.i.f.NetworkBus][viewAccepted] SiNOP-SERVICE found ? false
15-09-2022 10:38:57.293 [jgroups-105,SiNOPCluster_PROD_CLUSTER_DIMT,PROD_CLUSTER_DIMT_SiNOP-ONE_DVOSDBN011] DEBUG [c.t.s.s.s.i.f.NetworkBus][viewAccepted] PROD_CLUSTER_DIMT_SiNOP-ONE_DVOSDBN011 is member of the groups belonging to the view ? true
15-09-2022 10:38:57.293 [jgroups-105,SiNOPCluster_PROD_CLUSTER_DIMT,PROD_CLUSTER_DIMT_SiNOP-ONE_DVOSDBN011] DEBUG [c.t.s.s.s.i.f.NetworkBus][viewAccepted] Source PROD_CLUSTER_DIMT_SiNOP-ONE_DVOSDBN011 is unique member ? true (nb_members=1)
15-09-2022 10:38:57.293 [jgroups-105,SiNOPCluster_PROD_CLUSTER_DIMT,PROD_CLUSTER_DIMT_SiNOP-ONE_DVOSDBN011] DEBUG [c.t.s.s.s.i.f.NetworkBus][viewAccepted] PROD_CLUSTER_DIMT_SiNOP-ONE_DVOSDBN011 is the coordinator of this view ? true

Message SENT to SiNOP Service, but SiNOP Service is not present into the view member list

In another hand, the server logs "PROD_CLUSTER_DIMT_SiNOP-SERVICE_DVOSDBN011" :

initialHosts=${jgroups.tunnel.gossip_router_hosts:DVOSDBN011[15502]}
15-09-2022 09:34:03.671 [main] INFO [c.t.s.s.s.i.f.NetworkBus][start] NETWORKBUS START : gossipRouterConfigFilename = tunnel_PROD_CLUSTER_DIMT.xml
15-09-2022 09:34:03.672 [main] INFO [c.t.s.s.s.i.f.NetworkBus][start] NETWORKBUS START : create JChannel ...
15-09-2022 09:34:04.488 [main] INFO [c.t.s.s.s.i.f.NetworkBus][start] NETWORKBUS START : src_name = PROD_CLUSTER_DIMT_SiNOP-SERVICE_DVOSDBN011
15-09-2022 09:34:04.488 [main] INFO [c.t.s.s.s.i.f.NetworkBus][start] NETWORKBUS START : channel.getName() = PROD_CLUSTER_DIMT_SiNOP-SERVICE_DVOSDBN011
15-09-2022 09:34:04.488 [main] INFO [c.t.s.s.s.i.f.NetworkBus][start] NETWORKBUS START : SiNOPCluster connect ...
-------------------------------------------------------------------
GMS: address=PROD_CLUSTER_DIMT_SiNOP-SERVICE_DVOSDBN011, cluster=SiNOPCluster_PROD_CLUSTER_DIMT, physical address=10.149.40.125:61165
-------------------------------------------------------------------
15-09-2022 09:34:16.592 [main] INFO [c.t.s.s.s.i.f.NetworkBus][viewAccepted] ** view: [PROD_CLUSTER_DIMT_SiNOP-SERVICE_DVOSDBN011|0] (1) [PROD_CLUSTER_DIMT_SiNOP-SERVICE_DVOSDBN011]
15-09-2022 09:34:16.593 [main] DEBUG [c.t.s.s.s.i.f.NetworkBus][viewAccepted] SiNOP-SERVICE found ? true
15-09-2022 09:34:16.593 [main] DEBUG [c.t.s.s.s.i.f.NetworkBus][viewAccepted] PROD_CLUSTER_DIMT_SiNOP-SERVICE_DVOSDBN011 is member of the groups belonging to the view ? true
15-09-2022 09:34:16.593 [main] DEBUG [c.t.s.s.s.i.f.NetworkBus][viewAccepted] Source PROD_CLUSTER_DIMT_SiNOP-SERVICE_DVOSDBN011 is unique member ? true (nb_members=1)
15-09-2022 09:34:16.593 [main] DEBUG [c.t.s.s.s.i.f.NetworkBus][viewAccepted] PROD_CLUSTER_DIMT_SiNOP-SERVICE_DVOSDBN011 is the coordinator of this view ? true
15-09-2022 09:34:16.602 [main] INFO [c.t.s.s.s.i.f.NetworkBus][start] NETWORKBUS START : SiNOPCluster_PROD_CLUSTER_DIMT connected | State = CONNECTED
15-09-2022 09:34:16.602 [main] INFO [c.t.s.s.s.i.f.NetworkBus][start] NETWORKBUS START : channel.getAddressAsString() = PROD_CLUSTER_DIMT_SiNOP-SERVICE_DVOSDBN011

Message sent by " PROD_CLUSTER_DIMT_SiNOP-ONE_DVOSDBN011" were never received by "PROD_CLUSTER_DIMT_SiNOP-SERVICE_DVOSDBN011".

That's the main point of my problem. I don't understand why In the view or the merge view the destinator desappear or never appear.

Fred

Frederic

unread,

Sep 15, 2022, 5:08:25 AM9/15/22

to jgroups-dev

Bela,

On the other platform, I got lots of this king of message in the router logs :

2022-09-15 11:04:34 WARN dest 26224fde-e13a-48c5-8a59-187ce1273ab1 in cluster SiNOPCluster_CLUSTER_DIMT_14 not found
2022-09-15 11:04:36 WARN dest bf27e19e-724e-48ee-881e-4f4e88088269 in cluster SiNOPCluster_CLUSTER_DIMT_14 not found
2022-09-15 11:04:39 WARN dest bf27e19e-724e-48ee-881e-4f4e88088269 in cluster SiNOPCluster_CLUSTER_DIMT_14 not found
2022-09-15 11:04:39 WARN dest bf27e19e-724e-48ee-881e-4f4e88088269 in cluster SiNOPCluster_CLUSTER_DIMT_14 not found
2022-09-15 11:04:42 WARN dest 7ae2cd5b-a340-4537-bac0-02294d42fca4 in cluster SiNOPCluster_CLUSTER_DIMT_14 not found

Fred

Bela Ban

unread,

Sep 19, 2022, 10:40:22 AM9/19/22

to jgrou...@googlegroups.com

Why do you use initialHosts? Do you still have TCPGOSSIP configured?
That would be wrong; you need to have TUNNEL:PING, *not* TUNNEL:TCPGOSSIP!

The GossipRouter maintains a two-level hashmap of cluster names and
hashmaps, which cache the addresses of members (as keys) and IP
address:port values.

When a message is sent, the destination cluster is looked up and then
the GossipRouter forwards the message to the IP address:port of the
given destination, or to all IP addresses:ports if the destination is
null ('multicast message').

I suggest start only a single GossipRouter to which *all* members
connect. The discriminator is still the cluster name and the destination
address.

Of course, multiple routers can be started; clients will connect and
register themselves with all of them, and then load-balance their
traffic for redundancy purposes.

On 14.09.22 19:13, Frederic wrote:
> Hi Bela,
>
> Here is the complete picture of the architecture :
> - in one network side : one server with 6 gossip router running on
> different port number + one server app
>

> * 1st cluster :

> cluster_name = CLUSTER_DIMT_14
> initialHosts=${jgroups.tunnel.gossip_router_hosts:CANOPP01[15502]}
> channel.getName() = SiNOP-SERVICE_CANOPP01
> <TUNNEL
> gossip_router_hosts="${jgroups.tunnel.gossip_router_hosts:CANOPP01[15502]}"/>

> * 2nd cluster :

> cluster_name = CLUSTER_AITSAT_CE_14
> initialHosts=${jgroups.tunnel.gossip_router_hosts:CANOPP01[15505]}
> channel.getName() = SiNOP-SERVICE_CANOPP01
> <TUNNEL
> gossip_router_hosts="${jgroups.tunnel.gossip_router_hosts:CANOPP01[15505]}"/>

> * 3rd ... etc.

>
> - in another network side separeted by a firewall : several app
> clients on several computers
> The communication is established between all of them (client-to-server,
> server-to-client, client-to-client) thanks to tunnel configuration +
> gossip router
>
> Each Gossip Router has its own logs. We have into the logs on a specific
> router logs related to other clusters. example :
>

> * logs of gossip router : CLUSTER_DIMT_14

> * logs of gossip router : CLUSTER_AITSAT_CE_14

> https://github.com/belaban/JGroups/blob/jgroups-5.2.6.Final/conf/tunnel.xml <https://github.com/belaban/JGroups/blob/jgroups-5.2.6.Final/conf/tunnel.xml> <https://github.com/belaban/JGroups/blob/jgroups-5.2.6.Final/conf/tunnel.xml <https://github.com/belaban/JGroups/blob/jgroups-5.2.6.Final/conf/tunnel.xml>>

> >
> > --
> > Bela Ban | http://www.jgroups.org <http://www.jgroups.org>
> <http://www.jgroups.org <http://www.jgroups.org>>
> >
> > --
> > You received this message because you are subscribed to the Google
> > Groups "jgroups-dev" group.
> > To unsubscribe from this group and stop receiving emails from it,
> send
> > an email to jgroups-dev...@googlegroups.com
> > <mailto:jgroups-dev...@googlegroups.com>.
> > To view this discussion on the web visit
> >

> https://groups.google.com/d/msgid/jgroups-dev/17e068a7-f384-4b73-bcc5-eefb3e247830n%40googlegroups.com <https://groups.google.com/d/msgid/jgroups-dev/17e068a7-f384-4b73-bcc5-eefb3e247830n%40googlegroups.com> <https://groups.google.com/d/msgid/jgroups-dev/17e068a7-f384-4b73-bcc5-eefb3e247830n%40googlegroups.com?utm_medium=email&utm_source=footer <https://groups.google.com/d/msgid/jgroups-dev/17e068a7-f384-4b73-bcc5-eefb3e247830n%40googlegroups.com?utm_medium=email&utm_source=footer>>.
>
> --
> Bela Ban | http://www.jgroups.org <http://www.jgroups.org>

>
> --
> You received this message because you are subscribed to the Google
> Groups "jgroups-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to jgroups-dev...@googlegroups.com
> <mailto:jgroups-dev...@googlegroups.com>.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/jgroups-dev/237dd78a-31bc-4420-8e2c-6c58b4bf3797n%40googlegroups.com <https://groups.google.com/d/msgid/jgroups-dev/237dd78a-31bc-4420-8e2c-6c58b4bf3797n%40googlegroups.com?utm_medium=email&utm_source=footer>.

Bela Ban

unread,

Sep 19, 2022, 11:05:31 AM9/19/22

to jgrou...@googlegroups.com

Can you create a small test that reproduces the issue? If some members
can communicate with each other, and other can't, then I suspect a
firewall issue...

Also useful: run GossipRouter with option '-dump_msgs all'. This will
dump registrations and all messages that are routed.

Also note that you can use RouterStubGet to dump the registered members:
RouterStubGet [-host <host>] [-port <port>] [-cluster <cluster name
(default: draw>] [-nio true|false]

> --
> You received this message because you are subscribed to the Google
> Groups "jgroups-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to jgroups-dev...@googlegroups.com
> <mailto:jgroups-dev...@googlegroups.com>.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/jgroups-dev/e72044b7-ed01-45fa-8afd-53c93d51d613n%40googlegroups.com <https://groups.google.com/d/msgid/jgroups-dev/e72044b7-ed01-45fa-8afd-53c93d51d613n%40googlegroups.com?utm_medium=email&utm_source=footer>.

Bela Ban

unread,

Sep 19, 2022, 11:06:41 AM9/19/22

to jgrou...@googlegroups.com

So they're not registered; use RouterStubGet to dump the registrations.

> --
> You received this message because you are subscribed to the Google
> Groups "jgroups-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to jgroups-dev...@googlegroups.com
> <mailto:jgroups-dev...@googlegroups.com>.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/jgroups-dev/415d7b8d-d334-49c4-9cac-68e363568eban%40googlegroups.com <https://groups.google.com/d/msgid/jgroups-dev/415d7b8d-d334-49c4-9cac-68e363568eban%40googlegroups.com?utm_medium=email&utm_source=footer>.

Frederic Monclar

unread,

Sep 20, 2022, 11:50:08 AM9/20/22

to Bela Ban, jgrou...@googlegroups.com

Hi Bela,

I answers to yours questions above :

Why do you use initialHosts? Do you still have TCPGOSSIP configured? That would be wrong; you need to have TUNNEL:PING, *not* TUNNEL:TCPGOSSIP!

Fred : I don't use it, sorry, it is wrong log message. Here is the tunnel.xml configuration I use.

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<config xmlns:ns1="relay:1.0" xmlns="jgroups" xmlns:ns3="fork" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="jgroups xsd/jgroups.xsd">
<TUNNEL gossip_router_hosts="${jgroups.tunnel.gossip_router_hosts:CANOPP01[15504]}"/>
<PING/>
<MERGE3/>
<VERIFY_SUSPECT/>
<pbcast.NAKACK2 use_mcast_xmit="false"/>
<UNICAST3/>
<pbcast.STABLE/>
<pbcast.GMS/>
<UFC/>
<MFC/>
<FRAG2/>
<pbcast.STATE_TRANSFER/>
</config>

I suggest start only a single GossipRouter to which *all* members connect. The discriminator is still the cluster name and the destination address.

Fred :I'm going to do the test you suggest

Of course, multiple routers can be started; clients will connect and register themselves with all of them,

Fred : Do you mean that if a client is registered a router via one port number it will be registered to all other routers which are running on the same machine ? That would explain why the log "added member to group" is present in all router's logs for the same added member

and then load-balance their traffic for redundancy purposes.

Fred : Does the load balancing mechanism is running by default with several routers on the same VM ?

Can you create a small test that reproduces the issue? If some members can communicate with each other, and other can't, then I suspect a firewall issue...

Fred : communications between clients located on the same networks are working well, but communication with client that are not on the same give me lot of problems. The communication between the client which is on the same network that the router and several clients which are on the other network makes problems.

Also useful: run GossipRouter with option '-dump_msgs all'. This will dump registrations and all messages that are routed.

Fred : ok, I will do it

use RouterStubGet to dump the registrations.

Fred : ok, I'll do it

Fred

You received this message because you are subscribed to a topic in the Google Groups "jgroups-dev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jgroups-dev/UttnXiEUnZg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jgroups-dev...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jgroups-dev/cc3cc831-8542-7579-1fed-ecc9d409f770%40mailbox.org.

Bela Ban

unread,

Sep 20, 2022, 1:55:31 PM9/20/22

to Frederic Monclar, jgrou...@googlegroups.com

On 20.09.22 17:49, Frederic Monclar wrote:
> Hi Bela,

> * I suggest start only a single GossipRouter to which *all* members

> connect. The discriminator is still the cluster name and the
> destination address.

> o _Fred _:I'm going to do the test you suggest

OK

> * Of course, multiple routers can be started; clients will connect and

> register themselves with all of them,

> o _Fred _: Do you mean that if a client is registered a router via

> one port number it will be registered to all other routers which
> are running on the same machine?

No; what I meant is that you can run GossipRouters on multiple hosts and
list them all in TUNNEL.gossip_router_hosts

> That would explain why the log
> "added member to group" is present in all router's logs for the
> same added member

Perhaps you started routers on adjacent ports (e.g. 12001, 12002,
12003)? In that case, a TUNNEL.port_range > 1 would have caused clients
to register with multiple GossipRouters inadvertently.

However, running multiple GossipRouters on the same host makes no sense,
as redundancy is not provided when the host is down.

> * and then load-balance their traffic for redundancy purposes.
> o _Fred _: Does the load balancing mechanism is running by default

> with several routers on the same VM ?

Yes, by default: registration is with all GossipRouters, and traffic is
routed through one (randonly selected).

> * Can you create a small test that reproduces the issue? If some

> members can communicate with each other, and other can't, then I
> suspect a firewall issue...

> o _Fred _: communications between clients located on the same

> networks are working well, but communication with client that
> are not on the same give me lot of problems.

Perhaps turn off any firewalls, to see if this is the culprit?

> The communication
> between the client which is on the same network that the router
> and several clients which are on the other network makes problems.

This points to a network problem...

> * Also useful: run GossipRouter with option '-dump_msgs all'. This

> will dump registrations and all messages that are routed.

> o _Fred _: ok, I will do it
>
> * use RouterStubGet to dump the registrations.
> o Fred : ok, I'll do it

>
> Fred
>
> Le lun. 19 sept. 2022 à 17:06, 'Bela Ban' via jgroups-dev

> <jgrou...@googlegroups.com <mailto:jgrou...@googlegroups.com>> a

> <mailto:jgroups-dev%2Bunsu...@googlegroups.com>
> > <mailto:jgroups-dev...@googlegroups.com
> <mailto:jgroups-dev%2Bunsu...@googlegroups.com>>.

> > To view this discussion on the web visit
> >

> https://groups.google.com/d/msgid/jgroups-dev/415d7b8d-d334-49c4-9cac-68e363568eban%40googlegroups.com <https://groups.google.com/d/msgid/jgroups-dev/415d7b8d-d334-49c4-9cac-68e363568eban%40googlegroups.com> <https://groups.google.com/d/msgid/jgroups-dev/415d7b8d-d334-49c4-9cac-68e363568eban%40googlegroups.com?utm_medium=email&utm_source=footer <https://groups.google.com/d/msgid/jgroups-dev/415d7b8d-d334-49c4-9cac-68e363568eban%40googlegroups.com?utm_medium=email&utm_source=footer>>.
>
> --
> Bela Ban | http://www.jgroups.org <http://www.jgroups.org>

>
> --
> You received this message because you are subscribed to a topic in
> the Google Groups "jgroups-dev" group.
> To unsubscribe from this topic, visit

> https://groups.google.com/d/topic/jgroups-dev/UttnXiEUnZg/unsubscribe <https://groups.google.com/d/topic/jgroups-dev/UttnXiEUnZg/unsubscribe>.

> To unsubscribe from this group and all its topics, send an email to
> jgroups-dev...@googlegroups.com

> <mailto:jgroups-dev%2Bunsu...@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/jgroups-dev/cc3cc831-8542-7579-1fed-ecc9d409f770%40mailbox.org <https://groups.google.com/d/msgid/jgroups-dev/cc3cc831-8542-7579-1fed-ecc9d409f770%40mailbox.org>.

Frederic

unread,

Sep 20, 2022, 2:13:50 PM9/20/22

to jgroups-dev

Hi Bela,

Thanks for your answers.

I will post the results of the new tetsts soon.

Just one point taht concerned : RouterStubGet to dump the registration. On my local vm I got the list of members, but on the server which I have communication problems, I'm getting not output at all.

What does it mean ?

Thanks,

Fred

Bela Ban

unread,

Sep 21, 2022, 1:00:30 AM9/21/22

to jgrou...@googlegroups.com

On 20.09.22 20:13, Frederic wrote:
> Hi Bela,
> Thanks for your answers.
> I will post the results of the new tetsts soon.
>
> Just one point taht concerned : RouterStubGet to dump the registration.
> On my local vm I got the list of members, but on the server which I have
> communication problems, I'm getting not output at all.

Can you connect at all, or is there a communication problem? Do you pass
the correct cluster name (-cluster)?

jt RouterStubGet -h

RouterStubGet [-host <host>] [-port <port>] [-cluster <cluster name
(default: draw>] [-nio true|false]

> >
> > --
> > Bela Ban | http://www.jgroups.org <http://www.jgroups.org>
> <http://www.jgroups.org <http://www.jgroups.org>>
> >
> > --
> > You received this message because you are subscribed to a topic in
> > the Google Groups "jgroups-dev" group.
> > To unsubscribe from this topic, visit
> >

> https://groups.google.com/d/topic/jgroups-dev/UttnXiEUnZg/unsubscribe <https://groups.google.com/d/topic/jgroups-dev/UttnXiEUnZg/unsubscribe> <https://groups.google.com/d/topic/jgroups-dev/UttnXiEUnZg/unsubscribe <https://groups.google.com/d/topic/jgroups-dev/UttnXiEUnZg/unsubscribe>>.

> > To unsubscribe from this group and all its topics, send an email to
> > jgroups-dev...@googlegroups.com
> > <mailto:jgroups-dev%2Bunsu...@googlegroups.com>.
> > To view this discussion on the web visit
> >

> https://groups.google.com/d/msgid/jgroups-dev/cc3cc831-8542-7579-1fed-ecc9d409f770%40mailbox.org <https://groups.google.com/d/msgid/jgroups-dev/cc3cc831-8542-7579-1fed-ecc9d409f770%40mailbox.org> <https://groups.google.com/d/msgid/jgroups-dev/cc3cc831-8542-7579-1fed-ecc9d409f770%40mailbox.org <https://groups.google.com/d/msgid/jgroups-dev/cc3cc831-8542-7579-1fed-ecc9d409f770%40mailbox.org>>.
> >
>
> --

> Bela Ban | http://www.jgroups.org <http://www.jgroups.org>
>
> --

> You received this message because you are subscribed to the Google
> Groups "jgroups-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to jgroups-dev...@googlegroups.com

> <mailto:jgroups-dev...@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/jgroups-dev/1dc5b4ce-3b48-424a-9e92-605dfb34205an%40googlegroups.com <https://groups.google.com/d/msgid/jgroups-dev/1dc5b4ce-3b48-424a-9e92-605dfb34205an%40googlegroups.com?utm_medium=email&utm_source=footer>.

Frederic

unread,

Sep 21, 2022, 2:46:54 AM9/21/22

to jgroups-dev

Hi Bela,

I made a mistake with the clustername. The command is working fine.I got the list of the members.

Now I run only one GossipRouter with tunnel configuration with several clusters, but I still lost communications between hosts. So I try to get and to check the firewall configuration.

Thanks,

Bela Ban

unread,

Sep 21, 2022, 4:42:10 AM9/21/22

to jgrou...@googlegroups.com

On 21.09.22 08:46, Frederic wrote:
> Hi Bela,
>
> I made a mistake with the clustername. The command is working fine.I got
> the list of the members.
>
> Now I run only one GossipRouter with tunnel configuration with several
> clusters, but I still lost communications between hosts.

Right, try to see if a connection is established at all.

I suggest use netcat ('nc') to try to connect. The connection should be
established ok, but then it should be closed immediately by the
GossipRouter, as the (application-level) handshake will fail.

If the connection is never established, then this points to a firewall
issue.

Running multiple GossipRouters is a good idea if you want redundancy for
communication between members. However, I suggest fix the current issue
first, and only then increase the # of GRs.

> > >
> > > --
> > > Bela Ban | http://www.jgroups.org <http://www.jgroups.org>
> <http://www.jgroups.org <http://www.jgroups.org>>
> > <http://www.jgroups.org <http://www.jgroups.org>
> <http://www.jgroups.org <http://www.jgroups.org>>>
> > >
> > > --
> > > You received this message because you are subscribed to a topic in
> > > the Google Groups "jgroups-dev" group.
> > > To unsubscribe from this topic, visit
> > >
> >

> > > To unsubscribe from this group and all its topics, send an
> email to
> > > jgroups-dev...@googlegroups.com
> > > <mailto:jgroups-dev%2Bunsu...@googlegroups.com>.
> > > To view this discussion on the web visit
> > >
> >

> > >
> >
> > --
> > Bela Ban | http://www.jgroups.org <http://www.jgroups.org>
> <http://www.jgroups.org <http://www.jgroups.org>>
> >
> > --
> > You received this message because you are subscribed to the Google
> > Groups "jgroups-dev" group.
> > To unsubscribe from this group and stop receiving emails from it,
> send
> > an email to jgroups-dev...@googlegroups.com
> > <mailto:jgroups-dev...@googlegroups.com>.
> > To view this discussion on the web visit
> >

> https://groups.google.com/d/msgid/jgroups-dev/1dc5b4ce-3b48-424a-9e92-605dfb34205an%40googlegroups.com <https://groups.google.com/d/msgid/jgroups-dev/1dc5b4ce-3b48-424a-9e92-605dfb34205an%40googlegroups.com> <https://groups.google.com/d/msgid/jgroups-dev/1dc5b4ce-3b48-424a-9e92-605dfb34205an%40googlegroups.com?utm_medium=email&utm_source=footer <https://groups.google.com/d/msgid/jgroups-dev/1dc5b4ce-3b48-424a-9e92-605dfb34205an%40googlegroups.com?utm_medium=email&utm_source=footer>>.
>
> --

> Bela Ban | http://www.jgroups.org <http://www.jgroups.org>
>
> --
> You received this message because you are subscribed to the Google
> Groups "jgroups-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to jgroups-dev...@googlegroups.com
> <mailto:jgroups-dev...@googlegroups.com>.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/jgroups-dev/8da4349f-1718-41f8-84e0-7cb663b080den%40googlegroups.com <https://groups.google.com/d/msgid/jgroups-dev/8da4349f-1718-41f8-84e0-7cb663b080den%40googlegroups.com?utm_medium=email&utm_source=footer>.

Frederic

unread,

Sep 21, 2022, 6:43:45 AM9/21/22

to jgroups-dev

Hi Bela,

suggest use netcat ('nc') to try to connect. The connection should be established ok, but then it should be closed immediately by the GossipRouter, as the (application-level) handshake will fail.

- I tried from one PC that is located on a differnt network (with firewall between) and from another VM that is on the same network. I have no result. I mean that the program is not exited and the output is empty. I didn't find any log on the gossip router logs.

Fred

Bela Ban

unread,

Sep 21, 2022, 9:10:13 AM9/21/22

to jgrou...@googlegroups.com

[belasmacmini] /Users/bela$ nc -v localhost 12001
Connection to localhost port 12001 [tcp/entextnetwk] succeeded!
hello

GR:
15:09:46,417 [WARN] TcpServer: failed handling message
java.lang.ArrayIndexOutOfBoundsException: Index 111 out of bounds for
length 7
at org.jgroups.stack.GossipRouter.receive(GossipRouter.java:327)
~[classes/:?]
at org.jgroups.blocks.cs.BaseServer.receive(BaseServer.java:204)
~[classes/:?]
at
org.jgroups.blocks.cs.TcpConnection$Receiver.run(TcpConnection.java:307)
~[classes/:?]
at java.lang.Thread.run(Thread.java:834) ~[?:?]

On 21.09.22 12:43, Frederic wrote:
> Hi Bela,
>
> * suggest use netcat ('nc') to try to connect. The connection should

> be established ok, but then it should be closed immediately by the
> GossipRouter, as the (application-level) handshake will fail.

> o I tried from one PC that is located on a differnt network (with

> > > >
> > > > --
> > > > Bela Ban | http://www.jgroups.org <http://www.jgroups.org>
> <http://www.jgroups.org <http://www.jgroups.org>>
> > <http://www.jgroups.org <http://www.jgroups.org>
> <http://www.jgroups.org <http://www.jgroups.org>>>
> > > <http://www.jgroups.org <http://www.jgroups.org>
> <http://www.jgroups.org <http://www.jgroups.org>>
> > <http://www.jgroups.org <http://www.jgroups.org>
> <http://www.jgroups.org <http://www.jgroups.org>>>>
> > > >
> > > > --
> > > > You received this message because you are subscribed to a
> topic in
> > > > the Google Groups "jgroups-dev" group.
> > > > To unsubscribe from this topic, visit
> > > >
> > >
> >

> > > > To unsubscribe from this group and all its topics, send an
> > email to
> > > > jgroups-dev...@googlegroups.com
> > > > <mailto:jgroups-dev%2Bunsu...@googlegroups.com>.
> > > > To view this discussion on the web visit
> > > >
> > >
> >

> > > >
> > >
> > > --
> > > Bela Ban | http://www.jgroups.org <http://www.jgroups.org>
> <http://www.jgroups.org <http://www.jgroups.org>>
> > <http://www.jgroups.org <http://www.jgroups.org>
> <http://www.jgroups.org <http://www.jgroups.org>>>
> > >
> > > --
> > > You received this message because you are subscribed to the Google
> > > Groups "jgroups-dev" group.
> > > To unsubscribe from this group and stop receiving emails from it,
> > send
> > > an email to jgroups-dev...@googlegroups.com
> > > <mailto:jgroups-dev...@googlegroups.com>.
> > > To view this discussion on the web visit
> > >
> >

> >
> > --
> > Bela Ban | http://www.jgroups.org <http://www.jgroups.org>
> <http://www.jgroups.org <http://www.jgroups.org>>
> >
> > --
> > You received this message because you are subscribed to the Google
> > Groups "jgroups-dev" group.
> > To unsubscribe from this group and stop receiving emails from it,
> send
> > an email to jgroups-dev...@googlegroups.com
> > <mailto:jgroups-dev...@googlegroups.com>.
> > To view this discussion on the web visit
> >

> https://groups.google.com/d/msgid/jgroups-dev/8da4349f-1718-41f8-84e0-7cb663b080den%40googlegroups.com <https://groups.google.com/d/msgid/jgroups-dev/8da4349f-1718-41f8-84e0-7cb663b080den%40googlegroups.com> <https://groups.google.com/d/msgid/jgroups-dev/8da4349f-1718-41f8-84e0-7cb663b080den%40googlegroups.com?utm_medium=email&utm_source=footer <https://groups.google.com/d/msgid/jgroups-dev/8da4349f-1718-41f8-84e0-7cb663b080den%40googlegroups.com?utm_medium=email&utm_source=footer>>.
>
> --
> Bela Ban | http://www.jgroups.org <http://www.jgroups.org>

>
> --
> You received this message because you are subscribed to the Google
> Groups "jgroups-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to jgroups-dev...@googlegroups.com
> <mailto:jgroups-dev...@googlegroups.com>.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/jgroups-dev/8169684c-d70d-4f46-84e1-f47ed318374en%40googlegroups.com <https://groups.google.com/d/msgid/jgroups-dev/8169684c-d70d-4f46-84e1-f47ed318374en%40googlegroups.com?utm_medium=email&utm_source=footer>.

Frederic

unread,

Sep 21, 2022, 1:16:34 PM9/21/22

to jgroups-dev

Hi Bela,

On the VM where gossip router is running, I ran the following command and I got the following result :

C:\windows\system32>ncat -v localhost 15200
Ncat: Version 7.93 ( https://nmap.org/ncat )
libnsock ssl_init_helper(): OpenSSL legacy provider failed to load.

Ncat: Connection to ::1 failed: No connection could be made because the target machine actively refused it. .
Ncat: Trying next address...
Ncat: No connection could be made because the target machine actively refused it. .

The tunnel configuration is :

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>

Fred

Frederic

unread,

Sep 21, 2022, 1:20:47 PM9/21/22

to jgroups-dev

Another point that I don't understand is when I compare the list of members of the gossip router and the list of members of the views that the app server see are differnent. Shouldn't be the same ? The router list 14 members when the aa server list only 4 different members.

===> Result of :org.jgroups.tests.RouterStubGet -host CANOPP01 -port 15200 -cluster CLUSTER_DIMT_14 -nio true
CLUSTERNAME: CLUSTER_DIMT_14
01: c7149865-2c0f-48fa-9481-72f38bc18133, name=CLUSTER_DIMT_14_SiNOP-ONE_PC215407, addr=10.148.6.139:60673, server
02: 5620ba5c-138f-446b-8cfa-ab7c0ed08508, name=CLUSTER_DIMT_14_SiNOP-SERVICE, addr=10.149.135.6:61604, server
03: d6993c5e-71b3-41e8-8ad3-ca535081eebb, name=CLUSTER_DIMT_14_SiNOP-ONE_PC212684, addr=10.148.5.182:49510, server
04: 90611b8b-fe13-4882-874b-5015885fe1dd, name=CLUSTER_DIMT_14_SiNOP-ONE_PC215438, addr=10.148.6.109:62729, server
05: 12684047-69bb-4eca-a92a-902cbc6df5fc, name=CLUSTER_DIMT_14_SiNOP-ONE_PC710239, addr=10.148.5.33:63301, server
06: ca402298-ad5b-4447-bd21-dbe3d19f4b90, name=CLUSTER_DIMT_14_SiNOP-ONE_PC215432, addr=10.148.4.153:56429, server
07: 15992a4e-1713-4faf-9e9b-bfe259c162d9, name=CLUSTER_DIMT_14_SiNOP-ONE_PC215405, addr=10.148.0.80:54361, server
08: 6af75db4-e824-41c6-a1df-6adc94debfd0, name=CLUSTER_DIMT_14_SiNOP-ONE_PC215406, addr=10.148.1.189:63487, server
09: 807d2067-4a53-421b-84f1-7923c2fd5d30, name=CLUSTER_DIMT_14_SiNOP-ONE_PC215443, addr=10.148.1.147:62818, server
10: 661efba9-4820-49e9-805e-de77056b8d4c, name=CLUSTER_DIMT_14_SiNOP-ONE_PC215442, addr=10.148.1.155:52150, server
11: 2e5c4837-f0fb-41f8-abfd-4d2470c0f7e5, name=CLUSTER_DIMT_14_SiNOP-ONE_PC215399, addr=10.148.1.5:59824, server
12: ce0b70aa-8a35-4b36-871e-f752710dab74, name=CLUSTER_DIMT_14_SiNOP-ONE_PC215430, addr=10.148.0.46:53652, server
13: 23271550-74a9-4b13-bbad-ed28019771e5, name=CLUSTER_DIMT_14_SiNOP-ONE_PC215441, addr=10.148.5.55:59518, server
14: 29c5c9ae-02ac-481a-8d74-de610b76f402, name=CLUSTER_DIMT_14_SiNOP-ONE_PC215435, addr=10.148.6.167:54572, server
===> list of members of the view used by the App Server
** view: MergeView::[CLUSTER_DIMT_14_SiNOP-ONE_PC215438|341] (4) [CLUSTER_DIMT_14_SiNOP-ONE_PC215438, CLUSTER_DIMT_14_SiNOP-SERVICE, CLUSTER_DIMT_14_SiNOP-ONE_PC213798, CLUSTER_DIMT_14_SiNOP-ONE_PC215435], 1 subgroups: [CLUSTER_DIMT_14_SiNOP-ONE_PC215438|340] (4) [CLUSTER_DIMT_14_SiNOP-ONE_PC215438, CLUSTER_DIMT_14_SiNOP-SERVICE, CLUSTER_DIMT_14_SiNOP-ONE_PC213798, CLUSTER_DIMT_14_SiNOP-ONE_PC215435]

Fred

Frederic

unread,

Sep 21, 2022, 1:27:17 PM9/21/22

to jgroups-dev

I got something slightly different using hostname instead of localhost on the same vm:

C:\windows\system32>ncat -v canopp01 15200

Ncat: Version 7.93 ( https://nmap.org/ncat )
libnsock ssl_init_helper(): OpenSSL legacy provider failed to load.

Ncat: Connection to ::1 failed: No connection could be made because the target machine actively refused it. .
Ncat: Trying next address...

Ncat: Connected to 10.149.135.6:15200.

I don't see any error on GR logs.
Fred

Frederic

unread,

Sep 21, 2022, 1:40:16 PM9/21/22

to jgroups-dev

Finally, I succeeded to generate the GR error log :

2022-09-21 19:37:26 WARN failed handling message

java.lang.ArrayIndexOutOfBoundsException: Index 111 out of bounds for length 7

at org.jgroups.stack.GossipRouter.receive(GossipRouter.java:327) ~[jgroups-5.2.6.Final.jar:5.2.6.Final]

at org.jgroups.blocks.cs.BaseServer.receive(BaseServer.java:204) ~[jgroups-5.2.6.Final.jar:5.2.6.Final]
at org.jgroups.blocks.cs.TcpConnection$Receiver.run(TcpConnection.java:307) ~[jgroups-5.2.6.Final.jar:5.2.6.Final]
at java.lang.Thread.run(Unknown Source) ~[?:?]

Fred

nact-test-on-the-same-vm.png

Frederic

unread,

Sep 21, 2022, 1:47:44 PM9/21/22

to jgroups-dev

I got a good result also from another VM located on the same network behind the firewall.

Fred

Frederic

unread,

Sep 21, 2022, 1:57:24 PM9/21/22

to jgroups-dev

It seems to work also from the other network meaning that firewall is open on that port number

nact-test-on-the-another-network-through-firewall-3.png

Fred

Frederic

unread,

Sep 21, 2022, 2:11:47 PM9/21/22

to jgroups-dev

The next test is to run GR and app Client through the firewall in the same context of the previous ncat test :

on my PC, the app client succeed to create a channel on the same cluster "CLUSTER_DIMT_14", but it doen't appear in the list of the GR members

Does it mean something for you ?

run-app-client-on-another-network-through-firewall-4.png

Thanks,

Fred

Bela Ban

unread,

Sep 21, 2022, 4:31:37 PM9/21/22

to jgrou...@googlegroups.com

Use IPv4: nc -4 ...

> > > > > To unsubscribe from this group and all its topics, send an
> > > email to
> > > > > jgroups-dev...@googlegroups.com
> > > > > <mailto:jgroups-dev%2Bunsu...@googlegroups.com>.
> > > > > To view this discussion on the web visit
> > > > >
> > > >
> > >
> >

> > > > >
> > > >
> > > > --
> > > > Bela Ban | http://www.jgroups.org <http://www.jgroups.org>
> <http://www.jgroups.org <http://www.jgroups.org>>
> > <http://www.jgroups.org <http://www.jgroups.org>
> <http://www.jgroups.org <http://www.jgroups.org>>>
> > > <http://www.jgroups.org <http://www.jgroups.org>
> <http://www.jgroups.org <http://www.jgroups.org>>
> > <http://www.jgroups.org <http://www.jgroups.org>
> <http://www.jgroups.org <http://www.jgroups.org>>>>
> > > >
> > > > --
> > > > You received this message because you are subscribed to the
> Google
> > > > Groups "jgroups-dev" group.
> > > > To unsubscribe from this group and stop receiving emails from
> it,
> > > send
> > > > an email to jgroups-dev...@googlegroups.com
> > > > <mailto:jgroups-dev...@googlegroups.com>.
> > > > To view this discussion on the web visit
> > > >
> > >
> >

> > >
> > > --
> > > Bela Ban | http://www.jgroups.org <http://www.jgroups.org>
> <http://www.jgroups.org <http://www.jgroups.org>>
> > <http://www.jgroups.org <http://www.jgroups.org>
> <http://www.jgroups.org <http://www.jgroups.org>>>
> > >
> > > --
> > > You received this message because you are subscribed to the Google
> > > Groups "jgroups-dev" group.
> > > To unsubscribe from this group and stop receiving emails from it,
> > send
> > > an email to jgroups-dev...@googlegroups.com
> > > <mailto:jgroups-dev...@googlegroups.com>.
> > > To view this discussion on the web visit
> > >
> >

> >
> > --
> > Bela Ban | http://www.jgroups.org <http://www.jgroups.org>
> <http://www.jgroups.org <http://www.jgroups.org>>
> >
> > --
> > You received this message because you are subscribed to the Google
> > Groups "jgroups-dev" group.
> > To unsubscribe from this group and stop receiving emails from it,
> send
> > an email to jgroups-dev...@googlegroups.com
> > <mailto:jgroups-dev...@googlegroups.com>.
> > To view this discussion on the web visit
> >

> https://groups.google.com/d/msgid/jgroups-dev/8169684c-d70d-4f46-84e1-f47ed318374en%40googlegroups.com <https://groups.google.com/d/msgid/jgroups-dev/8169684c-d70d-4f46-84e1-f47ed318374en%40googlegroups.com> <https://groups.google.com/d/msgid/jgroups-dev/8169684c-d70d-4f46-84e1-f47ed318374en%40googlegroups.com?utm_medium=email&utm_source=footer <https://groups.google.com/d/msgid/jgroups-dev/8169684c-d70d-4f46-84e1-f47ed318374en%40googlegroups.com?utm_medium=email&utm_source=footer>>.
>
> --
> Bela Ban | http://www.jgroups.org <http://www.jgroups.org>

>
> --
> You received this message because you are subscribed to the Google
> Groups "jgroups-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to jgroups-dev...@googlegroups.com
> <mailto:jgroups-dev...@googlegroups.com>.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/jgroups-dev/a0fb07cd-b7c7-4e62-a223-65b8f9daefffn%40googlegroups.com <https://groups.google.com/d/msgid/jgroups-dev/a0fb07cd-b7c7-4e62-a223-65b8f9daefffn%40googlegroups.com?utm_medium=email&utm_source=footer>.

Frederic

unread,

Sep 22, 2022, 6:30:46 AM9/22/22

to jgroups-dev

Hi Bela,

There are more established connection in the GR logs thant the result of the RouterStubGet command ?

Fred

2022-09-22 11:44:04 DEBUG connection to 10.148.1.155:55802 established
2022-09-22 11:44:04 DEBUG connection to 10.148.0.150:57401 established
2022-09-22 11:44:06 DEBUG connection to 10.148.5.138:63756 established
2022-09-22 11:44:07 DEBUG connection to 10.148.1.16:55038 established
2022-09-22 11:44:09 DEBUG connection to 10.148.4.86:64539 established
2022-09-22 11:44:09 DEBUG connection to 10.148.0.45:51572 established
2022-09-22 11:44:10 DEBUG connection to 10.148.5.33:64459 established
2022-09-22 11:44:12 DEBUG connection to 10.148.5.55:51067 established
2022-09-22 11:44:20 DEBUG connection to 10.148.0.112:50975 established
2022-09-22 11:44:23 DEBUG connection to 10.148.1.65:51368 established
2022-09-22 11:44:45 DEBUG connection to 10.149.135.6:26652 established
2022-09-22 11:49:38 DEBUG connection to 10.149.135.6:27843 established
2022-09-22 11:49:52 DEBUG connection to 10.148.5.214:62164 established
2022-09-22 11:57:08 DEBUG connection to 10.148.5.92:51951 established
2022-09-22 12:01:39 DEBUG connection to 10.59.112.44:6675 established
2022-09-22 12:03:03 DEBUG connection to 10.148.5.92:52094 established
2022-09-22 12:10:50 DEBUG connection to 10.59.112.44:6905 established
2022-09-22 12:11:18 DEBUG connection to 10.59.112.44:6919 established
2022-09-22 12:19:37 DEBUG connection to 10.148.1.147:49923 established
2022-09-22 12:20:42 DEBUG connection to 10.148.6.109:57206 established

01: de54dac3-3323-4b86-830f-dc301014b1fe, name=CLUSTER_DIMT_14_SiNOP-SERVICE, addr=10.149.135.6:63122, server
02: df5d4e83-6999-4efd-92d1-4141fb176cd4, name=CLUSTER_DIMT_14_SiNOP-ONE_PC215431, addr=10.148.0.45:64612, server
03: 12684047-69bb-4eca-a92a-902cbc6df5fc, name=CLUSTER_DIMT_14_SiNOP-ONE_PC710239, addr=10.148.5.33:63301, server
04: ad40da0b-d961-43cf-8cd1-2be9deb9228f, name=CLUSTER_DIMT_14_SiNOP-ONE_PC215404, addr=10.148.1.16:61289, server
05: 7e8d6aad-f3f4-4576-816e-02955032cac6, name=CLUSTER_DIMT_14_SiNOP-ONE_PC215435, addr=10.148.0.150:49217, server
06: 3c0663d2-5730-4c4a-b0ee-eb0367c9cb84, name=CLUSTER_DIMT_14_SiNOP-ONE_PC215438, addr=10.148.6.109:61238, server
07: 661efba9-4820-49e9-805e-de77056b8d4c, name=CLUSTER_DIMT_14_SiNOP-ONE_PC215442, addr=10.148.1.155:52150, server
08: 59c5e53c-0080-4d49-ad69-08df8204b453, name=CLUSTER_DIMT_14_SiNOP-ONE_PC215429, addr=10.148.4.86:49666, server
09: 392e06bd-f14a-4799-b97a-7d6b0874019f, name=CLUSTER_DIMT_14_SiNOP-ONE_PC212618, addr=10.148.1.65:51091, server
10: f8b3ac89-cfe5-ef18-9003-62780e558f53, name=CLUSTER_DIMT_14_SiNOP-ONE_PC719472, addr=10.59.112.44:55386, server
11: 077ed278-3b87-4d2c-ba08-1d3e13018820, name=CLUSTER_DIMT_14_SiNOP-ONE_PC215443, addr=10.148.1.147:60733, server
12: f2b5abb8-f6f7-4788-b764-e8cea9037bc7, name=CLUSTER_DIMT_14_SiNOP-ONE_PC214452, addr=10.148.0.112:51828, server

Bela Ban

unread,

Sep 22, 2022, 6:55:07 AM9/22/22

to jgrou...@googlegroups.com

The GossipRouter lists all members which registered with it, e.g. 1-10.
However, members may have suspected and excluded each other, e.g. groups
{1,2,3}, {4,5,6,7} and {8,9,10}.

I can see that you have an issue with failure detection and merging, and
you had 342 view changes so far! Unless and until you're constantly
starting and stopping members, this is not normal.

> Ncat: Version 7.93 ( https://nmap.org/ncat <https://nmap.org/ncat> )

> > > > > To unsubscribe from this group and all its topics, send an
> > > email to
> > > > > jgroups-dev...@googlegroups.com
> > > > > <mailto:jgroups-dev%2Bunsu...@googlegroups.com>.
> > > > > To view this discussion on the web visit
> > > > >
> > > >
> > >
> >

> > > > >
> > > >
> > > > --
> > > > Bela Ban | http://www.jgroups.org
> <http://www.jgroups.org> <http://www.jgroups.org
> <http://www.jgroups.org>>
> > <http://www.jgroups.org <http://www.jgroups.org>
> <http://www.jgroups.org <http://www.jgroups.org>>>
> > > <http://www.jgroups.org <http://www.jgroups.org>
> <http://www.jgroups.org <http://www.jgroups.org>>
> > <http://www.jgroups.org <http://www.jgroups.org>
> <http://www.jgroups.org <http://www.jgroups.org>>>>
> > > >
> > > > --
> > > > You received this message because you are subscribed to
> the Google
> > > > Groups "jgroups-dev" group.
> > > > To unsubscribe from this group and stop receiving emails
> from it,
> > > send
> > > > an email to jgroups-dev...@googlegroups.com
> > > > <mailto:jgroups-dev...@googlegroups.com>.
> > > > To view this discussion on the web visit
> > > >
> > >
> >

> > >
> > > --
> > > Bela Ban | http://www.jgroups.org <http://www.jgroups.org>
> <http://www.jgroups.org <http://www.jgroups.org>>
> > <http://www.jgroups.org <http://www.jgroups.org>
> <http://www.jgroups.org <http://www.jgroups.org>>>
> > >
> > > --
> > > You received this message because you are subscribed to the
> Google
> > > Groups "jgroups-dev" group.
> > > To unsubscribe from this group and stop receiving emails
> from it,
> > send
> > > an email to jgroups-dev...@googlegroups.com
> > > <mailto:jgroups-dev...@googlegroups.com>.
> > > To view this discussion on the web visit
> > >
> >

> >
> > --
> > Bela Ban | http://www.jgroups.org <http://www.jgroups.org>
> <http://www.jgroups.org <http://www.jgroups.org>>
> >
> > --
> > You received this message because you are subscribed to the
> Google
> > Groups "jgroups-dev" group.
> > To unsubscribe from this group and stop receiving emails from
> it, send
> > an email to jgroups-dev...@googlegroups.com
> > <mailto:jgroups-dev...@googlegroups.com>.
> > To view this discussion on the web visit
> >

> https://groups.google.com/d/msgid/jgroups-dev/8169684c-d70d-4f46-84e1-f47ed318374en%40googlegroups.com <https://groups.google.com/d/msgid/jgroups-dev/8169684c-d70d-4f46-84e1-f47ed318374en%40googlegroups.com> <https://groups.google.com/d/msgid/jgroups-dev/8169684c-d70d-4f46-84e1-f47ed318374en%40googlegroups.com?utm_medium=email&utm_source=footer <https://groups.google.com/d/msgid/jgroups-dev/8169684c-d70d-4f46-84e1-f47ed318374en%40googlegroups.com?utm_medium=email&utm_source=footer>>.
>
> --
> Bela Ban | http://www.jgroups.org <http://www.jgroups.org>

>
> --
> You received this message because you are subscribed to the Google
> Groups "jgroups-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to jgroups-dev...@googlegroups.com
> <mailto:jgroups-dev...@googlegroups.com>.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/jgroups-dev/2c3aeda9-ee79-440b-8cbb-2f19664df0c3n%40googlegroups.com <https://groups.google.com/d/msgid/jgroups-dev/2c3aeda9-ee79-440b-8cbb-2f19664df0c3n%40googlegroups.com?utm_medium=email&utm_source=footer>.

Reply all

Reply to author

Forward