S3_PING protocole issue

[Meissa Sakho]

Hi all,
I'm struggling to set up a Keycloak cluster with aws S3 PING protocol.
I'm using RHBK 26 with Jgroups aws lib version 3.0.1.Final deployed in the providers folder.

My jgroups configuration is the following:

<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xmlns="urn:org:jgroups"
        xsi:schemaLocation="urn:org:jgroups http://www.jgroups.org/schema/jgroups-5.3.xsd">
    <TCP bind_port="7800"
         bind_addr="${jgroups.bind_addr:site_local}"
         recv_buf_size="${tcp.recv_buf_size:5M}"
         send_buf_size="${tcp.send_buf_size:5M}"
         max_bundle_size="64K"
         thread_pool.enabled="true"
         thread_pool.min_threads="2"
         thread_pool.max_threads="8"
         thread_pool.keep_alive_time="5000"/>
    <NON_BLOCKING_SENDS/>
    <aws.S3_PING region_name="eu-north-1"
                 bucket_name="rhbk-ec2-jgroups"
                 bucket_prefix="keycloak"/>

    <MERGE3 min_interval="10s"
            max_interval="30s"/>
    <FD_SOCK2/>
    <FD_ALL3 timeout="40s"
             interval="5s"/>
    <VERIFY_SUSPECT2 timeout="1.5s"/>
    <BARRIER/>
    <pbcast.NAKACK2 use_mcast_xmit="false"/>
    <UNICAST3/>
    <pbcast.STABLE desired_avg_gossip="50s"
                   max_bytes="4M"/>
    <pbcast.GMS print_local_addr="true"
                join_timeout="2s"/>
    <MFC max_credits="10M"
         min_threshold="0.4"/>
    <UFC max_credits="10M"
         min_threshold="0.4"/>
    <FRAG2 frag_size="60K"/>
    <pbcast.STATE_TRANSFER/>
  </config>

I've double checked all the AWS bucket setting and permission.

When I start my instance, I'm getting the following error message:

  2025-03-25 10:58:02,228 ERROR [org.infinispan.CONFIG] (Thread-5) ISPN000660: DefaultCacheManager start failed, stopping any running components: org.infinispan.commons.CacheConfigurationException: ISPN000541: Error while trying to create a channel using the specified configuration '[TCP(max_bundle_size=64K, send_buf_size=5M, thread_pool.max_threads=8, thread_pool.keep_alive_time=5000, thread_pool.enabled=true, bind_addr=site_local, bind_port=7800, recv_buf_size=5M, thread_pool.min_threads=2), NON_BLOCKING_SENDS(), aws.S3_PING(region_name=eu-north-1, bucket_prefix=keycloak, bucket_name=rhbk-ec2-jgroups), MERGE3(max_interval=30s, min_interval=10s), FD_SOCK2(), FD_ALL3(interval=5s, timeout=40s), VERIFY_SUSPECT2(timeout=1.5s), BARRIER(), pbcast.NAKACK2(use_mcast_xmit=false), UNICAST3(), pbcast.STABLE(desired_avg_gossip=50s, max_bytes=4M), pbcast.GMS(join_timeout=2s, print_local_addr=true), MFC(min_threshold=0.4, max_credits=10M), UFC(min_threshold=0.4, max_credits=10M), FRAG2(frag_size=60K), pbcast.STATE_TRANSFER()]'
    at
   
    CompletableFuture.java:1768)
    at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: java.lang.Exception: JGRP000002: unable to load protocol aws.S3_PING (either with relative - aws.S3_PING - or absolute - org.jgroups.protocols.aws.S3_PING - class name)
    at org.jgroups.util.Util.loadProtocolClass(Util.java:3328)
    at org.jgroups.conf.ProtocolConfiguration.loadProtocolClass(ProtocolConfiguration.java:93)

I've also double checked the dependencies and it seems like all the required libraries are packaged as expected.

There is obviously something that is missing but I can find it.

Any help would be greatly appreciated.

Thanks

[Bela Ban]

Seems as if your JAR containing S3_PING) is not found. I suggest make sure that you have all relevant JARs needed and place them in the same location as the JGroups JAR.

--
You received this message because you are subscribed to the Google Groups "jgroups-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jgroups-dev...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/jgroups-dev/CADELF5ssp0ur9Q_p2B%2Bk39j8xE1eN6%3Dg06oG0Ksivf%3D118zEVQ%40mail.gmail.com.

-- 
Bela Ban | http://www.jgroups.org

[Radoslav Husar]

Yes, this indeed looks like the jar containing the protocol class is
not in place where Keycloak is looking to load the protocol from. What
steps did you follow to install the protocol?

Rado

> To view this discussion visit https://groups.google.com/d/msgid/jgroups-dev/e15eff66-e447-4f96-9bc2-6d5869aea97d%40mailbox.org.

[mbs...@gmail.com]

I have adde the missing jar and now I can see from the logs that the access of the AWS bucket looks fine.:

2025-04-10 08:54:49,551 INFO  [org.jgroups.protocols.aws.S3_PING] (Thread-5) Using AWS S3 ping in region 'eu-north-1' with bucket 'rhbk-ec2-jgroups' and prefix 'rhbk/'.
2025-04-10 08:54:50,099 INFO  [org.jgroups.protocols.aws.S3_PING] (Thread-5) Found bucket 'rhbk-ec2-jgroups'.

I'm getting another error message that still prevent Keycloak to start and I don't know why.

I'm recieving repeatedly the following error message:

2025-04-10 08:54:52,515 DEBUG [org.jgroups.protocols.TCP] (TQ-Bundler-9,ISPN,ip-10-0-17-111-52254) JGRP000034: ip-10-0-17-111-52254: failure sending message to ip-10-0-26-177-51806: java.net.SocketTimeoutException: Connect timed out

TQ-Bundler-9,ISPN,ip-10-0-17-111-52254) JGRP000034: ip-10-0-17-111-52254: failure sending message to ip-10-0-17-97-45622: java.net.SocketTimeoutException: Connect timed out

My S3_PING configuration is the following:

<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xmlns="urn:org:jgroups"
        xsi:schemaLocation="urn:org:jgroups http://www.jgroups.org/schema/jgroups-5.3.xsd">
    <TCP bind_port="7800"
         bind_addr="${jgroups.bind_addr:site_local}"
         recv_buf_size="${tcp.recv_buf_size:5M}"
         send_buf_size="${tcp.send_buf_size:5M}"

         bundler.max_size="64K"

         thread_pool.enabled="true"
         thread_pool.min_threads="2"
         thread_pool.max_threads="8"
         thread_pool.keep_alive_time="5000"/>
    <NON_BLOCKING_SENDS/>
    <aws.S3_PING region_name="eu-north-1"
                 bucket_name="rhbk-ec2-jgroups"

                 bucket_prefix="rhbk"/>

    <MERGE3 min_interval="10s"
            max_interval="30s"/>
    <FD_SOCK2/>
    <FD_ALL3 timeout="40s"
             interval="5s"/>
    <VERIFY_SUSPECT2 timeout="1.5s"/>
    <BARRIER/>
    <pbcast.NAKACK2 use_mcast_xmit="false"/>
    <UNICAST3/>
    <pbcast.STABLE desired_avg_gossip="50s"
                   max_bytes="4M"/>
    <pbcast.GMS print_local_addr="true"
                join_timeout="2s"/>
    <MFC max_credits="10M"
         min_threshold="0.4"/>
    <UFC max_credits="10M"
         min_threshold="0.4"/>
    <FRAG2 frag_size="60K"/>
    <pbcast.STATE_TRANSFER/>
  </config>

Any idea about the reason why it's failing to start?