JDBC_PING issue with Keycloak or Wildfly?

1,949 views
Skip to first unread message

dc...@prosentient.com.au

unread,
Sep 2, 2021, 12:07:10 AM9/2/21
to Keycloak User

I recently mentioned that I noticed a problem with JDBC_PING when stopping Keycloak, and I’ve encountered it again with the latest Keycloak.

 

I’m guessing this is a Wildfly bug, but anyone care to comment?

 

 

04:04:26,980 INFO  [org.infinispan.CLUSTER] (ServerService Thread Pool -- 76) ISPN000080: Disconnecting JGroups channel ejb

04:04:26,985 INFO  [org.infinispan.CLUSTER] (ServerService Thread Pool -- 71) ISPN000080: Disconnecting JGroups channel ejb

04:04:27,013 ERROR [org.jgroups.protocols.JDBC_PING] (ServerService Thread Pool -- 71) JGRP000115: Could not open connection to database: java.sql.SQLException: javax.resource.ResourceException: IJ000470: You are trying to use a connecti

on factory that has been shut down: java:jboss/datasources/KeycloakDS

        at org.jboss.ironjac...@1.4.27.Final//org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:159)

        at org.jboss.a...@23.0.2.Final//org.jboss.as.connector.subsystems.datasources.WildFlyDataSource.getConnection(WildFlyDataSource.java:64)

        at org.j...@4.2.11.Final//org.jgroups.protocols.JDBC_PING.getConnection(JDBC_PING.java:302)

        at org.j...@4.2.11.Final//org.jgroups.protocols.JDBC_PING.delete(JDBC_PING.java:337)

        at org.j...@4.2.11.Final//org.jgroups.protocols.JDBC_PING.remove(JDBC_PING.java:175)

        at org.j...@4.2.11.Final//org.jgroups.protocols.FILE_PING.stop(FILE_PING.java:101)

        at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)

        at org.j...@4.2.11.Final//org.jgroups.stack.ProtocolStack.stopStack(ProtocolStack.java:899)

        at org.j...@4.2.11.Final//org.jgroups.JChannel.stopStack(JChannel.java:1085)

        at org.j...@4.2.11.Final//org.jgroups.JChannel.disconnect(JChannel.java:444)

        at org.jboss.as.cl...@23.0.2.Final//org.jboss.as.clustering.jgroups.subsystem.ChannelServiceConfigurator.accept(ChannelServiceConfigurator.java:122)

        at org.jboss.as.cl...@23.0.2.Final//org.jboss.as.clustering.jgroups.subsystem.ChannelServiceConfigurator.accept(ChannelServiceConfigurator.java:58)

        at org.wildfly.clu...@23.0.2.Final//org.wildfly.clustering.service.FunctionalService.stop(FunctionalService.java:73)

        at org.wildfly.clu...@23.0.2.Final//org.wildfly.clustering.service.AsyncServiceConfigurator$AsyncService.lambda$stop$1(AsyncServiceConfigurator.java:142)

        at org.jbos...@2.4.0.Final//org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)

        at org.jbos...@2.4.0.Final//org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1990)

        at org.jbos...@2.4.0.Final//org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486)

        at org.jbos...@2.4.0.Final//org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377)

        at java.base/java.lang.Thread.run(Thread.java:829)

        at org.jbos...@2.4.0.Final//org.jboss.threads.JBossThread.run(JBossThread.java:513)

Caused by: javax.resource.ResourceException: IJ000470: You are trying to use a connection factory that has been shut down: java:jboss/datasources/KeycloakDS

        at org.jboss.iro...@1.4.27.Final//org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:777)

        at org.jboss.ironjac...@1.4.27.Final//org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:151)

        ... 19 more

 

04:04:27,013 ERROR [org.jgroups.protocols.JDBC_PING] (ServerService Thread Pool -- 71) JGRP000215: Failed to delete PingData in database

04:04:27,027 INFO  [org.jboss.as] (MSC service thread 1-3) WFLYSRV0050: Keycloak 15.0.2 (WildFly Core 15.0.1.Final) stopped in 204ms

 

 

David Cook

Senior Software Engineer

Prosentient Systems

Suite 7.03

6a Glen St

Milsons Point NSW 2061

Australia

 

Office: 02 9212 0899

Online: 02 8005 0595

 

Schuster Sebastian (IOC/PAU1)

unread,
Sep 2, 2021, 4:12:53 AM9/2/21
to dc...@prosentient.com.au, Keycloak User

This is because when shutting down Keycloak, the database subsystem is shut down too early so it cannot update the JDBC_PING table.

We tried to fix that but without success. Since we switched to Kubernetes, we are using DNS_PING instead and we had zero problems since then…

 

Best regards,

Sebastian

 

Mit freundlichen Grüßen / Best regards

Dr.-Ing. Sebastian Schuster

Product Area User Management (IOC/PAU1)
Robert Bosch GmbH | Postfach 10 60 50 | 70049 Stuttgart | GERMANY | www.bosch.com
Tel. +49 30 726112-485 | Mobil +49 152 02177668 | Telefax +49 30 726112-100 | Sebastian...@bosch.io

Sitz: Stuttgart, Registergericht: Amtsgericht Stuttgart, HRB 14000;
Aufsichtsratsvorsitzender: Franz Fehrenbach; Geschäftsführung: Dr. Volkmar Denner,
Prof. Dr. Stefan Asenkerschbaumer, Filiz Albrecht, Dr. Christian Fischer, Dr. Stefan Hartung,
Dr. Markus Heyn, Harald Kröger, Rolf Najork

--
You received this message because you are subscribed to the Google Groups "Keycloak User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to keycloak-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/keycloak-user/0d2501d79faf%24f9c48ca0%24ed4da5e0%24%40prosentient.com.au.

dc...@prosentient.com.au

unread,
Sep 2, 2021, 9:54:36 PM9/2/21
to Schuster Sebastian (IOC/PAU1), Keycloak User

Thanks, Sebastian. I’ve been thinking about investigating some other options, as JDBC_PING has been fairly disappointing so far.

 

David Cook

Senior Software Engineer

Prosentient Systems

Suite 7.03

6a Glen St

Milsons Point NSW 2061

Australia

 

Office: 02 9212 0899

Online: 02 8005 0595

 

Niko Köbler

unread,
Sep 3, 2021, 1:47:44 AM9/3/21
to Keycloak User
From my experience, JDBC_PING just works fine, this exception occurs only when shutting the server down (which I usually only do in controlled reasons, otherwise Keycloak just runs...) and although the exception says that it isn't, the node is being deleted from the table.

- Niko

Thomas Darimont

unread,
Sep 3, 2021, 2:58:15 AM9/3/21
to Niko Köbler, Keycloak User
I also experienced similar problems with JDBC_PING in the past, especially with rolling upgrades (Kubernetes and other container solutions).

If the Keycloak system is rarely touched JDBC_PING can be a good option. But in my experience, other methods like DNS_PING or KUBE_PING are more robust 
in dynamic environments.

Cheers,
Thomas

To view this discussion on the web visit https://groups.google.com/d/msgid/keycloak-user/791171de-76a6-4d11-a383-66ea2e4bfaf1n%40googlegroups.com.

dc...@prosentient.com.au

unread,
Sep 5, 2021, 7:51:45 PM9/5/21
to Thomas Darimont, Niko Köbler, Keycloak User

Niko: There was a lot of evidence that the nodes weren’t being deleted from the table. Database was cluttered and nodes were trying to communicate with long since shut down nodes. Worked around it by adding the “remove_all_data_on_view_change” property though.

 

Thomas: Thanks for sharing that. Unfortunately, we haven’t been using containers. We’ve been deploying using VM images, and it was a problem before I added that “remove_all_data_on_view_change” property to the JDBC_PING. If I could do it all again, I think I’d probably go back and use containers instead.

 

David Cook

Senior Software Engineer

Prosentient Systems

Suite 7.03

6a Glen St

Milsons Point NSW 2061

Australia

 

Office: 02 9212 0899

Online: 02 8005 0595

 

Reply all
Reply to author
Forward
0 new messages