Infinispan errors on Keycloak startup

223 views
Skip to first unread message

L Vasseur

unread,
Jun 6, 2023, 6:09:29 AM6/6/23
to Keycloak User
Hello,

We have Infinispan errors during Keycloak restart and we don't understand them. Can you help us please?

Our cluster is composed of 4 Keycloak instances deployed on Kubernetes.

This is the infinispan configuration : 
            <cache-container name="keycloak" modules="org.keycloak.keycloak-model-infinispan">
                <transport lock-timeout="60000"/>
                <local-cache name="realms">
                    <heap-memory size="10000"/>
                </local-cache>
                <local-cache name="users">
                    <heap-memory size="10000"/>
                </local-cache>
                <local-cache name="authorization">
                    <heap-memory size="10000"/>
                </local-cache>
                <local-cache name="keys">
                    <heap-memory size="1000"/>
                    <expiration max-idle="3600000"/>
                </local-cache>
                <replicated-cache name="work">
                    <expiration lifespan="900000000000000000"/>
                </replicated-cache>
                <distributed-cache name="sessions" owners="2">
                    <expiration lifespan="900000000000000000"/>
                </distributed-cache>
                <distributed-cache name="authenticationSessions" owners="2">
                    <expiration lifespan="900000000000000000"/>
                </distributed-cache>
                <distributed-cache name="offlineSessions" owners="2">
                    <expiration lifespan="900000000000000000"/>
                </distributed-cache>
                <distributed-cache name="clientSessions" owners="2">
                    <expiration lifespan="900000000000000000"/>
                </distributed-cache>
                <distributed-cache name="offlineClientSessions" owners="2">
                    <expiration lifespan="900000000000000000"/>
                </distributed-cache>
                <distributed-cache name="loginFailures" owners="2">
                    <expiration lifespan="900000000000000000"/>
                </distributed-cache>
                <distributed-cache name="actionTokens" owners="2">
                    <heap-memory size="-1"/>
                    <expiration interval="300000" lifespan="900000000000000000" max-idle="-1"/>
                </distributed-cache>
            </cache-container>


The error observed is the following : 

05:49:03,730 ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (thread-29,ejb,prd-keycloak-69899bb5f9-nnxc2) ISPN000136: Error executing command PutMapCommand on Cache 'offlineSessions', writing keys [4e79fbca-1608-4b53-964c-34be06805b45]: java.lang.ClassCastException: class org.infinispan.container.entries.NullCacheEntry cannot be cast to class org.infinispan.container.entries.MVCCEntry (org.infinispan.container.entries.NullCacheEntry and org.infinispan.container.entries.MVCCEntry are in unnamed module of loader 'org.inf...@11.0.9.Final' @7e8984a2)
at org.inf...@11.0.9.Final//org.infinispan.interceptors.impl.CallInterceptor.lookupMvccEntry(CallInterceptor.java:520)
at org.inf...@11.0.9.Final//org.infinispan.interceptors.impl.CallInterceptor.visitPutMapCommand(CallInterceptor.java:484)
at org.inf...@11.0.9.Final//org.infinispan.commands.write.PutMapCommand.acceptVisitor(PutMapCommand.java:74)
at org.inf...@11.0.9.Final//org.infinispan.interceptors.impl.CallInterceptor.visitCommand(CallInterceptor.java:176)
at org.inf...@11.0.9.Final//org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:61)
at org.inf...@11.0.9.Final//org.infinispan.interceptors.DDAsyncInterceptor.handleDefault(DDAsyncInterceptor.java:53)
at org.inf...@11.0.9.Final//org.infinispan.interceptors.DDAsyncInterceptor.visitPutMapCommand(DDAsyncInterceptor.java:89)
at org.inf...@11.0.9.Final//org.infinispan.commands.write.PutMapCommand.acceptVisitor(PutMapCommand.java:74)
at org.inf...@11.0.9.Final//org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:59)
at org.inf...@11.0.9.Final//org.infinispan.interceptors.BaseAsyncInterceptor.lambda$new$0(BaseAsyncInterceptor.java:27)
at org.inf...@11.0.9.Final//org.infinispan.interceptors.InvocationSuccessFunction.apply(InvocationSuccessFunction.java:25)
at org.inf...@11.0.9.Final//org.infinispan.interceptors.impl.QueueAsyncInvocationStage.invokeQueuedHandlers(QueueAsyncInvocationStage.java:125)
at org.inf...@11.0.9.Final//org.infinispan.interceptors.impl.QueueAsyncInvocationStage.accept(QueueAsyncInvocationStage.java:88)
at org.inf...@11.0.9.Final//org.infinispan.interceptors.impl.QueueAsyncInvocationStage.accept(QueueAsyncInvocationStage.java:33)
at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859)
at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837)
at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)
at java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2073)
at org.inf...@11.0.9.Final//org.infinispan.remoting.transport.AbstractRequest.complete(AbstractRequest.java:67)
at org.inf...@11.0.9.Final//org.infinispan.remoting.transport.impl.MultiTargetRequest.onResponse(MultiTargetRequest.java:106)
at org.inf...@11.0.9.Final//org.infinispan.remoting.transport.impl.RequestRepository.addResponse(RequestRepository.java:52)
at org.inf...@11.0.9.Final//org.infinispan.remoting.transport.jgroups.JGroupsTransport.processResponse(JGroupsTransport.java:1402)
at org.inf...@11.0.9.Final//org.infinispan.remoting.transport.jgroups.JGroupsTransport.processMessage(JGroupsTransport.java:1305)
at org.inf...@11.0.9.Final//org.infinispan.remoting.transport.jgroups.JGroupsTransport.access$300(JGroupsTransport.java:131)
at org.inf...@11.0.9.Final//org.infinispan.remoting.transport.jgroups.JGroupsTransport$ChannelCallbacks.up(JGroupsTransport.java:1445)
at org.j...@4.2.11.Final//org.jgroups.JChannel.up(JChannel.java:784)
at org.j...@4.2.11.Final//org.jgroups.fork.ForkProtocolStack.up(ForkProtocolStack.java:135)
at org.j...@4.2.11.Final//org.jgroups.stack.Protocol.up(Protocol.java:309)
at org.j...@4.2.11.Final//org.jgroups.protocols.FORK.up(FORK.java:142)
at org.j...@4.2.11.Final//org.jgroups.protocols.FRAG3.up(FRAG3.java:165)
at org.j...@4.2.11.Final//org.jgroups.protocols.FlowControl.up(FlowControl.java:343)
at org.j...@4.2.11.Final//org.jgroups.protocols.pbcast.GMS.up(GMS.java:876)
at org.j...@4.2.11.Final//org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:243)
at org.j...@4.2.11.Final//org.jgroups.protocols.UNICAST3.deliverMessage(UNICAST3.java:1049)
at org.j...@4.2.11.Final//org.jgroups.protocols.UNICAST3.addMessage(UNICAST3.java:772)
at org.j...@4.2.11.Final//org.jgroups.protocols.UNICAST3.handleDataReceived(UNICAST3.java:753)
at org.j...@4.2.11.Final//org.jgroups.protocols.UNICAST3.up(UNICAST3.java:405)
at org.j...@4.2.11.Final//org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:592)
at org.j...@4.2.11.Final//org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:132)
at org.j...@4.2.11.Final//org.jgroups.protocols.FailureDetection.up(FailureDetection.java:186)
at org.j...@4.2.11.Final//org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:254)
at org.j...@4.2.11.Final//org.jgroups.protocols.MERGE3.up(MERGE3.java:281)
at org.j...@4.2.11.Final//org.jgroups.protocols.Discovery.up(Discovery.java:300)
at org.j...@4.2.11.Final//org.jgroups.protocols.TP.passMessageUp(TP.java:1396)
at org.j...@4.2.11.Final//org.jgroups.util.SubmitToThreadPool$SingleMessageHandler.run(SubmitToThreadPool.java:87)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at org.jboss.as.cl...@23.0.2.Final//org.jboss.as.clustering.context.ContextReferenceExecutor.execute(ContextReferenceExecutor.java:49)
at org.jboss.as.cl...@23.0.2.Final//org.jboss.as.clustering.context.ContextualExecutor$1.run(ContextualExecutor.java:70)
at java.base/java.lang.Thread.run(Thread.java:829)

05:49:03,731 WARN  [org.infinispan.CLUSTER] (thread-29,ejb,prd-keycloak-69899bb5f9-nnxc2) ISPN000071: Caught exception when handling command SingleRpcCommand{cacheName='offlineSessions', command=PutMapCommand{map={4e79fbca-1608-4b53-964c-34be06805b45=SessionEntityWrapper{version=e060b385-a0af-4599-b5b7-ebd6aac9d0dc, entity=UserSessionEntity [id=4e79fbca-1608-4b53-964c-34be06805b45, realm=master, lastSessionRefresh=1684948822, clients=[ccc01d7e-180b-43a4-80de-945cd7ad9344]], localMetadata={}}}, flags=[SKIP_CACHE_STORE, SKIP_CACHE_LOAD, IGNORE_RETURN_VALUES], metadata=EmbeddedExpirableMetadata{version=null, lifespan=900000000000000000, maxIdle=-1}, internalMetadata={}, isForwarded=true}}: java.lang.ClassCastException: class org.infinispan.container.entries.NullCacheEntry cannot be cast to class org.infinispan.container.entries.MVCCEntry (org.infinispan.container.entries.NullCacheEntry and org.infinispan.container.entries.MVCCEntry are in unnamed module of loader 'org.inf...@11.0.9.Final' @7e8984a2)
at org.inf...@11.0.9.Final//org.infinispan.interceptors.impl.CallInterceptor.lookupMvccEntry(CallInterceptor.java:520)
at org.inf...@11.0.9.Final//org.infinispan.interceptors.impl.CallInterceptor.visitPutMapCommand(CallInterceptor.java:484)
at org.inf...@11.0.9.Final//org.infinispan.commands.write.PutMapCommand.acceptVisitor(PutMapCommand.java:74)
at org.inf...@11.0.9.Final//org.infinispan.interceptors.impl.CallInterceptor.visitCommand(CallInterceptor.java:176)
at org.inf...@11.0.9.Final//org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:61)
at org.inf...@11.0.9.Final//org.infinispan.interceptors.DDAsyncInterceptor.handleDefault(DDAsyncInterceptor.java:53)
at org.inf...@11.0.9.Final//org.infinispan.interceptors.DDAsyncInterceptor.visitPutMapCommand(DDAsyncInterceptor.java:89)
at org.inf...@11.0.9.Final//org.infinispan.commands.write.PutMapCommand.acceptVisitor(PutMapCommand.java:74)
at org.inf...@11.0.9.Final//org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:59)
at org.inf...@11.0.9.Final//org.infinispan.interceptors.BaseAsyncInterceptor.lambda$new$0(BaseAsyncInterceptor.java:27)
at org.inf...@11.0.9.Final//org.infinispan.interceptors.InvocationSuccessFunction.apply(InvocationSuccessFunction.java:25)
at org.inf...@11.0.9.Final//org.infinispan.interceptors.impl.QueueAsyncInvocationStage.invokeQueuedHandlers(QueueAsyncInvocationStage.java:125)
at org.inf...@11.0.9.Final//org.infinispan.interceptors.impl.QueueAsyncInvocationStage.accept(QueueAsyncInvocationStage.java:88)
at org.inf...@11.0.9.Final//org.infinispan.interceptors.impl.QueueAsyncInvocationStage.accept(QueueAsyncInvocationStage.java:33)
at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859)
at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837)
at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)
at java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2073)
at org.inf...@11.0.9.Final//org.infinispan.remoting.transport.AbstractRequest.complete(AbstractRequest.java:67)
at org.inf...@11.0.9.Final//org.infinispan.remoting.transport.impl.MultiTargetRequest.onResponse(MultiTargetRequest.java:106)
at org.inf...@11.0.9.Final//org.infinispan.remoting.transport.impl.RequestRepository.addResponse(RequestRepository.java:52)
at org.inf...@11.0.9.Final//org.infinispan.remoting.transport.jgroups.JGroupsTransport.processResponse(JGroupsTransport.java:1402)
at org.inf...@11.0.9.Final//org.infinispan.remoting.transport.jgroups.JGroupsTransport.processMessage(JGroupsTransport.java:1305)
at org.inf...@11.0.9.Final//org.infinispan.remoting.transport.jgroups.JGroupsTransport.access$300(JGroupsTransport.java:131)
at org.inf...@11.0.9.Final//org.infinispan.remoting.transport.jgroups.JGroupsTransport$ChannelCallbacks.up(JGroupsTransport.java:1445)
at org.j...@4.2.11.Final//org.jgroups.JChannel.up(JChannel.java:784)
at org.j...@4.2.11.Final//org.jgroups.fork.ForkProtocolStack.up(ForkProtocolStack.java:135)
at org.j...@4.2.11.Final//org.jgroups.stack.Protocol.up(Protocol.java:309)
at org.j...@4.2.11.Final//org.jgroups.protocols.FORK.up(FORK.java:142)
at org.j...@4.2.11.Final//org.jgroups.protocols.FRAG3.up(FRAG3.java:165)
at org.j...@4.2.11.Final//org.jgroups.protocols.FlowControl.up(FlowControl.java:343)
at org.j...@4.2.11.Final//org.jgroups.protocols.pbcast.GMS.up(GMS.java:876)
at org.j...@4.2.11.Final//org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:243)
at org.j...@4.2.11.Final//org.jgroups.protocols.UNICAST3.deliverMessage(UNICAST3.java:1049)
at org.j...@4.2.11.Final//org.jgroups.protocols.UNICAST3.addMessage(UNICAST3.java:772)
at org.j...@4.2.11.Final//org.jgroups.protocols.UNICAST3.handleDataReceived(UNICAST3.java:753)
at org.j...@4.2.11.Final//org.jgroups.protocols.UNICAST3.up(UNICAST3.java:405)
at org.j...@4.2.11.Final//org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:592)
at org.j...@4.2.11.Final//org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:132)
at org.j...@4.2.11.Final//org.jgroups.protocols.FailureDetection.up(FailureDetection.java:186)
at org.j...@4.2.11.Final//org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:254)
at org.j...@4.2.11.Final//org.jgroups.protocols.MERGE3.up(MERGE3.java:281)
at org.j...@4.2.11.Final//org.jgroups.protocols.Discovery.up(Discovery.java:300)
at org.j...@4.2.11.Final//org.jgroups.protocols.TP.passMessageUp(TP.java:1396)
at org.j...@4.2.11.Final//org.jgroups.util.SubmitToThreadPool$SingleMessageHandler.run(SubmitToThreadPool.java:87)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at org.jboss.as.cl...@23.0.2.Final//org.jboss.as.clustering.context.ContextReferenceExecutor.execute(ContextReferenceExecutor.java:49)
at org.jboss.as.cl...@23.0.2.Final//org.jboss.as.clustering.context.ContextualExecutor$1.run(ContextualExecutor.java:70)
at java.base/java.lang.Thread.run(Thread.java:829)

Reply all
Reply to author
Forward
0 new messages