Rabbit MQ cliets gets disconnect automatically

97 views
Skip to first unread message

Siddu Y

unread,
Jul 21, 2022, 8:56:19 AM7/21/22
to rabbitmq-users
Dears,

One of client application gets disconnected after 5 min and we could see the below exceptions in logs and thread dump.

Can any one provide the solution to resolve the issue ... Thanks.

JVM Log:

WARNING [Thread-16] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [tomcat-api] appears to have started a thread named [org.springframework.amqp.rabbit.RabbitListenerEndpointContainer#7-1] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java...@11.0.15/java.lang.Object.wait(Native Method)
 java...@11.0.15/java.lang.Thread.join(Unknown Source)
 java...@11.0.15/java.lang.Thread.join(Unknown Source)
 java...@11.0.15/java.lang.ApplicationShutdownHooks.runHooks(Unknown Source)
 java...@11.0.15/java.lang.ApplicationShutdownHooks$1.run(Unknown Source)
 java...@11.0.15/java.lang.Shutdown.runHooks(Unknown Source)
 java...@11.0.15/java.lang.Shutdown.exit(Unknown Source)
 java...@11.0.15/java.lang.Runtime.exit(Unknown Source)
 java...@11.0.15/java.lang.System.exit(Unknown Source)
 org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.lambda$new$0(AbstractMessageListenerContainer.java:252)
 org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer$$Lambda$1978/0x0000000801780040.handle(Unknown Source)
 org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.run(SimpleMessageListenerContainer.java:1264)

====================================

Thread Dump:

org.springframework.amqp.rabbit.RabbitListenerEndpointContainer#7-1" #84 prio=5 os_prio=0 cpu=160497.93ms elapsed=227.98s tid=0x00007f295da39000 nid=0x9b4e
runnable  [0x00007f293dde3000]
   java.lang.Thread.State: RUNNABLE
        at org.apache.xerces.dom.DeferredDocumentImpl.synchronizeData(Unknown Source)
        at org.apache.xerces.dom.DeferredDocumentImpl.synchronizeChildren(Unknown Source)
        at org.apache.xerces.dom.CoreDocumentImpl.getDocumentElement(Unknown Source)
        at org.camunda.bpm.model.xml.impl.instance.DomDocumentImpl.getRootElement(DomDocumentImpl.java:48)
        - locked <0x0000000702f6dd00> (a org.apache.xerces.dom.DeferredDocumentImpl)
        at org.camunda.bpm.model.xml.impl.parser.AbstractModelParser.getSchema(AbstractModelParser.java:175)
        at org.camunda.bpm.model.xml.impl.parser.AbstractModelParser.validateModel(AbstractModelParser.java:156)
        at org.camunda.bpm.model.xml.impl.parser.AbstractModelParser.parseModelFromStream(AbstractModelParser.java:134)
        at org.camunda.bpm.model.dmn.impl.DmnParser.parseModelFromStream(DmnParser.java:76)
        at org.camunda.bpm.model.dmn.Dmn.doReadModelFromInputStream(Dmn.java:247)
        at org.camunda.bpm.model.dmn.Dmn.readModelFromStream(Dmn.java:159)
        at org.camunda.bpm.dmn.engine.impl.transform.DefaultDmnTransform.setModelInstance(DefaultDmnTransform.java:111)
        at org.camunda.bpm.dmn.engine.impl.transform.DefaultDmnTransform.modelInstance(DefaultDmnTransform.java:119)
        at org.camunda.bpm.engine.impl.dmn.deployer.DecisionRequirementsDefinitionDeployer.transformDefinitions(DecisionRequirementsDefinitionDeployer.java:6
1)
        at org.camunda.bpm.engine.impl.AbstractDefinitionDeployer.transformResource(AbstractDefinitionDeployer.java:99)
        at org.camunda.bpm.engine.impl.AbstractDefinitionDeployer.parseDefinitionResources(AbstractDefinitionDeployer.java:74)
        at org.camunda.bpm.engine.impl.AbstractDefinitionDeployer.deploy(AbstractDefinitionDeployer.java:64)
        at org.camunda.bpm.engine.impl.persistence.deploy.cache.CacheDeployer$1.call(CacheDeployer.java:54)
        at org.camunda.bpm.engine.impl.persistence.deploy.cache.CacheDeployer$1.call(CacheDeployer.java:51)
        at org.camunda.bpm.engine.impl.interceptor.CommandContext.runWithoutAuthorization(CommandContext.java:559)
        at org.camunda.bpm.engine.impl.interceptor.CommandContext.runWithoutAuthorization(CommandContext.java:547)
        at org.camunda.bpm.engine.impl.persistence.deploy.cache.CacheDeployer.deploy(CacheDeployer.java:51)
        at org.camunda.bpm.engine.impl.persistence.deploy.cache.DeploymentCache.deploy(DeploymentCache.java:71)
        at org.camunda.bpm.engine.impl.persistence.entity.DeploymentManager.insertDeployment(DeploymentManager.java:64)
        at org.camunda.bpm.engine.impl.cmd.DeployCmd.deploy(DeployCmd.java:254)
        at org.camunda.bpm.engine.impl.cmd.DeployCmd.lambda$doExecute$0(DeployCmd.java:155)
        at org.camunda.bpm.engine.impl.cmd.DeployCmd$$Lambda$1441/0x00000008012e7040.call(Unknown Source)
        at org.camunda.bpm.engine.impl.interceptor.CommandContext.runWithoutAuthorization(CommandContext.java:559)
        at org.camunda.bpm.engine.impl.interceptor.CommandContext.runWithoutAuthorization(CommandContext.java:547)
        at org.camunda.bpm.engine.impl.cmd.DeployCmd.doExecute(DeployCmd.java:141)
        at org.camunda.bpm.engine.impl.cmd.DeployCmd.execute(DeployCmd.java:101)
        - locked <0x000000071989d7d8> (a java.lang.Class for org.camunda.bpm.engine.ProcessEngine)
        at org.camunda.bpm.engine.impl.cmd.DeployCmd.execute(DeployCmd.java:81)
        at org.camunda.bpm.engine.impl.interceptor.CommandExecutorImpl.execute(CommandExecutorImpl.java:28)
        at org.camunda.bpm.engine.impl.interceptor.CommandContextInterceptor.execute(CommandContextInterceptor.java:110)
        at org.camunda.bpm.engine.spring.SpringTransactionInterceptor$1.doInTransaction(SpringTransactionInterceptor.java:72)
        at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140)
        at org.camunda.bpm.engine.spring.SpringTransactionInterceptor.execute(SpringTransactionInterceptor.java:70)
        at org.camunda.bpm.engine.impl.interceptor.ProcessApplicationContextInterceptor.execute(ProcessApplicationContextInterceptor.java:70)
        at org.camunda.bpm.engine.impl.interceptor.CommandCounterInterceptor.execute(CommandCounterInterceptor.java:35)
        at org.camunda.bpm.engine.impl.interceptor.LogInterceptor.execute(LogInterceptor.java:33)
        at org.camunda.bpm.engine.impl.RepositoryServiceImpl.deployWithResult(RepositoryServiceImpl.java:104)
        at org.camunda.bpm.engine.impl.repository.DeploymentBuilderImpl.deployWithResult(DeploymentBuilderImpl.java:270)
        at org.camunda.bpm.engine.impl.repository.DeploymentBuilderImpl.deploy(DeploymentBuilderImpl.java:266)
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java...@11.0.15/Native Method)
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java...@11.0.15/Unknown Source)
        at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java...@11.0.15/Unknown Source)
        at java.lang.reflect.Method.invoke(java...@11.0.15/Unknown Source)
        at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:169)
        at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:119)
        at org.springframework.amqp.rabbit.listener.adapter.HandlerAdapter.invoke(HandlerAdapter.java:75)
        at org.springframework.amqp.rabbit.listener.adapter.MessagingMessageListenerAdapter.invokeHandler(MessagingMessageListenerAdapter.java:261)
        at org.springframework.amqp.rabbit.listener.adapter.MessagingMessageListenerAdapter.invokeHandlerAndProcessResult(MessagingMessageListenerAdapter.jav
a:207)
        at org.springframework.amqp.rabbit.listener.adapter.MessagingMessageListenerAdapter.onMessage(MessagingMessageListenerAdapter.java:146)
        at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:1665)
        at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.actualInvokeListener(AbstractMessageListenerContainer.java:1584)
        at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer$$Lambda$1979/0x0000000801780040.invokeListener(Unknown Source)
        at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:1572)
        at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:1563)
        at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.executeListener(AbstractMessageListenerContainer.java:1507)
        at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.doReceiveAndExecute(SimpleMessageListenerContainer.java:967)
        at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.receiveAndExecute(SimpleMessageListenerContainer.java:914)
        at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.access$1600(SimpleMessageListenerContainer.java:83)

 

Arnaud Cogoluègnes

unread,
Jul 21, 2022, 9:48:15 AM7/21/22
to rabbitmq-users
Can you be more specific about the sequence of actions?

I guess a "client application" triggered an HTTP request, which triggered somehow the consumption of a message in Spring AMQP? If the message processing takes too long, the HTTP client can get disconnected because of a timeout setting somewhere in the chain (Tomcat, proxy, etc).

Reply all
Reply to author
Forward
0 new messages