java.util.concurrent.RejectedExecutionException, the taskqueue of hz.client_0.event-1 is full

[Saikat Sur]

Hi, 

Getting following Exception....

     Anyone, Please let me know the possible root causes.

java.util.concurrent.RejectedExecutionException: Task: com.hazelcast.client.spi.impl.listener.AbstractClientListenerService$ClientEventProcessor is rejected, the taskqueue of hz.client_0.event-1 is full

April 20th 2020, 13:48:30.104myservice-preprod - 

08:18:30.035 [hz.hz.client_0.IO.thread-in-0] WARN  c.h.client.spi.ClientListenerService - hz.client_0 [dev] [3.12] Event clientMessage could not be handled

April 20th 2020, 13:48:30.104myservice-preprod

 - 

at com.hazelcast.client.impl.protocol.util.ClientMessageDecoder.onRead(ClientMessageDecoder.java:70)

April 20th 2020, 13:48:30.104myservice-preprod

 - 

at com.hazelcast.internal.networking.nio.NioThread.processSelectionKey(NioThread.java:369)

April 20th 2020, 13:48:30.104myservice-preprod

 - 

at com.hazelcast.util.executor.StripedExecutor$Worker.schedule(StripedExecutor.java:204)

April 20th 2020, 13:48:30.104myservice-preprod

 - 

at com.hazelcast.util.executor.StripedExecutor$Worker.access$500(StripedExecutor.java:178)

April 20th 2020, 13:48:30.104myservice-preprod

 - 

at com.hazelcast.internal.networking.nio.NioThread.selectLoop(NioThread.java:280)

April 20th 2020, 13:48:30.104myservice-preprod

 - 

java.util.concurrent.RejectedExecutionException: Task: com.hazelcast.client.spi.impl.listener.AbstractClientListenerService$ClientEventProcessor@37650c9 is rejected, the taskqueue of hz.client_0.event-1 is full!

April 20th 2020, 13:48:30.104

myservice-preprod

 - 

at com.hazelcast.client.connection.nio.ClientConnection.handleClientMessage(ClientConnection.java:239)

April 20th 2020, 13:48:30.104myservice-preprod

 - 

at com.hazelcast.client.spi.impl.listener.AbstractClientListenerService.handleClientMessage(AbstractClientListenerService.java:183)

April 20th 2020, 13:48:30.104myservice-preprod

 - 

at com.hazelcast.internal.networking.nio.NioThread.processSelectionKeys(NioThread.java:354)

April 20th 2020, 13:48:30.104myservice-preprod

 - 

at com.hazelcast.internal.networking.nio.NioThread.selectLoop(NioThread.java:280)

April 20th 2020, 13:48:30.104myservice-preprod

 - 

at com.hazelcast.client.connection.nio.ClientPlainChannelInitializer$1.accept(ClientPlainChannelInitializer.java:68)

April 20th 2020, 13:48:30.104myservice-preprod

 - 

at com.hazelcast.client.spi.impl.listener.AbstractClientListenerService.handleClientMessage(AbstractClientListenerService.java:183)

April 20th 2020, 13:48:30.104myservice-preprod

 - 

at com.hazelcast.client.impl.protocol.util.ClientMessageDecoder.handleMessage(ClientMessageDecoder.java:112)

April 20th 2020, 13:48:30.104myservice-preprod

 - 

at com.hazelcast.internal.networking.nio.NioInboundPipeline.process(NioInboundPipeline.java:135)

April 20th 2020, 13:48:30.104myservice-preprod

 - 

at com.hazelcast.client.impl.protocol.util.ClientMessageDecoder.handleMessage(ClientMessageDecoder.java:112)

April 20th 2020, 13:48:30.104myservice-preprod

 - 

at com.hazelcast.util.executor.StripedExecutor$Worker.schedule(StripedExecutor.java:204)

April 20th 2020, 13:48:30.104myservice-preprod

 - 

at com.hazelcast.client.impl.protocol.util.ClientMessageDecoder.onRead(ClientMessageDecoder.java:70)

April 20th 2020, 13:48:30.104myservice-preprod

 - 

at com.hazelcast.client.impl.protocol.util.ClientMessageDecoder.handleMessage(ClientMessageDecoder.java:112)

[Ozan Kılıç]

The event queue is full so you need to increase its size like described here: https://docs.hazelcast.org//docs/3.12.6/manual/html-single/index.html#global-event-configuration

Another possibility is that your event threads are blocked so they cannot process the event queue and it keeps increasing. In this case, you should solve the problem with blocking. Check your all listener implementations and see if there is any place that would cause blocking such as DB connections, REST calls, etc... 

--
You received this message because you are subscribed to the Google Groups "Hazelcast" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hazelcast+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/hazelcast/0ea26dcb-5603-492d-abce-786937fb255d%40googlegroups.com.

--

Ozan KILIC Solutions Architect - Support       hazelcast®

This message contains confidential information and is intended only for the individuals named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. E-mail transmission cannot be guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. The sender therefore does not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. If verification is required, please request a hard-copy version. -Hazelcast

[Saikat Sur]

Thank you Ozan,

I have analysed into the possibilities and found the cause.  It was the case as of your second point solution.  The event threads were blocked on the event queue, since this map operation was dependent on another second maps entries.  Eventually the second map was getting destroyed on expired event, which was not intended implementation.  Fixed the same by removing unnecessary event listener from the second map.

.. Saikat

To unsubscribe from this group and stop receiving emails from it, send an email to haze...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/hazelcast/0ea26dcb-5603-492d-abce-786937fb255d%40googlegroups.com.