[Java Client] Rabbit MQ Thread Count

[Youtube Solace]

Hi All,

We have two environments where we are running the same setup which is using Rabbit MQ Java Client 3.5.3.

Environments: 

1. RHEL 6.6 Erlang 18.1.1 64 cpu - 2 threads per core - 130 G RAM

2. RHEL 6.6 Erlang 18.0.1 12 CPUs 1 thread per core. 64G RAM

For the first environment the Java Client (RabbitMQ Controlled threads only) created 129 threads.

The second environment created only 25 threads.

We know the threads are RabbitMQ controlled threads because of the naming convention used in the ThreadFactory which we have passed through via ConnectionFactory (com.rabbitmq.client).

Examining the dump, we saw that 129 threads of this nature:

"pool-1-thread-129" #278 daemon prio=5 os_prio=0 tid=0x00007fa464104800 nid=0x43bd waiting on condition [0x00007fa6222e1000]
   java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for  <0x00000000802c3b18> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

Is the client creating an x number of threads based on x number of CPUs available ? is there a way to control the number ? has anyone seen any differences like this before ?

We came across this problem because the differences in thread counts led to the 'max user processes' limit being reached for the user that it was running under.

Thanks

PP

[Michael Klishin]

On 4 December 2015 at 20:18:16, Youtube Solace (p300...@gmail.com) wrote:
> Is the client creating an x number of threads based on x number
> of CPUs available ? is there a way to control the number ?

Every connection uses several threads at various points of its lifecycle,
namely for I/O and consumer operation dispatch. The latter uses an executor
of size Runtime.getRuntime().availableProcessors() * 2 by default. 

You can provide a ThreadFactory for all created threads, and an executor for consumer
operation dispatch.

With recent changes temporary operations can be executed in a user-provided
executor:

https://github.com/rabbitmq/rabbitmq-java-client/issues/87
https://github.com/rabbitmq/rabbitmq-java-client/issues/94
--
MK

Staff Software Engineer, Pivotal/RabbitMQ