Disruptor vs Threadpool CPU and Memory performance

[Bobby Harsono]

Hello,

I am currently observing the cpu and memory usage of disruptor and threadpool in a message broker scenario,

1. A producer will generate and fire messages in certain amounts to ActiveMQ Queue
   
2. A single thread consumer will take the item from queue and pass it to the framework (Threadpool or Disruptor)
   
3. A simple database mapping with message's attributes will be done
   
4. A publisher will send the final message to another queue
   

I turned off my wifi connection, and didn't open any program when running the experiment on my laptop

I set up so either threadpool and disruptor will be using 1 thread only in consuming from queue, processing message until publishing it back..the only parameter i played is buffer size (disruptor) and working queue (threadpool)

I experimented by firing 60k messages and let my module process it by using disruptor and threadpool alternately, then i logged the cpu and memory usage per second and calculate the average values of both..

I see significant pattern in cpu % (although i haven't known why threadpool' usage is declining at certain queue size), however i can't say i see any pattern in memory %.. My colleagues also say the same thing, the graph say nothing at all...Any suggestions how can i know the comparison between cpu and memory usage of threadpool and disruptor? 

Whether i have wrong setup?

Actually my friend said that i need to know the effects / impacts / relationship between buffer size and queue size to memory, where should i start?

Thank you

[Michael Barker]

Hi,

Sorry about the delay in replying to your message.

I'm not really sure what question you are trying to answer here and can't really answer your question about the thread pool memory/cpu usage as I don't know much about the internals of the JDK threadpool.  However some information about the Disruptor memory sizing.  The Disruptor's memory use is proportional the size of the buffer and the size of the events used in the ring buffer.  These are preallocated at start up.  The best way to determine actual memory usage to take a full heap dump of your process and look at the retained size of the Disruptor of thread pool in question.

Regarding CPU usage, for the Disruptor this is primarily based on the selection of WaitStrategy.  This decision has to be based on your own latency v CPU usage requirements.  For lower latency the spin based strategies (BusySpin, Yield) are better, but will use much more CPU than the blocking or sleep based strategy.

Mike.

--
You received this message because you are subscribed to the Google Groups "Disruptor" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lmax-disruptor+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.