Re: [hawtdispatch] Question about how to implement function queueing

[Hiram Chirino]

At the end of the day your going to have multiple threads contending to pick up work from the queue manager.  How you handle/implement that work hand off is going to have a HUGE effect on your performance.  The simplest, most cache friendly, and most lockless, solution is probably going to give your the best performance.

I'd definitely start simple and and use a benchmark to compare how different implementations behave.

On Mon, Apr 1, 2013 at 1:33 PM, Saxo <jet...@web.de> wrote:

Hello,

I was thinking of developing something like HawtDispatch myself, because I needed something like it for my project, but couldn't find anything that suits the purpose until I found HawtDispatch. Am using it and am very happy with it.

Before I found HawtDispatch the way I thought to implement this is the way I will describe it in the following. What I would hope for would be some comment about its feasibility. Reason is that I'm not very into this problem domain and simply would like to get some feedback by someone competent.

Approach:

Make use of the JDK5 ThreadPool with a fixed size of threads. Whenever a queue is created it is added to some QueueManager. Once an item is added to a queue the QueueManager is notified. As soon as some Thread is finished with executing a function in the queue it asks the QueueManager for a new queue. The QueueManager returns one that has been waiting longest for processing or has the largest number of items in it or similar (whatever policy has been defined by the user). If there is no next queue to be processed the thread is blocked by the QueueManager till some new item has been added to a queue waiting for processing.

This approach is very simple I guess. It's what someone would come up with who has little experience as what performance is concerned in queueing systems. I would just be interested in some feedback how feasible that idea would be considered.

Thanks, Oliver

--
You received this message because you are subscribed to the Google Groups "hawtdispatch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hawtdispatch...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--

Hiram Chirino

Engineering | Red Hat, Inc.

hchi...@redhat.com | fusesource.com | redhat.com

skype: hiramchirino | twitter: @hiramchirino

blog: Hiram Chirino's Bit Mojo

[Saxo]

Hello Hiram,

thanks for your reply. The lock contention caused by the QueueManager will be a problem. I see. So I could build a hierarchy of QueueManagers to reduce the load. To do this with little locking is probably not easy ... I will not continue with that idea I mentioned. My question was merely out of interest.

Thanks, Oliver