Re: Disruptor vs ConcurrentLinkedQueue

[Michael Barker]

> One workaround that came to my mind to avoid messing with locks or entity
> update was to have multiple rings, that is:
> 1 - splitting data directly in the READER, filling RING_1.a, RING_1.b ... up
> to 4 rings
> 2 - then assigning a SINGLE Calculator as the one and only consumer of each
> RING_1
> 3 - also, grouping the Calculators as multiple Producers for the RING_2's
> (RING_2.a, RING_2.b)
> 4 - and having finally only one Writer as the consumer for each RING_2

That's a valid architecture. There are 2 other possible options.

- Use the WorkerPool class.
- Use some sort of key on the events going through a single Ring_1 and
use multiple subscribers, where each subscriber uses that key to
separate processing. Basically a sharding model.

One of the things that would recommend is if you can come up with a
way to avoid multiple producers then that approach is likely to
perform better. Being able to optimise for the single producer case
can bring nearly an order of magnitude performance improvement.

Mike.

[thompsri]

Hi - the work around you suggest seems to be quite a common pattern, we use this multilane approach quite heavily. By having the READER select which ring based on how busy the ring is (i.e the difference between the publish sequence and the consumer sequence) we see some good load balancing effects. We tend to find that (particularly if thread affinity is used) that just 1 or 2 of the Calculators can handle the load and the final ring in the sequence becomes effectively a single producer queue.

[jmarci]

Thanks so much mike, I'm now studying the WorkerPool!