Distribute Hazelcast topic events to multiple queues?

[ustc...@gmail.com]

Hello folks,

From the documentation, it seems that Hazelcast would internally send all the Reliable Topic events to one RingBuffer for consumers to poll from.

http://docs.hazelcast.org/docs/3.5/manual/html/reliabletopic.html

I wonder if there's a way to tell Hazelcast to create one RingBuffer for each consumer and distributes Reliable Topic events to all the RingBuffers. This way consumers can consumer and process messages at their own pace.

Thank you.

[Peter Veentjer]

Listeners can listen at their own pace anyway.

If you would create a ringbuffer per listener, it will only lead to having the same data in memory multiple times (+ all the remoting overhead to do so)

A ringbuffer is not like a queue; a take from a queue is destructive, but a read from a ringbuffer is not. So if one topic listener is very fast, it has no influence for the one that is slow.

[ustc...@gmail.com]

Thank you for the answer Peter. 

The issue I'm worrying about is that the gap between the fast listener and the slow listener is greater than the total capacity of the ringbuffer, say keep the application running for a few months and the fast listener is ahead of the slow listener for a few million messages. Is there any way to handle this situation gracefully? Thank you.

[Peter Veentjer]

It is always a worry if something is a lot slower than something else.

But a fast listener will not obstruct a slow listener or vice versa.

What you should worry about is a fast producer and a slow consumer. You need to favor one or have (infinitely) large buffer. So either the producer needs to slow down, or the listener will loose messages when it is too far.

--
You received this message because you are subscribed to the Google Groups "Hazelcast" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hazelcast+...@googlegroups.com.
To post to this group, send email to haze...@googlegroups.com.
Visit this group at https://groups.google.com/group/hazelcast.
To view this discussion on the web visit https://groups.google.com/d/msgid/hazelcast/8698b92d-d051-4048-b986-a54f81e4dfcb%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[dsukho...@gmail.com]

Is it possible to improve situation having several (slow) consumers?