Persistent Messages for Multiple Consumers

[Erez Hadad]

Hi folks,

I'm using RabbitMQ with golang.

My problem is this: I would like to define a fabric (combination of exchanges and queues) that has the following properties:

1. can be used to deliver messages from one producer to multiple consumers

2.  Consumers can connect at any point in time - even after the producer has disconnected from the queue

3. Each consumer, after being connected, should receive all the messages published by the producer, independently of other consumers, and in the publishing order

4. extra: At a later point in time, be able to remove the stored messages 

What I tried so far:

 One producer multiple consumers - using fanout exchange connected to queues. Each consumer when connected declares another queue that is bound to the same exchange. of the producer. This gives the one-to-many property 1, but not properties 2 and 3. Even if the messages are published as persistent, new consumers do not seem to receive messages that have been published before they connected.

I'd appreciate your help.

Regards,

-- Erez Hadad

[Michal Kuratczyk]

Hi,

Messages are routed to the queues when they are published. Therefore, while the customer can connect later and consume a message published when it wasn't connected, its queues and binding has to exist. If your consumers are "well known" you can define the queues and bindings for them upfront and then they will be able to connect and consume messages published while they were offline.

Another option is Stream in RabbitMQ 3.9+. Streams have non-destructive consumers. In this case, rather than publishing to multiple queues, one per consumer, you can just publish to the stream and multiple consumers can consume all messages in that stream at any time until the messages expire. This also gives you, at least partially, requirement 4. With queues, messages are deleted immediately after being consumed. With a stream, you define a retention policy and messages expire after the configured time.

Best,

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/bfb71300-53fc-4950-8489-bf5d54a15382n%40googlegroups.com.

--

Michał

RabbitMQ team