Fanout Exchange Message Reply

[Asadullah Hussain]

Fanout exchange can be used to send message from a single source to multiple receivers (subscriber). What is the mechanism to receive a "single" aggregated response from the receivers? 

I have two requirements:

1. Make sure message has been received by all subscribers (through some sort of ACK)

then

2. Each receiver replies an acknowledgement message (e.g., string "received") but I just want to send a single acknowledgement message to the client (because all subscribers reply with the same message)

Call_back queues: I understand that they can be defined for receiving replies but for a fanout exchange it will forward all the replies to the client. Whereas I just require one reply to be forwarded.

Any help in this regard will be appreciated.

[Alvaro Videla]

Hi,

You need to implement a message aggregator: http://www.eaipatterns.com/Aggregator.html

The aggregator will then reply with one message to the client.

You need to specify a reply_to queue and a correlation_id to the requests sent via fanout. Then your aggregator will put together all the replies arriving with the same correlation_id. Once it has the X amount of replies expected for said correlation_id, it aggregates them, and sends the reply to the client.

Keeps in mind that if you fire an aggregator waiting for say 5 replies, but then you bind a 6th queue to the fanout exchange, your aggregator will misbehave, since it won't know about that 6th reply.

Regards,

Alvaro

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Asadullah Hussain]

Thanks A lot. Do I have to change the source/configuration of fanout exchange for the aggregator to work or describing reply_to & correlation_id will do the job?

--

Cheers, 

Asadullah Hussain

[Alvaro Videla]

The core of the concept is the reply_to and correlation_id message properties.

[Asadullah Hussain]

On 21 August 2014 14:06, Alvaro Videla <videl...@gmail.com> wrote:

Your RPC servers should reply to the message aggregator queue, not to the same fanout exchange.

​That basically doesn't fulfill the dynamic binding of fanout exchange.​ I guess i'll have to stick with replying through fanout exchange and then add message aggregation at client & server ends.

On Thu, Aug 21, 2014 at 11:01 AM, Asadullah Hussain <asad...@gmail.com> wrote:

I tried using the RPC design from the tutorial to simply send a string "Hello" and receive it's reply "Reply". I am able to send the message & receive reply but here are a few design questions:

0. The client code to wait for the reply is :

    while response is None:
        connection.process_data_events()
    return str(response)

But even after the reply messages are received i.e., channel.basic_consume calls it's callback function. The "return str(response)" statement is not executed.

What may be the problem with the above code?

1. In the tutorial for RPC, the client (sender) only declares the callback queue (doesn't bind it because it is default exchange). I have to bind the call_back queue to exchange.

Now this causes a problem the exchange broadcasts all messages to all "binded queues" which means that both client & server receive both "Hello" & "Reply" messages.

How can I ensure that the client only receives "Reply" & server only receives "Hello"

--

Cheers, 

Asadullah Hussain

--

Cheers, 

Asadullah Hussain

[Alvaro Videla]

Let me see if I understand this correctly.

Let's say we have queues A, B, C bound to a fanout exchange. Consumers to those queues are your RPC servers.

Your client will send one message to the fanout exchange, so A, B and C will get the message, they will do something with it, and they they will have to reply back. That message will have a reply_to property set to "client_queue" and a correlation_id, for example "1" for the first request, "2" for the second request, and so on.

Your client will then wait for tree different replies (ones from each server), once it get the replies, it aggregates them, and do something with the aggregated data.

How does it know those three replies must be aggregated together? All three replies come with the same correlation_id. 

How do the RPC servers know where to reply? They check the reply_to property of the messages.

Is this what you want to implement?

Cheers,

Alvaro

[Asadullah Hussain]

​

On 21 August 2014 23:02, Alvaro Videla <videl...@gmail.com> wrote:

Let me see if I understand this correctly.

Let's say we have queues A, B, C bound to a fanout exchange. Consumers to those queues are your RPC servers.

Your client will send one message to the fanout exchange, so A, B and C will get the message, they will do something with it, and they they will have to reply back. That message will have a reply_to property set to "client_queue" and a correlation_id, for example "1" for the first request, "2" for the second request, and so on.

Your client will then wait for tree different replies (ones from each server), once it get the replies, it aggregates them, and do something with the aggregated data.

How does it know those three replies must be aggregated together? All three replies come with the same correlation_id. 

​You are right this would be a problem if I were to receive three different types of replies but in my case I have the multiple copies of ​the same program running on separate VMs. So all the instances send the exact same reply, for example client sends a http request with some parameters, it just needs one single reply that acknowledges that the http request has been processed by the server corresponding to your requested parameters (although the request was processed by multiple http servers).

So I am looking to make sure that only 1 reply is sent to the client, otherwise as you've mentioned it will be very difficult to aggregate messages at the client end.

 

How do the RPC servers know where to reply? They check the reply_to property of the messages.

Is this what you want to implement?

​So I am experimenting with a layered approach by running multiple exchanges e.g.,

Client--->Exchange(direct)----->Server​0

                                                       |

                                                       |____>Exchange(Fanout)---> Server (1,2,3,....)

--

Cheers, 

Asadullah Hussain

[Jon Zuilkowski]

This is pretty close to what I'm asking for.  Only I don't want an aggregate.