RPC with RabbitMQ (using correlationId)

[Ohad Avenshtein]

Hi,

I've read & implemented RPC using this tutorial: https://www.rabbitmq.com/tutorials/tutorial-six-dotnet.html .

Still, some things seems weird to me:

1. "CorrelationID" paragraph starts with the words: "In the method presented above we suggest creating a callback queue for every RPC request. That's pretty inefficient, but fortunately there is a better way - let's create a single callback queue per client.", and continues with: "... The client waits for data on the callback queue. When a message appears, it checks the correlationId property. If it matches the value from the request it returns the response to the application."

That actually means that each message is pulled out of the queue and being checked against generated correlationId.

But what if each message is pretty big ? This solutions becomes pretty insufficient. 

Does RabbitMQ have a mechanism to pull only message metadata OR correlate messages within RabbitMQ engine without the need to pull them out ?

2. It seems like this tutorial uses "autoAck = true" to pull messages out. That means - if a message that's not match a correlation is pulled out - it's actually lost.

Any help ?

Thanks,

Ohad.

[Michael Klishin]

It does not mean that "a message is pulled off a queue". Correlation ID matching is a consumer

(client application) feature (a set of conventions, really).

Message distribution happens at the routing stage, not once messages are enqueued.

Consumer acknowledgement mode [1] has no relation to it either.

Direct Reply-to works in a way that's closer to what you have in mind but all it uses is a bit of extra

channel state (basically, a map) and channels never keep messages around for long (they are passed on to

queues, discarded, delivered but all of those cases are transient in nature).

1. http://www.rabbitmq.com/confirms.html

2. http://www.rabbitmq.com/direct-reply-to.html

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

MK

Staff Software Engineer, Pivotal/RabbitMQ

[Ohad Avenshtein]

Thanks for your answer.

From what I understand - if we have a few clients sending messages simultaneously:

1. Client need to supply a reply-to queue in the client's request message (let's say reply-to is "aaa")

2. Server gets the message: sets exchange to "" and routing key to "aaa" (reply-to value)

3. The specific client from paragraph 1 gets the reply (without any need of correlation id).

Can we actually say that:

a. "replyTo" is useful to route response to correct client when multiple clients sends requests 

b. "correlationId" is used by a client that receives multiple responses to his requests

?

Thanks,

Ohad.

בתאריך יום שישי, 27 באוקטובר 2017 בשעה 17:36:49 UTC+3, מאת Michael Klishin:

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Michael Klishin]

On 29 Oct 2017, at 17:15, Ohad Avenshtein <ohadi...@gmail.com> wrote:

Thanks for your answer.

From what I understand - if we have a few clients sending messages simultaneously:

1. Client need to supply a reply-to queue in the client's request message (let's say reply-to is "aaa")

2. Server gets the message: sets exchange to "" and routing key to "aaa" (reply-to value)

3. The specific client from paragraph 1 gets the reply (without any need of correlation id).

Can we actually say that:

a. "replyTo" is useful to route response to correct client when multiple clients sends requests 

b. "correlationId" is used by a client that receives multiple responses to his requests

Yes, your understanding is exactly right.

Note that these two are basically conventions, there can be other uses for correlation ID, for instance, which we don’t discourage.

[Ohad Avenshtein]

Can you please tell me then why do you have this check in your docs (https://www.rabbitmq.com/tutorials/tutorial-six-dotnet.html) ?

"if (ea.BasicProperties.CorrelationId == correlationId)

            {

                respQueue.Add(response);

            } "

Is there any reason that "ea.BasicProperties.CorrelationId" would be different then "correlationId" ?

בתאריך יום ראשון, 29 באוקטובר 2017 בשעה 16:18:40 UTC+2, מאת Michael Klishin:

[Michael Klishin]

With a higher request/response rate and prefetch > 1 you can get multiple responses.

A concurrent server may or may not process and publish responses in the same order.

It’s a responsibility of the client to find out what request a response is for.

[Ohad Avenshtein]

I'm truely sorry but I still don't get it. 

How does "if (ea.BasicProperties.CorrelationId == correlationId)" helps to keep ordering ? In addition, if this condition is "false" - we won't lose this message ?

Can you please provide a case scenario example ?

בתאריך יום ראשון, 29 באוקטובר 2017 בשעה 16:35:28 UTC+2, מאת Michael Klishin:

[Michael Klishin]

It's just an example. Tutorials are just that, examples. They are meant to be simplistic

and demonstrate one major concept at a time.

`(ea.BasicProperties.CorrelationId == correlationId)` is a hint at the fact that clients should verify

what responses they receive (and correlation ID is one of the fields useful for that).

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.