HelloWorldLesson6RPC: why did we use correlationID if we are creating exclusive Q's for each request

[Rakib]

In the paragraph for "Callback Queue", it is presented that we use a callback back queue for every request sent out.

Then in the next paragraph for "Correlation Id" it is said that having to create a new callback queue for every RPC request is pretty inefficient. So we will use a single callback queue but identify one request from another via a Correlation Id.

Then at the end, we did end up creating a new exclusive callback queue for every RPC request and also tagged them with a unique Correlation Id. I don't understand why we did that? 

If i am creating a new exclusive callback queue for every RPC request, shouldn't a response on that queue automatically mean that it is a response for that request? Why do i need a correlation Id then? I am never having more than 1 item in a callback queue anyways regardless of how many times i fire the same client-side request from multiple terminal windows.

PS. I used the Javascript example to try out rabbitMQ with nodeJS.

[Michael Klishin]

On 26 October 2015 at 14:48:03, Rakib (syed.raki...@gmail.com) wrote:
> If i am creating a new exclusive callback queue for every RPC
> request, shouldn't a response on that queue automatically mean
> that it is a response for that request? Why do i need a correlation
> Id then? I am never having more than 1 item in a callback queue anyways
> regardless of how many times i fire the same client-side request
> from multiple terminal windows.

The above assumption isn’t true for every case. Correlation ID exists primarily
to provide more context to clients and really doesn’t introduce any noticeable overhead. 

Also, take a look at https://www.rabbitmq.com/direct-reply-to.html.
--
MK

Staff Software Engineer, Pivotal/RabbitMQ

[Rakib]

Thanks Michael,

I am still a bit confused here. 

The original notion of the author of helloWorld6RPCexample was that "to create a new callback queue for every RPC request is pretty inefficient. So we will use a single callback queue but identify one request from another via a Correlation Id"? 

However, we are ending creating a new callback queue for every RPC request by the end of that example. So how does this support the original notion that author made at the beginning of the article?

p.s. I am reading through the article you shared.

regards
Rakib

[Michael Klishin]

On 26 October 2015 at 15:13:00, Rakib (syed.raki...@gmail.com) wrote:
> However, we are ending creating a new callback queue for every
> RPC request by the end of that example. So how does this support
> the original notion that author made at the beginning of the article?

Of what article? You haven’t provided any links. 

[Rakib]

Sorry. I was referring to the rabbitmq getting started tutorial lesson no 6 for RPC. Here is the link. https://www.rabbitmq.com/tutorials/tutorial-six-javascript.html . My bad.

---
regards
Rakib
--- from mobile

[Rakib]

i was referring to the article at https://www.rabbitmq.com/tutorials/tutorial-six-javascript.html

The original notion of the author of this article was that "to create a new callback queue for every RPC request is pretty inefficient. So we will use a single callback queue but identify one request from another via a Correlation Id"? 

However, we are ending up creating a new callback queue for every RPC request by the end of the example anyways. So how does this support the original notion that the author made at the beginning of the article?

[Michael Klishin]

 On 27 October 2015 at 20:59:17, Rakib (syed.raki...@gmail.com) wrote:
> The original notion of the author of this article was that "to
> create a new callback queue for every RPC request is pretty inefficient.
> So we will use a single callback queue but identify one request
> from another via a Correlation Id"?
>
> However, we are ending up creating a new callback queue for every
> RPC request by the end of the example anyways. So how does this
> support the original notion that the author made at the beginning
> of the article?

In tutorial 6, one of the paragraphs says

«In the method presented above we suggest creating a callback queue for every RPC request.
That's pretty inefficient, but fortunately there is a better way - let's create a single callback queue per client.»

“One queue per request” and “one queue per client” are not the same thing: the latter can be dramatically more
efficient in a system with a high rate of requests.

I think what confuses you is the fact that “one queue per client” is explained conceptually but the following
code example in Putting it All Together is still for the first scenario: “one queue per request”.

Adding another code example would be possible but our tutorials typically cover only one approach.

Maybe we should move the section on correlation ID to the very end.

[Rakib]

I think you are right.

The order of the sections in the article is what is fueling this confusion. If the part saying "one queue per request is inefficient" is moved to the end, the article would be easier to understand for newcomers learning the RPC model.

However, then that would mean we don't really need to implement correlationId in this tutorial at all because a callback queue per request will, at most, contain only one item in it anyways.

On that case, probably the discussion about how "one queue per request" is inefficient, how "one queue per client" is a better way, how correlationId is used to achieve this should be a separate tutorial of it's own. Maybe?

---
regards
Rakib
--- from mobile

[Rakib]

i just tried writing the Javascript tutorial 6 exactly as it is in the tutorial (only without the correlationId concept). And it worked the same way as the tutorial (with CorrelationId). It's kind of difficult to understand from this tutorial that:

1. First we are presenting a basic RPC concept with only callback queues and no correlationId concept yet
2. Then we are saying that having a new-queue-pre-request model is very inefficient. We are saying there is a better way to do this CorrelationIds so that we can have a new-queue-per-client model instead of having a new-queue-per-request model.
3. Then we are writing client-server codes for this tutorial with a CorrelationId concept
4. But there is no way to test the impact of a CorrelationId in this tutorial. When we are firing multiple server scripts from multiple terminal windows and multiple client scripts from multiple terminal windows [trying to simulate multiple servers-and-clients from a single computer.], we are essentially seeing a one-queue-per-request model being implemented instead of actually seeing a one-queue-per-client model in action - at least the tutorial did not present any specific way of how the benefits of a CorrelationId can be assessed in a one-queue-per-client model. Please advise me if there is a better way to test the impact of CorrelationIds in the client-server code we just wrote from the tutorial.
5. Later I tried to remove the correlationId matters from the client-server codes of this tutorial. Then also, it behaved in exactly the same way as just wrote the client-server scripts from the tutorial with a correlationId
6. Point no 5 implies that the scripts shown in the tutorial was more like a basic traditional RPC model even though correlationIds were used in the request/response paddings. This somehow fails to reinforce the claim made in point no 2 earlier.

Hence, i think this lesson6 is more fitted for introducing upto a basic RPC concept with callback queues only. Therefore, the concept of CorrelationIds to make the RPC model more efficient should get an advanced tutorial of its own - let's say lesson 7 hopefully.

How can we make that happen? Or is there anything i am missing out in points 1-6.

regards

Rakib

On Wednesday, October 28, 2015 at 11:43:51 AM UTC+6, Rakib wrote:

I think you are right.

The order of the sections in the article is what is fueling this confusion. If the part saying "one queue per request is inefficient" is moved to the end, the article would be easier to understand for newcomers learning the RPC model.

However, then that would mean we don't really need to implement correlationId in this tutorial at all because a callback queue per request will, at most, contain only one item in it anyways.

On that case, probably the discussion about how "one queue per request" is inefficient, how "one queue per client" is a better way, how correlationId is used to achieve this should be a separate tutorial of it's own. Maybe?

---
regards
Rakib
--- from mobile

On Oct 27, 2015 10:36 PM, "Michael Klishin" <@pivotal.io> wrote:

[Rakib]

any feedback on this proposal?

[Michael Klishin]

On 2 November 2015 at 13:42:19, Rakib (syed.raki...@gmail.com) wrote:
> Hence, i think this lesson6 is more fitted for introducing upto
> a basic RPC concept with callback queues only. Therefore, the
> concept of CorrelationIds to make the RPC model more efficient
> should get an advanced tutorial of its own - let's say lesson 7
> hopefully.

I'm not sold on the idea of having 2 tutorials but if we add Direct Reply-to to tutorial 7,
then it might be enough information for a separate tutorial.

> How can we make that happen?

The website is open source:
https://github.com/rabbitmq/rabbitmq-website

If we introduce a new tutorial, it needs to be done for all languages,
so if you contribute just one and we have to cover 6, it will take a while.

[Rakib]

Ok. So, i think we should be able to at least modify the tutorial 6 to introduce CorrelationIds AFTER we are done writing the server-client scripts in a basic RPC model. Currently, the correlationId concept is introduced before writing the RPC server-client scripts, then the server-client scripts are written with CorrelationIds, but the CorrelationIds are really not being used for the parts of the tutorial as there is no way given to test the real impact of correlationIds. The concept of CorrelationIds can be given here as a brainteaser rather than including it in the scripts because impact of CorrelationId is unproven in the tutorial.

On Thursday, October 29, 2015 at 2:20:00 PM UTC+6, Rakib wrote: