Best strategy when for consume?

[Dimitri Pekarovsky]

Hi.

I write the app, http-amqp proxy, rack <-> bunny. I use the rack future to send the request to background (-1 as respond code) and then I use callback function to return the answer when it will be received from internal network.

What is the best strategy to work with rabbit (and bunny):

0) At application start i create connection and channel. Then:

a) for every http-request I create the queue, bind it with temporary key, create consumer and subscribe (less code in my app),

b) or it is not the good choice with internals of rabbit? I will use some more code, like ruby Queue for internal distribution of messages from one static queue + subscribtion?

//DP

[Michael Klishin]

Hi Dimitri,

I’m not sure what you mean by “the internals of rabbit”. Your own application’s design choices are much more important here.

If responses in your system have to be correlated to specific requests, you can use a temporary queue or Direct Reply-to [1][2].
The biggest hurdle with Rack, which was originally based on a synchronous request-response model, is how to await for a response
in a way that is not terribly inefficient or makes error communication really hard.

You can transfer responses received by a RabbitMQ consumer to the Rack app using a Ruby ::Queue or any other synchronisation
mechanism you prefer. If you intend to block the caller, they would largely all work the same.

What I’d do is to find a way to communicate back without blocking. This is not only more efficient but also a lot more natural. I am not
familiar with the state of modern Rack. Back when I used to do Web development the optimal answer was “use a WebSocket connection that acts as a proxy in your own app”. I suspect it is still one of the top choices but maybe there is a way to respond asynchronously with modern Rack.

Again, “RabbitMQ internals” or Bunny internals do not really matter here as the hard part is awaiting for. and returning a response/communicating
an error back to the HTTP client. Communication in the messaging protocols RabbitMQ supports is 100% asynchronous,
which doesn’t fit the narrative of HTTP particularly well but does fit WebSockets.

If you need Web apps to communicate to each other, rabbitmq-web-stomp and rabbitmq-web-mqtt would be good options [3].

In case you can afford to move this part of the system away from Rack, there is a pretty broad range of WebSocket-based options
to consider.

HTH.

1. https://www.rabbitmq.com/getstarted.html, see tutorial 6
2. https://www.rabbitmq.com/direct-reply-to.html
3. https://www.rabbitmq.com/web-stomp.html

> --
> Bunny: http://rubybunny.info
> March Hare: http://rubymarchhare.info
>
> IRC: #rabbitmq on irc.freenode.net
>
> Post to the group: ruby...@googlegroups.com | unsubscribe: ruby-amqp+...@googlegroups.com
> ---
> You received this message because you are subscribed to the Google Groups "Ruby RabbitMQ libraries" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to ruby-amqp+...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/ruby-amqp/8b35a4a2-ce89-4f80-8f6c-7f1d96df6275%40googlegroups.com.

[Dimitri Pekarovsky]

Thank you, Michael, for response.

I wonder to clarify my question.

I've read in docs, that rabbitmq don't like when clients makes multiple connections/disconnects many times (this is like my app will connect to rabbit for every http request and then disconnect).

Sure this is not the best strategy, it's just a starting point for my assumptions.

So I tought, will rabbit hurt from when making mutiple queues/deleting em for every http-request? You answered about temporary queues, thank you.

> The biggest hurdle with Rack, which was originally based on a synchronous request-response model, is how to await for a response
in a way that is not terribly inefficient or makes error communication really hard

I save env to global hash, then return [-1, {}, []], which means asynchronous and then I can call the env['async.callback'].call from the bunny consumer when response is ready. By such manner rack keeps the connection up until the call of the callback.

//DP

воскресенье, 15 сентября 2019 г., 17:37:13 UTC+3 пользователь Michael Klishin написал:

Hi Dimitri,

I’m not sure what you mean by “the internals of rabbit”. Your own application’s design choices are much more important here.

If responses in your system have to be correlated to specific requests, you can use a temporary queue or Direct Reply-to [1][2].
The biggest hurdle with Rack, which was originally based on a synchronous request-response model, is how to await for a response
in a way that is not terribly inefficient or makes error communication really hard.

You can transfer responses received by a RabbitMQ consumer to the Rack app using a Ruby ::Queue or any other synchronisation
mechanism you prefer. If you intend to block the caller, they would largely all work the same.

What I’d do is to find a way to communicate back without blocking. This is not only more efficient but also a lot more natural. I am not
familiar with the state of modern Rack. Back when I used to do Web development the optimal answer was “use a WebSocket connection that acts as a proxy in your own app”. I suspect it is still one of the top choices but maybe there is a way to respond asynchronously with modern Rack.

Again, “RabbitMQ internals” or Bunny internals do not really matter here as the hard part is awaiting for. and returning a response/communicating
an error back to the HTTP client. Communication in the messaging protocols RabbitMQ supports is 100% asynchronous,
which doesn’t fit the narrative of HTTP particularly well but does fit WebSockets.

If you need Web apps to communicate to each other, rabbitmq-web-stomp and rabbitmq-web-mqtt would be good options [3].

In case you can afford to move this part of the system away from Rack, there is a pretty broad range of WebSocket-based options
to consider.

HTH.

1. https://www.rabbitmq.com/getstarted.html, see tutorial 6
2. https://www.rabbitmq.com/direct-reply-to.html
3. https://www.rabbitmq.com/web-stomp.html

> On 15 Sep 2019, at 13:59, Dimitri Pekarovsky <dimitri....@gmail.com> wrote:
>
> Hi.
>
> I write the app, http-amqp proxy, rack <-> bunny. I use the rack future to send the request to background (-1 as respond code) and then I use callback function to return the answer when it will be received from internal network.
>
> What is the best strategy to work with rabbit (and bunny):
>
> 0) At application start i create connection and channel. Then:
>
> a) for every http-request I create the queue, bind it with temporary key, create consumer and subscribe (less code in my app),
>
> b) or it is not the good choice with internals of rabbit? I will use some more code, like ruby Queue for internal distribution of messages from one static queue + subscribtion?
>
> //DP
>
> --
> Bunny: http://rubybunny.info
> March Hare: http://rubymarchhare.info
>  
> IRC: #rabbitmq on irc.freenode.net
>  

> Post to the group: ruby...@googlegroups.com | unsubscribe: ruby...@googlegroups.com

> ---
> You received this message because you are subscribed to the Google Groups "Ruby RabbitMQ libraries" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to ruby...@googlegroups.com.

[Michael Klishin]

> On 16 Sep 2019, at 04:52, Dimitri Pekarovsky <dimitri.p...@gmail.com> wrote:
>
> Thank you, Michael, for response.
>
> I wonder to clarify my question.
> I've read in docs, that rabbitmq don't like when clients makes multiple connections/disconnects many times (this is like my app will connect to rabbit for every http request and then disconnect).
> Sure this is not the best strategy, it's just a starting point for my assumptions.
> So I tought, will rabbit hurt from when making mutiple queues/deleting em for every http-request? You answered about temporary queues, thank you.

Well, declaring and explicitly deleting a queue, as well as every binding, is a network round trip, so if you can rely on a predeclared topology,
that’s a lot less wasteful. However, it is not always possible. Modern RabbitMQ provide metrics of queue and binding churn.

Opening new connections and channels is unnecessary as the docs explain. Opening a new connection is *multiple* round trips and a TLS
handshake if you use TLS. High connection churn is a problematic scenario for every tool, in particular those that assumes long lived connections [1].

>
> > The biggest hurdle with Rack, which was originally based on a synchronous request-response model, is how to await for a response

> in a way that is not terribly inefficient or makes error communication really hard
>
> I save env to global hash, then return [-1, {}, []], which means asynchronous and then I can call the env['async.callback'].call from the bunny consumer when response is ready. By such manner rack keeps the connection up until the call of the callback.
>

OK, assuming that works as advertised this sounds like a good option. Would you be interested in putting together an example app, publishing
it on GitHub and announcing it on rabbitmq-users? I’m sure many present and future Ruby users would appreciate that :)

1. https://www.rabbitmq.com/networking.html#dealing-with-high-connection-churn

[Dimitri Pekarovsky]

> Would you be interested in putting together an example app,

Sure will do it, when I will not be ashamed of my code :)

//DP

понедельник, 16 сентября 2019 г., 20:55:53 UTC+3 пользователь Michael Klishin написал: