Help need in tuning RabbitMQ for performance

[Thirunavukkarasu Ravichandran]

Hi,

First of all, Thanks for the great tool - RabbitMQ!!!

We use RabbitMQ in our application and it is the backbone for your application. Our application composes of 12 micro services and each service communicate with each other using the RabbitMQ.

Currently, we have reached a situation, where our performance gets hindered because of RabbitMQ (correct me, if I am wrong). 

Let me first give you the context, we have 30-40 queues in the topic exchange and services will be listening to the corresponding queue. There is one component which orchestrates where all the message should go to. All the other component will be listening to a queue and once the message is there, it will process the message and put its response to queue of the orchestrator. The orchestrator will find the next queue and publish the message in the next queue. 

The problem here, since all the components will be publishing its response to the orchestrator queue, the queue is getting flooded and  the state of the queue is  changed to "Flow". We tried tuning the RabbitMQ configuration, but we are still facing the issue. 

Following are tuning we did,

• Increased the consumer count to 15 (this is because, we run our application in 16 core machine)
• Increased the prefetch count to 100

Can you please tell us, if RabbitMQ is slowing down the message rate when it is in flow state? If so, can you please suggest a way to overcome that problem?

Any other tip is also appreciated!!!

Thanks in advance!!!!

-- 

Regards

Thirunavukkarasu.R

[Michael Klishin]

Consumers are generally not throttled. Throttled

connections should not affect consumers. Channels in flow state shouldn't either.

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Michael Klishin]

See https://www.rabbitmq.com/blog/2014/04/14/finding-bottlenecks-with-rabbitmq-3-3/ for some measurable values

to be aware of and https://groups.google.com/d/msg/rabbitmq-users/p6CnpiBXf6E/-dvYlG7PBAAJ (there are many

more threads like that) for various throughput-related suggestions.

Your problem in general is that your system is designed in such a way that a single queue is a bottleneck

for all other components. This can only be optimized so far.

If you can trade off message ordering for parallelism, http://github.com/rabbitmq/rabbitmq-sharding might work

for your case. Otherwise all of the options will eventually hit the same wall: a single queue (and a specific node

that hosts its master, if it's mirrored). So perhaps it's worth investing time into redesigning the system

to not depend on a single queue.

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

MK

Staff Software Engineer, Pivotal/RabbitMQ

[Thirunavukkarasu Ravichandran]

Thanks Micheal!!! one more question.

Can I have number of consumers beyond the number of cores in client side?

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Michael Klishin]

It depends on what your consumers do and what client library is used.

> >> state of the queue is changed to *"Flow". *We tried tuning the RabbitMQ

> >> configuration, but we are still facing the issue.
> >>
> >> Following are tuning we did,
> >>

> >> - Increased the consumer count to 15 (this is because, we run our

> >> application in 16 core machine)

> >> - Increased the prefetch count to 100

> >>
> >> Can you please tell us, if RabbitMQ is slowing down the message rate when
> >> it is in flow state? If so, can you please suggest a way to overcome that
> >> problem?
> >>
> >> Any other tip is also appreciated!!!
> >>
> >>
> >> Thanks in advance!!!!
> >>
> >>
> >>
> >> --
> >>
> >> Regards
> >> Thirunavukkarasu.R
> >>
> >> --
> >> You received this message because you are subscribed to the Google Groups
> >> "rabbitmq-users" group.
> >> To unsubscribe from this group and stop receiving emails from it, send an
> >> email to rabbitmq-user...@googlegroups.com .
> >> To post to this group, send email to rabbitm...@googlegroups.com
> >> .
> >> For more options, visit https://groups.google.com/d/optout.
> >>
> >
> >
> >
> > --
> > MK
> >
> > Staff Software Engineer, Pivotal/RabbitMQ
> >
>
> --
> You received this message because you are subscribed to the Google Groups "rabbitmq-users"
> group.
> To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.

> To post to this group, send an email to rabbitm...@googlegroups.com.

[Thirunavukkarasu Ravichandran]

The consumer basically does a single DB call and the client library I use spring AMQP.

[Michael Klishin]

With Java client and brief processing time it's certainly not unreasonable to have twice as many
consumers as cores but you need to configure an adequate consumer work pool executor
on connection factory (so that it is of as many threads in size or dynamically growing).

Spring may have their own ways of doing the same thing, please consult the docs. 

On 11 August 2016 at 10:11:52, Thirunavukkarasu Ravichandran (thi...@sahajsoft.com) wrote:
> The consumer basically does a single DB call and the client library I use
> spring AMQP.
>
> On Thursday, August 11, 2016 at 12:31:14 PM UTC+5:30, Michael Klishin wrote:
> >
> > It depends on what your consumers do and what client library is used.
> >
> > On 11 August 2016 at 09:58:57, Thirunavukkarasu Ravichandran (

> > > To post to this group, send an email to rabbitm...@googlegroups.com