2,5 millions of updates/minute and 1,2 billion of updates/day is RabbitMQ the right solution ?

[piyush...@smarttra.de]

I have used rabbitMQ in the past but the amount of traffic that needed to be handle was not this much. 

I am being a bit skeptical and would like to know from the community, would RabbitMQ be able to handle so much messages that will be coming from a single source, basically a TCP/IP based application that is connecting to multiple source points. 

Looking forward to hear from the community. 

regards, 

Piyush

[Michael Klishin]

It depends.

2.5M a minute is 42K/second, which is what a reasonably beefy single node can do (with the most simplistic and naive workload you can imagine: "hello, world" style).

From there it all depends on your workload, message size, what your apps do, how long will messages be enqueued for, whether your consumers

can be parallelized, and so on.

RabbitMQ can go quite a bit higher than 42K/second but with a specific workload and set of apps:

https://blog.pivotal.io/pivotal/products/rabbitmq-hits-one-million-messages-per-second-on-google-compute-engine

However, that number alone is pointless because your apps probably won't be doing exactly the same thing.

See https://groups.google.com/d/msg/rabbitmq-users/p6CnpiBXf6E/-dvYlG7PBAAJ, for example.

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

MK

Staff Software Engineer, Pivotal/RabbitMQ

[V Z]

I've transmitted half a million messages per second without delivery guarantees (but no messages were lost), so it all depends on desired quality of service.

[Sujoy Ghosh]

Hi!!

We see to be stuck at a throughput of 1000msg/second on a 8core,32gb server.

We are serving an API endpoint in which we have 2 processes to be completed
1. Hitting another HTTP API to  
2. Save the response status to mongodb .

We are using Celery with RabbitMq and we are able to process 1k requests per second with 2 workers and 8 concurrencies. The configuration of server is 8 CPU 32 GB Ram.

How do we get to 30+k messages per second on this node.

Any help is appreciated.

[Luke Bakken]

Hello,

Please start a new message thread rather than replying to one that is over two years old.

I followed up here since I assume this is the same person - https://stackoverflow.com/questions/54401702/tune-rabbitmq-server-to-handle-more-request

There is no proof that RabbitMQ is the limiting factor in the (very limited) description of your environment.

Thanks,

Luke