Concurrent request betweem php amqplib and RabbitMQ

[tintu joseph]

 I would like to get a little bit of clarification regarding concurrent requests handling in RabbitMQ. At present I am involved in a project that requires to handle 1 lakh (0.1 million) concurrent requests. The receiving end of these requests is NGINX and from there we use CGI to forward the request to php (php-fpm). The next requirement is to get the payload of these message to a logging module using RabbitMQ. We are using phpamqplib client library for this. Upto 10k concurrent connections the rabbitmq passes messages without much delay. But as the concurrent request increases the delay to get all messages at consumer side increases. I tried implementing Singleton in php to reuse connection but it is not possible since state is not maintained between processes in PHP. Has anybody used phpamqlip to forward above 10k concurrent requests to RabitMq without delay in any method including Singleton in php?

[Michael Klishin]

Sounds like you actually need 10K connections. See rabbitmq.com/networking.html.

MK

On 23/7/2015, at 7:17, tintu joseph <joseph...@gmail.com> wrote:

 I would like to get a little bit of clarification regarding concurrent requests handling in RabbitMQ. At present I am involved in a project that requires to handle 1 lakh (0.1 million) concurrent requests. The receiving end of these requests is NGINX and from there we use CGI to forward the request to php (php-fpm). The next requirement is to get the payload of these message to a logging module using RabbitMQ. We are using phpamqplib client library for this. Upto 10k concurrent connections the rabbitmq passes messages without much delay. But as the concurrent request increases the delay to get all messages at consumer side increases. I tried implementing Singleton in php to reuse connection but it is not possible since state is not maintained between processes in PHP. Has anybody used phpamqlip to forward above 10k concurrent requests to RabitMq without delay in any method including Singleton in php?

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[tintu joseph]

@Michael Kishin

10k concurrent connection will be handled by RabbitMQ. The issue is when the  concurrent connection goes above 10k to 1 lakh. At that point the best approach is to reuse the connection and  the issue is reusing the connection between phpamqlib (php rabbitmq client library) and rabbitmq is not possible since singleton cannot be implement in php.

[Michael Klishin]

I'm not sure what 1 lakh is but get the idea.

Yes, it is a good idea to use fewer connections. I was saying that PHP makes that at least much harder. The guide linked to explains how to tune RabbitMQ and the kernel to let you reach 100s of thousands of connections. It's not trivial but doable.

MK