Simple publisher best practices
dc...@prosentient.com.au
Hi all,
I’m looking for some simple publisher best practices. I thought to write here after seeing Luke’s answer on https://stackoverflow.com/questions/54901637/wrap-rabbitmq-connection-in-a-singleton-class.
In my scenario, I have a few single-threaded (legacy) daemons where I’m wanting to publish a variable volume of messages.
So far, I am thinking that I’ll wrap the client library (https://metacpan.org/pod/Net::AMQP::RabbitMQ) with a singleton class so that I can only have a maximum of 1 connection per process, so that I get the benefits of having a persistent TCP connection.
However, since these daemons are single-threaded, I am worried about blocking too long while dealing with RabbitMQ (e.g. network timeout, latency of publishing a message, etc.), so I was thinking about spawning a child process to handle the messaging, but that’s just moving the same problem down the chain a bit, since then I’m just creating an internal queue for a publisher to send to the RabbitMQ queue.
Is there a best practice for RabbitMQ to resolve this situation, or is it just a case of legacy architecture just being too problematic? (In theory, the daemons should use threads or child processes to do the work instead of the master process doing everything. Then I could have 1 process for the daemon and use a different channel per thread, or a different connection per child process even.)
Thanks in advance for any advice or tips for an optimal setup for this particular scenario.
David Cook
Software Engineer
Prosentient Systems
Suite 7.03
6a Glen St
Milsons Point NSW 2061
Australia
Office: 02 9212 0899
Online: 02 8005 0595
dc...@prosentient.com.au
In terms of connections/channels, https://www.rabbitmq.com/channels.html seems to reinforce the idea of having 1 connection per process and then use a new channel per thread and maybe child process? The wording isn’t quite clear there. There’s talk about channel pooling although I imagine client library support for that varies.
David Cook
Software Engineer
Prosentient Systems
Suite 7.03
6a Glen St
Milsons Point NSW 2061
Australia
Office: 02 9212 0899
Online: 02 8005 0595
--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/01a501d72454%24ff9a8b10%24fecfa130%24%40prosentient.com.au.
dc...@prosentient.com.au
For my scenario, the publisher is part of a pipeline, so it doesn’t have any concept of batching. It prepares each message individually so it seems like I need to open/close channels on a per message basis. Or since I’m using single threaded processes, I could open a default channel (eg 1) at connection time and use that for the life of the connection?
To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/01b401d7245c%24ab6f0590%24024d10b0%24%40prosentient.com.au.