Multi-threaded consumers

[John Lilley]

I'm reading here:

https://www.rabbitmq.com/api-guide.html#consumer-thread-pool

specifically "Consumer threads (see Receiving below) are automatically allocated in a new ExecutorService thread pool by default".

This leads me to two questions:

-- Does this automatically set up a multi-threaded executor for the consumer?  Will my consumer need to be thread-safe?

-- If not, what is the best approach to a creating a multi-threaded consumer?

I also read that "sharing Channel instances between threads is something to be avoided", so this would imply that I need to create multiple Channels and create a separate consumer on each one?

Any guidance or RTFM in this area much appreciated.

Thanks

john

[John Lilley]

I did run across this: https://stackoverflow.com/questions/18531072/rabbitmq-by-example-multiple-threads-channels-and-queues

which suggests using an ExecutorService to submit tasks, which presumably carry out the work and return the reply.  Is this a recommended pattern?

john

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Michael Klishin]

If the executor service has a single thread (e.g. fixed in size), there can be no concurrency hazards.

Otherwise deliveries are dispatched to the pool in [per-channel] order but they can be handled concurrently.

Thread safety is a a property of the actions your consumers perform.

If your consumers share mutable state of any kind, it's a good idea to make sure they synchronise access to

that state or otherwise address any possible concurrency hazards.

On Mon, Jun 25, 2018 at 7:55 PM, John Lilley <whe...@gmail.com> wrote:

I did run across this: https://stackoverflow.com/questions/18531072/rabbitmq-by-example-multiple-threads-channels-and-queues

which suggests using an ExecutorService to submit tasks, which presumably carry out the work and return the reply.  Is this a recommended pattern?

john

On Mon, Jun 25, 2018 at 10:46 AM John Lilley <whe...@gmail.com> wrote:

I'm reading here:

https://www.rabbitmq.com/api-guide.html#consumer-thread-pool

specifically "Consumer threads (see Receiving below) are automatically allocated in a new ExecutorService thread pool by default".

This leads me to two questions:

-- Does this automatically set up a multi-threaded executor for the consumer?  Will my consumer need to be thread-safe?

-- If not, what is the best approach to a creating a multi-threaded consumer?

I also read that "sharing Channel instances between threads is something to be avoided", so this would imply that I need to create multiple Channels and create a separate consumer on each one?

Any guidance or RTFM in this area much appreciated.

Thanks

john

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

MK

Staff Software Engineer, Pivotal/RabbitMQ

[John Lilley]

I am struggling with this at a more basic level.  Like, of I do nothing to create or specify an ExecutorService, and just call basicConsume() on a channel with a simple handler method, how many dispatch threads are created by default?  One? I am unclear about whether any parallelism happens automatically, or whether I must explicitly set that up.  Can you point me to single-vs-multi-thread consumer examples?

Thanks

John

On Mon, Jun 25, 2018, 12:42 PM Michael Klishin <mkli...@pivotal.io> wrote:

If the executor service has a single thread (e.g. fixed in size), there can be no concurrency hazards.

Otherwise deliveries are dispatched to the pool in [per-channel] order but they can be handled concurrently.

Thread safety is a a property of the actions your consumers perform.

If your consumers share mutable state of any kind, it's a good idea to make sure they synchronise access to

that state or otherwise address any possible concurrency hazards.

On Mon, Jun 25, 2018 at 7:55 PM, John Lilley <whe...@gmail.com> wrote:

I did run across this: https://stackoverflow.com/questions/18531072/rabbitmq-by-example-multiple-threads-channels-and-queues

which suggests using an ExecutorService to submit tasks, which presumably carry out the work and return the reply.  Is this a recommended pattern?

john

On Mon, Jun 25, 2018 at 10:46 AM John Lilley <whe...@gmail.com> wrote:

I'm reading here:

https://www.rabbitmq.com/api-guide.html#consumer-thread-pool

specifically "Consumer threads (see Receiving below) are automatically allocated in a new ExecutorService thread pool by default".

This leads me to two questions:

-- Does this automatically set up a multi-threaded executor for the consumer?  Will my consumer need to be thread-safe?

-- If not, what is the best approach to a creating a multi-threaded consumer?

I also read that "sharing Channel instances between threads is something to be avoided", so this would imply that I need to create multiple Channels and create a separate consumer on each one?

Any guidance or RTFM in this area much appreciated.

Thanks

john

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

MK

Staff Software Engineer, Pivotal/RabbitMQ

--

You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.

[Michael Klishin]

Twice as many as there are available CPU cores, IIRC. You don't need to set anything

up for concurrent dispatch. [1] provides a reasonably detailed description of what is and isn't considered to be safe to do.

If consumers don't have any shared mutable state (including shared data stores and similar external resources), chances

are you don't have to worry about concurrency at all.

1. http://www.rabbitmq.com/api-guide.html#channel-threads

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

MK

Staff Software Engineer, Pivotal/RabbitMQ

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[John Lilley]

Thanks!  I'm starting to get a handle on this.

john

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

MK

Staff Software Engineer, Pivotal/RabbitMQ

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

MK

Staff Software Engineer, Pivotal/RabbitMQ

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.