SetConcurrentConsumerLimit hard-capped at 10 regardless of what I pass above 10?

[Tyler Austen]

Hi gang,

I'm seeing some baffling behavior w.r.t. the concurrent consumption of messages, and I'm hoping someone can jump in and say "that's because you need to tweak the wiggle-wobble setting".  Here's some context:

* 8 core CPU

* Windows 7 Ultimate

* MassTransit 2.7.x  (I haven't rolled back yet to see if it's happening with 2.1.x)

* RabbitMQ

We just noticed that there seems to be a hard limit of 10 concurrent messages consumed in our processes, eventhough we've been setting the limit to 100, 1000, etc..  If I change the call to SetConcurrentConsumerLimit to 2, this is honored...  but anything above 10 is capped to just 10.

I thought something might be messing with the .Net thread pool limits, so I queried ThreadPool and dumped these stats within my consumer and find:

Current workerThreads [32767], complPortThreads [1000]

MAX workerThreads [32767], complPortThreads [1000]

MIN workerThreads [8], complPortThreads [8]

So I know the default .Net thread pool is not forcing this limit.

In the past (and unfortunately I don't remember the timing and/or version of MassTransit we were using them), I've seen 100's and 1000's of concurrent messages going through my consumers...  but now it's just 10.

Any ideas what might be happening?  is there a newish config setting that I need to set in addition to the ConcurrentConsumerLimit?  anything system-wide that could be limiting MassTransit when trying to pull from Rabbit? etc..

The few remaining hairs on my head thank yas in advance for any help,

Tyler

[Travis Smith]

I know we're running with 10's to 100's on different services. But we're not at the latest version - so it's possible something has changed. Have you been able to replicate this with just a minimal example service? Regardless, opening an issue would be a good place to start. 

-Travis

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-dis...@googlegroups.com.
To post to this group, send email to masstrans...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/masstransit-discuss/c54fe06f-5113-425b-8f6f-da82c98f98eb%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Chris Patterson]

You need to up the prefetch count to an equal number in order to allow RabbitMQ to push more messages to the client, as it will limit the number of unack'd messages pushed to the client based on that prefetch count.

?prefetch=32

On the URI will give you 32 messages at the client, which should match the ConcurrentConsumerLimit(32).

To view this discussion on the web visit https://groups.google.com/d/msgid/masstransit-discuss/CAAqUTbm6smd8GKF-xrsrp8FaS%2BJs6604FixXw2%2BoVRTzGTvwqg%40mail.gmail.com.

[Tyler Austen]

Awesome!  Thanks you guys...  this was driving me nuts.  I'll chase down how to pass the prefetch bit through to rabbit.

-Travis

To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-discuss+unsub...@googlegroups.com.

To post to this group, send email to masstrans...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/masstransit-discuss/c54fe06f-5113-425b-8f6f-da82c98f98eb%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-discuss+unsub...@googlegroups.com.

To post to this group, send email to masstrans...@googlegroups.com.

[Chris Patterson]

You just add to the URI.

rabbitmq://host/vhost/queue?prefetch=32

To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-dis...@googlegroups.com.

To post to this group, send email to masstrans...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/masstransit-discuss/5590da59-bf8f-4f56-ac90-58b7d48ee20f%40googlegroups.com.

[Tyler Austen]

Thanks again Chris..  it works like a charm.

-Travis

To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-discuss+unsubscribe...@googlegroups.com.

To post to this group, send email to masstrans...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/masstransit-discuss/c54fe06f-5113-425b-8f6f-da82c98f98eb%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-discuss+unsubscribe...@googlegroups.com.

To post to this group, send email to masstrans...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/masstransit-discuss/CAAqUTbm6smd8GKF-xrsrp8FaS%2BJs6604FixXw2%2BoVRTzGTvwqg%40mail.gmail.com.

For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-discuss+unsub...@googlegroups.com.
To post to this group, send email to masstrans...@googlegroups.com.

[Wiktor Zychla]

It seems I have found something that looks like an issue in the implementation. Could someone from the team take a look at it, please?

Not only I have a workaround, I also have a possible culprit.

http://stackoverflow.com/questions/23997190/masstransit-capping-message-rates-at-10/31137140#31137140

Regards,
Wiktor