Polling Consumer

[Ozcan Bircan]

I have multiple workers listening on the same queue using RabbitMQ and MassTransit. These workers are responsible to handle long running background jobs. Depending on the specified prefetchCount and concurrentConsumerLimit, they share the workload as expected.

A new requirement requires to pull messages (job requests) on demand, something like a polling consumer. The idea is to not to bloat any worker with more messages than it can handle and expect any other worker to handle that message, so, I need to check CPU and memory consumption on each worker before fetching any new message from the queue. The concurrent message count that are being processed on each worker should depend on the CPU and memory consumption on the worker, rather than the predefined prefetch count.

I was thinking to fetch message first and then check the Memory and CPU consumption, and send it back using something like RetryLater(). I would assume this sends the message back to the queue and it will be pushed to the end of the queue. But, I want to keep it at the head of the queue. Is that possible to send a negative acknowledgement for a message when consuming the message to send it back to the head of the queue? Or is there any other way to handle this in MassTransit? 

[Ozcan Bircan]

Shortly, for one queue only I need to rely on pull API (Fetch messages as needed) rather than push API (Have messages delivered to them).

Is that possible to implement this using MT and RabbitMQ?

[Chris Patterson]

MT doesn't do polling, with either RabbitMQ or Azure Service Bus.

On Mon, Mar 21, 2016 at 1:58 PM, Ozcan Bircan <ozcan...@gmail.com> wrote:

Shortly, for one queue only I need to rely on pull API (Fetch messages as needed) rather than push API (Have messages delivered to them).

Is that possible to implement this using MT and RabbitMQ?

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-dis...@googlegroups.com.
To post to this group, send email to masstrans...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/masstransit-discuss/94583e25-d0e3-4de7-8413-5fd6381a589a%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Ozcan Bircan]

Thanks for the clarification Chris.

[Tomer Yoskovich]

On Tuesday, March 22, 2016 at 2:18:53 AM UTC+2, Ozcan Bircan wrote:
> Thanks for the clarification Chris.

Hello Ozcan,
We are interested in implementing a system that is very similar to what you are trying to accomplish
Did you end up finding a solution to the problem?

Thanks