Multiple consumers of the same message

[andrecarlucci]

Hello,

First of all, congrats for this great framework. I'm planning to use it extensively on the smart grid area :)

Is it possible to have one message being broadcast to many consumers, where each consumer gets a copy of the same message?

In my scenario I want to broadcast an entity update to all interested systems.

Thanks in advance,

André Carlucci

[Dru Sellers]

Yup. That's bread and butta' :)
-d

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To view this discussion on the web visit https://groups.google.com/d/msg/masstransit-discuss/-/sCdS2qzGqpEJ.
To post to this group, send email to masstrans...@googlegroups.com.
To unsubscribe from this group, send email to masstransit-dis...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/masstransit-discuss?hl=en.

[André Carlucci]

Great :)

So just to be clear: 

Every consumer has it's own queue name: everybody gets the same message.

Consumers sharing queue name: consumer competition.

Is that correct?

Also, do you think the version 2.x is already mature for production? I'm tempted to let thousand of power meters connect to it :)

André

--
André Carlucci
twitter/andrecarlucci

[Dru Sellers]

Every service.exe will have its own queue usually. and each 'consumer' in that 'service.exe' will get its own copy of the message.

Does that make sense?

What's your deploy timeline?

Personally, it is my opinion that you will be much better off on the 2.x line, if for no other reason problems will get fixed faster. The 1.x line you might have to fix yourself. Chris and I are both planning new projects that are going to be moving to the 2.0 line.

-d

[Travis Smith]

Competing consumers aren't really supported with MSMQ. You are welcomed to do that with RabbitMQ though. 

-Travis

[Dru Sellers]

and +1 million for RabbitMQ

[André Carlucci]

I got the idea, thanks.

Initially I don't need competing consumers, but I may in the future for load balancing.

The initial scenario will be two or three different data acquisition systems publishing meter data into the bus, and one consumer getting all the data and putting into an organized central repository.

After that, I'm planning to use the bus to share information between the systems too.

About the queue, I believe it's easier to start with MSMQ because it's available on every windows machine. If you would start a new project, would you pick RabbitMQ since the beginning? Is it easy to change it later?

Thanks,

André

[Dru Sellers]

I would pick rabbitmq for a host of reasons:


MUCH BETTER MGMT TOOLING (as in it exists)
secure (PKI encryption)
fewer ports
speed

its easy for your code, but you would have to think about it differently. we can talk in more depth, but RMQ all the way.

-d

[Travis Smith]

On MSMQ, you have other options for load balancing, I wrote the distributor for that specifically. 

Overall, I think RabbitMQ is much better messaging system than MSMQ. MSMQ hasn't really advanced in a number of years and it really shows when compared to newer MQs. 

-Travis

[André Carlucci]

Nice thoughts, I'll give a deeper look on RabbitMQ.

Tks for the thoughts guys, I really appreciated.

I let you know how the project goes.

Cheers,

André

[pasi...@georiot.com]

Hey guys,

I found this thread by way of http://stackoverflow.com/questions/9074411/how-to-implement-a-competing-consumer-solution

I'm looking at building out a competing consumer implementation, using RabbitMQ.  I have a multiple instances of a single logical service running, where each instance registers just one Consumer instance (which handles just one Message type), and each instance of this service is listening on the same queue (as I understand is possible with RabbitMQ).  

Everything was working just fine in my testing until now, but I just noticed two instances of my service picked up and processed the exact same message (at least once).  They were both sitting idle when the message was queued, and both got it at the same time.  That shouldn't happen, should it?  Any reasons you can think of that might have induced this state?

Thanks,

Jesse

[Travis Smith]

If two consumers are reading off the same queue, they shouldn't be able to consume the same message. The message will go into a 'busy' state until ack'ed. In the busy state, it's effectively gone from other consumers... What version of MT & RabbitMQ are you running? 

-Travis

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-dis...@googlegroups.com.

To post to this group, send email to masstrans...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/masstransit-discuss/e25d6d8c-5978-40d6-bff6-f97049c26d7d%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Chris Patterson]

You might just verify that everything is working properly. We use RabbitMQ and have many servers each running the same service pointing at the same queue without any duplicate deliveries (unless a consumer crashes, and then the message is redelivered to another node).

3.1.3/MT 2.8

[pasi...@georiot.com]

Looks like I accidentally forked the thread, due to some email client issues. (see other email for full description)  But yes, this got me thinking, and it looks like my consumer was actually throwing an exception due to some invalid input, causing the message to be retried several times on each host that was running.  Everything is working fine now. :)

On Wednesday, August 28, 2013 7:06:25 PM UTC-7, Chris Patterson wrote:

You might just verify that everything is working properly. We use RabbitMQ and have many servers each running the same service pointing at the same queue without any duplicate deliveries (unless a consumer crashes, and then the message is redelivered to another node).

3.1.3/MT 2.8

On Wed, Aug 28, 2013 at 5:55 PM, <pasi...@georiot.com> wrote:

Hey guys,

I found this thread by way of http://stackoverflow.com/questions/9074411/how-to-implement-a-competing-consumer-solution

I'm looking at building out a competing consumer implementation, using RabbitMQ.  I have a multiple instances of a single logical service running, where each instance registers just one Consumer instance (which handles just one Message type), and each instance of this service is listening on the same queue (as I understand is possible with RabbitMQ).  

Everything was working just fine in my testing until now, but I just noticed two instances of my service picked up and processed the exact same message (at least once).  They were both sitting idle when the message was queued, and both got it at the same time.  That shouldn't happen, should it?  Any reasons you can think of that might have induced this state?

Thanks,

Jesse

On Monday, August 29, 2011 1:04:53 PM UTC-7, andrecarlucci wrote:

Hello,

First of all, congrats for this great framework. I'm planning to use it extensively on the smart grid area :)

Is it possible to have one message being broadcast to many consumers, where each consumer gets a copy of the same message?

In my scenario I want to broadcast an entity update to all interested systems.

Thanks in advance,

André Carlucci

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-discuss+unsub...@googlegroups.com.

[Faisal Nadeem]

Thanks @Chris. Actually i have the same structure. I am running two same consumer services on two different machines. One of the consumer is consuming message but when i respond it is throwing the exception. Actually responding context trying to find the publisher endpoint which ins inaccessible. 

Any thoughts?

On Thursday, August 29, 2013 at 5:06:25 AM UTC+3, Chris Patterson wrote:

You might just verify that everything is working properly. We use RabbitMQ and have many servers each running the same service pointing at the same queue without any duplicate deliveries (unless a consumer crashes, and then the message is redelivered to another node).

3.1.3/MT 2.8

On Wed, Aug 28, 2013 at 5:55 PM, <pasi...@georiot.com> wrote:

Hey guys,

I found this thread by way of http://stackoverflow.com/questions/9074411/how-to-implement-a-competing-consumer-solution

I'm looking at building out a competing consumer implementation, using RabbitMQ.  I have a multiple instances of a single logical service running, where each instance registers just one Consumer instance (which handles just one Message type), and each instance of this service is listening on the same queue (as I understand is possible with RabbitMQ).  

Everything was working just fine in my testing until now, but I just noticed two instances of my service picked up and processed the exact same message (at least once).  They were both sitting idle when the message was queued, and both got it at the same time.  That shouldn't happen, should it?  Any reasons you can think of that might have induced this state?

Thanks,

Jesse

On Monday, August 29, 2011 1:04:53 PM UTC-7, andrecarlucci wrote:

Hello,

First of all, congrats for this great framework. I'm planning to use it extensively on the smart grid area :)

Is it possible to have one message being broadcast to many consumers, where each consumer gets a copy of the same message?

In my scenario I want to broadcast an entity update to all interested systems.

Thanks in advance,

André Carlucci

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-discuss+unsub...@googlegroups.com.