Queues/Topics in Azure Service Bus/MT3

[Phil]

Hi all.

I'm starting to use MassTransit 3 over Azure Service Bus, and trying to build an understanding of how MassTransit uses ASB queues/topics, how messages go from publisher to subscriber, and how this differs to a plain old Azure Service Bus configuration without MassTransit.

Consider I have a publisher called MyPublisher which sends a message called MyMessage. I subscribe to this message with MySubscriber.

Looking in Azure Service Bus I can see that queues/topics are created roughly in line with what is described at http://docs.masstransit-project.com/en/latest/overview/underthehood.html#design-benefits :

1. MyPublisher process sends MyMessage to a queue in ASB called xxx_MyPublisher_xxx (I'll call this the "publisher instance" queue)

2. The message is removed from the xxx_MyPublisher_xxx queue and is sent to the MyMessage topic (I assume that the MyPublisher process does this?)

3. The message is removed from the MyMessage topic and is sent to the MySubscriber queue (I'll call this the "subscription queue". I assume that ASB does this as I can see the subscription has a "forward to" configured)

4. The message is removed from the MySubscriber queue and is sent to a queue called xxx_MySubscriber_xxx (I'll call this the "subcriber instance" queue. I assume that the MySubscriber process does this?)

5. The message is removed from the xxx_MySubscriber_xxx queue by the MySubscriber process.

In comparison, the equivalent functionality using plain old ASB without MassTransit results in a single topic being created. Now, the end-to-end process is...

a. MyPublisher process sends MyMessage to the MyMessage topic

b. The message is removed from the MyMessage topic by the MySubscriber process.

What benefit do we get from MassTransit publishing to the "publisher instance queue", the "subscription queue" and the "subscriber instance" queue when using ASB? I can understand why this would be important if the instance queues were physically local to the publisher/subscriber process, as this would protect us from an unreliable network connection. However, given these queues/topics are all running in the cloud is there actually any benefit? Is this a fundamental part of the mass transit architecture or is there any way to configure this behaviour so that the ASB transport publishes straight to the ASB topic/subscriber?

I believe that the billing in ASB is per message per queue, so the fact that I'm now seeing 5 MassTransit steps rather than 2 ASB steps means that adopting MassTransit over ASB has a financial impact that might be difficult to justify.

Thanks,

Phil.

[Long Mai]

Phil, it feels like you are one step ahead of me. You may have identified how mass transit is behaving with ASB, which I was going to start trailing down to understand. 

Where have you progressed in understanding why the extra queue before going into the topic? 

Have you made a decision on using MassTransit with ASB? If so, what are your experiences?

I'm debating if one of the projects (at its early stages) that we will be adopting to develop on should continue to use ASB directly or have a framework on top of it right now before it gets too far.

If anyone else has input, please share your experiences. Thanks!

[Long Mai]

I had time to dig a little and this is what I have determined. Feel free to correct my understanding.

The Aha moment is when I found the subscription endpoint. I thought the receive endpoint was what I needed.

I was wanting to do a simple pub/sub and the receive endpoint was not the right way to go about it. Simply put, you would want to publish on the bus, and configure subscription endpoint on the consumer.

As for send on the bus, I saw this, which can be helpful to whomever: https://groups.google.com/forum/#!searchin/masstransit-discuss/send$20map%7Csort:relevance/masstransit-discuss/ur0PcHKJ5_g/_jqIEvKiAAAJ . In short, the mapping endpoint configuration may be used, but unknown if it is production ready functionality. Not sure what the state of it as of today, but no documentation of it yet.

When configuring a receive endpoint, the way it works right now is that the message will get sent to the topic for that type and a subscription will get created for the queue name of the receive endpoint. From there, the subscription autoforwards to the queue. I'm still internalizing this part, but it may not be important for me right now.

Long Mai

[Chris Patterson]

If you have a consumer that only consumes a single message type, you could use a subscriptionEndpoint on that topic to save the overhead of forwarding the message to the queue. However, if you need to send directly to the consumer, or if you have a consumer that consumes multiple message types (which are published, in this case), you should use a receive endpoint. The topology of MT will automatically create the subscriptions and forward to the queues.

You cannot send directly to a subscriptionEndpoint, FYI. Only published messages will be delivered, and only those of the type on which the subscriptionEndpoint is created.

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-discuss+unsub...@googlegroups.com.
To post to this group, send email to masstransit-discuss@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/masstransit-discuss/2f4efdf0-9c26-426c-9346-4dea1d8491e6%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Long Mai]

Thanks for replying Chris.

When would you use the sending on the bus? I thought it was to cover the scenario of sending a message directly to the consumer. Besides tighter coupling, I'm having a hard time seeing other usages of picking one vs the other if there are any.

To post to this group, send email to masstrans...@googlegroups.com.

[Chris Patterson]

Commands are Sent, events are Published.

It really is that simple. I realize that some like to decouple by publishing commands, honestly I think that's a fail. And why I'm adding routing on top of topology in upcoming versions of MT. Because I get the desire to have loose coupling via convention, but using pub/sub for commands can lead to some bad habits. And not everyone gets it, so a new team may totally screw up your happy code by not realizing the impact of adding consumers.

If you're building a carefully engineered system, and can control contract ownership to make sure developers don't do stupid things, using publish for commands is a reasonable approach. I just don't like it.

And I'm not always right, so there's that.

To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-discuss+unsubscribe...@googlegroups.com.

To post to this group, send email to masstrans...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/masstransit-discuss/2f4efdf0-9c26-426c-9346-4dea1d8491e6%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-discuss+unsub...@googlegroups.com.

To post to this group, send email to masstransit-discuss@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/masstransit-discuss/7af26bb2-fd5b-4369-8b9d-9b94bff73385%40googlegroups.com.