topic exchange

351 views
Skip to first unread message

tmsc...@gmail.com

unread,
Dec 14, 2015, 2:28:48 PM12/14/15
to masstransit-discuss
I don't have a lot of experience with RabbitMQ, but I'm considering using MassTransit on top of RabbitMQ for a project we're beginning work on.  The simplified environment/requirements...
  • a central service publishes events that are consumed by services/applications installed at customer locations 
    • a customer should not see another customer's data
  • services/applications at customer locations should be able to send requests the central service responds to
I know MassTransit supports request/response, as I've written a simple example to illustrate that, so I think the second requirement is covered.  However, I haven't figured out how to handle the message routing to meet the first requirement.  Based on my reading, I think I should probably create a topic exchange in RabbitMQ, but it seems that MassTransit creates/uses fanout instead.  Does anyone have any suggestions on how to address these issues using MassTransit?  Or is there another tool that might handle the requirements better?  

I appreciate the help.

Dru Sellers

unread,
Dec 15, 2015, 4:46:05 PM12/15/15
to masstrans...@googlegroups.com
So, the big thing with MassTransit is we make a lot of decisions on the routing for you. You subscribe to types (in the .Net sense) and MT will make sure you get it. MT is not JUST a rabbitmq library, it is an opinionated stack on how you should use rabbit as well in a .Net world. :)

-d

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-dis...@googlegroups.com.
To post to this group, send email to masstrans...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/masstransit-discuss/bf70fdb8-a73a-4360-adc3-5bc700453488%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Maverik Gately

unread,
Dec 16, 2015, 3:45:57 PM12/16/15
to masstransit-discuss
Hi there,

I've been trying to tackle a similar situation and uploaded my sample work at https://github.com/Venomed/MassTransit-MultitenantCredentialValidator

Have a look and maybe you can find some ideas on how to do your implementation.

Cheers,
Maverik

tmsc...@gmail.com

unread,
Dec 17, 2015, 5:37:08 PM12/17/15
to masstransit-discuss
Thank you for your responses, Dru and Maverick.  It seems that just about every library I've run across that interfaces with RabbitMQ is opinionated. :)  

I've spent a bit more time on this, and I really like a lot of things MassTransit has to offer.  I just don't know if I can force the square peg into the round hole.  (That's what your solution feels like, Maverick.)  I'll think about it some more, go over more requirements with our team, and see if I can come up with something on our end that fits better with what MassTransit offers.  If I can't, I'll have to move on.  

Again, I appreciate your responses.

Dru Sellers

unread,
Dec 18, 2015, 9:45:36 AM12/18/15
to masstrans...@googlegroups.com
The RabbitMQ client from RabbitMQ is going to be the only unopinionated one is my guess.

  • a central service publishes events that are consumed by services/applications installed at customer locations 
does this mean WAN? or will the apps at the customer locations connect to YOUR rabbitmq instance
    • a customer should not see another customer's data
easy enough
  • services/applications at customer locations should be able to send requests the central service responds to
you can do that too


Based on your simple requirements there is no reason MT can't fit this bill. :)


Talk to me more about "Message Routing" - ok, so the question is how to get the routing from the central server to the client applications.

so, your customers could connect to Virtual Hosts - that would keep customers from seeing other data.

@chris - do you have any fancy pants stuff for VHOSTs?

@tmschlot: you might have to build a bridge that has the smarts to route from the general to the specific - which wouldn't be that hard and would make your routing nice and clean.

-d


--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-dis...@googlegroups.com.
To post to this group, send email to masstrans...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/masstransit-discuss/ae0410fc-e047-4bec-b496-90f30d14c50c%40googlegroups.com.

tmsc...@gmail.com

unread,
Dec 18, 2015, 11:40:50 AM12/18/15
to masstransit-discuss
You're getting my hopes up, Dru!  

I believe all of the customer applications and services would be connected to our RabbitMQ instance via VPN.  It could end up being quite a few connections (we're planning to limit it to one per customer location), but this path seems straightforward.

Here are some simplified examples of messages/interactions that we think we want to do...
  • Request/response from customer
    • When customer A's service starts up, it sends a request for users updated since it was last online.  A service in our data center responds back to client A with the list of users. 
  • Events from central server
    • Someone creates a new user for customer A.  
      • All other services in the data center that are interested/subscribed should be notified
      • Customer A should be notified
      • Customer B should not be notified
I've already put together a proof-of-concept that demonstrates MassTransit in some common scenarios within our data center.  And we've talked about possibly using some kind of bridge to route messages from our data center to customer locations.  The logic isn't that difficult; I'm not seeing how to set that part up with MassTransit, though.  Obviously, receiving the messages is just like anything else in our data center.  But without using RabbitMQ's client, I don't see how to easily forward filtered messages to another exchange or queue.  And any bridge cannot get in the way of request/response from the customers. 

Does that clarify some of the requirements?  Any suggestions are more than welcome.

Thanks for helping me think through this!



To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-discuss+unsub...@googlegroups.com.

To post to this group, send email to masstrans...@googlegroups.com.

Chris Patterson

unread,
Dec 18, 2015, 11:59:23 AM12/18/15
to masstrans...@googlegroups.com
If you want to use RabbitMQ for this, you might want to check out Federation, but you'd have a ton of RabbitMQ instances.

The thing is, credentials are not super easy to manage -- but neither are 1000 clusters... So it's a pick your poison approach in that case.


To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-dis...@googlegroups.com.
To post to this group, send email to masstrans...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-dis...@googlegroups.com.

To post to this group, send email to masstrans...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/masstransit-discuss/d7363181-45ed-4bb2-9eef-ec9bee569265%40googlegroups.com.

Dru Sellers

unread,
Dec 18, 2015, 2:07:44 PM12/18/15
to masstrans...@googlegroups.com
So, you could send a command to the "router" it would take that message and then look at which customer it should be directed to. From there it could connect to the VHOST for that customer and dump it in a known queue that the client instance would listen on. Do its work and then communicate back to the central server.

Its more like a request/response and direct sends then publishing but it should work quite well.

@chris: am I off my rocker?

-d


To view this discussion on the web visit https://groups.google.com/d/msgid/masstransit-discuss/CAHX4WYegbZZ-vRPPFfN5-2joh96KcTq1zCMZGG%2B5hNTtPNvyxw%40mail.gmail.com.

tmsc...@gmail.com

unread,
Dec 21, 2015, 9:22:38 AM12/21/15
to masstransit-discuss
I'm having a bit of trouble with this.  Could you tell me what I'm doing wrong?
  • Service is generating events on virtual host "/"
  • Bridge is...
    • consuming events on virtual host "/"
    • filtering events based on content
    • attempting to send filtered content to virtual host "/xyz-virtualhost/xyz-exchange"
  • Consumer is consuming events at "/xyz-virtualhost/xyz-queue"
From RabbitMQ's management UI, I see that the consumer appears to be setting up exchanges and queues as it normally would in the "/xyz-virtualhost" space.  As I begin execution, I see the bridge pick up events and attempt to send them, but I see an error when sending...

MassTransit.Pipeline.Filters.RescueReceiveContextFilter<MassTransit.ReceiveContext> Error: 0 : Rescuing exception, MassTransit.EndpointNotFoundException: The endpoint address specified an unknown host: rabbitmq://localhost/xyz-virtualhost/xyz-exchange

Again, I'm attempting to route the messages to the consumer by having the bridge send to "rabbitmq://localhost/xyz-virtualhost/xyz-exchange".  I have not set up any other users, and I gave guest access to the new virtual host when I set it up.  The consumer is using guest to connect to the new virtual host, also.  

My code to "forward" the event looks like...

Uri destinationAddress = new Uri("rabbitmq://localhost/xyz-virtualhost/xyz-exchange");
ISendEndpoint destination = await context.GetSendEndpoint(destinationAddress);
await destination.Send(myEvent);

What did I screw up?

Thanks!
To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-discuss+unsub...@googlegroups.com.
To post to this group, send email to masstrans...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-discuss+unsub...@googlegroups.com.

To post to this group, send email to masstrans...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-discuss+unsub...@googlegroups.com.

To post to this group, send email to masstrans...@googlegroups.com.

Chris Patterson

unread,
Dec 21, 2015, 10:29:10 AM12/21/15
to masstrans...@googlegroups.com
The easiest way to track this (and most flexible, if you add additional virtual hosts) is to create a bus for each host since each virtual host requires a separate RabbitMQ connection. Of course, you could just add all the hosts to a single bus, and then send using the proper virtual host, but that's going to require the bus to be restarted for each virtual host added.

Then just keep a cache of the bus instances by virtual host name, and pull the proper one and either publish and/or send to the bus for the appropriate virtual host.



To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-dis...@googlegroups.com.
To post to this group, send email to masstrans...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-dis...@googlegroups.com.

To post to this group, send email to masstrans...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-dis...@googlegroups.com.
To post to this group, send email to masstrans...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-dis...@googlegroups.com.

To post to this group, send email to masstrans...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/masstransit-discuss/5b031e04-9b6f-4c48-bd29-04f9fdf71299%40googlegroups.com.

tmsc...@gmail.com

unread,
Dec 21, 2015, 10:59:30 AM12/21/15
to masstransit-discuss
Chris

I don't think I understand what you are suggesting.  Are you saying to create an exchange (I'm using RabbitMQ language) for each end-consumer on the default virtual host ("/") and route that to an exchange in that consumer's virtual host via a Send?  Or create an exchange for each end-consumer in their own virtual host and Send to that?  

Based on your comments related to needing a separate RabbitMQ connection for each virtual host, I'm guessing that might be the problem.  Right now, I have a connection to the "/" virtual host only.  I'll attempt to create and keep a reference to the "other" virtual host and attempt to Send or Publish using that.  (I think this is where your "cache" comment comes from too... correct?)

Thanks.
To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-discuss+unsub...@googlegroups.com.
To post to this group, send email to masstrans...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-discuss+unsub...@googlegroups.com.

To post to this group, send email to masstrans...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-discuss+unsub...@googlegroups.com.
To post to this group, send email to masstrans...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-discuss+unsub...@googlegroups.com.

To post to this group, send email to masstrans...@googlegroups.com.

Chris Patterson

unread,
Dec 21, 2015, 11:27:01 AM12/21/15
to masstrans...@googlegroups.com
The latter part of your response is where I was going, you need a separate bus for each virtual host, so that you are able to publish/send to that virtual host. For the reverse direction, you could also setup a receive endpoint on each of those buses to get messages back from that virtual host and bridge them into the root virtual host.

We actually built a system similar to this years ago that did that very same thing - to also ensure that credentials per remote system were unique. And we bridged messages between the virtual hosts, adding the source credentials to the message as part of the bridge.

To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-dis...@googlegroups.com.
To post to this group, send email to masstrans...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-dis...@googlegroups.com.

To post to this group, send email to masstrans...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-dis...@googlegroups.com.
To post to this group, send email to masstrans...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-dis...@googlegroups.com.

To post to this group, send email to masstrans...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-dis...@googlegroups.com.

To post to this group, send email to masstrans...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/masstransit-discuss/8918cb4f-4772-4846-b726-2e0a2b65ca16%40googlegroups.com.

tmsc...@gmail.com

unread,
Dec 21, 2015, 2:49:12 PM12/21/15
to masstransit-discuss
Appreciate the help, Chris.  I've got this working, and it's pretty straight-forward.  This is essentially what I was hoping I could do.  Thanks again!

To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-discuss+unsub...@googlegroups.com.
To post to this group, send email to masstrans...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-discuss+unsub...@googlegroups.com.

To post to this group, send email to masstrans...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-discuss+unsub...@googlegroups.com.
To post to this group, send email to masstrans...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-discuss+unsub...@googlegroups.com.

To post to this group, send email to masstrans...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-discuss+unsub...@googlegroups.com.

To post to this group, send email to masstrans...@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages