Multi Tenancy Design

[William Wilson]

We are building a multi tenant solution utilizing RabbitMQ as the message broker.  The teams that make use of the message broker are internal teams.  The solution is multi tenant in that each customer will be logically (and at some layers, physically) separated from one another, however, they are all effectively clones (they'll have the same set of exchanges and queues).  Is this a good use-case for virtual hosts?  I'm concerned that each virtual host requires a separate connection, so we won't be able to effectively use a connection pool and as the number of services grows that utilizes the broker, the broker itself will have a large number of connections open.  We are looking at tens of thousands of tenants potentially.

The alternative we've discussed is having a single virtual host and creating separate queues per tenant.  On the publishing side this would allow us to pool connections and channels, but on the consumer side we'd still be creating a large number of subscriptions (due to the separate queues per tenant) so we'd end up with a large number of subscriptions per channel, or a large number of channels per connection (and as I'm sure you're well aware, there are limits here in amqp).

Any advice or suggestions are greatly appreciated.  If further details are needed I can provide more.  This is our first production use of RabbitMQ and while we intend to evolve the solution as it grows, I'm looking to benefit from the community experience if I can to avoid a headache later.

[vikinghawk]

I had a similar usecase (although 100s of tenants not 10000s) and went with a topic exchange per tenant all running out of 1 vhost. The main reason for doing so was the same as you mention... I wanted the ability to share the connection and channel pools between tenants. Also in your case, creating and managing 10000s of vhosts may be additional overhead.

[Michael Klishin]

In 3.7.x and future versions each virtual host has its own message stores and generally some stateful parts that consume

additional memory. 100K virtual hosts is going to be very wasteful and admittedly it's not a workload the current design is optimized for.

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

MK

Staff Software Engineer, Pivotal/RabbitMQ