Proxying internal traffic to Azure Service Bus

[cocowalla]

What I'm trying to achieve is interoperability between RabbitMQ clients in an internal network, and Azure Service Bus consumers running in Azure.

The RabbitMQ clients need to publish and subscribe, as do the Azure Service Bus consumers - so I need some kind of 'bi-directional proxy'. A diagram (and gist incase it gets garbled):

                                      +
                    Internal network  |  Azure
                                      |
                                      |
+--------+                            |                               +----------+
| Client +---+                        |                           +---+ Consumer |
+--------+   |                        |                           |   +----------+
             |                        |                           |
             | +-----------------+    |    +-------------------+  |
             +-+ RabbitMQ Broker +---------+ Azure Service Bus +--+
             | +-----------------+    |    +-------------------+  |
             |                        |                           |
+--------+   |                        |                           |   +----------+
| Client +---+                        |                           +---+ Consumer |
+--------+                            |                               +----------+
                                      |
                                      |
                                      |
                                      +

AFAIK, both the RabbitMQ broker and Azure Service Bus can do AMQP 1.0. I've looked at the rabbit shovel plugin, but I *think* this would only handle publishing of messages from the internal clients to Azure, and wouldn't allow the clients to subscribe to messages published by the Azure consumers? Or have I got this wrong, and shovel will work for this?

If shovel won't work, is there any other way to achieve this?

(NOTE: I also posted this in rabbitmq-discuss, but afterwards noticed it's rather quiet there, so I guess this is where I should have posted in the first place!)

[Karl Nilsson]

Hi,

The rabbitmq-shovel is bi-directional so you can both send and receive messages from remote endpoints. Interacting with Azure can be a bit tricky due to the additional headers it requires. Some more details can be found in this thread: https://groups.google.com/d/msg/rabbitmq-users/vUKM8MIsmEM/XL3VYZZoBAAJ

Cheers

Karl

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Karl Nilsson

Pivotal/RabbitMQ

[cocowalla]

Wow, so I can just install shovel inside the network - without installing anything else in Azure - and it can effectively act as a 'proxy' to another broker? That sounds perfect! Is there any special configuration needed for this?

BTW, thanks for that link - I doubt I'd ever have figured out the partition key stuff myself!

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Karl Nilsson

Pivotal/RabbitMQ

[Karl Nilsson]

The shovel is a plugin to RabbitMQ - you want to use the AMQP 1.0 protocol option. The docs are here: https://www.rabbitmq.com/shovel.html

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Karl Nilsson

Pivotal/RabbitMQ

[cocowalla]

I've got some reading to do - many thanks for your help!

[Michael Klishin]

Shovel acts as a client (both consumer and publisher), so as long as it can connect to Azure, you can run

it on a node that's not hosted on Azure.

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

MK

Staff Software Engineer, Pivotal/RabbitMQ