Local/Remote Broker bridging?

[Roman Gaufman]

Hi there,

I am interested in MQTT for it's simplicity and use case in applications like the internet of things, but I'm struggling to find any tutorials on how to have local brokers bridges to other local brokers and remote brokers. Is there a tutorial or blog post or more detailed documentation? - Will this scale to tens of thousands of local brokers bridging to a remote broker over an unreliably connection?

For example, imagine there are 3 machines:

1. Raspberry Pi in charge of alarm with some motion sensors
2. Linux box in charge of recording from CCTV cameras and home automation
3. Central server online with a web interface.

Some example cases are:

1. The Pi and Linux box message the central server that alarm is armed or motion was detected on the cameras
2. The central server messages the Linux box to stop recording or the Pi to arm the alarm remotely
3. The Pi messages the Linux box that alarm was armed, for the Linux box to set the thermostat to a lower settings (this needs to work even if the central server is unreachable).

How would you achieve this kind of bi-directional communication without any message loss? - Would MQTT be a good fit for this kind of application?

Right now we are using AMQP with RabbitMQ which works but seems a bit heavy. The RabbitMQ server takes up significant CPU just idling and requires Erlang to be installed which is quite a beast. What I do like about the current setup is once you set up federation between the brokers, you can subscribe to arbitrary topics and RabbitMQ handles the bridging automatically for you and ensures messages are passed to all relevant brokers that are subscribed to receive those messages (using a topic). Is something similar possible with MQTT?

Thank you in advance :)

[Ian Craggs]

Roman,

bridging between servers is not currently part of the MQTT specification.  Because of that, any bridging capability is MQTT-server specific, so the information about bridging would be found in the documentation for a particular server.

Very simple bridging, as implemented in Mosquitto (and RSMB) is primarily intended to allow an MQTT connection to an enterprise server.  You can have several Mosquitto servers linked together, but you would have to handle the configuration yourself.  This is very lightweight, but not particularly scalable, certainly not to tens of thousands of brokers.

There are enterprise servers (like IBM WebSphere MQ) which do have sophisticated clustering capabilities which will work with MQTT, but the more scalable the implementation, the less likely it is to be really lightweight, in my opinion. 

Even so, I'm not aware of any messaging servers that could scale to tens of thousands of nodes for any messaging protocol.  Anyone out there who does?

Ian Craggs (IBM)

[Vatsal Shah]

Hi Roman,

As Ian said, MQTT server bridging is still server specific.

Bridging carries lot of network overhead. Tens of thousands are tough to implement.

We have developed (Litmus Loop) clustering at large scale but certainly not lightweight !

I would suggest implementing mesh MQTT PubSub instead of bridging.

(Define few streams for PubSub and all clients communicate bidirectionally)

Vatsal Shah

Litmus Automation

[Roger Light]

Hi Roman,

There was a post to the mosquitto-users mailing list that you may find
interesting which discusses lots (3000) of bridges to a single broker.

The link is https://lists.launchpad.net/mosquitto-users/msg00335.html

Regards,

Roger

> --
> To learn more about MQTT please visit http://mqtt.org
> ---
> You received this message because you are subscribed to the Google Groups
> "MQTT" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to mqtt+uns...@googlegroups.com.
> To post to this group, send email to mq...@googlegroups.com.
> Visit this group at http://groups.google.com/group/mqtt.
> For more options, visit https://groups.google.com/d/optout.

[Karl P]

On 05/05/2014 09:21 PM, Vatsal Shah wrote:
> Hi Roman,

>
> Bridging carries lot of network overhead. Tens of thousands are tough to implement.
> We have developed (Litmus Loop) clustering at large scale but certainly not
> lightweight !

How so? What makes bridging more overhead than just mqtt clients? Tens of
thousands of anything is tough, but I'm not sure where you get the idea that
bridging is any worse?

Cheers,
Karl P

[anku...@gmail.com]

Hi,

I am new in MQTT . Just wanted to know what is the exact use case for using MQTT bridges. Does it add to scalability ?

Suppose I have many front line brokers which in turn will connect to a central broker. What advantage can I get from this? Can any one explain this! Thanks in advance.