MQTT brokers behind a load balancer

[Ian Su]

We're looking to build a highly available/scalable MQTT system in the AWS cloud. Many publishers will publish to a single elastic load balancer (ELB) endpoint, with many brokers behind it. In order to subscribe to and consume the messages, instead of bridging all the brokers, which will grow and shrink based on usage in an auto-scaling group (ASG), we are considering simply having one subscriber per broker on the same instance consuming the messages. 

It seems to me that this is a common scenario (very similar to this), is this a valid way of doing things, and are there things we need to look out for? E.g. is it an issue that when publishing, two messages from the same client might in reality be hitting two different brokers behind the same ELB?

[Dominik Obermaier]

Hi Ian,

Mqtt brokers in the amazon cloud with ELBs are a common scenario. We've done several projects with that setup.

There are some gotchas when using AWS, though.

* ELBs disconnect TCP connections after a minute if there was no data transfer (like a PINGREQs), sp you have to make sure your CONNECTs keepalive is small.

On Thursday, 24. October 2013 at 06:43, Ian Su wrote:

We're looking to build a highly available/scalable MQTT system in the AWS cloud. Many publishers will publish to a single elastic load balancer (ELB) endpoint, with many brokers behind it. In order to subscribe to and consume the messages, instead of bridging all the brokers, which will grow and shrink based on usage in an auto-scaling group (ASG), we are considering simply having one subscriber per broker on the same instance consuming the messages. 

It seems to me that this is a common scenario (very similar to this), is this a valid way of doing things, and are there things we need to look out for? E.g. is it an issue that when publishing, two messages from the same client might in reality be hitting two different brokers behind the same ELB?

--
--
To learn more about MQTT please visit http://mqtt.org
 
To post to this group, send email to mq...@googlegroups.com
To unsubscribe from this group, send email to
mqtt+uns...@googlegroups.com
 
For more options, visit this group at
http://groups.google.com/group/mqtt
 
---
You received this message because you are subscribed to the Google Groups "MQ Telemetry Transport" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mqtt+uns...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Dominik Obermaier]

Sorry, I hit the submit button too early. Damn smartphone UIs.

Hi Ian,

Mqtt brokers in the amazon cloud with ELBs are a common scenario. We've done several projects with that setup.

There are some gotchas when using AWS, though.

* ELBs disconnect TCP connections after a minute if there was no data transfer (like a PINGREQs), sp you have to make sure your CONNECTs keepalive is small.

* you might consider clustering instead of bridging when using AWS

We had great success with clustering on AWS with the HiveMQ MQTT broker. It dynamically builds clusters and can do the cluster node discovery with Amzon S3 buckets, Amazon RDS or with statically configured IPs/hostnames.

With clustering, all state is replicated between cluster nodes, so your clients can connect to any node they like (or get connected via ELB). Also, every message is distributed clusterwide, so you can publish to any node and all other clients, even if they are connected to another node, receive the messages (if they are subscribed to the correct topics)

(Note I'm a bit biased since I am one of HiveMQs developers)

Best regards,

Dominik

[Elad Nava]

It should be noted that today AWS Elastic Load Balancers support configuring an Idle Timeout of up to 3,600 seconds to support MQTT connections with keep-alive intervals of up to 60 minutes:

http://docs.aws.amazon.com/elasticloadbalancing/latest/classic/config-idle-timeout.html

[Uday Subbarayan]

Hi Ian,

    We have a purpose built MQTT loadbalancer and can help you. Pl let me know. (ud...@elasticbeam.com)

http://www.hivemq.com/blog/scalable-and-secure-mqtt-load-balancing-with-elastic-beam-and-hivemq/

Thanks,

Uday Subbarayan,

Elastic Beam.