Building Chat Infrastructure

[Nils Sch]

Hello,

we are in the middle of building a mobile chat application (iOS) and looking right now into our options for the backend infrastructure. 

One solution we came up with was using MQTT (RabbitMQ broker) as a protocol and Node.js as our backend server. After benchmarking the number of concurrent connection (>50.000) on a clustered Node instance we are happy with the performance, but we are afraid of using MQTT, because scaling the brokers and message persistence is not well documented for our use cases (Client-to-Client) and we have time contraints.

As a alternative we are thinking to use a HTTP stack to send message and EventSource to receive messages, because we know how to scale an application like this.

Can you give us any good reason to stick with MQTT? (Performance, Battery, Architecture, Broker Scaling)?

For us it seems right now, using a tech stack we are very familiar with seems to be the better solution, rather than using a new protocol which is kind of an unknown to us.

Thanks for the Help

Nils

[Dave Locke]

On the server side there are good examples of MQTT servers that scale for instance the MessageSight appliance from IBM can handle 1 million concurrently connected devices and message rates of 14 million non persistent messages and 400 thousand persistent messages per second.

On the client side there are good reasons for using MQTT.  HTTP is a request reply paradigm whereas MQTT uses a publish subscribe paradigm. As a result an MQTT client app expresses interest in receiving messages on a particular topic / subject.  After expressing an interest  (subscribing) it can go to sleep. When the server receives a message that matches the topic that one of more clients are interested in the message is  pushed to the client(s).   There is no polling involved.   As there is no polling the mobile app consumes very little network and CPU and as a result it is frugal on battery usage.  

A good example of a mobile application that uses MQTT is facebook messenger.  You can see the reasons MQTT was chosen on the blog of the facebook engineer here: https://www.facebook.com/notes/facebook-engineering/building-facebook-messenger/10150259350998920


All the best
Dave

--
To learn more about MQTT please visit http://mqtt.org
---
You received this message because you are subscribed to the Google Groups "MQTT" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mqtt+uns...@googlegroups.com.
To post to this group, send email to mq...@googlegroups.com.
Visit this group at http://groups.google.com/group/mqtt.
For more options, visit https://groups.google.com/d/optout.

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

[Nils Sch]

Hey Dave,

thanks for your fast response. For the server side we want stick with OpenSource technology so, IBM MessageSight isn't option for us and it seems like there is no scalable reference implentation based on Ruby, Node.js or Python out there yet. We saw that HiveMQ posted some details about how to scale MQTT brokers, but nothing really about message persisting, etc.

On the client side, we can archive similar a effect with a combination of POST requests and EventSource connections (sleep & retry's). Scaling this kind of setup is well documented and we can keep our toolchain and use our knowledge in that domain. Our team knows that scaling an app is not a trivial task and doing it with an unmature/less documented technology seem risky.

We read that Facebook was using MQTT but other than this, the short blog article doesn't give any actionable details. We were hoping to find out more about how to Scale MQTT brokers for Chat application. It seems like even if we would use MQTT we had to build more infrastrucuture around message persistence and other features we want to support, and at the point the advantages with MQTT fading away compare to a classical HTTP setup.

[Bart Van Der Meerssche]

Hello Nils,

You might want to take a look at Fubar [1], a scalable, open-source MQTT broker written in Erlang.

[1] https://github.com/jinnipark/fubar

Cheers

/Bart

[Nils Sch]

Thanks for the tip icarus75, I'll take a look later. 

Just for reference: We made more benchmarks around comparing a "Server Send EventS" (EventSource) setup and an MQTT setu. With MQTT we were able ti connect up to 30,000 stable MQTT clients (server in Node.js) and with EventSource up to 20,000 clients. We are right now benchmarking message sending. We hope to publish these benchmarks in the future. Let us know if there is interest in it.

On Saturday, July 19, 2014 2:16:17 AM UTC-7, icarus75 wrote:

Hello Nils,

You might want to take a look at Fubar [1], a scalable, open-source MQTT broker written in Erlang.

[1] https://github.com/jinnipark/fubar

Cheers

/Bart

On Friday, July 18, 2014, Nils Sch <nils.sc...@gmail.com> wrote:

Hey Dave,

thanks for your fast response. For the server side we want stick with OpenSource technology so, IBM MessageSight isn't option for us and it seems like there is no scalable reference implentation based on Ruby, Node.js or Python out there yet. We saw that HiveMQ posted some details about how to scale MQTT brokers, but nothing really about message persisting, etc.

On the client side, we can archive similar a effect with a combination of POST requests and EventSource connections (sleep & retry's). Scaling this kind of setup is well documented and we can keep our toolchain and use our knowledge in that domain. Our team knows that scaling an app is not a trivial task and doing it with an unmature/less documented technology seem risky.

We read that Facebook was using MQTT but other than this, the short blog article doesn't give any actionable details. We were hoping to find out more about how to Scale MQTT brokers for Chat application. It seems like even if we would use MQTT we had to build more infrastrucuture around message persistence and other features we want to support, and at the point the advantages with MQTT fading away compare to a classical HTTP setup.

--
To learn more about MQTT please visit http://mqtt.org
---
You received this message because you are subscribed to the Google Groups "MQTT" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mqtt+unsubscribe@googlegroups.com.

[Karl P]

Yes, Yes, there's a lot of interest in more scaling reports. What server are
you using in node.js, is that Mosca?

On 07/23/2014 06:59 PM, Nils Sch wrote:
> Thanks for the tip icarus75, I'll take a look later.
>
> Just for reference: We made more benchmarks around comparing a "Server Send
> EventS" (EventSource) setup and an MQTT setu. With MQTT we were able ti connect
> up to 30,000 stable MQTT clients (server in Node.js) and with EventSource up to
> 20,000 clients. We are right now benchmarking message sending. We hope to
> publish these benchmarks in the future. Let us know if there is interest in it.
>
>
>
> On Saturday, July 19, 2014 2:16:17 AM UTC-7, icarus75 wrote:
>
> Hello Nils,
>
> You might want to take a look at Fubar [1], a scalable, open-source MQTT
> broker written in Erlang.
>

> [1] https://github.com/jinnipark/fubar <https://github.com/jinnipark/fubar>
>
>
> Cheers
> /Bart

> an email to mqtt+uns...@googlegroups.com.

> To post to this group, send email to mq...@googlegroups.com.
> Visit this group at http://groups.google.com/group/mqtt

> <http://groups.google.com/group/mqtt>.

> For more options, visit https://groups.google.com/d/optout

> <https://groups.google.com/d/optout>.

>
> --
> To learn more about MQTT please visit http://mqtt.org
> ---
> You received this message because you are subscribed to the Google Groups "MQTT"
> group.
> To unsubscribe from this group and stop receiving emails from it, send an email

> to mqtt+uns...@googlegroups.com <mailto:mqtt+uns...@googlegroups.com>.

> To post to this group, send email to mq...@googlegroups.com

> <mailto:mq...@googlegroups.com>.

[Ashutosh Agrawal]

Did you try evaluating ActiveMQ or ApolloMQ?

Ash