Redis vs. MQTT vs. ZMQ vs. Longpoll

[Xiangrong Fang]

Hi There,

I am evaluating products to setup a notification service. The client will be mobile phones running Android or iOS.   Products under consideration are:

* Redis

* MQTT (mosquitto)

* NginX with long-poll plugin

* ZeroMQ

The primary concern for the server side is SCALABILITY (rather than throughput), i.e. I hope that a single server can support as many concurrent clients as possible, although most of the clients are just waiting, or have low amount of messages passing to them.   The communication will be a list style (BRPOP/BLPOP), i.e. NOT broadcasting, but one-to-one.

Other criteria are: RELIABILITY especially under bad network conditions.

Which product would you recommend?

Thanks,

Shannon

[Felix Gallo]

You're comparing a database, an embedded telemetry system, a web server, and a unix sockets developer convenience layer.

The problem you are facing is not in product selection, but in question selection.

F.

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+u...@googlegroups.com.
To post to this group, send email to redi...@googlegroups.com.
Visit this group at http://groups.google.com/group/redis-db.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Javier Guerra Giraldez]

On Sun, Jul 7, 2013 at 8:59 PM, Felix Gallo <felix...@gmail.com> wrote:
> You're comparing a database, an embedded telemetry system, a web server, and
> a unix sockets developer convenience layer.

my thoughts too.

still, you can build a notification system on top of any of these, so
the question sorta makes sense on a _very_ early design phase.

but the first thing you have to remember is that both Redis and ZeroMQ
are 'private lan only', never for wide internet.

I don't know about MQTT, but HTTP is certainly scalable and
universally supported. Also, there are _lots_ of examples about how
to do it. Nginx is a good platform, but far from being the only one.

in short, unless you have good experience on MQTT, i would stay with HTTP.

--
Javier

[Josiah Carlson]

Your notification system is already built, it's called push notifications, and they are available for iOS and Android today through their respective Apple or Google APIs. Or you can go the simple route and just use Urban Airship, which offers an even further simplified API.

 - Josiah

On Sun, Jul 7, 2013 at 6:41 PM, Xiangrong Fang <xrf...@gmail.com> wrote:

--

[Vijay]

Hi Shannon

I have experience using MQTT for push notifications. MQTT uses a TCP connection and hence as long as your app is "connected" to the mqtt server(waiting for notifications), it incurs exchange of TCP packets which increases the data usage on the mobile(unless you are on a wifi). Secondly, the TCP connection would considerably drain the battery. While the handset is moving across different locations, its ip address might change dynamically which can break the tcp connection. You can have your application to reconnect when the connection breaks, but if a notification arrives during this period, it might be lost. If you have workaround for the above problems, you can go ahead with MQTT.

However, if real time notifications are not required, you can periodically poll a http service to fetch the notifications pending for that handset. You can use a redis db to store the pending notifications (till they are delivered) in redis. The http service, probably written in php(or any other language) could just "query" redis and respond with the notification data. The app can "acknowledge" the receipt of the notification to the http service which in turn can delete the notification from redis thus achieving reliability. 

Going thorugh the Apple/Google api is clean, safe and simplifies your application development.

Going through the MQTT/redis/ other will make your hands dirty but can give a valuable learning experience.

-- Vijay

[Xiangrong Fang]

Thanks.  To reply some of the suggestions above:

1) Why redis is not for internet but only for lan? I don't know if this is because of security reason or not? If I am to use redis, it will be a dedicated instance, will there still be problems?

2) I won't use apple or google's existing service, mainly because we want to make it uniformly, also, there are lot of potential customers whose android phone do not have google service framework installed, nor will they ever signup and use a google account.

3) will polling consume even more battery than push? also, we do want message to arrive as fast as possible, just like in chatting.  Also, the message is very simple, like "you have you message", and the client will call a normal api via https to get the message itself.

2013/7/8 Vijay <vis...@gmail.com>

[Josiah Carlson]

On Sun, Jul 7, 2013 at 11:51 PM, Xiangrong Fang <xrf...@gmail.com> wrote:

Thanks.  To reply some of the suggestions above:

1) Why redis is not for internet but only for lan? I don't know if this is because of security reason or not? If I am to use redis, it will be a dedicated instance, will there still be problems?

Redis doesn't handle anything related to security. Any client that knows that it is connected to Redis can send arbitrary data or even cause it to shut down if you don't rename all of your commands.

 

2) I won't use apple or google's existing service, mainly because we want to make it uniformly, also, there are lot of potential customers whose android phone do not have google service framework installed, nor will they ever signup and use a google account.

That's too bad. Urban Airship is an awesome product and makes all of this super easy.

3) will polling consume even more battery than push? also, we do want message to arrive as fast as possible, just like in chatting.  Also, the message is very simple, like "you have you message", and the client will call a normal api via https to get the message itself.

Polling eats battery. A sustained TCP connection uses far less battery - that's what is going on with iOS and Android push notifications. That said... you are limited on iOS as only somewhat recent versions of iOS let you create connections that persist over time while in the background. You will have fewer issues on Android (as multitasking was built in from the start), but both will arbitrarily kill your app if it decides to.

If possible, you really want to send as much of your message as possible when doing the push. Establishing a new connection will eat more battery, especially https. Just have your persistent connection be https already, and you should be fine. Heck, if you can find a proper websocket client, nginx does websockets now, and that's all over https.

 - Josiah

[Felix Gallo]

Urban Airship is terrible.  A large client asked for UA to be involved in a similar solution I implemented and at around 50k interactive-style sessions we were throttled, called, and told that we were significantly impacting their systems.  So that is a nonstarter and a really bad way to waste a month of your time.

The 'right thing to do' in this space appears to be 

1) when in the app, either use something like Pusher (http://pusher.com), which is specifically designed to be scalable for that particular use case, or if you are mature enough, to roll your own websocket server with, e.g., erlang's excellent cowboy framework.

2) when not in the app, use raw APN/GCM but batched serverside and without the UA passthrough.  If clients can't use APN/GCM then, I don't know, text them?

Also, Nginx doesn't 'do websockets', it knows about websockets.  You still need an application layer.

In any case, none of this discussion is about redis.

F.

[Xiangrong Fang]

> Redis doesn't handle anything related to security. Any client that knows that it is connected to Redis can send arbitrary data or even cause it to shut down if you don't rename all of your commands.

I do plan to rename ALL commands. And the "channel" for clients to listen is its object id (a uuid). I think it shoud be safe?

[Josiah Carlson]

Safe from arbitrary command execution, sure. But not safe from any kind of DOS attack. I can think of several off the top of my head that have zero methods of mitigation within Redis (which are trivially mitigated in other platforms).

 - Josiah

On Mon, Jul 8, 2013 at 3:54 PM, Xiangrong Fang <xrf...@gmail.com> wrote:

> Redis doesn't handle anything related to security. Any client that knows that it is connected to Redis can send arbitrary data or even cause it to shut down if you don't rename all of your commands.

I do plan to rename ALL commands. And the "channel" for clients to listen is its object id (a uuid). I think it shoud be safe?

[Xiangrong Fang]

My original question is related to redis. I would like to compare the following aspects of the 4 products in a message pushing app:

1. scalability
2. throughput
3. security
4. delay
5.whatever suggestions you may have...

Thanks.

[Javier Guerra Giraldez]

On Mon, Jul 8, 2013 at 5:54 PM, Xiangrong Fang <xrf...@gmail.com> wrote:
> I do plan to rename ALL commands. And the "channel" for clients to listen is
> its object id (a uuid). I think it shoud be safe?

i wouldn't dismiss the possibility of some viciously malformed
protocol edgecases killing the redis process.

--
Javier

[Josiah Carlson]

On Mon, Jul 8, 2013 at 4:00 PM, Xiangrong Fang <xrf...@gmail.com> wrote:

My original question is related to redis. I would like to compare the following aspects of the 4 products in a message pushing app:

1. scalability

Redis and ZeroMQ will not have any issues. I don't know about the others 

2. throughput

That's sort of the same thing as #1, you'll likely be limited by your internet connection well before Redis and/or ZeroMQ fall over.

3. security

Redis has little/none. I don't know about the others. 

4. delay

Redis has no built-in delay, you are limited by outgoing bandwidth here. 

5.whatever suggestions you may have...

If you could use the standard Google/iOS push notifications, you should follow Felix' advice. Since you can't... I'd suggest against Redis due to it's vulnerability to DOS attacks without reworking. Another option is to just use one of the standard XMPP clients/servers that support SSL. They were more or less built for this.

 - Josiah

[Xiangrong Fang]

2013/7/9 Josiah Carlson <josiah....@gmail.com>

2. throughput

That's sort of the same thing as #1, you'll likely be limited by your internet connection well before Redis and/or ZeroMQ fall over.

I probably used the wrong word.  In fact, I do not really care about "throughput" as important as "scalability", i.e.  the number of messages processed per second is NOT important (because it won't be super high in my app), but the number for CONCURRENT user that hooked on the server waiting for message really matters.

3. security

Redis has little/none. I don't know about the others. 

I think Javier's advice about protocol  edge cases are very important, based on this, I will not use redis as the messaging solution.   However, I am still interested in how the "other" tools such as mqtt or https longpoll mitigate the threats of DOS attack?  Which is the same problem as my #1 concern - the number of concurrent user.

5.whatever suggestions you may have...

If you could use the standard Google/iOS push notifications, you should follow Felix' advice. Since you can't... I'd suggest against Redis due to it's vulnerability to DOS attacks without reworking. Another option is to just use one of the standard XMPP clients/servers that support SSL. They were more or less built for this.

How can SSL help in prevent DOS attacks? Could you please explain a bit?

In your previous email, you suggested that the real message should be sent along with the notification to avoid another http request.  But my doubt is the risk of losing a message, if the notification is merely a notification, the client can query via another API about new messages since its last query attempt.  What do you think?

Thanks,

Shannon

[Javier Guerra Giraldez]

On Mon, Jul 8, 2013 at 6:52 PM, Josiah Carlson <josiah....@gmail.com> wrote:
> Another option is to just use one of the standard XMPP clients/servers that
> support SSL.

+1

also SIP, which is not only for VoIP, but mainly for sending the
notification that somebody wants to establish a voice connection.

--
Javier

[Felix Gallo]

Oh man, sorry to be so contrary today Josiah, but XMPP is also awful, pessimally designed, and a tremendous waste of both bandwidth and processing power (verbose XML + XML parsers in order to implement packet sending).  Never use XMPP for any reason unless you need a multi user chat server; and even then, don't use it.  Especially not on mobile.  This was another hard lesson learned from production mobile experience with the same customer.

Fang, to cut to the heart of the matter, what you need is, as far as we can tell from your partially described requirements, a multi tier solution.  

In the first tier, you need something capable of terminating a large number of socket connections -- probably websocket connections.  Your best choices here are (a) Pusher; (b) the 'standard' web app front ends like nginx/haproxy; or (c) the 'industrial strength exotics' like Cowboy on Erlang.  That thing needs to talk to:

The second tier: the application tier, which is where the logic of your application lives.  That application tier needs to be horizontally scalable, because generally the 'business logic' involves computation and your computation may be (or may grow to be) an intensive part of your round trip communication.  If you chose (a) or (b) above then your app tier can be anything, but popular choices include Ruby frameworks sitting on Unicorn servers, JVM framework languages like JRuby or Java (with Spark) or Clojure using Netty and sitting on containers like Resin.  If you chose (c) then you are already writing in Erlang and so you might as well continue writing in Erlang and exploiting OTP.

That tier is served by:

The final tier: the database tier.  This is where Redis and PostgreSQL and Riak, and their lesser cousins MySQL and MongoDB and whatever else, reside.  Notice that they are three layers deep, hidden well away from the internet.  Depending on the size of your data set and the complexity of each query, each of the holy trinity of databases may be best for your particular use case.  None of them are perfect for everything.  All of them can be coerced into doing what you want but it will hurt more or cost more depending on which way they are being coerced.

It makes no sense to compare front-tier technology against back-tier technology.  You can't switch *those* things for each other, it makes no sense.

First and most important I recommend that you hire someone immediately who has some amount of experience with internet architecture, so that you don't hurt yourself trying to get a good architecture done on the cheap from mailing lists.  Systems offering extremely high scalability to hundreds of thousands of mobile clients are not simple and there are no magic bullets.  If there were I would be selling hot dogs on Santa Monica Pier.

F.  

[Xiangrong Fang]

Thanks Felix.  What I need is not as complex as you think.   Yes I do want scalability everywhere, but I am now only talking about the message pushing system.  For the APIs (business logic), we are a PHP shop, we use Redis + PHP + NginX. 

I agree with Josiah that before we hit the ceiling on NginX or Redis, or our server hardware, the bandwidth will be filled up first.

2013/7/9 Felix Gallo <felix...@gmail.com>

[gradetwo]

 Use iOS's APN and android's GCM instead of building it from scratch,

check this project https://github.com/uniqush/uniqush-push

BTW. GCM have some problems in some country like China, u can build a android push notification system with Websockets+Redis

[gradetwo]

XMPP is too heavy, notification system should be lightweight.

[Josiah Carlson]

On Mon, Jul 8, 2013 at 6:58 PM, Felix Gallo <felix...@gmail.com> wrote:

Oh man, sorry to be so contrary today Josiah, but XMPP is also awful, pessimally designed, and a tremendous waste of both bandwidth and processing power (verbose XML + XML parsers in order to implement packet sending).  Never use XMPP for any reason unless you need a multi user chat server; and even then, don't use it.  Especially not on mobile.  This was another hard lesson learned from production mobile experience with the same customer.

I do not believe that you actually feel sorry for being contrary, but that's okay. I've never actually used XMPP in production, though I'm sad that it's so awful in practice. That stuff was supposed to make federated chat possible... Maybe a dumbed-down IRC over SSL. Most IRC daemons nowadays should be pretty solid, and there is not XML to be seen :)

(As a tangent, I'm surprised that XML is still an issue. At least on the Python side of things, there are a couple really fast XML parsers that make it less painful to use. Perhaps not as convenient as json or even standard http GET/POST requests, but not bad all the same.)

 

Fang, to cut to the heart of the matter, what you need is, as far as we can tell from your partially described requirements, a multi tier solution.  

In the first tier, you need something capable of terminating a large number of socket connections -- probably websocket connections.  Your best choices here are (a) Pusher; (b) the 'standard' web app front ends like nginx/haproxy; or (c) the 'industrial strength exotics' like Cowboy on Erlang.  That thing needs to talk to:

The second tier: the application tier, which is where the logic of your application lives.  That application tier needs to be horizontally scalable, because generally the 'business logic' involves computation and your computation may be (or may grow to be) an intensive part of your round trip communication.  If you chose (a) or (b) above then your app tier can be anything, but popular choices include Ruby frameworks sitting on Unicorn servers, JVM framework languages like JRuby or Java (with Spark) or Clojure using Netty and sitting on containers like Resin.  If you chose (c) then you are already writing in Erlang and so you might as well continue writing in Erlang and exploiting OTP.

That tier is served by:

The final tier: the database tier.  This is where Redis and PostgreSQL and Riak, and their lesser cousins MySQL and MongoDB and whatever else, reside.  Notice that they are three layers deep, hidden well away from the internet.  Depending on the size of your data set and the complexity of each query, each of the holy trinity of databases may be best for your particular use case.  None of them are perfect for everything.  All of them can be coerced into doing what you want but it will hurt more or cost more depending on which way they are being coerced.

It makes no sense to compare front-tier technology against back-tier technology.  You can't switch *those* things for each other, it makes no sense.

First and most important I recommend that you hire someone immediately who has some amount of experience with internet architecture, so that you don't hurt yourself trying to get a good architecture done on the cheap from mailing lists.  Systems offering extremely high scalability to hundreds of thousands of mobile clients are not simple and there are no magic bullets.  If there were I would be selling hot dogs on Santa Monica Pier.

+1

On a completely unrelated note, we should get a beer some time. We're both on the west side.

 - Josiah

[Luis Romero]

I suggest to use zeromq as pub sub server and another queue server it's fast enough.