MQTT vs AMQP, OpenMAMA and DNP3. Why do we really need MQTT?

[Mandar]

I saw Andy's recent MQTT video presentation but I fail to
understand why IBM chose to develop yet another protocol i.e. MQTT and
is choosing to make it a open standard.

What exactly is IBM trying to accomplish? What are the advantages
that MQTT is supposed to deliver? The problem of reliable messaging
with minimal bandwidth use over unreliable/slow network links has
already been around and tackled by these protocols and providers.

The example comparison of MQTT vs HTTP in the video presentation is
an apple to oranges comparison and does not fit.

Can IBM please explain exactly why MQTT should be used when other
solutions and protocols exist that solve the same issue, and please
provide some hard comparison data.

Thanks

- Mandar

[Sy]

I can't say for AMQP or openMAMA as I have no knowledge of those
protocols. DNP3 is a protocol designed for transmission of
instrumentation data such as analogue signals etc. over low bandwidth
networks, I've used it offshore to get data from platforms back to the
beach for display in control systems. Its not a general purpose
messaging system.

Unlike a lot of messaging systems MQTT allows subscribers to declare
interest in a specific topic it then notifies them of updates whenever
they are available. A lot of message protocols have to poll for
updates which isn't very effecient.

I agree MQTT and HTTP are very different and designed for completely
different purposes.

The best explanation I can offer is its like restaurants, there are
always lots of restaurants around and its your choice which one you
eat at. Just because there are a few already doesn't mean you should
open another.

Regards,
Simon

[Mandar]

Thanks for the insight.

It would be helpful if there was a comparison of features and pros/
cons of MQTT vs AMQP (www.amqp.org) vs DDS (www.omgwiki.org/dds/) in
particular so implementors can select a protocol best suited for their
needs. These three seem to be the most alike in their intended target
applications i.e. real time, low bandwidth message based pub/sub
transmission. Erlang/OTP (www.erlang.org) is probably a worthwhile
mention as well.

Some interesting threads on scalability:

http://www.quora.com/What-should-I-choose-between-XMPP-and-AMQP-for-building-a-scalable-Twitter-clone

- Mandar

[Raphael Cohn]

Mandar,

There's quite a difference between them.

Essentially, DDS is apt for stream event processors and 'in-data centre' applications; it excels when in memory messaging is needed.

OpenMAMA is intended as the defacto Linux API to messaging, like JMS is to Java. There will be an AMQP implementation ('bridge') in OpenMAMA soon.

MQTT is a simple, lightweight protocol ideal for basic interactions. It doesn't have multiple queues or containers, and so isn't suitable when these are needed, eg for multi-hosting, security permission partitioning, etc, or when one needs more complex messaging scenarios, such as transactions, routing, non-topic filtering, etc. MQTT currently has weak security. There's absolutely no way you should be using it for communications on public networks of anything personally identifiable (eg GPS locations). There's also no way currently of sending large messages fragmented, so if you've got a dodgy link, you can't do resends.

AMQP, on the other hand, does address all these concerns.

Hope that helps,

Raph

Raphael Cohn
Chief Architect
raphae...@StormMQ.com
StormMQ Limited

UK Office:
Gateshead int'l Business Centre, Mulgrave Terrace, Gateshead, NE8 1AN, United Kingdom
Telephone: +44 845 3712 567

Registered office:
16 Anchor Street, Chelmsford, Essex, CM2 0JY, United Kingdom
StormMQ Limited is Registered in England and Wales under Company Number 07175657
StormMQ.com

--
To learn more about MQTT please visit http://mqtt.org

To post to this group, send email to mq...@googlegroups.com
To unsubscribe from this group, send email to
mqtt+uns...@googlegroups.com

For more options, visit this group at
http://groups.google.com/group/mqtt

[andyp...@gmail.com]

Hi All, thanks for the discussion here and apologies that I've only
just had a chance to get around to participating, given that it was
addressed to my presentation!

First of all - IBM and Eurotech have chosen to donate MQTT to Eclipse
Paho, join the M2M Industry Working Group over there to *co-operate*
on M2M standards and tools, and some months ago signalled an intent to
take the MQTT specification to a standards body. It's a protocol with
12 years of real production usage behind it already - it's not
something newly-developed to challenge existing things. So "What
exactly is IBM trying to accomplish?" and "Can IBM please explain..."
style questions can be asked, implying there is some kind of
masterplan in this - that's just not the case, it's an open donation
with transparent organisations and it's a community discussion. I'm
happy to address some of these questions as a member of that
community.

It is not the purpose of this group to list and debate alternatives -
we know there are different technologies around! Clarity around use
cases is definitely helpful, and it's certainly a good place to think
about possible future requirements as the specification moves towards
standardisation.

I've made some very broad comments about the kinds of comparisons you
mention in my blog post, without zeroing in on specific differences
http://andypiper.co.uk/2011/11/04/mqtt-goes-free-a-personal-qa/

In general it's worth being careful in any comparison, as the domains
tend to be different. For instance, AMQP (a protocol, not an API) is -
primarily but not solely - aimed at the reliable enterprise
transactional messaging space. OpenMAMA (an API, not a protocol) was
originally from the low latency messaging space and thus has
significantly different semantics to some of the other traditional
messaging APIs. We'll see how well they get adopted. I personally feel
that AMQP is likely to be rather heavyweight when dealing with the
kinds of systems that have been put together using MQTT in the past,
and that because it covers a range of "enterprise" topologies and use
cases it isn't likely to scale down easily.

The reason I talked about HTTP in the videos you mention was not to
come up with an "apples to oranges" comparison, but simply to
highlight the fact that although HTTP is pervasively available in many
devices (embedded systems and smartphones) it is not always
appropriate even when a RESTful interface is available. It wasn't an
intent to rubbish HTTP - there are times and places for both that and
MQTT. It's fairly easy to compare MQTT and HTTP and demonstrate
greater network efficiency of the former. We've got some battery life
/ usage data out on a blog about MQTT as well. If you look at where
MQTT comes from - unlike AMQP et al, *specifically engineered for
constrained devices* - and talk to companies like Eurotech who have
been implementing it in embedded systems for many years, the goal is
to break data out of the very proprietary data jails such as those
found when dealing with SCADA systems, and making it more available
more widely across an enterprise.

The comment was made in this thread that MQTT is very simple and
lightweight - indeed it is - in fact I've described it as deliberately
simple - it's almost like the UNIX philosophy of doing one thing and
doing it well. That makes it easy to integrate into different
solutions, whilst maintaining a lot of flexibility. On "weak security"
- there are a range of security options beyond simple user/pw on the
API, including SSL for some implementations, the ability to encrypt or
sign in-application, and as this space moves forward that may well
change (I know OAuth and other ideas have been mentioned) - we'll have
to see.

The upshot is that you have a choice of protocols, and IBM and
Eurotech were (as far as I know!) happy to contribute a "known good" /
proven protocol to the community.

Andy

--
Andy Piper | Farnborough, Hampshire (UK)
blog: http://andypiper.co.uk   |   skype: andypiperuk
twitter: @andypiper  |  images: http://www.flickr.com/photos/andypiper

[Raphael Cohn]

Good comments. I'd agree HTTP is almost never appropriate in a world where connectivity is poor and data important.

I'd take issue with AMQP being heavyweight and inappropriate for constrained devices. We've deployed AMQP, with TLS, in 64K black boxes in electric trucks for Smith Electric Vehicles; our actual stack is less than that. To date, it's the worlds largest electric truck telemetry solution; we won the business because the cost of an IBM based solution was 10x the TCO of what they could justify. In the 18 months we've been live, the solution has delivered over 1.6 billion statistics over some extremely poor links. Without AMQPs message re-transmission and fragmentation, Smiths just couldn't have done that.

Smith's have saved over $2m a year in preventative maintenance, and can now direct filtered truck data directly to their customers, all with one protocol.

AMQP scales down extremely well. And we've got the evidence to prove it.

Raph

Raphael Cohn
Chief Architect
raphae...@StormMQ.com
StormMQ Limited

UK Office:
Gateshead int'l Business Centre, Mulgrave Terrace, Gateshead, NE8 1AN, United Kingdom
Telephone: +44 845 3712 567

Registered office:
16 Anchor Street, Chelmsford, Essex, CM2 0JY, United Kingdom
StormMQ Limited is Registered in England and Wales under Company Number 07175657
StormMQ.com

[Zvi]

Can you please post the link to the video?
Thanks,
Zvi

[andyp...@gmail.com]

I believe the original reference was to a talk I gave at the
PubSubHuddle in September, which I think you may have seen before ;-)
http://skillsmatter.com/podcast/home/andy-piper-talk

On Sun, Dec 25, 2011 at 20:08, Zvi <zvi.a...@gmail.com> wrote:
> Can you please post the link to the video?
> Thanks,
> Zvi

--