Alternative to TCP for MQTT

[lightweight]

Hi All,

I am curious to know if there is any plans to support UDP in place of TCP for MQTT. I know this will 

require revisiting the protocol from scratch. But this will help making MQTT lighter. 

I know about GpsGate(that's tracking only protocol), which supports TCP,UDP and even sms.

Regards,

Manish

[Nicholas O'Leary]

HI Manish,

there is already a variant of MQTT for non-TCP networks, called MQTT-S.

It is linked off http://mqtt.org/documentation

As a protocol, it is not yet released under a proper royalty-free
license like MQTT is, but this is being addressed.

Regards,
Nick

> --
> --
> To learn more about MQTT please visit http://mqtt.org
>
> To post to this group, send email to mq...@googlegroups.com
> To unsubscribe from this group, send email to
> mqtt+uns...@googlegroups.com
>
> For more options, visit this group at
> http://groups.google.com/group/mqtt
>
>
>

[Raphael Cohn]

Interesting question... and one that comes quite often for messaging protocols.

A messaging protocol needs delivery of messages in order. TCP gives that without any "effort" required on behalf of the protocol designer. UDP makes that harder and requires protocol-level support.

It is also difficult with UDP to deal with networking unreliability - a common problem. For example, many routers in commercial use to date have simply dropped UDP packets when their buffers overflow, meaning that a receiver only becomes aware of message loss after processing, say, the next 1,000 packets. If that message loss was for a packet that should have been received much earlier, the receiver's window for message recovery (eg his own moving window of partially completed message states) may be unavailable (eg due to limited memory).

Similar things occur when the receiving broker is near or at maximum load.

Unreliable networks can be overcome by putting reliability g'tees into the protocol atop of UDP - effectively making it very similar to TCP but without flow control... And it is naive TCP flow control that often hinders TCP's use on things like 2G networks, hence the persistent interest in UDP.

So an UDP based messaging protocol is possible, but works best in a controlled environment, such as a LAN, rather than the wilds of the internet. In the latter case, UDP is normally a far 'heavier' protocol in network traffic and bandwidth consumed for greatly increased complexity in userspace clients and brokers - not ideal for small devices.

Rather than use UDP, a better choice that gives most of the benefits of it whilst maintaining ordering is SCTP. May be that is the way MQTT could go at some point, but there is possibly not  enough widespread support (IIRC Windows doesn't out of the box, for instance).

Oh, and I'll toss in PGM whilst I'm here just to spice things up...

Raphael Cohn
Chief Architect, StormMQ

Secretary, OASIS AMQP Standard
raphae...@StormMQ.com

+44 7590 675 756

UK Office:
Hamblethorpe Farm, Crag Lane, Bradley BD20 9DB, North Yorkshire, United Kingdom
Telephone: +44 845 3712 567

Registered office:
16 Anchor Street, Chelmsford, Essex, CM2 0JY, United Kingdom
StormMQ Limited is Registered in England and Wales under Company Number 07175657
StormMQ.com

On 3 October 2012 18:48, lightweight <manish...@googlemail.com> wrote:

--

[lightweight]

Amazing reply. 
I was quickly interested to know more about SCTP, wondering why this has not become so popular, atleast

it could have been in Embedded products space.

I wish this should be there in MQtt specification.

[Raphael Cohn]

lighweight,

Yeah, it's a shame about SCTP, but it will come - I know that several large organisations are very keen on using it for sensor networks, not least the UCSD's Ocean Observatory. Mind you, some embedded stacks struggle to even offer half decent TCP implementations after 30 years, so who knows when it will be usable! 

Raph

Raphael Cohn
Chief Architect, StormMQ

Secretary, OASIS AMQP Standard
raphae...@StormMQ.com

+44 7590 675 756

UK Office:
Hamblethorpe Farm, Crag Lane, Bradley BD20 9DB, North Yorkshire, United Kingdom
Telephone: +44 845 3712 567

Registered office:
16 Anchor Street, Chelmsford, Essex, CM2 0JY, United Kingdom
StormMQ Limited is Registered in England and Wales under Company Number 07175657
StormMQ.com

--

[Roger Light]

On Thu, Oct 4, 2012 at 7:53 AM, Raphael Cohn <raphae...@stormmq.com> wrote:

> Rather than use UDP, a better choice that gives most of the benefits of it
> whilst maintaining ordering is SCTP. May be that is the way MQTT could go at
> some point, but there is possibly not enough widespread support (IIRC
> Windows doesn't out of the box, for instance).

Funnily enough I had a little play with SCTP the other week. I can't
remember what prompted it. I got as far as forcing all socket() calls
to use SCTP in the mosquitto clients and broker. I'm happy to report
that everything just worked (on Linux). I've not got any further
partially because userland support isn't quite there yet. Even netstat
doesn't show SCTP sockets yet.

Cheers,

Roger

[Dave Locke]

its a little different but I have started a page on the Paho wiki about what it would mean provide MQTT over WebSockets in an inter-operable way.   i.e. have first class MQTT support direct from a modern web browser where the app is written in HTML and JavaScript.  The page can be found here: http://wiki.eclipse.org/Paho/Paho_Websockets


All the best
Dave Locke

Senior Inventor, Pervasive and Advanced Messaging Technologies

lo...@uk.ibm.com Dave Locke/UK/IBM@ibmgb 7-246165 (int) +44 1962816165 (ext) 37274133 (mobex) +44 7764132584 (ext)

Fringe Bluepages with fuel injection My Cattail: Share files in IBM and save your in-box

--
--
To learn more about MQTT please visit

http://mqtt.org

To post to this group, send email to mq...@googlegroups.com
To unsubscribe from this group, send email to
mqtt+uns...@googlegroups.com

For more options, visit this group at
http://groups.google.com/group/mqtt

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

[Raphael Cohn]

The big win we get with SCTP, apart from reliable multi-homing (good for dodgy links) is sensible framing - which can be exploited at a protocol level to remove the need for sizes, etc and so make for less on the wire AND more efficient broker / client implementations (need less socket read / write calls).

Raphael Cohn
Chief Architect, StormMQ

Secretary, OASIS AMQP Standard
raphae...@StormMQ.com

+44 7590 675 756

UK Office:
Hamblethorpe Farm, Crag Lane, Bradley BD20 9DB, North Yorkshire, United Kingdom
Telephone: +44 845 3712 567

Registered office:
16 Anchor Street, Chelmsford, Essex, CM2 0JY, United Kingdom
StormMQ Limited is Registered in England and Wales under Company Number 07175657
StormMQ.com

[Raphael Cohn]

Dave,

That's very interesting! Fancy collaborating on a Javascript client?

Raph

Raphael Cohn
Chief Architect, StormMQ

Secretary, OASIS AMQP Standard
raphae...@StormMQ.com

+44 7590 675 756

UK Office:
Hamblethorpe Farm, Crag Lane, Bradley BD20 9DB, North Yorkshire, United Kingdom
Telephone: +44 845 3712 567

Registered office:
16 Anchor Street, Chelmsford, Essex, CM2 0JY, United Kingdom
StormMQ Limited is Registered in England and Wales under Company Number 07175657
StormMQ.com

[Andy Piper]

Feel free to collaborate with us on Eclipse Paho - we are looking at getting a javascript client added soon :-)

-- 
Andy Piper

Sent with Sparrow

On Thursday, 4 October 2012 at 11:55, Raphael Cohn wrote:

Dave,

That's very interesting! Fancy collaborating on a Javascript client?

Raph

Raphael Cohn
Chief Architect, StormMQ

Secretary, OASIS AMQP Standard
raphae...@StormMQ.com

+44 7590 675 756

UK Office:
Hamblethorpe Farm, Crag Lane, Bradley BD20 9DB, North Yorkshire, United Kingdom
Telephone: +44 845 3712 567

Registered office:
16 Anchor Street, Chelmsford, Essex, CM2 0JY, United Kingdom
StormMQ Limited is Registered in England and Wales under Company Number 07175657
StormMQ.com

On 4 October 2012 16:29, Dave Locke <lo...@uk.ibm.com> wrote:

its a little different but I have started a page on the Paho wiki about what it would mean provide MQTT over WebSockets in an inter-operable way.   i.e. have first class MQTT support direct from a modern web browser where the app is written in HTML and JavaScript.  The page can be found here: http://wiki.eclipse.org/Paho/Paho_Websockets


All the best
Dave Locke

Senior Inventor, Pervasive and Advanced Messaging Technologies

<Untitled>

[Matteo Collina]

2012/10/4 Andy Piper <andyp...@gmail.com>

Feel free to collaborate with us on Eclipse Paho - we are looking at getting a javascript client added soon :-)

I would like to collaborate too.

 

Have you started developing, or have you got a discussion about it?

Matteo

[Roger Light]

Hi Gary,

> I am not sure if requiring binary format is a good idea. Not all browsers
> can use it,

But it's a fair bet that they will do eventually. Not all browsers
provide even basic websockets support at the moment.

> plus it can be a real headache for someone used to coding where
> ints and chars are almost interchangeable to start thinking at the byte
> level.

I don't think this is an issue. The end user of an MQTT javascript
implementation won't have to think at the byte level, they'll just be
dealing with their own payloads in whatever format they are. Is it
more of a problem for the people doing the implementation? Sure, but
not significantly. The javascript MQTT client that I whipped up with
no prior JS experience was easy enough. I presume someone who knew
what they were doing could implement it much better.

> Also a lot of websocket server implementations have their own custom
> formatting, for example http://jwebsocket.org/ is a java websocket server
> and javascript client. While it can be used to build a simple websocket
> server, it uses plugins to allow one server to route traffic to many
> different brokers. This is partially done using special data encoding
> where the data is sent inside a json encoded packet with additional framing
> data to direct it to the correct plugin.

I like the model where the websockets url provides the information on
what to do with a connection. The jwebsocket model sounds as though it
is deciding which protocol to use per message based on metadata which
sounds... messy.

> As such, I'd extend the protocol name to be more clear, ie:
> Must use "mqtt" as a suffix for the websocket protocol name.
> Should include the supported mqtt version after the suffix
> Must include a descriptive label in the protocol name.
> For example:
> mqtt-v3.1-raw-binary
>
> MQTT protocol version 3.1, raw binary payload
>
> mqtt-v3.1-json-base64
> MQTT protocol version 3.1, json encoded packets of some sort, base64 encoded
> payloads

This is worth considering to at least include the non-binary frames,
although I'd avoid any sort of definition that included "of some sort"
in it :)

Cheers,

Roger

[Raphael Cohn]

If you...
- Only want at most once messaging (QoS 0);
- Have tiny messages;
- Don't care about data loss;
- Don't care about order of data (ie don't care about it being a queue)
- Don't want or can handle less security needs (and UDP + TLS solutions aren't much fun to get working)

then one can use UDP. But then that sort of use case is fairly specialised, and at that point, I wouldn't even use MQTT. I'd simply send UDP packets of known structure. And may be use some of zeromq to help me out.

On the other hand, if you're transmitting a lot of telemetry data (and we have clients doing 1.6bn messages or more a month), you'll want custom data compression. One of the easiest ways of doing that for many statistics is key-framing - sending only what's changed since the last 'key' message. There are many permutations, one of which can involve MQTT's RETAIN. But that then requires ordering, ie queuing, requiring protocol-level support, etc.

But even where data loss is somewhat tolerable, the recovery mechanisms rapidly put in complexity, and the load for doing it ends up in userspace. Using TCP-like approaches means the load ends up in kernel-space, where it arguably belongs. There are higher level flow control problems (eg a rogue client firing 1000s of messages a second, preferential clients paying more, etc) which have to be dealt with in user space - but if they are significant one would use a different protocol, eg AMQP, which is designed more for that sort of thing.

As for 'alerting someone there's a problem when a connection times out' - well, it might work for a small set up, but with large volumes or once reliably unreliable networks are involved (eg 2G), then it just doesn't scale in our experience. Actually, you'd probably want to be able to queue those alerts somewhere reliably, and run a moving window processor to suppress repeats... The other metrics you mention might be very appropriate. Typically, we find things like that are usually heuristically derived by measuring the application over a week or a month, so one can build useful alerts based on steady state models (rates of change, acceleration, etc) rather than static numbers (queue depth).

The challenge is that metrics like these then put the message retry logic squarely into the application programmer's space, when really they should be dealt with as close to the connection as possible - by a broker or client. What happens if a client can't retransmit because of a limited buffer (or no buffer)? Only the client knows that, for instance, and so should communicate it implicitly with his connection (by using QoS, say).

Raphael Cohn
Chief Architect, StormMQ

Secretary, OASIS AMQP Standard
raphae...@StormMQ.com

+44 7590 675 756

UK Office:
Hamblethorpe Farm, Crag Lane, Bradley BD20 9DB, North Yorkshire, United Kingdom
Telephone: +44 845 3712 567

Registered office:
16 Anchor Street, Chelmsford, Essex, CM2 0JY, United Kingdom
StormMQ Limited is Registered in England and Wales under Company Number 07175657
StormMQ.com

On 5 October 2012 02:33, garyamort <me+...@gary.mort.net> wrote:

On Thursday, October 4, 2012 2:53:52 AM UTC-4, Raphael Cohn wrote:

Interesting question... and one that comes quite often for messaging protocols.

A messaging protocol needs delivery of messages in order. TCP gives that without any "effort" required on behalf of the protocol designer. UDP makes that harder and requires protocol-level support.

I'm not following why would be needed.  Considering that the size of an MQQT packet tends to be very small, there is little need for packets to arrive in order as out of order packets can be ignored.  As such, it sees to me that the MQQT protocol could be used directly over serial connections or embedded in udp.

At least, that's how I'm working on it with MQQT on a launchpad.  For my test project, the launchpad will act as an mqqt server - waiting for a connection on it's serial port.

The client runs on my pc, it connects to the launchpad via a wireless virtual com port and then submits a subscribe fridge-temp

It then waits some pre-determined time and if it does not receive a suback or a message, it will retry the subscription.

Once it has confirmed subscription, it consumes any published temperatures and logs them for future analysis.

For topic fridge-temp it's 10 chars from the original ascii set, so 10 bytes of data.  The temperature can be sent in a wasteful human readable format of 

+999.999F to -999.999F, 9 chars so 9 bytes.  Add in the packet framing of 2 bytes and it comes to 21 bytes, so far below the maximum packet length of udp in use today that fragmentation would never be a reasonable concern.

As long as I'm using QOS 0, UDP is a fine protocol for publishing a stream of information, such as temperature readings taken every 10 seconds.  If a bunch of udp packets get dropped at some point, there is no problem as it is not important for the data to be reliably collected every minute.  Instead, reliability can be put in place using a combination of:

MQQT client using ping requests to periodically ensure the connection is still alive.  If it does not receive 3 out of 5 acknowledgements on pings then it can send an alert to inform someone of a problem.

Additional logic can periodically check to ensure that there have been at least 20 readings received in the last our.  1/3rd of the readings is an acceptable number for this use case.

All of that is application logic.  MQQT doesn't need to concern itself with, or even care, about that data.  As I read the protocol, as long as QOS is 0, there is no requirement for responses for most packets.

[Raphael Cohn]

I don't see it as a headache... I very much doubt we'll see more than one or two javascript libraries providing MQTT over WebSockets, and if they're any good they'll abstract away the connection details and simply pass opaque message payloads. You can then gzip or base64 or lzma or use plain text or whatever you want at that point. As for the MQTT connection state data (eg subscriptions) one would hope that it uses a simple javascript, anaemic object model whose prototype can be augmented by developers to provide the necessary behaviours and callbacks.

I'd not be in favour of base64-encoding or JSON-encoding the transmission unless it was impossible otherwise. Adding 1/3 bloat cost to our bandwidth and slowing JS down even further isn't something I'd want to do unless we had to do.

Raphael Cohn
Chief Architect, StormMQ

Secretary, OASIS AMQP Standard
raphae...@StormMQ.com

+44 7590 675 756

UK Office:
Hamblethorpe Farm, Crag Lane, Bradley BD20 9DB, North Yorkshire, United Kingdom
Telephone: +44 845 3712 567

Registered office:
16 Anchor Street, Chelmsford, Essex, CM2 0JY, United Kingdom
StormMQ Limited is Registered in England and Wales under Company Number 07175657
StormMQ.com

On 5 October 2012 02:54, garyamort <me+...@gary.mort.net> wrote:

I am not sure if requiring binary format is a good idea.  Not all browsers can use it, plus it can be a real headache for someone used to coding where ints and chars are almost interchangeable to start thinking at the byte level.

Also a lot of websocket server implementations have their own custom formatting, for example http://jwebsocket.org/ is a java websocket server and javascript client.  While it can be used to build a simple websocket server, it uses plugins to allow one server to route traffic to many different brokers.   This is partially done using special data encoding where the data is sent inside a json encoded packet with additional framing data to direct it to the correct plugin.

As it is both simple and standard practice to base64 encode binary data, the mqqt packets could be base64 encoded and embedded in json.  It would not be until everything is stripped off and delivered to the mqqt server or client that it is back in binary format..as long as those 2 pieces get the data properly formatted - they should not need to care about what might have happened to the data while it was being transported between the..base64 encoded in json, or written out on punch cards and sent via carrier pigeon - the transport mechanism is irrelevant.

As such, I'd extend the protocol name to be more clear, ie:

Must use "mqtt" as a suffix for the websocket protocol name.

Should include the supported mqtt version after the suffix

Must include a descriptive label in the protocol name.

For example:

mqtt-v3.1-raw-binary

MQTT protocol version 3.1, raw binary payload

mqtt-v3.1-json-base64
MQTT protocol version 3.1, json encoded packets of some sort, base64 encoded payloads

On Thursday, October 4, 2012 11:30:20 AM UTC-4, Dave Locke wrote:

its a little different but I have started a page on the Paho wiki about what it would mean provide MQTT over WebSockets in an inter-operable way.   i.e. have first class MQTT support direct from a modern web browser where the app is written in HTML and JavaScript.  The page can be found here: http://wiki.eclipse.org/Paho/Paho_Websockets

[Gary Mort]

On Friday, October 5, 2012 10:20:10 AM UTC-4, Raphael Cohn wrote:

If you...
- Only want at most once messaging (QoS 0);
- Have tiny messages;
- Don't care about data loss;
- Don't care about order of data (ie don't care about it being a queue)
- Don't want or can handle less security needs (and UDP + TLS solutions aren't much fun to get working)

then one can use UDP. But then that sort of use case is fairly specialised, and at that point, I wouldn't even use MQTT. I'd simply send UDP packets of known structure. And may be use some of zeromq to help me out.

Ah, but by using MQTT you now have the ability to scale from hobbyist level to enterprise.  It means your hobbyist can set everything up using UDP - or even a straight serial connection - and honestly most likely they will have no error recovery - it runs, it gathers data, and if it fails... *shrug*.

But as long as everything is abstracted out, with very little change the app can be scaled.

So on a launchpad I might be using:

payload_protocol.c <-- my data format

mqtt_protocol.c  <-- mqtt protocol framing functions

tilaunchpad_uart_connection.c <-- communication link

Stick an RF boosterpack on there and the only programming change is

tilaunchpad_rf_connection.c

To go from hobbyist to enterprise, swap out the processor  for an msp430f5329 and now the app becomes a bit more complicated:

msp430f5x_tcp_connection.c 

plus adding in contikiOS in order to get some a full tcp stack and some additional functionality

http://www.ti.com/product/msp430f5329

The main purpose in using a somewhat common protocol[MQTT] is it allows easy scaling and testing.  And avoiding locking in to specific underlying protocols allows for easy testing. [If you know websocket support is there, then you can run a simple MQTT server application in a chrome web browser on your desktop in order to test/view the device status.]

And, at least to me, it's fun...and fun means more people trying out different things and adding to the eccosystem.

[Andy Piper]

It's on the Paho wiki - please come and discuss on the Paho mailing list.

-- 
Andy Piper

Sent with Sparrow

--

[Jaspar Stach]

So you did a working MQTT Protocol over a SCTP Protocol?
Would be interesting for me to know, what's the trick with it.

Greetings
Jaspar

[Roger Light]

Hi Jaspar,

Essentially I just added the SCTP line below when opening a socket.

hints.ai_family = PF_UNSPEC;
hints.ai_flags = AI_PASSIVE;
hints.ai_socktype = SOCK_STREAM;

+ hints.ai_protocol = IPPROTO_SCTP;

getaddrinfo(listener->host, service, &hints, &ainfo);

Terribly boring and not much point unless you go on an use some of the
features that sctp provides.

Cheers,

Roger

[Jaspar Stach]

Ok. I am using python instead of C and my problem is, that the packets are sent via TCP only the socket connects on SCTP and the other SCTP Packets I can see (on WireShark) are Heartbeat Packets. But all the MQTT relevant Data is send via TCP and the MQTT Message Packets are send via TCP, too. Instead of MQTT, like before SCTP connection.

[Roger Light]

Hi Jaspar,

It sounds like your code isn't doing what you think :) My guess would
be you've got a separate connection set up that is using TCP. I've not
done any Python and SCTP though, so can't really say more.

Cheers,

Roger

[nobletrout]

What would the exact benefit be for using SCTP in this space? Roaming capabilities? I've always struggled to figure out what the benefit is of it over TCP.

[Jaspar Stach]

We need to change the transport protocol for research in university.
We use MQTT and CoAP and need to change Protocol for each to DCCP and SCTP.