MQTT vs IPC

[AF]

Hi everyone, I'm very interested in the MQTT protocol, but I'm looking at it with a slightly different use case than most of you will probably have. I'm thinking of extending the MQTT network inside a single system. So I'm thinking of having a server/broker and multiple clients running on the same box. Basically I'm thinking of using it as a form of IPC. I quite like the simple publish/subscribe API and the fact that MQTT has very low overhead. I will potentially push out some MQTT messages to another machine but the main question I have is about using it for communication on a single machine. Usually MQTT runs over TCP/IP. I could do it like that as well on the machine for something like inter-process communication, but creating an IP tunnel looping data on a single machine seems overkill and unnecessary much performance overhead. Now my question is - has someone tried to use MQTT over something like Unix sockets instead of an IP connection? Would that be a good idea? If not, has anyone got a better idea of extending MQTT on a single system? I'm also interested in the advertise/search mechanism of MQTTs which would require broadcast and therefore could not really run over Unix sockets. Is there any other way to extend MQTT within the system?

Any ideas are welcome.

[Stefano Costa]

Hi AF,
we do use MQTT client/broker on the same host, as a variant of a IPC in
several different situations. We actually stay with a TCP communication
and this enables to use several tricks, like using the same application
locally or on remote hosts, adding external hosts for debug purposes,
replicate some local GUI on a remote host, up to having machines in the
field that will replicate local messages on remote hosts for field tests
and so on.

MQTT and related existing applications (the mosquitto suite, and Paho)
are very light and will easily survive even "poor" platforms giving
advantages.

--
Stefano Costa, R&D Manager
M +39 335 6565749
Skype stefanocosta.bluewind
Twitter @stefanobluewind
http://www.bluewind.it

[Dave Locke]

Using MQTT to decouple applications and adapters on the same box is a common pattern.  Most implementations I am aware of use TCP primarily as that is what all MQTT client and server implementations  use.  TCP stacks these days have been optimised for local intra server comms and incurs little overhead.   That said there have been implementations that use different comms, for instance we use to have an implementation of a Java client and server that used Java Streams to provide intra Java runtime comms between the client and the server.

One other benefit of using a small footprint MQTT server is as well as providing intra box comms it can also providing bridging to servers and clients on other boxes.


All the best
Dave

--
--
To learn more about MQTT please visit http://mqtt.org

To post to this group, send email to mq...@googlegroups.com
To unsubscribe from this group, send email to
mqtt+uns...@googlegroups.com

For more options, visit this group at
http://groups.google.com/group/mqtt

---
You received this message because you are subscribed to the Google Groups "MQ Telemetry Transport" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mqtt+uns...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.




Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

[Raphael Cohn]

It's worth pointing out that it's probably as trivial change to any of the C implementations (client and broker) to use Unix domain sockets. You'll need to modify the one or two lines of source code that open a new socket (the call to socket() ), changing the constant AF_INET to AF_UNIX and using slightly different code for bind() - e.g. http://beej.us/guide/bgipc/output/html/multipage/unixsock.html  should be a good steer.

For a Java implementation that DOESNT use NIO, you can use a different socket factory with a third party Unix domain sockets library. If it does use NIO, you're probably stuffed.

That said, I agree with Dave Locke - local loopback TCP stacks on linux are really rather good, although the folks at Dolphin SuperSockets reckon they can do better... and the advantages as Stefano and Dave say of then also being able to go off box quite out weigh the overhead.

Raphael Cohn
Chief Architect, stormmq

Co-Chair, OASIS MQTT Standard

Secretary, OASIS AMQP Standard
raphae...@stormmq.com

+44 7590 675 756

UK Office:
Hamblethorpe Farm, Crag Lane, Bradley BD20 9DB, North Yorkshire, United Kingdom
Telephone: +44 845 3712 567

Registered office:
16 Anchor Street, Chelmsford, Essex, CM2 0JY, United Kingdom
StormMQ Limited is Registered in England and Wales under Company Number 07175657
StormMQ.com

[Stefano Costa]

On 08/02/2013 02:31 PM, Raphael Cohn wrote:
> It's worth pointing out that it's probably as trivial change to any of
> the C implementations (client and broker) to use Unix domain sockets.
> You'll need to modify the one or two lines of source code that open a
> new socket (the call to socket() ), changing the constant AF_INET
> to AF_UNIX and using slightly different code for bind() - e.g.
> http://beej.us/guide/bgipc/output/html/multipage/unixsock.html should
> be a good steer.
>
> For a Java implementation that DOESNT use NIO, you can use a different
> socket factory with a third party Unix domain sockets library. If it
> does use NIO, you're probably stuffed.
>
> That said, I agree with Dave Locke - local loopback TCP stacks on
> linux are really rather good, although the folks at Dolphin
> SuperSockets reckon they can do better... and the advantages as
> Stefano and Dave say of then also being able to go off box quite out
> weigh the overhead.
>

Two rather interesting examples of what we're saying here, taken from real:

(1) "Black box" that should enable integration between two home
automation related words (two different protocol standards, physical
networks, groups of devices). Inside the box two demons talk to the two
worlds and a local MQTT broker is the endpoint of all messages after
"smart" translations. Each demon pubs / subs and can make its own use of
messages. A third rather complex demon is being developed with the aim
of doing calculations, prediction algorithms, etc etc so that the two
worlds can share not only rough data but also mutual "knowledge" of the
environment. BUT: the algorithms are evolving as Matlab simulations
running on a much powerful platform (or even automated Excel sheet...)
and it's a matter of a few hours (or minutes) to connect the simulation
ins/outs to the local broker so that it behaves exactly like the
(future) third demon.

(2) "White good" with a rather complex user interface that will, may be,
in the future be duplicated as a Smartphone app. But in the meantime
having used a local broker as IPC inside the product, it's very easy to
generate an identical user interface that will run on smartphone-like
platforms and magically connects over Internet (with a few tricks and
far from optimized).

Hope this is useful in order to understand how (re)using something that
proved to be flexible and smart in other projects, is always a good idea!

[AF]

I don't think it is such a trivial change. As far as I understand the advertise and search messages of MQTTs require some form of broadcast transport which does not exist for UX sockets? So some other transport would have to be used for this pattern of discovering MQTT servers and clients. In general, the whole principle of discovering endpoints is easier solved with TCP/IP as with using UX sockets. For UX sockets you will always have to fix socket names and hard-code them in the clients. Otherwise you will have to implement some sort of naming service on top of MQTT. How can clients and servers find each other otherwise? It is potentially easier on an IP network where techniques for this are deployed anyway. For IPC it isn't that straight forward.

[Caio]

Hi,

I had the same idea in my project, but I found a ZeroMq project, that
could be interesting solution for this case, zeromq use local socket
(just in Unix systems).
You could find more information at http://api.zeromq.org/2-1:zmq-ipc

Best Regards,

Caio Pereira

2013/8/5 AF <fis...@gmail.com>:

--
----------------------------------------------
Caio Pereira