Is it possible to deliver message to just one subscriber

[Sandra Krneta]

Hi all

I was wondering did anybody have similar problem, and did he found some solution for that.
We are trying to integrate MQTT communication channel  in existing application which was using REST.
One of requirement is that our application can be easily scaled over multiple servers,
right now we have 2 servers, and devices were sending data using REST posts, load balancer
was routing requests on one or another server.

Now we need two way communication and decided to go with MQTT for newer devices,
but now if we have data sent with some topic from device to broker, and our application is subscribed
to that topic form both servers, that message will be delivered to both servers (and stored twice).
Is there some smart way to handle this situation and deliver message to just one subscriber?

I was thinking about subscribing on odd end even serial numbers (because we are using 2 servers now, or some similar strategy for 3 or 4 servers)
 to balance load on our servers and to be sure that message sent from one device will end up on just one application server,
but it if  application server, which is taking care of odd serial number devices, is down second one won't start handling messages form devices with odd serial numbers
without some custom logic which will be checking states of application servers, and sent signal to live server to take over.

Did anyone have similar problem, and do you have some idea how to solve this?
Any idea is welcome.

[Stefano Costa]

Sandra,
(to me) the scenario is not so clear... correct me if I'm wrong.

- one MQTT broker
- all devices will publish to that broker
- two (or more) HTTP servers
- both servers subscribe to all topics on the (same) broker
- the HTTP servers are there in order to give redundancy / load balancing

The short answer is that MQTT by itself (as a protocol) has no
provisioning for rules like "if more than one client subscribed to this
topic, then send the related messages only to (the first?) one".

It may be possible that one of the existing implementations of a broker
has this, not to my knowledge.

You could play with $SYS statistics (on Mosquitto) and try to understand
of the HTTP server is connected as a client to the broker etc etc or
better, to stay on the safe side and not be linked to one particular
implementation, each server should specify a "last will" message when
subscribing to topics on the broker and a very simple watchdog demon
(ruby, python, anything) should react to the relevant message saying
"server #1 has disconnected!" with a logic, ex. start republishing the
messages that went to server #1 so that they will be processed by server
#2 now.

Something useful can also be obtained by playing with more brokers and a
bridge rule between them.

--
Stefano Costa, R&D Manager
M +39 335 6565749
Skype stefanocosta.bluewind
Twitter @stefanobluewind
http://www.bluewind.it

[ನಾಗೇಶ್ ಸುಬ್ರಹ್ಮಣ್ಯ (Nagesh S)]

>>>One of requirement is that our application can be easily scaled over multiple servers,
right now we have 2 servers, and devices were sending data using REST posts, load balancer
was routing requests on one or another server.

I think, you should still retain your load balancer; except that, it should subscribe to all posts from the MQTT broker and then carry on with its existing logic to distribute the publications to HTTP servers. So, you would have devices -PUB-> MQTT broker -SUB-> balancer -Distribute-> server1, server2 ... serverN.

N

--
--
To learn more about MQTT please visit http://mqtt.org
 
To post to this group, send email to mq...@googlegroups.com
To unsubscribe from this group, send email to
mqtt+uns...@googlegroups.com
 
For more options, visit this group at
http://groups.google.com/group/mqtt
 
---
You received this message because you are subscribed to the Google Groups "MQ Telemetry Transport" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mqtt+uns...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Paul Fremantle]

Sandra

What you are describing is a common pattern. Effectively those two servers are acting as a single subscriber and sharing the messages between them.

In a normal full-messaging environment you have both queues and topics available, so what you would do is to have the message distributed to the topic, and you have a queue as a subscriber to the topic. The queue represents the single subscription for the two servers. Then the servers pull messages from the queue. The nice aspect of this is that the servers auto-load balance - they pull work when they can. If one goes down, the other keeps pulling all the work.

That isn't easy to replicate in a pure pub-sub model. One approach is that described above, where you take the mqtt messages, and then HTTP post them via the load balancer into the server. 

If you don't care about what happens if a server dies, you could create two topics and then subscribe each server to half the messages, but if one dies, the other won't take over. 

I'm sure there are some other clever things to do, but they aren't popping into my head right now. 

Paul

--

Paul Fremantle
CTO and Co-Founder, WSO2
OASIS WS-RX TC Co-chair, Apache Member

UK: +44 207 096 0336
US: +1 646 595 7614

blog: http://pzf.fremantle.org
twitter.com/pzfreo
pa...@wso2.com

wso2.com Lean Enterprise Middleware

Disclaimer: This communication may contain privileged or other confidential information and is intended exclusively for the addressee/s. If you are not the intended recipient/s, or believe that you may have received this communication in error, please reply to the sender indicating that fact and delete the copy you received and in addition, you should not print, copy, retransmit, disseminate, or otherwise use the information contained in this communication. Internet communications cannot be guaranteed to be timely, secure, error or virus-free. The sender does not accept liability for any errors or omissions.

[Sandra Krneta]

Yes, you understood it correctly.
I am just investigating possibilities of brokers,
and what can be achieved with bridging more brokers.
And this thing with "last will" seems like good idea.
Thanks

[Sandra Krneta]

That came to my  mind also, to have single subscriber which will put all messages in the queue, and then servers will pull messages from there,
but I wanted to see is there some other solution.
Another idea is to have one subscriber which will get messages from broker as they arrive, and then check both servers states(like load balancer do) and republish that message
with new topic, on which just one server is subscribed e.g /server1/originalTopic. But it doesn't look so nice to me.