MQTT Implementation - Dilemma on strategy of implementation, experienced advice needed.

[Point85]

Hi, 

I have an IOT project I am working on and I have a dilemma on how to implement the control.

We have a GSM device which connects on a periodic interval and then shuts down. When it boots it sends a status packet with various parameters over MQTT which are stored in a database. This is working well.

What I need to do at this point is occasionally pass back some configuration variables.

In an ideal world I would have a response code such as 0 - Failed, 1- Succeeded, 2- Succeeded config packet to download. If a 1 was received in response to the MQTT then the device would not look for a config. If a 2 was received the device would download config information.

Now MQTT doesn't let me send response codes like that....

I am now thinking that the device on boot-up should subscribe to a unique settings endpoint (something like /s-(device serial number)).

Should then send the information packet.

   The server should then process this packet into the database, and if a configuration is needed to be sent to the device publish it to  /s-(device serial number)

There would need to be a delay in the device shutting down after sending the packet, to wait for a potential response being published to /s-(device serial number). Something like 10 seconds.

While this sounds like it would work in theory, how reliable will this be once scaled? There is the potential for hundreds of thousands of devices... Can an MQTT message sit on the queue for a long period before being sent (longer than the shutdown delay)?

It sounds like a scaling nightmare to me....

Is there a better solution??

[Ranjith Kumar Diraviyam]

The objective of the MQTT is to keep the devices connected to the Server continuously.  I am not sure about your exact use case and the reason for your connect and disconnect for each message send.  This will cause problem when you scale up. 

if you still want to follow the shutdown method. Try writing the shutdown on the message received. if your flow is purely publish -- receive -- shutdown. 

Hope this helps. 

Ranjith@Bevywise.

[Dustin Sallings]

On Sat, Oct 12, 2019 at 6:31 AM Point85 <alien...@gmail.com> wrote:

 

I am now thinking that the device on boot-up should subscribe to a unique settings endpoint (something like /s-(device serial number)).

Should then send the information packet.

   The server should then process this packet into the database, and if a configuration is needed to be sent to the device publish it to  /s-(device serial number)

There would need to be a delay in the device shutting down after sending the packet, to wait for a potential response being published to /s-(device serial number). Something like 10 seconds.

While this sounds like it would work in theory, how reliable will this be once scaled? There is the potential for hundreds of thousands of devices... Can an MQTT message sit on the queue for a long period before being sent (longer than the shutdown delay)?

It sounds like a scaling nightmare to me....

Is there a better solution??

I like using retained messages for configuration of devices like this.  The latest configuration is always available to any subscribers (e.g., both your device and any management system) at any time, and whenever the device connects and asks for it, it'll get it. 

[Andy Stanford-Clark]

You might want to look at the request-response pattern over MQTT…

device subscribes to a unique topic (maybe based on its serial number).

devices sends a message to a “service” topic (e.g. “configuration/request” or something like that) asking for any config updates, *and including the secret “reply-to” topic that it is subscribed to*

Server replies to the device’s private topic with whatever information is needed, or some other response to show nothing new to send.

Device can unsubscribe from the private topic at this point if it wants to.

Andy

--
To learn more about MQTT please visit http://mqtt.org
---
You received this message because you are subscribed to the Google Groups "MQTT" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mqtt+uns...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mqtt/d6c49539-0bb8-4873-a7fb-3f5ba053046a%40googlegroups.com.

[anton prokofiev]

Hello, this could be done quite simple if you just change your pattern a little bit:

1. You have 2 topics:

Devices/Id/data

Devices/Id/config

Where Id a unique identifier of a device. 

Your back end send configuration messages to the topic  devices/id/config. 

Message could be: ok, error,

Update-config with a list of configuration parameters to be updated. 

You should send RETAINED messages to this topic, see mqtt specification.  

1. Your device connec to the broker 

And subscribe to the config topic. 

At first connection there are no messages.  

2. Device  send data to the data topic, disconnect and shut down.

3, server process message and send reply to the device config topic.  As message is retained. It is kept in the queue. 

4. Device booted. 

Connect to the config topic and got  server reply for it previous message. 

Then it has options:

If ok, it can send new message,

If config it update config data based on the parameters it got from the message. 

If error probably re-send last message and send a new one.   

Then it disconnects.  

So cycle repeat and you have stable flow of messages.  

So this is close to your original idea, with some improvements:

Device should not wait to get reply from the server, as it will get it on a next connect. 

As you use RETAINED messages, MQTT will ensure that your device will get it on a next connection. 

One more remark: as you could see from documentation: retained message will be sent to all new connections to the given topic. 

So you can use it to share basic configuration data with all you devices, effectively reducing configuration parameters from millions of messages you should store to just on.  

Of course this depends on you usage pattern. 

Regards, Anton 

To view this discussion on the web visit https://groups.google.com/d/msgid/mqtt/DEF81C92-AB51-47AF-9273-C38498E0271A%40stanford-clark.com.