Disabling MQTT input node

[Tijl Dullers]

Hi,

Is there an easy way to disable an MQTT input or output node programmatically? E.g. you read some stuff from Mqtt to process it and dump it into a DB, in case the DB is offline you want to stop porcessing incoming Mqtt messages so they remain queued on the broker.

[Julian Knight]

No I don't believe there is. It certainly would be an interesting option. I came across a related issue recently when running a development environment on my laptop connected to a broker at home. When away from home, there is no connection and it would be nice to be able to temporarily turn off the connection.

In the meantime, two possible approaches I can think of quickly.

1. If the broker is on a different machine, use a command line script, triggered from NR to change the local machines firewall to prevent access (though how you would know when to reconnect would be an issue maybe)
2. Cache the incoming messages using NR. So set an in-memory flag when you've seen the DB go offline and use that to redirect incoming MQTT messages to a temporary store. Dump the store to the DB when it comes back online

Not sure either of those is useful to you and I'm sure there are other ways too.

[Colin Law]

On 27 September 2016 at 09:32, Julian Knight <j.kni...@gmail.com> wrote:
> No I don't believe there is. It certainly would be an interesting option. I
> came across a related issue recently when running a development environment
> on my laptop connected to a broker at home. When away from home, there is no
> connection and it would be nice to be able to temporarily turn off the
> connection.

This is an interesting area that has been concerning me, I have been
meaning to run some tests. If node-red is running on the client
machine and the broker is running on a remote server then if an MQTT
output node is configured with a QoS of 1 or 2 then my understanding
is that (eventual) delivery of the message should be guaranteed even
if there is no connection to the server at the time the message. The
mqtt client should buffer messages if necessary. I think that if this
does not happen then this is a bug in the mqtt client s/w underlying
the mqtt out node. As I said I have not actually tried this yet.

Colin

[Julian Knight]

Good point. You are right of course that the sender should keep trying until the receiver acknowledges receipt. Of course, that is only true until the sender is reset - unless, presumably, the retain flag is used.

Don't think this helps Tijl though because in his case, it is the DB going offline that I think he is interested in rather than the broker? So he wants to temporarily disconnect a subscribe.

[Colin Law]

On 27 September 2016 at 16:12, Julian Knight <j.kni...@gmail.com> wrote:
> Good point. You are right of course that the sender should keep trying until
> the receiver acknowledges receipt. Of course, that is only true until the
> sender is reset - unless, presumably, the retain flag is used.
>
> Don't think this helps Tijl though because in his case, it is the DB going
> offline that I think he is interested in rather than the broker? So he wants
> to temporarily disconnect a subscribe.

You are correct I think.
Tijl - in what way is the DB going offline? is it on a different
machine to node-red?

Colin

>
> On Tuesday, 27 September 2016 15:36:35 UTC+1, Colin Law wrote:
>>
>> On 27 September 2016 at 09:32, Julian Knight <j.kni...@gmail.com> wrote:
>> > No I don't believe there is. It certainly would be an interesting
>> > option. I
>> > came across a related issue recently when running a development
>> > environment
>> > on my laptop connected to a broker at home. When away from home, there
>> > is no
>> > connection and it would be nice to be able to temporarily turn off the
>> > connection.
>>
>> This is an interesting area that has been concerning me, I have been
>> meaning to run some tests. If node-red is running on the client
>> machine and the broker is running on a remote server then if an MQTT
>> output node is configured with a QoS of 1 or 2 then my understanding
>> is that (eventual) delivery of the message should be guaranteed even
>> if there is no connection to the server at the time the message. The
>> mqtt client should buffer messages if necessary. I think that if this
>> does not happen then this is a bug in the mqtt client s/w underlying
>> the mqtt out node. As I said I have not actually tried this yet.
>>
>> Colin
>

> --
> http://nodered.org
>
> Join us on Slack to continue the conversation: http://nodered.org/slack
> ---
> You received this message because you are subscribed to the Google Groups
> "Node-RED" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to node-red+u...@googlegroups.com.
> To post to this group, send email to node...@googlegroups.com.
> Visit this group at https://groups.google.com/group/node-red.
> To view this discussion on the web, visit
> https://groups.google.com/d/msgid/node-red/65d56d86-1529-4a61-900d-e2fef3e2fac9%40googlegroups.com.
>
> For more options, visit https://groups.google.com/d/optout.

[Tijl Dullers]

So, allow me to Clarify:

- Case: I  already use message retainment and QOS 1 for all my MQTT flows, so whenever a MQTT input node crashes, or I want to disable a nodered instance for maintenance, the messages get buffered on the broker, and I get them in next time I'm connecting without losing anything. This works pretty nice, but it is natively built into the MQTT protocol, so that's not a surprise.

- Case: I have a flows in node-red that reads input from MQTT node, processes some stuff in function node and then writes it to a DB. But, suppose that e.g. the DB has an issue and I cannot insert data into the DB, the function node connected to the MQTT in, will already have processed a message and presumably, the MQTT in node (client) already confirmed message acknowledgement to the broker.

So, this behaviour leads to 2 options:

1) The 'dirty' way: Store the SQL statements I cannot write to the DB to a temporary file and flush it. I have that working with json-file-db and yes, it works but it is more complex than it should be.

2) I check with e.g. isonline if my database is online, or I can even make a specific query to the DB to assure it functions correctly, if not I should be able to set a value that tells the MQTT in node to disconnect so it does not process any new messages until the DB is back operational after which I will get all buffered messages from the broker. This assures I don't loose any messages and do not have to implement a cache to file mechanism for each case.

[Colin Law]

I think option 1 should work, though I would store the mqtt messages
rather than the SQL. Just push each new message into a queue (whether
the db is online or not) and release them from the queue when they
have been actioned see [1] for example which looks as if it might to
the job. If you needed persistence over reset then you would have to
use a file based version. I am a bit surprised there is not a
node-red-contrib-queue that would do that.

Your option 2 may not be safe, as the db might go offline just after
you receive a message and before you can disable the mqtt node, so you
would still have at least one message to buffer.

Easier to go with option 1 I think. This sounds like an ideal
opportunity to write a node-red-contrib-queue node.

Colin


[1] http://flows.nodered.org/flow/cea8afa28b7a93ebdc0f

>
>
> --
> http://nodered.org
>
> Join us on Slack to continue the conversation: http://nodered.org/slack
> ---
> You received this message because you are subscribed to the Google Groups
> "Node-RED" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to node-red+u...@googlegroups.com.
> To post to this group, send email to node...@googlegroups.com.
> Visit this group at https://groups.google.com/group/node-red.
> To view this discussion on the web, visit

> https://groups.google.com/d/msgid/node-red/b86dd611-1282-4066-a34b-3d749d0b732a%40googlegroups.com.