How to initialize flow and global context before anything else (Don't we need an Init node ?)

[Jan Van den Audenaerde]

Hi,

Currently I am wondering how I can properly initialize flow context variables and global context variables before any node becomes executed that is reading/updating these flow/global context variables.

Usually I do this with an inject node that is triggered at start which calls a function node that initializes all the flow and global context variables.

But this doesn't guarantee that the initialization is happening before any of the other nodes becomes executed.

Maybe we need to create a specific function node "init" having no inputs and outputs.

Node-Red will assure that the javascript specified in this "init" node will be completely executed at the start before any other node (that is not an init node).

It might also be interesting to specify a priority level (= a number) for the "init" nodes so that "init" nodes with a higher priority level become executed before "init" nodes with a lower priority level.

This way it becomes also possible to control the order of execution of the "init" nodes if this is needed.  

What do you think ?

kr
jan.

[Colin Law]

Hi Jan

The first thing I would try to do is get rid of flow and global
context variables. In fact there are very few occasions where they are
necessary, and in the long run I believe it is better to avoid their
use whenever possible. It is better to pass data around as necessary.
If you give an example of how you are using them I may be able to
suggest an alternative. In all my flows (so far) I have zero flow or
global context variables. Others will say that I am overzealous in
this and it is easier just to use the context and initially this may
appear to be the case, but in the long run they often cause more
problems than they save. A working lifetime writing software and
seeing the problems that global data can cause has led me to avoid
them whenever practical.

However I believe that global context may be initialised in
settings.js using the functionGlobalContext setting.

Colin

On 15 February 2017 at 07:49, Jan Van den Audenaerde

> --
> http://nodered.org
>
> Join us on Slack to continue the conversation: http://nodered.org/slack
> ---
> You received this message because you are subscribed to the Google Groups
> "Node-RED" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to node-red+u...@googlegroups.com.
> To post to this group, send email to node...@googlegroups.com.
> Visit this group at https://groups.google.com/group/node-red.
> To view this discussion on the web, visit
> https://groups.google.com/d/msgid/node-red/8df02a81-4016-43b7-bc5f-e0897b68d8d5%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Max Hadley]

A similar situation arises on the Beaglebone. Node-RED has to run as root to be able to acces GPIO devices. However, once the device files have been created, it can drop back to being a non-privileged user OK. Currently I do this by using an inject node 'once at start', a delay, and a function node. But there is no way to guarantee that all the BBB nodes have initialised except by setting a very long delay, which increases the window of vulnerability. Ideally I'd like to guarantee it was no longer root before starting any internet exposed nodes.

[Jan Van den Audenaerde]

Hi Colin,

I am very well aware of issues when using global variables and I agree if it can be fixed by passing it in the message payload then it is better to do so,

but in this case I am speaking about the initialization : at that time there are no messages that contain the information I need.

Some examples where flow context / global context might be used and should of course be properly initialized

1. when a static mapping table is used by multiple nodes.
2. when you want to maintain a flow state.

I understand that global context might also be initialized in settings.js using the functionGlobalContext setting but I think it is more beautiful if this could be done by an "init" node:

as in that case all relevant information about the flow can be found in the flow itself.

kr

Jan

[Colin Law]

Hi Jan

When you say maintaining flow state do you mean things like the state
of switches?

Not sure what you mean by a mapping table.

Colin

On 15 February 2017 at 12:19, Jan Van den Audenaerde

> --
> http://nodered.org
>
> Join us on Slack to continue the conversation: http://nodered.org/slack
> ---
> You received this message because you are subscribed to the Google Groups
> "Node-RED" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to node-red+u...@googlegroups.com.
> To post to this group, send email to node...@googlegroups.com.
> Visit this group at https://groups.google.com/group/node-red.
> To view this discussion on the web, visit

> https://groups.google.com/d/msgid/node-red/edd7dabd-aa6e-42fb-98e8-b460b58ed3bc%40googlegroups.com.

[Jan Van den Audenaerde]

Hi Colin,

1/ State can be anything that you can store in a flow context or global context.  E.g. you can store the timestamp of the last message received by a certain function node in a flow context variable.

    This will make it possible if that new function node receives a new message to calculate the time between the received message and the previous message.

2/ A mapping table can be e.g. a javascript variable like   ip2name =  { "192.168.1.2" : "pi_one", "192.168.1.3": "pi_two", "192.168.1.1" : "my_wifi_router" };

    This variable can then be used by the different function nodes to convert IP addresses to meaningful names.

kr

jan

[Colin Law]

On 15 February 2017 at 12:52, Jan Van den Audenaerde
<jan.vanden...@gmail.com> wrote:
> Hi Colin,
>

> 1/ State can be anything that you can store in a flow context or global
> context. E.g. you can store the timestamp of the last message received by a
> certain function node in a flow context variable.
> This will make it possible if that new function node receives a new
> message to calculate the time between the received message and the previous
> message.

I don't see how it is necessary to initialise context of that sort.
What are you going to initialise it to? Also why save that in the flow
context rather than the node context?

>
> 2/ A mapping table can be e.g. a javascript variable like ip2name = {
> "192.168.1.2" : "pi_one", "192.168.1.3": "pi_two", "192.168.1.1" :
> "my_wifi_router" };
> This variable can then be used by the different function nodes to
> convert IP addresses to meaningful names.

I would put that in settings.js, I do see your point about keeping
everything in the flow though. On the other hand there is an argument
for saying that it is *better* in settings as it means it is not
necessary to edit the flow in order to change the mapping.

Colin

> https://groups.google.com/d/msgid/node-red/63ab4f75-49ff-4cac-8528-0cf63fe8620e%40googlegroups.com.

[Jan Van den Audenaerde]

On Wednesday, February 15, 2017 at 2:28:47 PM UTC+1, Colin Law wrote:

On 15 February 2017 at 12:52, Jan Van den Audenaerde
<jan.vanden...@gmail.com> wrote:
> Hi Colin,
>
> 1/ State can be anything that you can store in a flow context or global
> context.  E.g. you can store the timestamp of the last message received by a
> certain function node in a flow context variable.
>     This will make it possible if that new function node receives a new
> message to calculate the time between the received message and the previous
> message.

I don't see how it is necessary to initialise context of that sort.
What are you going to initialise it to? Also why save that in the flow
context rather than the node context?

In the simple example above I only wanted to respond to your question what I mean by state.

Indeed - like you mention - for this simple example you could have stored it in the node context instead of the flow context

but that just moves the initialization problem from the flow context to the node context.

To come back to the initialization issue which is also applicable for the simple scenario as we can not calculate the 

time difference if no previous message has been received.

So to fix this I would like to initialise the node or flow state as follows.

   state = { has_previous_message_received : 0 };

Once the flow or node receives a message the state becomes updated as:

   state = { has_previous_message_received : 1, last_received_message_timestamp :  <timestamp of last received message> };

when the flow or node wants to calculate the time difference between the current and the previous message it must first check

if (state.has_previous_message_received===1) otherwise we are calculating the time difference using an unitialized variable.

In this simple example we just wanted to know if a variable is initialized or not

but there are more complex cases where we want to initialize a variable or set of variables at specific values.

E.g. imagine where you are controlling a light switch or the temperature of your heating system.  

So when your raspberry pi reboots (e.g. due to temporarily power failure when you are on Holiday)

you would like it to set the light switch and the temperature of your heating system properly.

In those scenarios it would be good if you can explicitly set the state at initialization. 

>
> 2/ A mapping table can be e.g. a javascript variable like   ip2name =  {
> "192.168.1.2" : "pi_one", "192.168.1.3": "pi_two", "192.168.1.1" :
> "my_wifi_router" };
>     This variable can then be used by the different function nodes to
> convert IP addresses to meaningful names.

I would put that in settings.js, I do see your point about keeping
everything in the flow though.  On the other hand there is an argument
for saying that it is *better* in settings as it means it is not
necessary to edit the flow in order to change the mapping.

Colin

 

In my cases these mapping table are always specific for a certain flow so I would like to keep it together with the flow.

Note also that if I want to migrate the flow to another raspberry pi then I can export and import the complete flow without any need to copy parts of the settings.js files.

So having it in the flow - for me is definitely better than having it in the settings file.

[Colin Law]

On 15 February 2017 at 14:28, Jan Van den Audenaerde

Can you not just do, in the node where it is used
var lastTime = context.get('lastTime')
if (typeof lastTime === 'undefined) {
// initialisation code
}
...

In fact where I have exactly this situation I have done
var now = Date.new();
var lastTime = context.get('lastTime') || now
and then I make sure my code copes sensibly with a time difference of
zero, which is good practice anyway just in case it ever happens.

>
> In this simple example we just wanted to know if a variable is initialized
> or not
> but there are more complex cases where we want to initialize a variable or
> set of variables at specific values.
> E.g. imagine where you are controlling a light switch or the temperature of
> your heating system.
> So when your raspberry pi reboots (e.g. due to temporarily power failure
> when you are on Holiday)
> you would like it to set the light switch and the temperature of your
> heating system properly.
> In those scenarios it would be good if you can explicitly set the state at
> initialization.

In those cases I strongly recommend using MQTT with Retain = true.
Then the mqtt server will automatically feed you the current value
when node-red restarts. This has the additional great advantage that
you can have multiple instances of the UI open at once and they will
all remain in sync via mqtt.

>
>> >
>> > 2/ A mapping table can be e.g. a javascript variable like ip2name = {
>> > "192.168.1.2" : "pi_one", "192.168.1.3": "pi_two", "192.168.1.1" :
>> > "my_wifi_router" };
>> > This variable can then be used by the different function nodes to
>> > convert IP addresses to meaningful names.
>>
>> I would put that in settings.js, I do see your point about keeping
>> everything in the flow though. On the other hand there is an argument
>> for saying that it is *better* in settings as it means it is not
>> necessary to edit the flow in order to change the mapping.
>>
>> Colin
>
>
> In my cases these mapping table are always specific for a certain flow so I
> would like to keep it together with the flow.
> Note also that if I want to migrate the flow to another raspberry pi then I
> can export and import the complete flow without any need to copy parts of
> the settings.js files.
> So having it in the flow - for me is definitely better than having it in the
> settings file.

In fact in the particular case you show it might be better to put them
in the servers hosts file, then you can refer to them by name
everywhere, not just in node-red.

Colin

[Jan Van den Audenaerde]

For simple cases like the one above we can indeed check if the typeof the context variable is defined or use the OR (= ||) operator.

 

>
> In this simple example we just wanted to know if a variable is initialized
> or not
> but there are more complex cases where we want to initialize a variable or
> set of variables at specific values.
> E.g. imagine where you are controlling a light switch or the temperature of
> your heating system.
> So when your raspberry pi reboots (e.g. due to temporarily power failure
> when you are on Holiday)
> you would like it to set the light switch and the temperature of your
> heating system properly.
> In those scenarios it would be good if you can explicitly set the state at
> initialization.

In those cases I strongly recommend using MQTT with Retain = true.
Then the mqtt server will automatically feed you the current value
when node-red restarts.  This has the additional great advantage that
you can have multiple instances of the UI open at once and they will
all remain in sync via mqtt.

You are assuming that I am using MQTT which actually is not true

:e.g. I am using html requests to activate/deactivate a pump.

>
>> >
>> > 2/ A mapping table can be e.g. a javascript variable like   ip2name =  {
>> > "192.168.1.2" : "pi_one", "192.168.1.3": "pi_two", "192.168.1.1" :
>> > "my_wifi_router" };
>> >     This variable can then be used by the different function nodes to
>> > convert IP addresses to meaningful names.
>>
>> I would put that in settings.js, I do see your point about keeping
>> everything in the flow though.  On the other hand there is an argument
>> for saying that it is *better* in settings as it means it is not
>> necessary to edit the flow in order to change the mapping.
>>
>> Colin
>
>
> In my cases these mapping table are always specific for a certain flow so I
> would like to keep it together with the flow.
> Note also that if I want to migrate the flow to another raspberry pi then I
> can export and import the complete flow without any need to copy parts of
> the settings.js files.
> So having it in the flow - for me is definitely better than having it in the
> settings file.

In fact in the particular case you show it might be better to put them
in the servers hosts file, then you can refer to them by name
everywhere, not just in node-red.

 

It was just an example.  One of the real cases was the mapping between MAC addresses and device names like "Jan's phone".

[Colin Law]

On 15 February 2017 at 16:24, Jan Van den Audenaerde

<jan.vanden...@gmail.com> wrote:
> On Wednesday, February 15, 2017 at 4:24:49 PM UTC+1, Colin Law wrote:

>> ...

>> In those cases I strongly recommend using MQTT with Retain = true.
>> Then the mqtt server will automatically feed you the current value
>> when node-red restarts. This has the additional great advantage that
>> you can have multiple instances of the UI open at once and they will
>> all remain in sync via mqtt.
>>
> You are assuming that I am using MQTT which actually is not true
> :e.g. I am using html requests to activate/deactivate a pump.

If I were in that situation I would still use MQTT (for the
persistence and other advantages it offers). From the flow that
determines the activation/deactivation send the result to MQTT, then
separately pick it up from MQTT and send it to the pump. Note that
the latter operation need not even be in the same machine as the flow
controlling the logic.

As a matter of interest what device have you go that accepts html
requests to drive it?

Colin

[Mike Bell]

Hello,

My two cents (or two thoughts). In principle, I agree with Colin, but I have at least once backed myself into a situation where I needed to initialize a flow context and couldn't justify re-factoring the code to use messages. I suggest Jan take a look at the config node (node-red-contrib-config), as it may do what he needs.

Mike

[Jan Van den Audenaerde]

On Wednesday, February 15, 2017 at 5:59:19 PM UTC+1, Colin Law wrote:

If I were in that situation I would still use MQTT (for the
persistence and other advantages it offers).  From the flow that
determines the activation/deactivation send the result to MQTT, then
separately pick it up from MQTT and send it to the pump.  Note that
the latter operation need not even be in the same machine as the flow
controlling the logic.

As a matter of interest what device have you go that accepts html
requests to drive it?

 

http://openenergymonitor.blogspot.be/2015/08/3ch-wifi-relay-control-board.html

I didn't use the MQTT interface as it was not reliable in my setup for some reason. 

Colin

[Julian Knight]

Hi Max, wouldn't it be better to fix the permissions on the GPIO? This isn't an uncommon requirement.

Also, we shouldn't forget that global vars can be set in settings.js which is simply another Node.JS file and is run before the flows start.

[Julian Knight]

It seems to me that several issues are being conflated here.

Initialisation of flows can be problematic if you need something to happen before the flows start. However, you do have the capability to do some things, certainly to set global variables, using the settings.js which is run before the flows start.

As for keeping state, as has been said, most of us are using (misusing?) MQTT but there are plenty of other options varying from a simple file, JSON formatted file, SQLite, etc. depending on need/complexity.

[Dean Cording]

node-red-contrib-config would be what you are after. It's available from npm

Dean

[Peter Scargill]

Putting stuff in settings.js is really not the answer - that's going way away from our simple visual interface.  Now at one time when we had this discussion - either you or Dave (or someone) told me that if you put stuff in the LEFTMOST TAB it would be run first.....  so I think of the left most tab as the init tab. I have never gotten around to proving definititely whether or not this works every time - I should imagine if there's a callback involved it probably won't.....

[Dave C-J]

When you deploy (and we write) the flows file we write it global config nodes first (including subflow definitions), then tab by tab left to right, - and it is likewise read and instantiated in that order. (if you set the settings.js to save in pretty format you can edit the flows file and see for yourself). As long as the first flow on the first page does not have any async calls in then it should at least be called first (after any config nodes). 

[Joseph Martine]

I am not in no way desiring to highjack this thread, but want to give another example.   I am working on a configurable dosing pump for my aquarium.  As part of the process, it goes through a calibration process that give me a ml/sec that is then stored and used each day when it comes time to pump that day's dose of chemicals.   This value needs to be settable via the ui/calibration flow, stored if the PI is rebooted, and quickly/easily accessible on demand.  The Config node shared above is nice, but does not accept programmatic input.  I have seen some other things from smart guys like Scargill that persist the context to a json file as things change and can be loaded at startup into a single global and then is accessed by something like global.config.item.

[Colin Law]

If I understand the question correctly, I solve such problems by
sending the value to an MQTT topic with Retain true. Then on restart
it is immediately available.

Colin

> --
> http://nodered.org
>
> Join us on Slack to continue the conversation: http://nodered.org/slack
> ---
> You received this message because you are subscribed to the Google Groups
> "Node-RED" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to node-red+u...@googlegroups.com.
> To post to this group, send email to node...@googlegroups.com.
> Visit this group at https://groups.google.com/group/node-red.
> To view this discussion on the web, visit

> https://groups.google.com/d/msgid/node-red/34cff2f5-c547-4a40-a42f-8bdaf942f627%40googlegroups.com.

[Peter Scargill]

You received this message because you are subscribed to a topic in the Google Groups "Node-RED" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/node-red/e_nO4L2W5P8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to node-red+u...@googlegroups.com.
To post to this group, send an email to node...@googlegroups.com.

Visit this group at https://groups.google.com/group/node-red.

To view this discussion on the web, visit https://groups.google.com/d/msgid/node-red/CAL%3D0gLu-82RHrxBDQ-6KfGkLe%2BNQ-YfEGDWQq8aU8cafzYz-uw%40mail.gmail.com.

[Peter Scargill]

Trust me - that needs explaining. Like string theory - once you understand...

-----Original Message-----
From: node...@googlegroups.com [mailto:node...@googlegroups.com] On Behalf Of Colin Law
Sent: 14 March 2017 21:50
To: node...@googlegroups.com
Subject: Re: [node-red] Re: How to initialize flow and global context before anything else (Don't we need an Init node ?)

You received this message because you are subscribed to a topic in the Google Groups "Node-RED" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/node-red/e_nO4L2W5P8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to node-red+u...@googlegroups.com.
To post to this group, send an email to node...@googlegroups.com.

Visit this group at https://groups.google.com/group/node-red.

To view this discussion on the web, visit https://groups.google.com/d/msgid/node-red/CAL%3D0gLu-82RHrxBDQ-6KfGkLe%2BNQ-YfEGDWQq8aU8cafzYz-uw%40mail.gmail.com.

[Joseph Martine]

From the answers on this thread and others (going back to 2015 on all the forums) I have read over the previous weeks, I feel that this question is still largely unanswered.  I understand the short answer of what Colin had replied.  And at the same time and concerned about the comments from Julian about the time/place to use MQTT.  I have read lots of comments from the main contributors about there is something in the works, but for the life of me cannot find the end result or status.  

Definitions:

  1.) CI = Config Item = any value that the user or system will reference over time 

        a.) Current state of something (light - On/Off, CurrentFluidLevel - 1020mL) - can receive input from many sources.

        b.) Attribute - ContainerCapacity - 1500mL - seldom if ever changes, 

        c.) Calculated Flow R

Here is what we have:

  1.) Server?/global/flow variables that are manipulated with getters and setters

  2.) DB - read/write using existing nodes  - 

  3.) MQTT - retain for initialization, pub/sub for current state - how do you query current state in a function?

  4.) Persist JSON file to drive, copy to global, access using global./your var name/./your attribute name/

  

Here are some potential Requirements:  

  1.)  Restore the state of all CIs on an event

  2.)  Values can be user editable, then persisted

  3.)  Read values at runtime - inside a function be able to access current value as is done with Context.get()

I am thinking all of this should maybe go  like this:

None of the above is original solutions, except maybe a mashup of what I have read????  Maybe we don't need a node, but maybe a design pattern?  My little picture doesn't  guarantee first initialization - especially if what Peter mentioned about the left most tab turns out to be false when tested or worse, able to be thwarted but a require or callback.

I am new to Node Red so be kind :-)

(BTW, Julian I am really interested in the MQTT "message bus" layout you had in your GitHub concerning Home Automation).  

[Peter Scargill]

So that left-most tab turns out to be false?   Is that right?  I can think of at least two people who are likely to disagree…. Unless the action taken is asynchronous…

To view this discussion on the web, visit https://groups.google.com/d/msgid/node-red/5f239ae3-ab63-481f-9759-708478fe3391%40googlegroups.com.

[Csongor Varga]

Can I also give another example to Colin? I created a simple report screen in UI, where I have some dropdowns to set some filter values and a Refresh button. So the idea is that the user can use the filter buttons and click on Refresh to update the report. All my dropdowns save the selected value into global. The code behind the Refresh button checks the global values and construct the SQL select accordingly which goes to a sqlite node -> template -> UI.

How would you have done that without global/flow variables?

[Colin Law]

Csongor, when you say "the code behind the refresh button" do you mean
you are doing that in the browser rather than in node red? If in fact
the refresh button sends a message to a node that constructs the SQL
then you can send the filter button outputs to that node too.

Colin

> --
> http://nodered.org
>
> Join us on Slack to continue the conversation: http://nodered.org/slack
> ---
> You received this message because you are subscribed to the Google Groups
> "Node-RED" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to node-red+u...@googlegroups.com.
> To post to this group, send email to node...@googlegroups.com.
> Visit this group at https://groups.google.com/group/node-red.
> To view this discussion on the web, visit

> https://groups.google.com/d/msgid/node-red/61b541dc-81d1-4c01-904f-a60bbfb8eaea%40googlegroups.com.

[ajaykas...@gmail.com]

I was hoping we could have something like an Init Node too, because I have a scenario, where I need to make an http request out at the startup and then get a list of say devices and I store the device list in the global context in a map object as thru' out the flow I need to track the health of these devices, some kind of house keeping I need to perform. I also store this list in a json file, so incase the NR reboots i can retrieve this list back. Getting this list needs to happen before any data starts flowing from those devices and I was also hoping to able to create the necessary device list and its correspond objects prior to any data flowing in and rest of the NR flows begin and there is no way to guarantee this based on what I have read on this thread. Also is there is any synchronization mechanism between access to common resources that are stored in a global context object?

One of the comments mentioned is that these initialization's can be done from the settings.js file, how would I achieve that?

Thanks,

Ajay.

[Sebastian Barwe]

Hi,

see https://nodered.org/docs/configuration under "Node configuration"

or you may find the "node-red-contrib-config" useful to initially setup context variables (Peter put a post how to use on http://tech.scargill.net/node-red-global-flow-and-context/) .

[Csongor Varga]

Yes, the button send a message to a function node where I construction the SQL, send to the a sqlite node, template node, and ui_template node.

OK, I send the filter button outputs to the function which construct the SQL, but how I stored the value of filter? E.g. I have a filter for time (last 24 hours, last week, etc.) and another filter for status. I set the time for last 24 hours, I update the list. Than I set the status to something. I would like the list to show me items in the last 24 hours and in particular status. In that case i would need to store the time, who when the status filter message comes in I can combine the two.

[Dave C-J]

the settings.js file is indeed just that ... a javascript file... that gets read and executed early in the startup phase... hence you could put things in here that you want initialised before the rest of the flow gets going... Of course these ought to be simple setting type things really rather than asynchronous function calls as they still may not finish before the rest starts.

[Colin Law]

Store them in the node, in the node context. Use the topic to identify
what you are setting up so in the function node test the topic and if
not 'refresh' (for example) then save the payload in the node context
context.save(msg.topic, msg.payload);
Then when a refresh comes in the values can be read from the context
and used to setup the SQL

Colin

> https://groups.google.com/d/msgid/node-red/df983fbf-8c78-47d3-b497-ff62362f4532%40googlegroups.com.

[Peter Scargill]

I'm just thinking - I really am not happy about messing with settings.js like that especially when it might be necessary to UPDATE such info....   but it occurred to me this morning after my first coffee (so it may be flawed)  that an INCLUDE  in the settings.js might be better - to, say a file called defaults.js or something - which could then be over-written by Node-Red itself in the event that stored values need changing without fear of screwing up the settings.js file.  Is there a way to do this?

[Dave C-J]

Well you could use settings.js to include another file... but as we don't overwrite settings.js anyway that's up to you. On first install/run we copy the default settings.js to your user dir if it doesn't exist. If it exists we don't touch it.

[Peter Scargill]

Thanks for the confirmation Dave – I guess we could put a marker in there and simply update any info past that – I’ll have a play with that – clearly need to be able to update that from Node-Red… hmmm. That config node works but it only has a one-line input field which is a pain when working with large objects…

 

From: node...@googlegroups.com [mailto:node...@googlegroups.com] On Behalf Of Dave C-J
Sent: 24 March 2017 09:58
To: node...@googlegroups.com
Subject: Re: [node-red] Re: How to initialize flow and global context before anything else (Don't we need an Init node ?)

 

Well you could use settings.js to include another file... but as we don't overwrite settings.js anyway that's up to you. On first install/run we copy the default settings.js to your user dir if it doesn't exist. If it exists we don't touch it.

--

http://nodered.org
 
Join us on Slack to continue the conversation: http://nodered.org/slack
---

You received this message because you are subscribed to a topic in the Google Groups "Node-RED" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/node-red/e_nO4L2W5P8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to node-red+u...@googlegroups.com.

To post to this group, send email to node...@googlegroups.com.
Visit this group at https://groups.google.com/group/node-red.

To view this discussion on the web, visit https://groups.google.com/d/msgid/node-red/CACXWFwLV2H9UMV9pzd%3DzsdK1kcYsxCB%2BEFzUMtAGLH-KFMi8XQ%40mail.gmail.com.

[Dave C-J]

you can always write to a file from Node-RED... if that file happens to be the one you reference from settings then so be it.

[Nick O'Leary]

Personally, I wouldn't go about trying to dynamically modify your settings file. I would stick to the original idea of importing a separate file.

More explicitly, in your settings file you can do:

var mySettings = require("./path/to/my/own/file");

You can then write to that custom file however you choose - as long as it is kept as valid JSON or a valid node.js module file.

Nick

On 24 March 2017 at 10:17, Dave C-J <dce...@gmail.com> wrote:

you can always write to a file from Node-RED... if that file happens to be the one you reference from settings then so be it.

--

http://nodered.org
 
Join us on Slack to continue the conversation: http://nodered.org/slack
---

You received this message because you are subscribed to the Google Groups "Node-RED" group.
To unsubscribe from this group and stop receiving emails from it, send an email to node-red+unsubscribe@googlegroups.com.

To post to this group, send email to node...@googlegroups.com.
Visit this group at https://groups.google.com/group/node-red.

To view this discussion on the web, visit https://groups.google.com/d/msgid/node-red/CACXWFwJPrvDaw4%2B2n%3Dm-7hFFvTDK%2BtXFBTbBXexXGbAYY1C9ww%40mail.gmail.com.

[Peter Scargill]

AHAH that’s what I want but I read some CLEARLY INCORRECT docs somewhere saying you could not import files into a JS file..

 

Yes, something like /home/pi/mysettings.js with a block of Jason in it will absolutely do it – that can be updated from Node-Red without chancing messing up the settings file… I don’t suppose you can put a try-catch around that include???

 

From: node...@googlegroups.com [mailto:node...@googlegroups.com] On Behalf Of Nick O'Leary
Sent: 24 March 2017 10:55
To: Node-RED Mailing List <node...@googlegroups.com>
Subject: Re: [node-red] Re: How to initialize flow and global context before anything else (Don't we need an Init node ?)

 

Personally, I wouldn't go about trying to dynamically modify your settings file. I would stick to the original idea of importing a separate file.

 

More explicitly, in your settings file you can do:

 

var mySettings = require("./path/to/my/own/file");

 

You can then write to that custom file however you choose - as long as it is kept as valid JSON or a valid node.js module file.

 

Nick

On 24 March 2017 at 10:17, Dave C-J <dce...@gmail.com> wrote:

you can always write to a file from Node-RED... if that file happens to be the one you reference from settings then so be it.

--
http://nodered.org
 
Join us on Slack to continue the conversation: http://nodered.org/slack
---
You received this message because you are subscribed to the Google Groups "Node-RED" group.

To unsubscribe from this group and stop receiving emails from it, send an email to node-red+u...@googlegroups.com.

To post to this group, send email to node...@googlegroups.com.
Visit this group at https://groups.google.com/group/node-red.
To view this discussion on the web, visit https://groups.google.com/d/msgid/node-red/CACXWFwJPrvDaw4%2B2n%3Dm-7hFFvTDK%2BtXFBTbBXexXGbAYY1C9ww%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

 

--

http://nodered.org
 
Join us on Slack to continue the conversation: http://nodered.org/slack
---

You received this message because you are subscribed to a topic in the Google Groups "Node-RED" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/node-red/e_nO4L2W5P8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to node-red+u...@googlegroups.com.

To post to this group, send email to node...@googlegroups.com.
Visit this group at https://groups.google.com/group/node-red.

To view this discussion on the web, visit https://groups.google.com/d/msgid/node-red/CAF%3DvhqcMvRXJ1EWC5iDidtRSoG%2B1M_eecmr_xU2f_ifGAQ8Qyg%40mail.gmail.com.

[Dave C-J]

Yes - adding a try catch would be a good thing.

[Csongor Varga]

Thank you very much. Actually I am doing almost the same with the global, use topic as the key and payload as the value. To be honest, somehow the node context skipped my attention when I was reading the docs. Did not know that is exist.

[Sebastian Barwe]

@peter / changeconfig node : regarding JSON input, that should be much  better with next node red version (https://trello.com/c/3uzZQaS6/93-json-expression-editor)

[Peter Scargill]

Ok, following on from this conversation – I have put this together

 

http://tech.scargill.net/node-red-initialisation/

 

So this WORKS – but I’m not happy with the format of the data in my new redvars.js file – I cannot figure out how to just put JSON data in there (which can be created from a stringify command)… and attempts to wrap the “require” with a try-catch have failed purely because of flaws in my JS knowledge.

 

HOWEVER it works – and solves the problem of initialisation without messing with settings.js other than initial setup – and you can of course now safely mess with vars in the redvars file without screwing up settings.js

 

If anyone has time to make those two improvements (error checking and putting only a chunk of json in the redvars file) I would be very grateful and other can benefit from it as I get LOTS of people on that blog now (2,600 visitors yesterday alone).

 

Thanks for any help in advance guys.

 

Pete.

 

 

From: node...@googlegroups.com [mailto:node...@googlegroups.com] On Behalf Of Nick O'Leary

Sent: 24 March 2017 10:55

To: Node-RED Mailing List <node...@googlegroups.com>
Subject: Re: [node-red] Re: How to initialize flow and global context before anything else (Don't we need an Init node ?)

 

Personally, I wouldn't go about trying to dynamically modify your settings file. I would stick to the original idea of importing a separate file.

 

More explicitly, in your settings file you can do:

 

var mySettings = require("./path/to/my/own/file");

 

You can then write to that custom file however you choose - as long as it is kept as valid JSON or a valid node.js module file.

 

Nick

On 24 March 2017 at 10:17, Dave C-J <dce...@gmail.com> wrote:

you can always write to a file from Node-RED... if that file happens to be the one you reference from settings then so be it.

--
http://nodered.org
 
Join us on Slack to continue the conversation: http://nodered.org/slack
---
You received this message because you are subscribed to the Google Groups "Node-RED" group.

To unsubscribe from this group and stop receiving emails from it, send an email to node-red+u...@googlegroups.com.

To post to this group, send email to node...@googlegroups.com.
Visit this group at https://groups.google.com/group/node-red.
To view this discussion on the web, visit https://groups.google.com/d/msgid/node-red/CACXWFwJPrvDaw4%2B2n%3Dm-7hFFvTDK%2BtXFBTbBXexXGbAYY1C9ww%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

 

--

http://nodered.org
 
Join us on Slack to continue the conversation: http://nodered.org/slack
---

You received this message because you are subscribed to a topic in the Google Groups "Node-RED" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/node-red/e_nO4L2W5P8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to node-red+u...@googlegroups.com.

To post to this group, send email to node...@googlegroups.com.
Visit this group at https://groups.google.com/group/node-red.

To view this discussion on the web, visit https://groups.google.com/d/msgid/node-red/CAF%3DvhqcMvRXJ1EWC5iDidtRSoG%2B1M_eecmr_xU2f_ifGAQ8Qyg%40mail.gmail.com.

[Nick O'Leary]

Hi Peter,

don't forget the settings.js file is a JavaScript file itself, so you can do lots of things in it:

The basic structure of settings.js (and your redvars.js) file is as a node.js module, for example:

    module.exports = {

        uiPort: 1880,

        functionGlobalContext: {

             mySettings:require("/home/pi/redvars.js")

        }

    }

To put a try/catch around the loading of your redvars file, you should try loading it outside of the module.exports declaration:

    var mySettings;

    try {

        mySettings = require("/home/pi/redvars.js");

    } catch(err) {

        mySetting = {};

    }

    module.exports = {

        uiPort: 1880,

        functionGlobalContext: {

             mySettings:mySettings

        }

    }

As for the contents of redvars, just prepend your JSON with `module.exports=` :

    module.exports = { "a": 1, "b": 2}

Nick

On 24 March 2017 at 13:50, 'Peter Scargill' via Node-RED <node...@googlegroups.com> wrote:

Ok, following on from this conversation – I have put this together

 

http://tech.scargill.net/node-red-initialisation/

 

So this WORKS – but I’m not happy with the format of the data in my new redvars.js file – I cannot figure out how to just put JSON data in there (which can be created from a stringify command)… and attempts to wrap the “require” with a try-catch have failed purely because of flaws in my JS knowledge.

 

HOWEVER it works – and solves the problem of initialisation without messing with settings.js other than initial setup – and you can of course now safely mess with vars in the redvars file without screwing up settings.js

 

If anyone has time to make those two improvements (error checking and putting only a chunk of json in the redvars file) I would be very grateful and other can benefit from it as I get LOTS of people on that blog now (2,600 visitors yesterday alone).

 

Thanks for any help in advance guys.

 

Pete.

 

 

From: node...@googlegroups.com [mailto:node-red@googlegroups.com] On Behalf Of Nick O'Leary
Sent: 24 March 2017 10:55
To: Node-RED Mailing List <node...@googlegroups.com>
Subject: Re: [node-red] Re: How to initialize flow and global context before anything else (Don't we need an Init node ?)

 

Personally, I wouldn't go about trying to dynamically modify your settings file. I would stick to the original idea of importing a separate file.

 

More explicitly, in your settings file you can do:

 

var mySettings = require("./path/to/my/own/file");

 

You can then write to that custom file however you choose - as long as it is kept as valid JSON or a valid node.js module file.

 

Nick

On 24 March 2017 at 10:17, Dave C-J <dce...@gmail.com> wrote:

you can always write to a file from Node-RED... if that file happens to be the one you reference from settings then so be it.

--
http://nodered.org
 
Join us on Slack to continue the conversation: http://nodered.org/slack
---
You received this message because you are subscribed to the Google Groups "Node-RED" group.

To unsubscribe from this group and stop receiving emails from it, send an email to node-red+unsubscribe@googlegroups.com.

To post to this group, send email to node...@googlegroups.com.
Visit this group at https://groups.google.com/group/node-red.
To view this discussion on the web, visit https://groups.google.com/d/msgid/node-red/CACXWFwJPrvDaw4%2B2n%3Dm-7hFFvTDK%2BtXFBTbBXexXGbAYY1C9ww%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--
http://nodered.org
 
Join us on Slack to continue the conversation: http://nodered.org/slack
---

You received this message because you are subscribed to a topic in the Google Groups "Node-RED" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/node-red/e_nO4L2W5P8/unsubscribe.

To unsubscribe from this group and all its topics, send an email to node-red+unsubscribe@googlegroups.com.

To post to this group, send email to node...@googlegroups.com.
Visit this group at https://groups.google.com/group/node-red.
To view this discussion on the web, visit https://groups.google.com/d/msgid/node-red/CAF%3DvhqcMvRXJ1EWC5iDidtRSoG%2B1M_eecmr_xU2f_ifGAQ8Qyg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--

http://nodered.org
 
Join us on Slack to continue the conversation: http://nodered.org/slack
---
You received this message because you are subscribed to the Google Groups "Node-RED" group.

To unsubscribe from this group and stop receiving emails from it, send an email to node-red+unsubscribe@googlegroups.com.

To post to this group, send email to node...@googlegroups.com.
Visit this group at https://groups.google.com/group/node-red.

To view this discussion on the web, visit https://groups.google.com/d/msgid/node-red/027b01d2a4a5%249965cb30%24cc316190%24%40gmail.com.

[Peter Scargill]

Hi Nick

 

Tested, works - updating the blog now – in half an hour it should be done – and that, for now I think goes a GOOD way to solving this particular issue.

 

Thanks to you and Dave once again. Marvelous – no more undefined vars for me….

 

Pete.

 

From: node...@googlegroups.com [mailto:node...@googlegroups.com] On Behalf Of Nick O'Leary
Sent: 24 March 2017 14:03
To: Node-RED Mailing List <node...@googlegroups.com>
Subject: Re: [node-red] Re: How to initialize flow and global context before anything else (Don't we need an Init node ?)

 

Hi Peter,

To unsubscribe from this group and stop receiving emails from it, send an email to node-red+u...@googlegroups.com.

To post to this group, send email to node...@googlegroups.com.
Visit this group at https://groups.google.com/group/node-red.
To view this discussion on the web, visit https://groups.google.com/d/msgid/node-red/CACXWFwJPrvDaw4%2B2n%3Dm-7hFFvTDK%2BtXFBTbBXexXGbAYY1C9ww%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--
http://nodered.org
 
Join us on Slack to continue the conversation: http://nodered.org/slack
---

You received this message because you are subscribed to a topic in the Google Groups "Node-RED" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/node-red/e_nO4L2W5P8/unsubscribe.

To unsubscribe from this group and all its topics, send an email to node-red+u...@googlegroups.com.

To post to this group, send email to node...@googlegroups.com.
Visit this group at https://groups.google.com/group/node-red.
To view this discussion on the web, visit https://groups.google.com/d/msgid/node-red/CAF%3DvhqcMvRXJ1EWC5iDidtRSoG%2B1M_eecmr_xU2f_ifGAQ8Qyg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
http://nodered.org
 
Join us on Slack to continue the conversation: http://nodered.org/slack
---
You received this message because you are subscribed to the Google Groups "Node-RED" group.

To unsubscribe from this group and stop receiving emails from it, send an email to node-red+u...@googlegroups.com.

To post to this group, send email to node...@googlegroups.com.
Visit this group at https://groups.google.com/group/node-red.
To view this discussion on the web, visit https://groups.google.com/d/msgid/node-red/027b01d2a4a5%249965cb30%24cc316190%24%40gmail.com.

For more options, visit https://groups.google.com/d/optout.

 

--

http://nodered.org
 
Join us on Slack to continue the conversation: http://nodered.org/slack
---
You received this message because you are subscribed to a topic in the Google Groups "Node-RED" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/node-red/e_nO4L2W5P8/unsubscribe.

To unsubscribe from this group and all its topics, send an email to node-red+u...@googlegroups.com.

To post to this group, send email to node...@googlegroups.com.
Visit this group at https://groups.google.com/group/node-red.

To view this discussion on the web, visit https://groups.google.com/d/msgid/node-red/CAF%3Dvhqed-mk-AhwhzwGCH59rW%2BjniKyDfTzhmc1gvqgOGYFLTA%40mail.gmail.com.

[Joseph Martine]

Thank you SOOOOO Much for your work in this space.   Your blog was one of the early ones I stumbled upon early on.   It has help tremendously in my understanding of several topics.  I have "stolen" several ideas and snippets as I have been battling to get my first project up and off the ground.  This problem of initialization and status/settings management has been my largest roadblock.  I will dl redvars and give it a try today!

Once again thanks!

[Joseph Martine]

I think I may have found a typo.

s.. Here’s a better way. At the VERY START of setting.js add this..

var mySettings;

try {

mySettings = require("/home/pi/redvars.js");

    } catch(err) {

mySetting = {};

    }

I think mySetting should be mySettings???? 

[Peter Scargill]

Correct - you found a bug - and when I went in I found lots of things that could be improved in the description - so - welcome to the NEW version of http://tech.scargill.net/node-red-initialisation/

Thanks for taking the time to check and report.

Pete.

[Joseph Martine]

could you show me a snippet of your redvars.js?

[Joe Morris]

Hi Pete,

Thanks for this post and doing the nice job on redvars.js I am thinking this will not work for me and would like some thoughts.

I have a set of device commands in a MySql table. On deployment I read the table and set global variables to the commands. I then use these commands throughout the flows to initiate actions on the devices. I pair generic commands to the specific device command structures.

I am using the sql table because I have a number of devices that perform the same sets of actions but use different command structures. So the actual query is based upon the device chosen that I wish to take the actions upon and I go and get those device specific command structures.

So unless I execute the sql query outside of node-red and update the redvars.js and then restart node-red I do not think that I have a viable solution yet.

Joe

[Julian Knight]

This is not so different to my own use case. I have a number of different types of device that I both receive from and send commands to.

On receive, I normalise the data and republish to a new topic structure.

For commands, I use a standard command structure that is published and only translated in the final flow before sending to physical devices. However, I keep any translation data in global variables that are set on startup but can be reset at any time. So simply add another flow that re-reads your SQL data (I keep all mine in retained MQTT messages) into the global variable and always consume from there.

[Joe Morris]

Yes, that is my plan. How I actually got to this thread was through my searching on trying to find a way to ensure that the global variable load process ran to completion prior to any other flows starting. I am running into the problem of the global variables trying to be used prior to them being created by the mysql flow.

[Julian Knight]

Perhaps a hybrid approach. Run an initial query in settings.js so that you know the variable is populated before your flows run but then have something in your flows to do further updates.