message cloning (again?) - objects damaged when a node output is connected to two destinations.

[Simon H]

Hi all,

working on a bit of native opencv in Node-Red, and hit a familiar issue...

using node-opencv, an image is represented by an object which includes executable functions.  If a function node has more than one connection from an output, only one connection will get the original msg object, the others will get cloned objects.

In my case, because of the cloning, the image object is incorrect..  and functions on it no longer execute (like .toBuffer() to encode the img to jpeg).

Suggestion:  if variables within a msg object start with '_' (or some other magic character/prefix), they could be excluded from the clone and just the reference duplicated?

Then we'd have the option of cloning or not... and it would be fairly easy to document when it would or would not be appropriate to use such objects?

thoughts?

Simon

[Nick O'Leary]

Hi Simon,

I've always resisted doing something like this for a couple reasons:

 - it makes the message passing less efficient as the runtime now has to check every property and decide whether it should be cloned or not.

 - with an eye on the future, it wouldn't work if a flow was distributed over multiple runtimes as the messages would have to be JSON encodable to traverse the network

Happy to discuss further; just explaining why we've purposefully avoided doing something like this already.

Nick

--
http://nodered.org
 
Join us on Slack to continue the conversation: http://nodered.org/slack
---
You received this message because you are subscribed to the Google Groups "Node-RED" group.
To unsubscribe from this group and stop receiving emails from it, send an email to node-red+unsubscribe@googlegroups.com.
To post to this group, send email to node...@googlegroups.com.
Visit this group at https://groups.google.com/group/node-red.
To view this discussion on the web, visit https://groups.google.com/d/msgid/node-red/0411f594-c2d9-4962-8826-ccff6ecd6f3d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Simon H]

Hi Nick,

yes, we've discussed before; at length I seem to remember :).

NR has uses beyond small data; for example streampunk are using it for real time video processing (rightly or wrongly; it's very impressive, and BBC approved!).

In many of these extreme uses, the data is large, and cloning an 8GByte buffer would kill the ability to run.

In my case, (now looking at opencv), if the video frames are passed as something duplicatable (which I don't know if they are :) - but they don't work after cloning), again cloning would be very bad - the worst thing being that at the moment its not obvious to an average user why adding a second cable totally kills the performance... (if it worked).

NOT cloning is also bad - I've had my own bad experiences when I've modified message content in one part of a divided flow, not anticipating it's effect in another part of the flow; again as a newbie at the time, the effects were not anticipated and lost me a good few days of diagnosis to sort out.

So, what are the options?

Do we just document heavily that multiple wires will clone - but you still don't know which wire will be the clone?

Do we create a std duplicator node, on which multiple outputs can be configured as cloned or not? (there is already skargill's Diode, which I guess is a cloner; I have my own duplication function node approach to avoiding cloning).

Do we allow every output of any node to be configured to be one of 'system decide/always clone/never clone'?

Should we make the wires themselves have a configuration for cloning? - or maybe, we just change the colour of wires which will contain a cloned message?

I do tend to just think as I type; so sorry for the disjointed thoughts!. I really like that last one - just colour the wire to indicate it will be cloned! (Q: would THAT wire always be the cloned one, even after a restart or an export/import?).  This would also be a solution for multi-platform, as the wires which end up cross platform could be highlighted (if we end up knowing/being able to predict which nodes will be on what platform).

Thinking about multi-platform - there will always be cases where you want to control which links are cross platform, and which are not, and we'll need profiling tools which highlight links not suitable/ideal for cross-platform use....

I'll get back to my opencv now; going ok, but maybe by the end of the evening flying....

very best,

Simon

[steve rickus]

I'm just wondering if this would a good candidate for extending the output port properties, similar to the port labels. Perhaps a "Clone msg object?" column of checkboxes next to each port label -- by default, the first port would be unchecked, and the rest would be checked, so that old flows would continue to work the same way they have in the past. Is that too oversimplified to be a good solution?

--

Steve

On Monday, October 16, 2017 at 4:17:35 AM UTC-4, Nick O'Leary wrote:

I've always resisted doing something like this for a couple reasons:

 - it makes the message passing less efficient as the runtime now has to check every property and decide whether it should be cloned or not.

On 16 October 2017 at 08:19, 'Simon H' via Node-RED <node...@googlegroups.com> wrote:

[Julian Knight]

Some good thoughts there Simon.

I like the idea of an option connected to the output port that would force a clone where that is necessary. As Nick points out, it isn't necessary for most use cases so we wouldn't want it as a default. 

I'm not convinced that adding complexity to the wire is a good idea as I think that would add significant complexity to Node-RED itself - at least to the flow definitions.

I do like the idea of a colour or shape indication though it could be on the port rather than the wire which might be more consistent with current Node-RED design?

Or maybe this whole thing should go on the "nice to have" pile for someone to pick up via a pull request when they have the time and energy. Since we can already handle this via a node for now.

[Bart Butenaers]

Hi folks,

Last week I had a similar issue, while testing my new contribution (multipart decoder).  

The decoder is decoding an endless MJPEG stream from an IP camera, generating messages (containing a camera image in the msg.payload):

The first wire (1) carries the original image, to display it in my dashboard.

The message on the second wire (2) is a clone of the first message, which is only used to measure the number of images per second.

This means that a lot of data (= image) is being copied, and I don't even need the images in my speed node.  Just want to measure the speed...

A silly problem for me at the moment, since I spend lot of time getting my decoder node optimized (both for cpu and memory). 

I thought that the new pluggable wire in the future could solve my issue, by letting me develop a new wire that allows to deactivate cloning.

But I have the impression that I misunderstood the new concept, and this will not be possible ???

Disabling the (default) cloning on a wire, input port, or output port would be a great feature. 

If someone can convince Nick with a good solution, I'm willing to develop a pull request (if I get the correct specs) !!!!!!!!

Bart

[Julian Knight]

If you want the image stream and the stream rate, why not add a second output port? Surely more efficient to do that in the node rather than having to take a clone of the stream?

[Dave C-J]

or if just for debug - just set node.status text

[Simon H]

Hi Bart,

As per Julian's response,

The simplest short term solution is a function node with two outputs containing:

node.send([msg, null]);

node.send([null, msg]);

return;

then (currently) there will be no cloning, and you control the order of msg delivery to the downstream nodes.

In the longer term, you may not be able to control the order of delivery if async messaging becomes the default; if this is important to you join in the discussion.  I don't think cloning will become the default for a single msg delivery, as this will kill performance.

s

[Bart Butenaers]

Hi Dave, Julian,
thanks for the workarounds but it was just a simplified example to explain why I would appreciate if the cloning could be disabled on the ports or on the wires.

More general I want 1 single output port to be wired to N input ports (speed, license plate recognition, motion detection, ...). And suppose I'm sure that none of these won't change my message data. Then it would be a big performance boost if I could avoid cloning.

But I'm pretty sure that Nick has good reasons to avoid this kind of things. Could imagine that the results could become weird if multiple nodes would change the same data in an unpredictable order??

And the nodes I'm using don't pass the original input message on an output port, so I cannot simply connect them in one large series. Or is that some design pattern that I missed during custom node development?

Bart

[Bart Butenaers]

Hi Simon,

Some time ago I have been playing around with Java's FBP dataflow framework.  There I had also a similiar issue: the allow many(outputs)-to-one(input) but not one(output)-to-many(inputs).  I have always loved the fact that Node-Red supports both.  In FBP you had to workaround this one-to-many lack by adding an extra node in between (which is responsible for the routing and cloning):

• a demultiplexer node which clones the input message to every output
  
• a load balancer node which sends the input message to one of the outputs (depending on some criteria)

Do you mean that is the way I should proceed?  If so is there already some node available that supports this kind of behaviour?

Anyway still a bummer that I have to implement this by adding an extra node in between, instead of adding some adjustable behaviour to the ports or wires ;-(

And could you please explain 'you may not be able to control the order of delivery if async messaging becomes the default' in dummy language?  Don't have enough knowledge yet of the whole emit/async story...  Does this mean that even your function node approach (N x sending same message) would not work correctly anymore in the future ??

Thanks a lot !!!!

Bart

[Simon H]

Hi Bart,

I just use a normal function node with two (or more) outputs and the three lines of code in it :).

In terms of forcing cloning, you have node-contrib-diode - I've not used it but I think that's what it does...

Basically, if async delivery becomes the norm, then the node would fire out two messages, but both would wait for the next tick in nodejs before calling the input of the next node; and I'm not sure you could guarantee the order.

I would assume that you could still prevent cloning by sending the messages separately, so the approach above would still work for you.

You can force an async version of the simple function node using:

setTimeout(function(){node.send([msg]}, 0);
setTimeout(function(){node.send([null, msg]}, 0);
return;

s

p.s. have you done any OpenCV - sounds like a personal project of mine is close to what you are doing.

I'd like to process pi camera video stream, and do opencv tasks on it (currently masked motion detection - I started to document the NR opencv use at the bottom) - I have some python doing it at the moment, with NR fronting some of the UI.

But I am stumbling with OpenCV in node; I've chosen an OpenCV3 based node-opencv, and it's confusing me at every task - the API is not well documented.