Node-red as an "image/video processing orchestrator"

[Seb D]

Hello,

I've been looking into and training on Node-red for a couple of weeks. Now I'm seeking people who know node-red better than me and have experienced it more than I have. I hope you could give me your opinion and if Node-red is a good (or at least not bad) choice.

Here's the project big picture:

• Offer a "service platform" on which I could draw/design differrent image processing graphs. Image processiong nodes of the graph could be :
• a custom node that does the job:
• through a Node C++ Addon that uses an image processing lib
• through a Python script that uses Tensorflow for example (eventually through with this npm python wrapper nodejs package)
• by doing the processing directly in Javascript if it's not too heavy
• a call to another service that will do the image processing operation (as in this sample : https://flows.nodered.org/node/node-red-contrib-cognitive-services)
• The images to process could be get on a filesystem accessible to the node-red instance or get through a web service call.
• The node-red graph should expose a web API to receive orders about images to process and send back the results that could be images or reports containing detection scores for example.

In my opinion all this could be achieved and scaled through a cluster of node-red Docker containers. But I'm a bit worried by the performance issues especially if I start dealing with video processing.

I've seen several nodes in the node-red library related to Streampunk Media's Dynamorse project. I still have to give it a try but apparently it deals with audio and video streams over a node-red graph.

My performance worries partly come from the following quote of this FBP author's article  J Paul Morrison:

"NoFlo is based on Node.js and is written in JavaScript and CoffeeScript. These languages basically support a single-threaded implementation, although they can achieve some asynchronism by the use of "callbacks". Although NoFlo and its relatives can simulate asynchronism to some extent, only one thing is happening at a time, and they are limited to using only a single processor."

Which obvisouly concerns Node-red too. But for heavy processing it should not be done by the node-red runtime itself.

So I'm very interesting in the opinion of the community :

• Considering the purpose of my project: am I choosing the wrong tech ? The key point is I do want this FBP approach and so forth being able to easily build different processing flows through the node-red graph editor
• Considering I'll scale the solution through a Docker container cluster: am I right to be afraid about potential performance issues ?

[Julian Knight]

Not sure I'd ever consider JavaScript/Node.JS as being the best platform for doing heavy duty processing of this sort, especially where the process lends itself to high levels of parallelism. However, your options for visual programming are really quite limited. I don't think Node-RED was designed for this kind of processing though that doesn't mean it can't do it. But as you've correctly identified, Node-RED, being built over Node.JS is primarily single-threaded. Node.JS can do multiple threads I believe but multi-threaded program design is always tricky.

The normal approach would be to share the processing out across multiple instances & again, while you can do that with Node-RED, it does take some setup. I've not tried it so I don't know how much performance gain you would get but you come back to the same problem, can you break down your flow sufficiently to allow processing to be shared.

It might be more sensible to use Node-RED purely for the control interface and offload all the complex processing to languages better suited to parallel processing. In a microservice architecture, you could, of course, have many parallel API services running that listen for commands, do a slice of work and return the data. Much as I love using Node-RED, I doubt I would personally choose Node-RED to provide the API microservices.

[Sebastian Barwe]

I agree to Juian, that using Node-RED as a controller would be more common I think.

Regarding node.js it is single threaded by design, but you have possibiliy to use "fork" instances. So you just create new procecess and it is up to the scheduler how to handle this with your SMP system.

To make interprocess communication more convenient, nodejs provides the cluster module. But nevertheless it means a new V8 engine per new process ..  on the other hand everything is isolated, can be distributed etc. this is why nodejs is supposed to be "scaling well".

But this needs to be designed into the architecture of your application ... and in Node-RED it not...

For example: One flow could be a childprocess and the link-node could be IPC-capable. This would give us the benefits that you could the spread the work of a flow over different processes or even machines. An errornous flow would not lead to a total downtime of the runtime but only for the affected flow. By errornous this are not unhandled exceptions, this could be aunresponsive process due to an infinite loop or an outpassing event queue as well. From a point of system architecture you seem to get very much: scalability, robustness, transparency and control. 

Downside would be a higher requirement on ressources and additional latency accross flow to flow message passing.

Some nice introduction on Multiprocessing with nodejs: http://blog.carbonfive.com/2014/02/28/taking-advantage-of-multi-processor-environments-in-node-js/

Though this not multithreading as new processes do not share the same memory/adressspace etc. 

But as you can be build wrappers for native code for node.js you can incorporate native threads into nodejs. But they need to be synchronized to the nodejs main thread. 

So it will be something like ... "calculate this stuff here on the GPU, give me the result etc."  You are decoupling tasks from the event loop of nodejs with the possibilty of synchronization. and this is nothing else as "control" an other task.

I think this is the concept of webworkers-thread and you can find the module for node here: https://github.com/audreyt/node-webworker-threads

It follows the Webworker API and use message passing for synchronization with nodejs mainthread.

What I do not really understand from these concepts is how to be performant with a high volume data stream.

For example if it would be possible to share a nodejs Buffer (DoubleBuffer) with native code as a uin8 array where the thread can access to, but it would only be synchronized by message (e.g. by just providing the info: new data in buffer 1)

I do not know if this possible, but that may make it more performant between both worlds. I would need to drive deeper into this...

br

Seb

[Sebastian Barwe]

Appendix to my previous post 

Regarding Buffer share between native and nodejs: https://community.risingstack.com/using-buffers-node-js-c-plus-plus/

and node-webworker-threads should get a Promise API (and not the odd postMessage stuff)  around so even async/await possible (run on thread, syncrhonized to eventloop)... but could not findsomething...

So with this background, I would think to choose Node-RED as a controlling instance in means of how to synchronize a multithreaded dataflow and offloading the work and sharing the data with above methods is worth something to look at 

[Seb D]

Thank you both and the links on webworkers and buffer sharing between the NodeJS and C++ runtimes will probably be very helpful!

The fact that Node-red authors made a design choice by limiting nodes to a single input is something else I'm concerned about but I did read the google group discussion about it and there're different alternatives to this (I kind of lie this one which appears to be the same as this one IMHO).

I partially share Julian Knight opinion but I'll give Node-red a try on the proof of concept and may switch to another FBP runtime (may be CppFBP) and use a graph editor that supports FBP protocol like DrawFBP or Noflo-ui

[Nick O'Leary]

There is also the built-in Join node (that didn't exist when those others were first published)

Nick

--
http://nodered.org
 
Join us on Slack to continue the conversation: http://nodered.org/slack
---
You received this message because you are subscribed to the Google Groups "Node-RED" group.
To unsubscribe from this group and stop receiving emails from it, send an email to node-red+unsubscribe@googlegroups.com.
To post to this group, send email to node...@googlegroups.com.
Visit this group at https://groups.google.com/group/node-red.

To view this discussion on the web, visit https://groups.google.com/d/msgid/node-red/6381ee27-c5b9-4612-b415-5475087ce0a6%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Sebastian Barwe]

Hi Seb D,

you should definitely try a noflo example with multiple outputs and inputs. When using node red at the beginning I had the wish to have multiple inputs as well.

But after using noflo I go for Node RED from a perspective of user experience. 

At the end it was more convenient to follow a flow (e.g. with subflows ) with node-red as with one input the nodes tend to be more concentrate on one function - which is a good thing when looking at separation principle of software design.

The noflo designs seemed a lot of more complex and because the wire routing is not optimal you have a lot of crossing lines.

I know that has nothing todo with the functionality and the runtime it is just user experience of the editor. But this an important part to make the work with it more fun.

So I will stick to Node-RED because it really makes fun to use and is just simple

But this is totally my own oppinion (and of some customers).

br

Seb B