Noflo component providing argument to process() before starting network

[Santosh Kumar]

Hi,

I am designing a browser application and the architecture is such that:

1. DOM will be handled by React
2. Data Visualization by a separate library
3. Data processing by noflo----for this purpose I will have 100 plus custom components. 

I am looking into how to provide arguments to process before start of the network and how to fetch data at the other end of flow into react/reduct while network is running. After reading the docs, I believe It has to be a asCallback. I have following questions:

1. How will the component be loaded: currently I am loading them by keeping js code in the component folder
   
   
2. Currently noflo-component-loader is under devDependencies, Do I need to install that in dependencies?
   
   
3.  noflo-component-loader is designed to automatically pick up and register the components...How can I utilize it for dynamic component.
   
   
4. How arguments can be passed to process() of dynamic components?: By the end of design user will provide it through a UI before running the network

Please let me know how to go about it. 

Thanks,

Santosh.

[Henri Bergius]

Hi,

Great questions, as always. Keep them coming! Though it might be better to post them on Stack Overflow (https://stackoverflow.com/questions/tagged/noflo) to keep the answers more discoverable :-)

On Fri, 10 Nov 2017 at 10:03 Santosh Kumar <lsantos...@gmail.com> wrote:

I am looking into how to provide arguments to process before start of the network and how to fetch data at the other end of flow into react/reduct while network is running. After reading the docs, I believe It has to be a asCallback.

There are two ways to look at this:

1. The main control flow of your application is in regular JavaScript-land, and you're using your NoFlo graphs as essentially a "library".

In that case, asCallback is great for "batch-style" operation where you send some data to NoFlo, and then get results back.

For more continuous usage, you could maybe look at https://github.com/noflo/noflo-wrapper or simply mount the component and bind its ports manually, like so:

const loader = new noflo.ComponentLoader();

loader.load('my-project/MyGraph', (err, instance) => {

  instance.start((err) => {

    // Here you can bind to ports. Using just in/out as example, but can be more

    const out = noflo.internalSocket.createSocket();

    const ins = noflo.internalSocket.createSocket();

    // React to results from outport

    out.on('ip', (ip) => {

      // Received an information packet

    });

    // Send something

    ins.send('Some data');

  });

}); 

This way you're fully in control of the network lifecycle, and can keep a single instance of your graph running as long as your application needs it.

Depending how your graph is made, you can send/receive as long as you need.

2. Have NoFlo manage the control flow, in which case you'll want to expose your UI as "components" in NoFlo land. Typically in these cases you'd have a circular dataflow where your UI generator component can send actions as IPs to the NoFlo graph, and IPs from the rest of the graph get set as props of your UI component.

For a simplistic example, see:

https://app.flowhub.io/#gist/c111454c9fd2f74d37d1e8a4e739adfd

This is also the architecture we used for building Flowhub/noflo-ui, basicallt implementing a Redux-like flow with pure NoFlo. See notes in:

https://github.com/noflo/noflo-ui#architecture

 

1. How will the component be loaded: currently I am loading them by keeping js code in the component folder
   

That's the default approach. We find all components from the components/ folder (and components/ folders of your NPM deps) at build-time, and make them available to NoFlo ComponentLoader. However, we also support "custom loaders". More on that below (sorry for the funky numbering, my mail editor seems to dislike the formatting).

 

2. Currently noflo-component-loader is under devDependencies, Do I need to install that in dependencies?
   

devDependencies is the right place, since noflo-component-loader is only used at build time. What it does is replace NoFlo's built-in component discovery implementation (which looks up components from filesystem, an approach that obviously won't work in browsers) with a generated one that has a statically configured list of components.

2.  noflo-component-loader is designed to automatically pick up and register the components...How can I utilize it for dynamic component.
   

Ok, so in NoFlo we have a feature called "custom component loaders". When noflo-component-loader builds the component loader, it adds a list of all static components (from the components/ folders, as stated above) to the generated component loader.

However, it can also register any custom loaders modules implement. These are registered to noflo-component-loader by adding the following to your package.json:

"noflo": {

  "loader": "path/to/my/CustomLoader.js"

}

These loaders get called when ComponentLoader lists components at start-up. The loader should expose the following interface:

module.exports = (loader, callback) => {}

There the loader argument is the ComponentLoader instance, meaning that you can do whatever you need to discover your dynamic components, and then register them using

loader.registerComponent('libraryname', 'ComponentName', implementation);

 

There are some examples of this out there. The most relevant to your case is probably noflo-polymer, where we discover available WebComponents at runtime, and make those available as NoFlo components:

https://github.com/noflo/noflo-polymer

1. 
   
2. How arguments can be passed to process() of dynamic components?: By the end of design user will provide it through a UI before running the network

Generally any data you want to use in a component should be passed to it as IP packets. Components can have multiple inports, and can wait for data on many of them before activating.

https://github.com/noflo/noflo-physics/blob/master/components/Spring.coffee might be a good example with bunch of (optional) configuration ports for how it will behave.

 

Santosh.

/Henri 

[Santosh Kumar]

Hi Henri,

Thanks a ton for taking your time to provide such a wonderful answer. I will try to post on stack-overflow, though in past I had not so nice experience there. Some moderators are quickly flagging these kind of questions as too broad before it reaches the right audience. I am finding google FBP community great. 

Once I am done this startup project of mine by December end; I plan to create a series of tutorials/use cases on integrating noflo with React/Angular and will try to contribute towards noflo documentation. I come from Visual Effects and C++ background and I believe FBP is going to be the future in many industries including Big Data very soon.

Thanks to your efforts that I do not need to code tools for my dream application.

Regards,

Santosh

 

--
You received this message because you are subscribed to a topic in the Google Groups "Flow Based Programming" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/flow-based-programming/7CQDqLbTsXY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to flow-based-programming+unsub...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.