FPB In Javascript - framework minimal version.

236 views
Skip to first unread message

Fernando Rivero

unread,
Sep 27, 2020, 10:28:04 AM9/27/20
to Flow Based Programming
Hello FBP group! 

my name is Fernando and I am a software developer from Punta del Este, Uruguay (South America).

I found FBP few months ago and now I am building my own framework for JS and deciding if use PHP or NODJS for backend side. (I don't know NODJS yet).
I was always trying to build my own "no code" system and this is the missing piece that I was looking for! Thank you!
But to be honest I am still understanding the concept, I started reading the Paul Morrison's book but I found it a quite difficult to understand the total idea, even for a person with more than 10 years in the software development field and with an strong background in electronic (I developed firmwares for microchip devices in Assembler and C).

In my personal opinion those are the problems:
- Too much dependency with Java language, in my case I know JavaScript (basic level) and PHP (basic level). I mean dependency because in the book and other examples I found in the web, there is almost always examples in Java, so I can't follow the idea.
- There is no a super minimal expression of the concept, or I didn't found it yet.
- Examples are linked to already made solutions, like NOFLO or Sr. Morrison repository, In my case I don't want to use any third party framework like NOFLO, it is very difficult to me to understand and use it, i prefer to build my own micro-solution.

For your information the closest and useful guide that I found (and the only one I found for JS) was this example:
Which is a great contribution but still quite complex for a newbie.

One specific point which I can't understand yet is about the loop in the components and their input ports.
The software need to do a main loop in all the elements for ever?
Or I just need to start the network, then I need to do a loop inside each element for ever?
Would be great is some one can just write a basic diagram showing how an MVP of the FBP works.
Of course and example in JS could be the final solution that I am looking for (and i believe so many other are too) .
Sorry if my terms are too vague, I hope you get the point.
Thank you so much for this great concept!

Fernando Rivero / XENIO

Paul Morrison

unread,
Sep 27, 2020, 11:57:46 AM9/27/20
to Flow Based Programming
Hi Fernando!

First, thanks for the link to David Brooks' article - I thought it was great!  As he says in "The End" of his article:

"While FBP is certainly useful, and fun to create programs with, and has many advantages over textual programming, I think the real lesson to be learned here is: "how should we think about our programs?" Can we write functions/modules that can be viewed as "black boxes"? Sure. Can we write them in such a way that they can be infinitely reconfigurable, connected externally? Yep. Can we model our programs on factory machines and conveyor-belts, focusing on data? Yes! When we do, we gain many of the benefits of flow-based programming.

And I think that's worth at least thinking about."

Quite agree!  The "fun" aspect is one that we keep running into with FBP users!


However, I don't quite know what to make of your own post - maybe some of the other readers would care to weigh in!

A couple of comments, however:

-  you say you started reading my book - if you had read Chap. 3, I feel it would have at least got you started on the concepts... 

- don't know why you say there is too much Java - the book has very few references to Java.  JavaFBP is one implementation - I like it because it plays nicely with DrawFBP, but FBP is language-agnostic... 

- I don't understand the appeal of JavaScript - I find it almost unmaintainable!    If I have to use JS, I prefer to base my software on node-fibers - see https://github.com/jpaulm/jsfbp .  Without node-fibers, I find JS, with all its extensions, very hard to work with - plus, I really don't like typeless!   There is an interesting discussion between some JS gurus about "promises", etc.  in https://github.com/jpaulm/jsfbp/issues/14 ...  You might also want to take a look at  https://jpaulm.github.io/fbp/concat.html , which tries to highlight the difference between doing a rather basic  FBP function in "real" FBP  on the one hand, and "FBP-inspired" on the other.  David mentions my paper contrasting these, but I will repeat the link here: http://www.jpaulmorrison.com/fbp/noflo.html .

- You ask, "Or I just need to start the network, then I need to do a loop inside each element for ever?"  I would have thought this was obvious - in a factory, do you ask if the bottling machine runs forever?   It just runs until it requires maintenance, or is switched off on weekends, or whatever - user's choice!  That's what an FBP app is: multiple machines running concurrently!

Lastly, I understand why David built his own micro-FBP, but he says, "... in other words do not use rhei.js for anything."!  I thought that was great!   In comparison, I do not understand why people (other than David) so often decide to  build their own implementations - .  I would strongly recommend using DrawFBP and JavaFBP - DrawFBP lets you draw your network, automatically generate the Java for it, compile and run it!  What's not to like?!   Bob Corrick in England has produced a number of easy to follow YouTube videos demonstrating this process (do a find on DrawFBP).   Thanks, Bob!

Cheers,

Paul M.


Ged Byrne

unread,
Sep 28, 2020, 5:07:52 AM9/28/20
to flow-based-...@googlegroups.com
Hi Fernando,

In repsonse to your comment that "there is no a super minimal expression of the concept, or I didn't found it yet, " please let me try an describe a simple version of the concept based on standard patterns.

The first pattern is the pipes and filter, which looks like this in the EIP definition:
image.png
And it is described like this: "Use the Pipes and Filters architectural style to divide a larger processing task into a sequence of smaller, independent processing steps (Filters) that are connected by channels (Pipes)" 


Pipes and filters is a common pattern that serves the foundation for both REST and UNIX.  In the dissertation that introduced the concept of REST fielding says: 

"In a pipe and filter style, each component (filter) reads streams of data on its inputs and produces streams of data on its outputs, usually while applying a transformation to the input streams and processing them incrementally so that output begins before the input is completely consumed. This style is also referred to as a one-way data flow network [6]. The constraint is that a filter must be completely independent of other filters (zero coupling): it must not share state, control thread, or identity with the other filters on its upstream and downstream interfaces."

Rather than the metaphor of pipes and filters Paul talks about conveyor belts and workstations within a factory, but they are two metaphors describing the same thing: "visualize an application built up of many such main-line programs running concurrently, passing IPs around between them. This is very like a factory with many machines all running at the same time, connected by conveyor belts. Things being worked on (cars, ingots, radios, bottles) travel over the conveyor belts from one machine to another. . In fact, there are many analogies we might use. "  

The metaphors change, but the drawing stay fairly consistent:
image.png
 

http://www.jpaulmorrison.com/fbp/concepts.shtml  (This is the original 1994 book that is available online.  I'd recommend buying the updated version.)

This is the basic concept, and it is a recurring theme that appears throughout software.  For example, read this description from Von Neumann's essay on EDVAC, written in 1945, the computer is described in terms of telegraph relays: http://web.mit.edu/STS.035/www/PDFs/edvac.pdf

This is a diagram describing a repeater.  Look familiar?

image.png

In 1978 John Backus published the essay: "Can programming be liberated from the von Neumann style?"

So this is the foundations for flow based programming.  We stop thinking about our code as one big wall of text that gets executed on line at at time.

Instead we build systems as many separate processors that have ports for input and ports for output.  

These processors are connected together so that the output from one component is the input for the next component.

The data is sent and received between ports in packets of information, information packets (IPs).  We keep track our our packets to ensure that none get lost along the way.

The "pipe" or "conveyor belt" connecting our two processors is actually a buffer has a limited capacity.  Once that buffer is full then a processor can no longer send packets.  It is blocked waiting for a space to become available.  That space becomes available when the upstream processor receives it's next packet, removing it from the buffer and making a space available.  This creates 'back pressure' that keeps all of the our processors in synchronised: https://medium.com/@jayphelps/backpressure-explained-the-flow-of-data-through-software-2350b3e77ce7

For me these are the fundamentals for FBP.  If this is clear so far I'll go on to describe the different implementation strategies by various frameworks.

Regards, 


Ged





On Sun, 27 Sep 2020 at 15:28, Fernando Rivero <river...@gmail.com> wrote:
Hello FBP group! 

my name is Fernando and I am a software developer from Punta del Este, Uruguay (South America).

I found FBP few months ago....


Fernando Rivero

unread,
Sep 28, 2020, 3:02:45 PM9/28/20
to Flow Based Programming
Thank you so much Paul Morrison and Ged!
Paul:
The reason why I need to create my own SIMPLE JavaScript FBP implementation is because the Apps that I am developing run on mobile browsers and need to run even OFFLINE, so I am using PWA which is JavaScript (https://developers.google.com/web/updates/2015/12/getting-started-pwa).
( Just FYI I am developing small in-house apps for small and medium companies ).
So I need to get one or more fbp networks running in a mobile web browser, there is no room for any server side technology when the app is OFFLINE.
Of course when the app is ONLINE I can run other fbp networks in the backend side using nodjs or PHP.

After I had read your and Ged answers I can see more clearly the reason why you think JavaScript is not a good fit for fbp, it is synchronic and single-threaded!
Unfortunately I can't use fiber or even NODJS because some of my fbp networks will be running in a web browser.
But as I can see it (please correct me if am I wrong), anyway I can build a "fbp inspired" implementation in JS and get at least part of the benefits, the black boxes concept, infinitely reconfigurable and connected externally through INPUT/OUTPUTS ports.
I believe I can't get the port's buffer strategy covered but that is something that I should let go, at least in my "front end" networks.

Ged:
Firs of all I just bought the updated Morrison's book, can't wait to read it! Thanks for the suggestion.
The pipes and filter explanation helped me a lot in order to understand the concept.
Also, would be great to hear different implementation strategies from your end!

As a wrap one of the main pains that I wish to get fixed with fbp is the source code reutilization between projects.

All the best!

Fernando Rivero / XENIO

Paul Morrison

unread,
Sep 28, 2020, 4:39:17 PM9/28/20
to Flow Based Programming
Thanks, Fernando, that's very clear!   However, I don't think mobile browsers rule out the "real" FBP approach.  And, as I said, FBP is language-agnostic.  And IIUC, so are sockets...

So I suggest that you take a look at https://github.com/jpaulm/javafbp-websockets ... In spite of the name, the client side doesn't use Java - it uses HTML (5?)  and JavaScrpt, and is very simple.   The JS is in the 'resources' directory. 

The server side could be written in any language that understands web sockets and has (or could have) an FBP implementation...  This one happens to be Java and so is in the 'java' directory.

I'd just like to point out that Facebook has recently adopted the topology shown in the diagrams, and they call their technology "Flux"...  They have a video at https://www.youtube.com/embed/i__969noyAM  .

Cheers,

Paul M.

Ged Byrne

unread,
Sep 29, 2020, 5:52:21 AM9/29/20
to flow-based-...@googlegroups.com
Hi Fernando,

To review, the foundational principle for FBP is that we treat our program as a series of independent machines that communicate with each other by means of bounded buffers.

Each component can have ports that are coming in or going out.

Each component can have out-ports that can be used to send information packets out to the connected buffer, if there is space available.  If the buffer is full then the component will wait for space to become available.

Each component can have in-ports that can receive information from the connected buffer, if there is an information packet already in the buffer.  If the buffer is empty then the component will wait for an information packet to become available.

We can draw this as a diagram showing our components spread out in space.  Like this:

image.png
However, we have a problem.  Our code must be able to run on a single CPU and CPUs can only do one thing at a time.

We do not have two CPUs, one for the filter and another for the selector.  We have just one CPU that must do the job of both.

This is the problem that an FBP implementation must solve.

The simplest solution is to use multithreading.  Using multithreading we can timeshare our CPU across the two threads.  This is how the Java and C# implementations work.  Each component gets its own thread and from a coding perspective that looks exactly the same as having a dedicated CPU.

You wrote: "One specific point which I can't understand yet is about the loop in the components and their input ports.  The software need to do a main loop in all the elements for ever?  Or I just need to start the network, then I need to do a loop inside each element for ever?"

This is an aspect of the mutli-threaded solution.  Each thread has it's own main loop, and the loop does something this:
1) Recieve from a port.  Remember, if the port is empty then the thread is blocked and cannot proceed.  This is when the CPU will put the thread to sleep and move on to another.
2) Process the input until it is ready to send something.
3) Send to a port.  Again, if the port is full the thread will block and wait for the next component to receive and create space.
4) Loop back to 1.

Obviously we can have much more complex loops than this.  In the above example the FILTER will have  more Receives than Sends because it will just drop the packets it is filtering out.  The selector will receive from one port and send to two different ports.  We might have a RANDOM component that has not in-ports.  Is just looks like this:

while true {
  Out.Send( Random() );
}
  
This is the "classic" FBP described in Paul's book.  It uses the threading capabilities of the underlying OS to achieve a simple, elegant solution.

The problem is that Javscript does not support multi-threading.  So what can we do?

Paul has already pointed out his soltuion, so I will look at what can be done in standard JS.

I'll quickly skim over some possible solutions that work with standards JS.  Before going deeper I'd like to hear about how you would like to approach this.

  1. Single Activation
    • This is the approach taken by Node.JS.  Our component is a javascript modeule with a process command that gets called when the component is activated.  The component runs until it is finished.  We do not block on in-ports because the component will only activate is packets are ready and waiting.  Where there are multiple in-ports it may be necessary to test to see which one has content.  More details here: https://noflojs.org/documentation/components/
  2. Event Driven
  3. Observable
  4. Co-Routines
There are all the approaches I'm aware of.  Am I missing any?

What I'd like to do next is to consider how we can move on from these implementation methods is to consider how we might implement a zero-code approach within a disconnected javascript single web page application.  How do you envisage this working?

Regards, 


Ged







--
You received this message because you are subscribed to the Google Groups "Flow Based Programming" group.
To unsubscribe from this group and stop receiving emails from it, send an email to flow-based-progra...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/flow-based-programming/2fa6802f-df5a-4584-b4ec-579415c052b6n%40googlegroups.com.

Paul Morrison

unread,
Sep 29, 2020, 10:06:07 AM9/29/20
to Flow Based Programming
Hi Ged, you did ask if you were missing any...!  So I have to add the JSFBP solution, which uses Marcel Laverdet's node-fibers, as this results in very clear coding for JSFBP components.

Unfortunately, it doesn't look as though node-fibers will be accepted into the JS ecology, and now even Marcel is recommending against its use - see https://github.com/laverdet/node-fibers .  However,  JSFBP components are so clear and understandable, that I'd love to be able preserve this concept in some way! 

Here is the infamous "concat":

'use strict';


module.exports = function concat() {

     var array = this.openInputPortArray('IN');

     var outport = this.openOutputPort('OUT');

     var ip = null;


     for (var i = 0; i < array.length; i++) {

         while (true) {

             ip = array[i].receive();

             if (ip === null) {

                     break;

             }

         outport.send(ip);

         }

     }

};

Ged Byrne

unread,
Sep 29, 2020, 10:27:37 AM9/29/20
to flow-based-...@googlegroups.com
This is one to add to the Coroutines category:"Fibers, sometimes called coroutines, are a powerful tool,"

Tom Robinson's proof of concept is directly inspired by JS FBP.  Lines 61 onwards are direct ports of the JSFBP code: https://gist.github.com/tlrobinson/c8407e0e3f31a0da5400#file-collate-js-L61

image.png

It needs more work.

Regards, 


Ged






--


You received this message because you are subscribed to the Google Groups "Flow Based Programming" group.


To unsubscribe from this group and stop receiving emails from it, send an email to flow-based-progra...@googlegroups.com.


Paul Morrison

unread,
Sep 29, 2020, 10:53:22 AM9/29/20
to Flow Based Programming
Thanks, Ged!   Tom Robinson's use of Bluebird looks like a promising solution to Fernando's requirements!

Cheers,

Paul

Ruud Steltenpool

unread,
Sep 29, 2020, 2:05:31 PM9/29/20
to Flow Based Programming
I heard GoLang can be compiled to WASM to run parallelism in some web browsers ...
If that's true, I guess it can deliver what JS can't.
Good luck Googling :-)

Fernando Rivero

unread,
Sep 29, 2020, 3:48:12 PM9/29/20
to Flow Based Programming
Thanks again Ged and Paul.

Paul: I did see your websocket implementation (https://github.com/jpaulm/javafbp-websockets), it looks nice and good to see how websockets are easy to implement (at least in the front end side).
However, I see the diagram showing 2 components in the client's side (Send and OnMessage):
Screen Shot 2020-09-29 at 4.13.00 PM.png

I do believe in the client's side it is just an "fbp inspired" code, correct?
So, no ports nor buffers.

Ged:
Thank you again for such a detailed instructions, I am very excited about what I am learning with you guys!
From your email I have a doubt:
You wrote: "Each component can have out-ports that can be used to send information packets out to the connected buffer, if there is space available.  If the buffer is full then the component will wait for space to become available."
My question is:
Both ports (IN & OUT) have buffers? Or only the IN port have a buffer?

"Each component can have in-ports that can receive information from the connected buffer, if there is an information packet already in the buffer.  If the buffer is empty then the component will wait for an information packet to become available."
My question is:
Are you referring to the component's input buffer, correct?

This is what I understand by now:
FBP_NETWORK.jpg
Please let me know.

Regarding to the JS approach and based on my basic JS knowledge and also the project requirement, I would prefer to go with #1 (Single activation) and in second place, the #2 (event driven).
I did read your reference information for all the approaches and I believe those 2 are something that I can afford at this moment :)

All the best!

Fernando Rivero / XENIO



Paul Morrison

unread,
Sep 29, 2020, 4:17:49 PM9/29/20
to Flow Based Programming
I admit it, Fernando: in my diagram on the client side, I thought it would be cute to show the client diagrammatically as two FBP components, but, if you look at the code, you will see both of these are functions with a resources/scripts/chatx.html member...  So it is not even FBP-inspired code - just regular old JS!   Sorry if it was confusing!

Re the questions directed at Ged - if I might jump in - ports don't have buffers!  A port (input or output) is a connector between a component instance ("process") and a "connection", each of which does have exactly one buffer.    However, since "real" FBP does not allow one-to-many connections, there is a one-to-one relationship between connections and input ports...

Ged, I would appreciate your feedback: IIUC, only the Bluebird approach will provide "real" FBP, right?  Although I assume with "green" threads, rather than "red" threads...? And maybe Golang, Ruud?

Fernando, try "real" FBP - you will enjoy it!

Paul

Ged Byrne

unread,
Sep 29, 2020, 4:25:45 PM9/29/20
to flow-based-...@googlegroups.com
Hi Fernando, 

The buffer is the line connected the OUT port of cone component to the IN port of another.

The buffer has a fixed capacity.

image.png

Imagine that Component #1 a receives one word at a time and sends each letter one character at a time.

Component 2 sleeps for 1 second and then receives 4 letters before going back to sleep and doing the same again.

#2 Starts sleeping
#1 Receives Hello
#1 Sends H

image.png
#1 Sends E
image.png
#1 Sends L
 
image.png
#1 Wants to send the seccon L but it cannot.  There is no space in the buffer.  So #1 is blocked waiting.

#2 Wakes and receives H
image.png
#1 is no longer blocked because a space has become available so it sends L.
image.png
#2 Receives E
image.png
#1 Sends O.   Since it has no more to send so it returns to its import and blocks as it waits for the next work.

image.png
#2 Receives L
image.png  
#2 Receives L then sleeps for one second.
image.png
#2 Received O then blocks waiting for #1 to send the next letter
image.png
This is how classic FBP behaves.

if you are going with Single Activation then there is no bounded buffer.

#1 is called with 'HELLO'.  It iterates through the word sending each letter, filling the output array with five letters.  Then #1 is done and the process method completes.

#2 is then called with the array containing the 5 letters, which it can iterate over.

Regards, 


Ged


Ruud Steltenpool

unread,
Sep 29, 2020, 4:39:18 PM9/29/20
to Flow Based Programming
Ha, you got me thinking there Paul:
I actually meant building FBP on top of WASM (Web AsSeMbly) directly, but maybe a route 'through' GoLang is also an option.

But after some quick (non-thorough) Googling myself:
- I wonder if  this presentation I saw actually mentioned this WASM-feature that was already implemented, or only proposed :-(
- This might be interesting though:  https://qvault.io/2020/09/23/running-go-in-the-browser-with-wasm-and-web-workers/  

Ged Byrne

unread,
Sep 29, 2020, 4:39:51 PM9/29/20
to flow-based-...@googlegroups.com
Hi Paul,

Yes, only Co-Routines will give you real FBP because this is the only approach that will allow you to suspend the execution and resume at that point later.

The means that you cannot use your code to act at a state machine: https://eli.thegreenplace.net/2009/08/29/co-routines-as-an-alternative-to-state-machines

The other methods always require the code to enter at the same point, which means that you have to add extra complexity  to work out what the state is.  The component is like Guy Pearce's character in the film Memento who forgets everything when he goes to sleep.  He has to write messages to himself to read in the morning: https://en.wikipedia.org/wiki/Memento_(film)

Threads and co-routines return to the point where they left off with every variable in local storage intact.  It can then carry on where it left off.

This is why the code is so much cleaner.

Regards, 


Ged

Ged Byrne

unread,
Sep 29, 2020, 4:44:00 PM9/29/20
to flow-based-...@googlegroups.com

Ged Byrne

unread,
Sep 29, 2020, 5:10:56 PM9/29/20
to flow-based-...@googlegroups.com
Lua has also been compiled to JavaScript.  It much lighter than GoLang. 


On Tue, 29 Sep 2020 at 21:17, Paul Morrison <paul.m...@rogers.com> wrote:

Ged Byrne

unread,
Sep 29, 2020, 5:22:02 PM9/29/20
to flow-based-...@googlegroups.com
Hi Fernando,

A lot of this  discussion around approaches may not be relevant foe your implementation because it’s a no code approach.  When the framework and components are both written in JS it is an issue.

How will your components be written?

For example, you could implement them as graphical state machines, mini spreadsheets or decision tables.  

You could mix many different implementing styles.  

Regards,


Ged

On Tue, 29 Sep 2020 at 20:48, Fernando Rivero <river...@gmail.com> wrote:

Fernando Rivero

unread,
Sep 29, 2020, 5:38:16 PM9/29/20
to Flow Based Programming
Hello,

Paul: thanks for explaining, it is clear now. 
Regarding to your quote " try "real" FBP - you will enjoy it!", yes, I am starting to understand how powerful a real fbp implementation is, but unfortunately i can't do it by now, due to my current JS knowledge and also the project requirements, for example it need to work even offline.
But no worries, at some point I will do a real fbp implementation for sure!

Ged: your buffer information is more than clear, no doubts there, thanks!
In the "event driven" JS implementation, there is no buffer as well?

Also: yes, in my case I will write both, the framework and the components using simple JS, I will do 2 frameworks, one for the front end using JS (it will run in a web browser as explained before) and other framework for the backend side using NODEJS.
No graphical UI in the beginning, just the networks, components, processes and connections.

I didn't get this question:
"How will your components be written?
For example, you could implement them as graphical state machines, mini spreadsheets or decision tables. "

Best,

Fernando Rivero / XENIO


Paul Morrison

unread,
Sep 29, 2020, 6:48:32 PM9/29/20
to flow-based-...@googlegroups.com
Yes, Ged, I was looking for a scripting language that would talk to my C++/Boost FBP implementation, and found Lua - very nice! 


So Lua by itself: green threads; Lua on top of C++FBP: red threads!

Ged Byrne

unread,
Sep 30, 2020, 4:23:54 AM9/30/20
to flow-based-...@googlegroups.com
I've always wondered: does C++FBP launch a separate Lua instance for each component?

--
You received this message because you are subscribed to the Google Groups "Flow Based Programming" group.
To unsubscribe from this group and stop receiving emails from it, send an email to flow-based-progra...@googlegroups.com.

Paul Morrison

unread,
Sep 30, 2020, 10:45:44 AM9/30/20
to flow-based-...@googlegroups.com
That's a good question, Ged!  I had to go into the code, as it's been quite a while since I wrote it... but I think the answer is no! 

The component code for each CppFBP component running Lua is   https://github.com/jpaulm/cppfbp/blob/master/CppFBPComponents/Components/ThLua.cpp  . 

The first thing it does is issue a call to luaL_newstate() , which initializes a new Lua state.  Lua does support multiple threads, but I don't use that.  Since Lua is fully reentrant, we just have to make sure each FBP Lua process has a unique Lua state. 

Hope this makes sense!  My test cases work... but if I have a conceptual error in my implementation, hopefully someone will point this out!

Regards,

Paul 
Reply all
Reply to author
Forward
0 new messages