Flow Based Programming as a Fractal Approach to Software Development

[Ron Lewis]

I think Paul Morrison never claims that Flow Based Programming (FBP) is a "fractal" approach to software development that replaces all other approaches.

By "fractal" I mean a procedure that is iterative, with each iteration refining the software design (or the software). 

The tree in my front yard is a fractal sort of thing. Starting from the trunk, main branches come out, from each main branch, smaller branches come out, and so on iteratively until arriving at the twigs. Then you have the leaves stopping the iteration. 

I think top-down, step-wise refinement is fractal. I consider bottom-up programming to be fractal, even though going from detail to general. In the tree in my front yard, we can also start from the twigs, and merge until we get to the trunk. We still get a tree.

I think object-oriented programming can be fractal because objects are composed of other objects which are in turn composed of other objects.

I think my original Google searches were to find some kind of better fractal approach to developing software. That's how I discovered Paul Morrison and FBP. 

And I think that FBP is not an approach for the lowest-level programming, meaning machine code, assembler, and so forth. And I think that there is no claim that FBP is an approach for developing the lowest level of routines.

As we build up our programs without FBP, at some point the executable becomes very large, complex, and unwieldy. And, if I am understanding, that is where FBP comes to the rescue. FBP is a good glue for gluing separate programs together into a network of programs.

And we do not need to wait until are programs are very large and becoming costly to debug and enhance. We can keep our programs very small and simple using FBP to make them work together as a network.

FBP could be used to network together multiple FBP networks. That is, an FBP network of FBP program-components could be considered an FBP component, which is then a component in a larger FBP network. So, maybe FBP is a little fractal. But FBP is not so fractal that FBP can be the only approach used. I am thinking the lowest levels use some other approach such as object oriented, structured, top-down, bottom-up, or even spaghetti-style (spaghetti working fine for trivial programs). 

I am thinking that my motivation to seek a fractal approach is a fractal approach feels so elegant. A fractal approach is elegant to be able to apply the same simple methodology over and over again to develop whole programs of arbitrary size with arbitrarily complex requirements. But when the program becomes very large, the fractal methodology may not work so well. Using FBP would be a practical recognition that what works easily for small simple programming no longer works for large, complex software systems.

So, am I on the right track? Does FBP make any claim to be a fractal approach that has universal application from the smallest, elemental programs through to the most arbitrarily complex giga-systems?

[Paul Morrison]

I would agree that FBP is a bit fractal - that was in fact one of the things that appealed to me about it!  Wayne Stevens talked in a number of his books about "consistent application view", running from micro to macro.  In Chap. 15 of mine - http://www.jpaulmorrison.com/fbp/method.shtml -  I say ,"Figure 15.6 shows a network which implements what used to be called a “half adder”, using IPs to represent bits and FBP processes to model boolean operations – while this worked fine as a simulation, I do not suggest that we could build a real machine on it (although we might be able to build a slow simulation of a machine)!"  While I do not advocate writing low-level components in FBP - at some point, as Ron says, you have to change to another idiom - hardware does have asynchronous characteristics, and in fact is becoming more like FBP networks.  At the macro end you have business flows, and there is a growing literature about these.  Also, as I understand it, SQL requests are being implemented as data flows (Mike Beckerle talks about this briefly in Chap. 27 of the 2nd edition).  And so on...

So Ron, would you be OK with *almost* universal? :-)

--
http://jpaulmorrison.blogspot.com/

[Ron Lewis]

Yes, I can see almost universal when talking about size of program or system. 

In other discussions, such as "Flows and State", maybe some of the participants were looking as I was for FBP to be a "fractal" type of methodology applying all the way down to the smallest granule of a program.

Seems like a database program (SQL Server, Oracle, MySQL) is managing a shared state of the database. The database receives inserts, updates, and deletes from multiple sources. The database program would be a processing component in a FBP network. The processing component stands guard over the shared state.

Maybe people have different definitions of "shared state". Regardless of the differences, maybe the FBP way of handling "shared state", however defined, is to have a processing component represent the shared state. 

My thought in having this be a separate discussion thread is that in several discussion threads, maybe others besides myself have attempted to make FBP a fractal paradigm applying to all levels of a system/program. 

Instead of being a fractal paradigm, maybe an important strength of FBP is blending effortlessly with other paradigms. Another strength is that FBP blends multiple paradigms in into a coherent system. Thus, a program written with the Functional Programming paradigm issues  IPs to a program written the Object Oriented paradigm or most any other paradigm.

[Ged Byrne]

Ron,

For me there is an important distinction between the concept of "Flow"
and the flow based networks that we define in an FBP program.

I think that Flow is a concept that applies at all levels, from the
microprocessor to complex enterprise architectures. At all level flow
can be identified.

FBP deals with one of those levels of flow, as Paul touches upon in
the OO chapter. I'll quote from the earlier version for ease of
reference:

"One important topic I want to address is the issue of
granularity. All discussions of both OO and FBP eventually come up
against this topic: how "big" should FBP processes and OO classes be?
The lower end of FBP granularity is determined by the fact that IPs
normally have multiple fields and often represent objects in the
outside world. You could chop a business IP up into one IP per field,
but then you would have to pay a lot of overhead to recombine it to
write it on a file, data base, screen or report. The granularity of a
language like C++ is approximately the same as that of FBP: objects
very often correspond to file records. Many Smalltalk objects are at
this level, but Smalltalk also makes much smaller pieces of data
objects, such as amounts of money."
- http://www.jpaulmorrison.com/fbp/oops.shtml

At other levels of granularity other languages become more
appropriate. I believe this refers to time as well as scale.
Database diagrams and UML class diagrams tend to be very static in
nature, reflecting the entire lifespan of the data. FBP networks
reflect change over time - the information packets may change
radically between the beginning and end of a flow network.

I think there is more to this than FBP just being a glue language. I
believe that the concept of flow can be used to start to make sense of
which language should be used when. To look at a problem space and
carve it up into components saying "this component will be implemented
using this language" and "That component will be implemented in that
language."

When building a bridge a designer the properties of the material of
known. For example, it is known that cast iron "is strong under
compression, but not under tension." (http://en.wikipedia.org/wiki/
Cast_iron#Historical_uses). While the the materials may change at
different scales, the physical forces are the same. Gravity forms the
galaxys in their clusters as well as the bridges spanning a river.
This is the fractal nature of the universe.

I think that there are similar forces at play within software. They
are present in the problem space itself, which is why the same ideas
keep reoccurring in so many different places at different levels of
granularity.

It this is true, then Flow shouldn't just be a glue that binds
together languages based on other paradigms. It should be the
organising principle behind all of the languages. What exactly that
means in practical terms is difficult to say.

Regards,


Ged



On Feb 16, 2:14 am, Ron Lewis <rle...@indinfer.com> wrote:
[...]

> In other discussions, such as "Flows and State", maybe some of the
> participants were looking as I was for FBP to be a "fractal" type of
> methodology applying all the way down to the smallest granule of a program.

[...]

[Ron Lewis]

Ged,

I see what you are saying that FBP is more than just a glue. I agree. Seems that FBP is very flexible to be just at the top level "gluing" disparate programs into a network. And it can be used at almost the lowest level, on up.

I've been looking over "Thinking Forth" by Leo Brodie, from the link you provided. Seems that at the lowest level, Forth word-definition is assembly language or machine code. So, if I am understanding, at the lowest level, Forth also incorporates a foreign (non-Forth) language (assembly or machine code). And FBP also necessarily relies on some other language or paradigm for at least its lowest elements. Is that one of the comparisons you were making?

In any case, FBP is indeed a good paradigm to guide design and development, rather than just being a glue.

[John Cowan]

Ron Lewis scripsit:

> I've been looking over "Thinking Forth" by Leo Brodie, from the link you
> provided. Seems that at the lowest level, Forth word-definition is assembly
> language or machine code. So, if I am understanding, at the lowest level,
> Forth also incorporates a foreign (non-Forth) language (assembly or machine
> code). And FBP also necessarily relies on some other language or paradigm
> for at least its lowest elements. Is that one of the comparisons you were
> making?

Forth code can also be seen as an FBP pipeline, where what passes through
the pipeline is the stack. Likewise, pure functional programs can be seen
as a tree-shaped FBP network that passes values from the roots of the tree
up to the top.

--
John Cowan co...@ccil.org http://ccil.org/~cowan
You cannot enter here. Go back to the abyss prepared for you! Go back!
Fall into the nothingness that awaits you and your Master. Go! --Gandalf

[Ged Byrne]

Yes, and the Joy Programming Language is a purely functional variant of Forth:

http://www.complang.tuwien.ac.at/anton/euroforth/ef01/thomas01a.pdf

Regards, 

Ged

[Ged Byrne]

Ron,

In Forth it is necessary to build the language on top of Machine code because of the underlying hardware: the Von Neumann architecture.  

    "The time is now ripe for a new paradigm to replace the von Neumann model as the bridging model between hardware and software. The one we will be describing is similar to the one Valiant proposes (I'll talk about his in more detail in Chapter 27) and in fact seems to be one of a family of related concepts which have appeared over the last few years in the literature. The common concept underlying much of this work is basically that, to solve these problems, we have to relax the tight sequential programming style characteristic of the von Neumann machine, and structure programs as collections of communicating, asynchronous processes."

    - http://www.jpaulmorrison.com/fbp/intro.shtml

It is this structuring of programs of collections of communicating, asynchronous processes that is the guiding principle of flow.

To see how this affects thinking in Forth take a look at Thinking Forth's third chapter, Preliminary Design / Decomposition, which directly contrasts a component-based approach that focuses on the flow of data and a traditional approach concerned primarily with control flow.  Chapter 8 discusses minimising control structure.

GreenArray's chips are not based on the von Neumann architecture.  Instead they have an array of small computers, each with it's own memory and Forth built into the microcode.  They are based on the principles of flow at every level.

Regards

[John Cowan]

Ged Byrne scripsit:

> Yes, and the Joy Programming Language is
> a purely functional variant of Forth:
> http://www.complang.tuwien.ac.at/anton/euroforth/ef01/thomas01a.pdf

Indeed it is, as I should know. I modified Manfred's original C
implementation to use the BDW collector and to add a lot of new words.
I have a half-finished implementation in Scheme I want to get back to
sometime.

I like Joy because unlike Lisp it really does exhibit code-data
equivalence: every list is a function and every function is a list.

--
BALIN FUNDINUL UZBAD KHAZADDUMU co...@ccil.org
BALIN SON OF FUNDIN LORD OF KHAZAD-DUM http://www.ccil.org/~cowan

[Sam Watkins]

There are real graphical FBP systems where the components are:
- arithmetic ops (in an educational system)
- signal processors (labview)
- sound processors (musos like to wire things together!)

The first two are fairly low level, and the 'arithmetic ops' one
persuades me that graphical FBP can 'stand alone', without an underlying
textual language.

I can see three challenges with using FBP at a very low level:
- people may want to type expressions, not draw graphs of them
- we want speed, but it's not easy to write an optimizing compiler
- we might need to make changes to the FBP model, for low level coding

None of these is really a major obstacle. We can make it easy to draw
graphs by typing, and we can automatically convert textual expressions
to neat graphs and back again. We can knucle down and write a compiler
that targets C, javascript, assembly or whatever. FBP really does seem
to be a 'fundamental force of nature' (to misquote Paul's book): we can
discover more of it without getting too complex.

Perhaps the biggest challenge (for me) is to spend less time talking
and thinking, and more time building useful stuff! a challenge I've not
overcome yet, it seems!

I'd like to make some example programs for DrawFBP, to show what can be
done with it. The app I want to do is a web app that helps someone
learn another language, by reading carefully chosen chunks out of free
books in that language. I can break this app into small 'example
programs', but I might need some help getting to grips with DrawFBP /
JavaFBP.

I've been focusing on using FBP at low level, to represent mathematical
and structural relationships. It's stretching the definition of FBP,
but many of the concepts are the same, and the graphs look very similar,
with components, ports, array ports, links (or channels).

Believe it or not I independently thought to call my (dreamware) FBP
system "AMPS", when I was maybe 15 years old! My "AMPS" stands for
"Advanced Modular Programming System" (rather than Processing!).
This might be a funny example of convergent thinking, the scarcity
of good acronyms, or perhaps telepathy!

Anyway, I think FBP can be well applied at the very lowest levels up to
the very highest levels. FBP is essentially graphical, and can
represent structure much more faithfully than textual languages do.

I'm thinking to write a graph editor in JavaScript with RaphaelJS.
It might not be so fast as a Java or C app, but it would be very
accessible to the casual web surfer.

And that is one sweet little graphics library! for example:

http://raphaeljs.com/graffle.html
http://raphaeljs.com/fonts.html

It looks as good as flash, without the flash.

Sam

[Paul Morrison]

The AMPS story is fascinating! When I came up with AMPS in the late
60s, I remember thinking that 3-character acronyms were pretty much
played out, so I went to 4 characters! My guess is telepathy -
probably with a time-warp thrown in!

Re getting an app working, Bob Corrick has built up a fair amount of
operational experience with DrawFBP and JavaFBP, and I am sure would
be happy to share it with you and anyone else who wants to give them a
try as well. Hope this is OK, Bob!

[Graham Chiu]

note http://home.ccil.org/~cowan/joy.tar.gz gives a 404

--
Graham Chiu

[Bob]

Yes, although I am a novice at this! Working from time to time on a solution for Ralf's AppKata...

[John Cowan]

Graham Chiu scripsit:

> note http://home.ccil.org/~cowan/joy.tar.gz gives a 404

That was only an interim location. Joy1 (which is my update
to Manfred's Joy0) is downloadable from the main Joy page at
http://www.latrobe.edu.au/phimvt/joy.html .

One nice thing that Joy1 has is "monadic" I/O; the "read" word expects
to find a file object on the stack and pushes back the file object
followed by the object that's read. By the same token, the "write"
word expects to find a file object and the object to be written, and
pushes back the file object only.

--
John Cowan http://www.ccil.org/~cowan co...@ccil.org
if if = then then then = else else else = if;

[Graham Chiu]

and development ceased in 2005?

--
Graham Chiu

[John Cowan]

Graham Chiu scripsit:

> and development ceased in 2005?

Pretty much. Implementing was a fun challenge, but I'm not much of a
concatenative programmer -- I love me my variable names. If you want
a severely practical concatenative language, try Factor.

--
I don't know half of you half as well John Cowan
as I should like, and I like less than half co...@ccil.org
of you half as well as you deserve. http://www.ccil.org/~cowan
--Bilbo