F B P i s F P

[Paul Morrison]

The Wikipedia article on Functional Programming - https://en.wikipedia.org/wiki/Functional_programming - says:

"Eliminating side effects, i.e. changes in state that do not depend on the function inputs, can make it much easier to understand and predict the behavior of a program, which is one of the key motivations for the development of functional programming."

In FBP interactions between processes are only allowed via Information Packets, which must have unique ownership, so side effects are largely eliminated. 

Individual processes can also be expressed functionally, using Friedman and Wise's "lazy cons"  - as described in http://www.jpaulmorrison.com/fbp/recurs.shtml .  Look for G'(x,y)in this chapter for a purely functional process specification.

[Paul Morrison]

In the referenced Wikipedia article - https://en.wikipedia.org/wiki/Functional_programming - one tip-off is the word "computation" in the second line.   Most of the languages and approaches listed here seem to be related to computation, and assume that this is the main task achieved by programming.  Flow-based programming concentrates on the data, rather than on computation.  FBP uses an "assembly line" image of program design, rather than numeric computation.  So the data being processed can naturally be thought of as "things" - hence, probably, the growing interest in combining IoT and FBP...

Obviously in a data flow environment, data and its transforms can be seen as dual.  Interestingly, in Fig. 1 of the Word Count example for Google Cloud Pipelines - https://cloud.google.com/dataflow/examples/wordcount-example - the data collections (roughly analogous to streams and substreams in FBP) are shown as blocks, while the transforms (analogous to FBP processes) are shown as arrows.

I guess the missing concept that so often doesn't get verbalized is why it is helpful to be able to chop a program into individual processes, and that has to do with maintainability.  A complex single-threaded application is extremely hard to maintain, and people entering the programming field for the first time usually don't realize that they are going to spend 80-90% of their working careers maintaining code - their own or other people's!   If the code has been designed for maintainability from the start, this is much easier!  You cannot add maintainability after the fact!  This is also why I am not enthusiastic about languages that claim to speed up development - especially if they reduce maintainability.

In the early days of my career, many of the programs we worked on were Batch Updates.  Here is the generic logic chart for Updates: http://www.jpaulmorrison.com/fbp/Trad_update.png . This was a paper chart, called a "flow-chart", and it was the only assistance given to programmers on how to build Updates!  There was no reusable code!  Now compare that with http://www.jpaulmorrison.com/fbp/update.jpg - here Collate is a precoded, pretested component, as are the Readers and Writers - only "Process Merged Stream" has to be written for a given application, and it is much simpler!  In fact, its general logic is given in my book - http://www.jpaulmorrison.com/fbp/substrs.shtml - Fig. 9.4.  Straightforward... and highly maintainable!

And, as I said in the previous post, individual processes can be expressed functionally, because they have no side effects!  One (actually zero) or more streams in, zero or more streams out - that's it!

Cheers,

Paul M.

[Paul Morrison]

I also should have stressed that, in FBP, no process knows who its neighbours are.  Communication is via connections, identified in turn via named ports.

This is absolutely key to the improved maintainability of FBP applications, as compared with applications built with other technologies.

[Ged Byrne]

Hi Paul,

What separates FBP from FP is state. In FBP the IP is mutable, its contents can be altered. In FBP the process has memory, maintaining state over time.

When a function takes a parameter the function is a clean slate, with no memory of the past. This makes it easy to parallelise the functions. If I have 100 pieces of data I can pass them to 100 independent functions that can exist on separate machines. No matter how Indistribute them I am guaranteed the same result.

In FBP when a processor READS an IP it is state full. There is state and the READ could come from one of many positions in the code. That position, and the historical events that led to being in that position, are significant.

If I have 100 IPs how I distribute them across multiple processes is important. Changing the distribution changes the result.

This is how the world we encounter behaves. This makes it easier to understand FBP for the non-expert. The virtual processes behave like physical processes, so all the experience we have gained in the physical world can be applied when considering a flow based program.

FP does not behave like the physical world. Before it can be understood it is necessary to forget what we know about the physical reality and learn the abstract principles that apply.

This makes FP harder to understand.

My hypothesis: Given any solvable problem a solution can be developed using both FP and FBP. However, more people will be able to understand and maintain the FBP solution, and they will require less time to do so.

The problem of maintaining FP is well known. For example both Yahoo and Reddit had to rewrite their original LISP implementations.

http://discuss.fogcreek.com/joelonsoftware/default.asp?cmd=show&ixPost=31402

https://www.reddit.com/r/programming/comments/13ccbh/reddit_was_originally_written_in_lisp_when_it_was/

Regards,


Ged

--
You received this message because you are subscribed to the Google Groups "Flow Based Programming" group.
To unsubscribe from this group and stop receiving emails from it, send an email to flow-based-progra...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Paul Morrison]

Hi Ged, great stories about LISP rewrites! 

I agree that IPs and processes both store state, but, in the case of IPs, that is their function!   You take a "customer" IP and run it through various transforms before storing or displaying the result. 

In the case of processes, the programmer has the choice of writing them as "loopers" or "non-loopers": the latter behave exactly like functions, as they do not remember anything from the processing of the previous incoming IP.

I agree that "loopers" require a bit more care:  Collate, e.g. receives IPs from 2 (1 actually) or more input streams, and merges them based on sort key values.  It is hard to imagine writing Collate without internal storage - and, anyway, this is a reusable component, so should have been debugged thoroughly before releasing it into the wild!  Another typical "looper" might be a Writer, where the only local storage is really a reference to the file or database involved.  This seems eminently "debuggable"!

If two Writers reference the same file or database, they still do not share internal data - there is the possibility of a deadlock, but this is exactly the same deadlock as can arise when two conventional programs share a database or file... and is solved in the exact same way!

To unsubscribe from this group and stop receiving emails from it, send an email to flow-based-programming+unsubscri...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Flow Based Programming" group.

To unsubscribe from this group and stop receiving emails from it, send an email to flow-based-programming+unsubscri...@googlegroups.com.

[Ged Byrne]

Hi Paul,

Regarding the idea of two writers sharing a database table FBP has a clear, elegant solution. Create a single process that has exclusive write access to the table. Then have the other processes send their updates IPs to that Process to be written asynchronously.

This isn't necessary for reads because they don't cause deadlock.

This solution is obvious in FBP, but in conventional programming it needs a fancy name: https://martinfowler.com/bliki/CQRS.html

Regards,


Ged

To unsubscribe from this group and stop receiving emails from it, send an email to flow-based-progra...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Flow Based Programming" group.

To unsubscribe from this group and stop receiving emails from it, send an email to flow-based-progra...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Flow Based Programming" group.

To unsubscribe from this group and stop receiving emails from it, send an email to flow-based-progra...@googlegroups.com.

[Paul Morrison]

Hi Ged, you're absolutely right about this solution, and it's one I've used many times.  I was trying to highlight the idea that separate processes have separate states, and even if two processes need update access to the same file or database, this is a problem that has been solved in conventional (non-FBP) programming - which solutions can then of course be used in FBP.

To unsubscribe from this group and stop receiving emails from it, send an email to flow-based-programming+unsub...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Flow Based Programming" group.

To unsubscribe from this group and stop receiving emails from it, send an email to flow-based-programming+unsub...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Flow Based Programming" group.

To unsubscribe from this group and stop receiving emails from it, send an email to flow-based-programming+unsub...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Flow Based Programming" group.

To unsubscribe from this group and stop receiving emails from it, send an email to flow-based-programming+unsub...@googlegroups.com.

[Paul Tarvydas]

On 2017-09-14 05:35 AM, Ged Byrne wrote:
> Hi Paul,
>
> Regarding the idea of two writers sharing a database table FBP has a
> clear, elegant solution. Create a single process that has exclusive
> write access to the table. Then have the other processes send their
> updates IPs to that Process to be written asynchronously.
>
> This isn't necessary for reads because they don't cause deadlock.
>
> This solution is obvious in FBP, but in conventional programming it
> needs a fancy name: https://martinfowler.com/bliki/CQRS.html

Haha, I thought this would be called <xxx>aaS.

Ged, you often have interesting insights, or anyone else, how would
Collate be done in FP?

My feeling is that the promise of FP is substitutability
(pin-compatible) and composability. To my mind, absence of state is
simply an implementation detail, usually espoused by reactionaries who
typically don't understand what the point of FP is.

I believe that when you can compose systems from components and
substitute components at will, then "state" is moot - no one needs to
care what happens inside a component (since any component can be
replaced by a like component, regardless of implementation).

OOP claimed to have such magic powers, but fell down when it was
realized control-flow was not composable in OOP.

pt

[Ged Byrne]

Hi Paul,

I think the common pattern for collation  is Map-Reduce: https://en.m.wikipedia.org/wiki/MapReduce

Google was built on this pattern, until they switched to DataFlow: https://www.quora.com/Why-did-Google-stop-using-MapReduce-and-start-encouraging-Cloud-Dataflow

I think state is essential, and trying to get rid of it is the problem with FP.  The world is stateful.  This is why FP works so well for abstract problems but makes solving real life problems so difficult. 

The big problem with OOP was that it makes it easy to build systems with objects that correlate with reality within a single session, but not when those objects need to persist over time.  

It's great for building UIs where everything happens in a single session. As long as the object model can remain in memory it all is fine. Once the computer is turned off, things get complicated. 

That's why OOP has been so dependant upon RDBMS despite the object-relational impedance mismatch: https://en.m.wikipedia.org/wiki/Object-relational_impedance_mismatch

For me one of FBPs biggest strengths is the ability to 'time stretch' processes. 

Time is an essential property of reality.  Entities in our world except exist over time, and their state changes.  

FBP processes can be executed over the time scales of real world processes.  Weeks, months and years.  Not just the minutes and hours of a single session.  

Regards,

Ged

[Paul Morrison]

On Thu, Sep 14, 2017 at 6:49 PM, Paul Tarvydas <paulta...@gmail.com> wrote:

I believe that when you can compose systems from components and substitute components at will, then "state" is moot - no one needs to care what happens inside a component (since any component can be replaced by a like component, regardless of implementation).

I agree, "inside" is the key word - in FBP, side effects are minimized because every process has its own working storage - even when there are multiple occurrences of the same component in a single network.  The only other place where data is stored is in Information Packets, which have a single, unique "owner" at any one point in time, or are in transit, when no process can affect them.

(In the interests of full disclosure, FBP does support "named globals", but these should only used in very specific situations, and very carefully - and they are completely trackable!) 

Regards,

PaulM

[John Cowan]

On Fri, Sep 15, 2017 at 9:26 AM, Paul Morrison <jpau...@gmail.com> wrote:

I agree, "inside" is the key word - in FBP, side effects are minimized because every process has its own working storage - even when there are multiple occurrences of the same component in a single network.  The only other place where data is stored is in Information Packets, which have a single, unique "owner" at any one point in time, or are in transit, when no process can affect them.

Some components have to be stateful, of course, like "sort all packets" (in a system without an explicit end-of-stream packet, this would be "sort all packets between a left bracket and a right bracket), which inherently must keep all the packets being sorted in private state before it can emit any of them.

 

(In the interests of full disclosure, FBP does support "named globals", but these should only used in very specific situations, and very carefully - and they are completely trackable!)  

Can you say more about this?  It would be simple to have a key-value component that allows you to send lookup and update packets to it.

-- 

John Cowan          http://vrici.lojban.org/~cowan        co...@ccil.org

Knowledge studies others / Wisdom is self-known;

Muscle masters brothers / Self-mastery is bone;

Content need never borrow / Ambition wanders blind;

Vitality cleaves to the marrow / Leaving death behind.    --Tao 33 (Bynner)

[Paul Morrison]

On Fri, Sep 15, 2017 at 11:13 AM, John Cowan <co...@ccil.org> wrote:

On Fri, Sep 15, 2017 at 9:26 AM, Paul Morrison <jpau...@gmail.com> wrote:

I agree, "inside" is the key word - in FBP, side effects are minimized because every process has its own working storage - even when there are multiple occurrences of the same component in a single network.  The only other place where data is stored is in Information Packets, which have a single, unique "owner" at any one point in time, or are in transit, when no process can affect them.

Some components have to be stateful, of course, like "sort all packets" (in a system without an explicit end-of-stream packet, this would be "sort all packets between a left bracket and a right bracket), which inherently must keep all the packets being sorted in private state before it can emit any of them.

Quite true!  However, without brackets, even without explicit end of stream packets (as in my GitHub implementations), Sort just has to receive all its input IPs, sort them, and then output the result.  Currently, I just store the IPs to be sorted in Sort's working storage (the 'execute' method's local variables), but I thought it might be interesting to implement some of the famous old-time Sorts using disk storage, like Fibonacci! 

 

(In the interests of full disclosure, FBP does support "named globals", but these should only used in very specific situations, and very carefully - and they are completely trackable!)  

Can you say more about this?  It would be simple to have a key-value component that allows you to send lookup and update packets to it.

You're right - however, I didn't want to clutter up the diagram with the pathways to and from the Global Manager process.  Although that would sort of restrict the use people might make of this facility!

Currently FBP allows references to any object to be stored in a "global" hashmap (getGlobal and putGlobal), but I am thinking that maybe we should only allow IPs to be stored - this might be more in keeping with FBP philosophy!  What do people think?

Regards,

PaulM

[John Cowan]

On Fri, Sep 15, 2017 at 4:38 PM, Paul Morrison <jpau...@gmail.com> wrote:

Quite true!  However, without brackets, even without explicit end of stream packets (as in my GitHub implementations), Sort just has to receive all its input IPs, sort them, and then output the result.

Yes, true.

 

Currently FBP allows references to any object to be stored in a "global" hashmap (getGlobal and putGlobal), but I am thinking that maybe we should only allow IPs to be stored - this might be more in keeping with FBP philosophy!  What do people think?

Makes sense to me.  However, it won't work if/when you want to distribute FBP networks across multiple OS processes, to say nothing of multiple machines.  You need a distributed key-value store like Redis.

-- 

John Cowan          http://vrici.lojban.org/~cowan        co...@ccil.org

You let them out again, Old Man Willow!

What you be a-thinking of?  You should not be waking!

Eat earth!  Dig deep!  Drink water!  Go to sleep!

Bombadil is talking.

[Paul Morrison]

On Fri, Sep 15, 2017 at 1:48 AM, Ged Byrne <ged....@gmail.com> wrote:

Hi Paul,

I think the common pattern for collation  is Map-Reduce: https://en.m.wikipedia.org/wiki/MapReduce

Google was built on this pattern, until they switched to DataFlow: https://www.quora.com/Why-did-Google-stop-using-MapReduce-and-start-encouraging-Cloud-Dataflow

Assuming Cloud Dataflow follows many of the concepts of FBP (and it looks like it does), it seems much more flexible than MR... 

I think state is essential, and trying to get rid of it is the problem with FP.  The world is stateful.  This is why FP works so well for abstract problems but makes solving real life problems so difficult.

The big problem with OOP was that it makes it easy to build systems with objects that correlate with reality within a single session, but not when those objects need to persist over time.  

It's great for building UIs where everything happens in a single session. As long as the object model can remain in memory it all is fine. Once the computer is turned off, things get complicated. 

Absolutely!

 

That's why OOP has been so dependant upon RDBMS despite the object-relational impedance mismatch: https://en.m.wikipedia.org/wiki/Object-relational_impedance_mismatch

I looked at this article, and I can see it philosophically, but we used Java.sql very successfully on our Electronic Brokerage JavaFBP app...  I guess the key was to access rows mostly as IPs, and any complex fields were serialized as Strings.  As I describe in http://www.jpaulmorrison.com/busdtyps.shtml , all business data was implemented using objects, so for instance when storing Currency objects on the database, they were linearized as e.g. CAD29.95.

IIRC the one significant mismatch was Time vs. Date - do a find on "Date, TimeStamp and Time Zone (TimeTz) classes" in the above web page.  As described there, we built our own Date and Time Stamp classes, using what we thought were appropriate units and encodings.

We did store dates and timestamps using Java.sql, so our Date and TimeStamp classes looked after these conversions (Java.sql calls their thing a Timestamp, with lower case 's').

For me one of FBPs biggest strengths is the ability to 'time stretch' processes. 

And you can "stretch" them over multiple machines also...

Time is an essential property of reality.  Entities in our world except exist over time, and their state changes.  

FBP processes can be executed over the time scales of real world processes.  Weeks, months and years.  Not just the minutes and hours of a single session.  

Agree 100%!  That to me is the one of the biggest problems with most of what we are calling FBP-like or FBP-inspired systems.

Regards,

PaulM

Regards,

Ged

To unsubscribe from this group and stop receiving emails from it, send an email to flow-based-programming+unsub...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Flow Based Programming" group.

To unsubscribe from this group and stop receiving emails from it, send an email to flow-based-programming+unsub...@googlegroups.com.

[Paul Morrison]

Interesting!  I'll look into Redis...

If we switch getGlobal and putGlobal to only support IPs, we should probably pick different names... we don't know if anyone is currently using these functions as presently defined!

Thanks John,

PaulM

--
You received this message because you are subscribed to the Google Groups "Flow Based Programming" group.

To unsubscribe from this group and stop receiving emails from it, send an email to flow-based-programming+unsub...@googlegroups.com.

[ern0]

> Currently FBP allows references to any object to be stored in a "global"
> hashmap (getGlobal and putGlobal), but I am thinking that maybe we should
> only allow IPs to be stored - this might be more in keeping with FBP
> philosophy! What do people think?

Globals are evil. (It's not only my fantasy, I'm working on a code
which has 650 Mbyte of globals.)

1. We should set up components, which store values by key, and other
components are calling them (setValue(k,v) and v=getValue(k)).
- pro: separate storage components can store different parts' data, no collision
- con: needs different graphical representation
- con: it's re-inventing the OOP

2. In my dataflow system I've used a third type of port: property
(other two are consumer and producer). It can act as consumer (in my
implementation, I needed to add setter consumer for each property, but
the result is the same), so other components can set its value by
sending a message.
- pro: it's more FPBish than a global storage
- pro: component's storage variables are visible
- con: if several components share the same value, all of them
contains it, redundant storage
- con: a single variable invokes lot of wiring, makes the graph big
- con: no automatic sync-value-to-all-components, must be drawn every time
--
ern0
dataflow evangelist

[Ged Byrne]

Hi Ern0,

"Evil" is a strong word. Using a single global address space without discipline is foolish, but I don't think anybody is going to hell for it.

Global is clearly an inaccurate word because it only covers one machine. These days we are building systems that truest global in nature. This single machine centric view is something we have to lose.

There does seem to be a need for a globally visible repository for information that needs to be widely shared.

DNS provides a truly global registry. Is that evil? Could the internet work without It?

Regards,


Ged

[ern0]

"Evil" is a term for something, which may cause pain in programmers'
ass. "Pain in the ass" is a term for the mental status, when one hits
something, which is uncomfortable, not acceptable, and he or she wants
to ignore/round/forget it, but can't. We'er using strong words, see
"I'm a programmer - sorry what I said when I was debugging".

Global _variables_ are evil. Just imagine your favorite IDE, and when
you start typing a variable name, the typing aid popup contains _all_
the variable names in an endless list.

There are several techniques avoid global variables: namespaces (its
definitive goal is to set up a hierarchy for variables), objects (as
"side effect"), container objects and so on.

--
ern0
dataflow evangelist

[Ged Byrne]

The problem with declaring things as evil is that it limits our thinking. 

In a paticlar style of programming the use of Global variables is troublesome. 

However, thinking in terms of data flow rather than development activity the Global _scope_ exists and is not only useful but often essential. 

New solutions to the global scope problems keep arising, such as the Singleton pattern, only to be shot down as "Evil". 

https://blogs.msdn.microsoft.com/scottdensmore/2004/05/25/why-singletons-are-evil/

Lack of clear thinking in this area has lead to such abominations as the Windows Registry.   Nobody has developed best practices because obviously 'best evil' is oxymoronic. So when the global scope is unavoidable we end up with something bad. 

I wish we could apply some strong engineering discipline to the trade offs of shared scope instead of simplistic moralising. 

Everything in global scope vs nothing in global scope is a false dichotomy.   Everything in its proper scope is what we should be striving for. 

Regards,

Ged

[Ged Byrne]

A good example of essential global scope ia language.

You define "Evil" as something that causes programmer pain. The word Evil already had a meaning, so why is it being redefined?

Googles old motto was 'Don't be evil.'

Did that simply mean 'don't cause programmers pain?'

Actually, that could explain a lot.

Is the whole of Silicon Valley blind to the universal concept of Evil because it has been overridden in their scope?

[Paul Morrison]

On Sat, Sep 16, 2017 at 5:37 AM, Ged Byrne <ged....@gmail.com> wrote:

 

  Everything in its proper scope is what we should be striving for. 

Amen!  This should be every programmer's mantra!

PaulM

[Paul Morrison]

On Saturday, September 16, 2017 at 3:11:28 AM UTC-4, ern0 wrote:

2. In my dataflow system I've used a third type of port: property
(other two are consumer and producer). It can act as consumer (in my
implementation, I needed to add setter consumer for each property, but
the result is the same), so other components can set its value by
sending a message.
- pro: it's more FPBish than a global storage
- pro: component's storage variables are visible
- con: if several components share the same value, all of them
contains it, redundant storage
- con: a single variable invokes lot of wiring, makes the graph big
- con: no automatic sync-value-to-all-components, must be drawn every time
--
ern0
dataflow evangelist

In my earlier post, where I speculated about just using the "global" facility for IPs, I realized afterwards that this goes against the "unique owner" property of IPs...   And then John pointed me at Redis... but then I realized that most of what it does can be done with simple FBP components - caching, replication, load-balancing, etc.!

Since I have only used putGlobal and getGlobal very occasionally over the years, it doesn't seem to make sense to build a heavy-weight facility to support this across machines. 

So now I'm thinking that the pragmatic (as opposed to beautiful and theoretical) solution is to leave things as they are (getGlobal and putGlobal within a single machine), and anyone who needs a truly inter-system global can build their own using current FBP facilities - as per Ged's suggestion.

This is in line with my rants about not trying to include both inter- and intra-system functions in FBP connections.  So I think I'm being consistent... 

Regards,

PaulM

[Paul Morrison]

I seem to be having trouble explaining why chopping a program up into separate asynchronous processes makes maintenance so much easier. 

I had a sort of "Aha!" moment (late '60s), when I was trying to create a reusable "template" for batch Updates (at that time probably the most common batch program type), and at the same time playing with a simple coroutine mechanism.  I had the coroutines communicate via bounded buffers - that seemed pretty obvious - and had the coroutines reference the bounded buffers via ports - also obvious (to me!).  Initially the ports were just numbered, and that worked fine.  (Later we changed to named ports, some of which had to be array ports, and that was even better!)  So I thought: why not use this coroutine/buffer/port combo to write an Update, and I found that: a) it looked totally different, b) it yielded a precoded, pretested component (Collate), and c) the prototype network could be extended indefinitely by adding more coroutines, reconfiguring the network, etc., while still remaining highly maintainable...  My boss at that time said "That's a nice way to document programs", and I said, "No, you can actually build them this way!"

Since I never was able to develop a non-FBP Update template, while the FBP solution was incredibly easy to work with (extend, test, etc.) - and even performed better than conventional programs - it's no wonder that I've been an FBP enthusiast ever since!  Oh, and FBP networks can be tested and instrumented; prototypes and/or simulations do not have to be thrown away when you grow the prototype to a complete program, etc.

I thought I had posted a picture of the non-FBP and FBP versions of Update, but I can't find them on this Group, so I have posted them above. I know the first one doesn't have any legends, but I've lost the tatty bit of paper that was the only guidance that old-time programmers ever received for writing Updates - and that's the basic Update, with only two input files!  (The green circles were meant to represent line connections; blue ellipses are "start" and "stop")

Cheers,

Paul M

 

[Sam Watkins]

On Sat, Sep 23, 2017 at 07:48:49AM -0700, Paul Morrison wrote:
> I seem to be having trouble explaining why chopping a program up into
> separate asynchronous processes makes maintenance so much easier.

I think it's like a factory, where each person has a clear and simple task to
do then the product moves to the next station in the production line, versus
trying to build cars by yourself in a relatively disorganized manner, running
around trying to do every different thing at the right time.

It's clear enough that separating programs into modular sub-programs is a good
approach and makes things much simpler, compared to a conventional
single-threaded program where the execution point bounces chaotically all over
the place.

[Tom Young]

On Sat, Sep 23, 2017 at 10:48 AM, Paul Morrison <paul.m...@rogers.com> wrote:

I seem to be having trouble explaining why chopping a program up into separate asynchronous processes makes maintenance so much easier. 

​If you can convince others that less overall complexity makes maintenance easier​ and also agree on some definition of program complexity,  your troubles might be over.    

One might, for instance, agree the more tests are required, the greater the complexity and therefore the more difficult the program will be to understand and maintain.     For FBP programs, I suggest that the number of tests is equal to the sum of the number of tests (this will likely be greater than the cyclomatic complexity) required for the components employed plus some low number, like 1, to test the connections between them. 

It should be possible to prove that this Flow Based complexity number will always be less than or equal to the equivalent monolithic program. 

Cheers, 

    --- twy 

 

 

-- 

[paul tarvydas]

I've got John R. Allen's book "Anatomy of Lisp" in my hands. It contains, IIRC, the code for a completely functional (FP) compiler (that I implemented on a 22K[sic] Z80).

Thinking aobut this, I am reminded that a central issue in FP dogma is the rule "thou must not change anything once it has been created". Miranda and Denotational Semantics had this problem - if they wanted to "change" something, they would have to copy it and effect the change during the copying operation (e.g. to change one element in an array, copy the whole array, except with a new element at the "changed" position).

Rich Hickey, in one of his earlier talks, describes the very interesting data structure used in Clojure to effect such dogma.

FBP follows this rule, but uses language that is offensive to FP dogmatists to describe implementation details, . "State" (in IPs) and "change" (effected by Components). Bzzzt, clink, ears and eyes shut tight. Political correctness, these words must not be used, ever.

FBP manages this FP rule by using the "single owner" paradigm. It is *impossible* for any component to detect whether an IP has been changed or whether is has been copied. When a component "changes" an IP and passes it along, it loses reference to the IP and can no longer "see" it (nor change it further). That's the central point of FP.

It seems that FBP's rules are simply optimization details that make FP practical.

Maybe we should drag some Rust advocates into this group (possibly after conferring with the Fractalide people).

pt

[Paul Morrison]

Thanks, Sam, that's a good analogy!  So... say you normally tackle jobs using the single-threaded model, how do you transition to the other approach?!

I think I have said this before: catch kids while they are young!

Paul M.

--
You received this message because you are subscribed to the Google Groups "Flow Based Programming" group.

To unsubscribe from this group and stop receiving emails from it, send an email to flow-based-programming+unsub...@googlegroups.com.

[Paul Morrison]

Hi Tom,

Yes, Chapter 2 of my book:

"In fact, there is a complexity measure used quite widely in the industry called McCabe’s Cyclomatic complexity measure, which is based very directly on the number of binary decisions in a program. In comparison, in our work, we discovered that the amount of logic in conventional programming is in fact reducible, because much of the logic in conventional programming has to do with the synchronization of data, rather than with business logic. Since FBP eliminates the need for a lot of this synchronization logic, this means that FBP actually does reduce the amount of logic in programs."

Now, can we prove the suggestion in your last paragraph?!

Cheers,

Paul M.

--

You received this message because you are subscribed to the Google Groups "Flow Based Programming" group.

To unsubscribe from this group and stop receiving emails from it, send an email to flow-based-programming+unsub...@googlegroups.com.

[Paul Morrison]

Thanks, PaulT,

"It seems that FBP's rules are simply optimization details that make FP practical." 

Are they "optimization details", or have we added a (non-obvious) complementary concept?

Cheers,

Paul

pt

--
You received this message because you are subscribed to the Google Groups "Flow Based Programming" group.

To unsubscribe from this group and stop receiving emails from it, send an email to flow-based-programming+unsub...@googlegroups.com.

[Paul Morrison]

Here are a couple of Update algorithms:  http://cs.uni.edu/~east/teaching/cobol/topics/seq_update/algorithms.html

Have fun!

Paul M

[Paul Morrison]

Re testing, prototyping, scaffolding, cleaving, etc., I thought I would post this slide from my Powerpoint presentation - http://www.jpaulmorrison.com/fbp/FBPnew.ppt .  It is probably a bit cryptic, but remember that I talk for several minutes when giving my presentation - on this slide alone.  Remember: "Picture worth 1K words", as some guru said.

Nate Edwards describes "configurable modularity" as the key attribute of any system that may be called "engineered"... which presumably excludes most conventional programming!

PaulT or others, is this orthogonal to FP?

Cheers,

Paul M

 

[Paul Tarvydas]

On 2017-09-24 11:33 AM, Paul Morrison wrote:
> Thanks, PaulT,
>
> "It seems that FBP's rules are simply optimization details that make
> FP practical."
>
> Are they "optimization details", or have we added a (non-obvious)
> complementary concept?

I was thinking about the subject line (FBP is FP) and trying to state
why most others don't see the similarities.

Is FBP a complementary concept? I'm not sure, I would be very happy to
read others' ideas.

There is one thing that FBP has that FP doesn't - concurrency. Order of
operations cannot matter in FBP.

I see concurrency as a design paradigm / rule - thinking concurrently
during design produces better encapsulation.

Is this a complementary concept or the culmination of FP?

pt

[John Cowan]

On Mon, Sep 25, 2017 at 3:02 PM, Paul Tarvydas <paulta...@gmail.com> wrote:

There is one thing that FBP has that FP doesn't - concurrency. Order of operations cannot matter in FBP.

It can't matter in FP either.  There is no explicit concurrency, but the whole thing is implicitly concurrent.

-- 

John Cowan          http://vrici.lojban.org/~cowan        co...@ccil.org

On the Semantic Web, it's too hard to prove you're not a dog.

        --Bill de hOra

[Paul Tarvydas]

On 2017-09-25 03:09 PM, John Cowan wrote:

On Mon, Sep 25, 2017 at 3:02 PM, Paul Tarvydas <paulta...@gmail.com> wrote:

There is one thing that FBP has that FP doesn't - concurrency. Order of operations cannot matter in FBP.

It can't matter in FP either.  There is no explicit concurrency, but the whole thing is implicitly concurrent.

So, does concurrency inoculate sequential code with FP-ness?

pt

[John Cowan]

So, does concurrency inoculate sequential code with FP-ness?

If only it did.  No, the potential concurrency is a result of the statelessness.  Without state, concurrency is a matter of implementation choices, because there is nothing to share or prevent sharing.  If you say "Compute the sum of the results of calling A and B", then it does not matter whether you call A first or B first or both simultaneously, provided there is no shared state between them.  This can be written as the functional expression a + b or an FBP addition box.

-- 

[Paul Morrison]

As in...

is the same as

              c = G(F(a),F(b))

?

Cheers,

PaulM

[John Cowan]

On Mon, Sep 25, 2017 at 3:37 PM, Paul Morrison <jpau...@gmail.com> wrote:

As in...

is the same as

              c = G(F(a),F(b))

?

Exactly.  The trick is to be able to encode multiple-output boxes in this style, something like this:

let (a, b) = F(c, d).

Note that variables in this notation are not only immutable but linear: if you are going to use a variable more than once, a duplication box with one input and n identical outputs has to be introduced.  See <http://home.pipeline.com/~hbaker1/LinearLisp.html> for Henry Baker's paper on linear programming (not to be confused with the optimization strategy with the same name).

-- 

John Cowan          http://vrici.lojban.org/~cowan        co...@ccil.org

At the end of the Metatarsal Age, the dinosaurs abruptly vanished.

The theory that a single catastrophic event may have been responsible

has been strengthened by the recent discovery of a worldwide layer of

whipped cream marking the Creosote-Tutelary boundary.  --Science Made Stupid

[Paul Morrison]

On Monday, September 25, 2017 at 4:23:09 PM UTC-4, John Cowan wrote:

... The trick is to be able to encode multiple-output boxes in this style, something like this:

let (a, b) = F(c, d).

If  F is a component (not a subnet), what might be an appropriate functional notation to define it?

I didn't come up with a good solution in my chapter on functional notation using "lazy cons" - http://www.jpaulmorrison.com/fbp/recurs.shtml ...

Thanks,

PaulM

[Sam Watkins]

FBP is the industrial revolution waiting to happen to software development.

> > email to flow-based-progra...@googlegroups.com.

[Sam Watkins]

On Sat, Sep 23, 2017 at 06:24:27PM -0400, paul tarvydas wrote:
> Thinking aobut this, I am reminded that a central issue in FP dogma is the rule "thou must not change anything once it has been created". Miranda and Denotational Semantics had this problem - if they wanted to "change" something, they would have to copy it and effect the change during the copying operation (e.g. to change one element in an array, copy the whole array, except with a new element at the "changed" position).

Computers are based on registers and ram with mutable state, end of story. If you want FP, and there are many advantages to it, it's easier and more efficient to allow mutable state, and take a copy if and only if you still need the unmodified object for something else, rather than requiring a copy in all cases.

Imagine poking pixels to a pixmap one by one, it is simply ludicrous to try to do this in functional programming where the pixmap cannot be modified unless you copy it. Yeah, they try to make it more efficient with trees and other data structures, which could be good in some cases, but actually for the simple case we need to allow the thing to be modified, and take a copy only if the unmodified pixmap is still needed for something. Count references, it's not hard. So you can have semantically functional programming with efficient mutable state also.

Also in FBP you can consider each channel to be a growing sequence of immutable IPs, where the future part of the sequence sort of already exists but is not known yet. I believe this is an acceptable immutable object for functional or relational programming. You don't modify an IP, but add knowledge of an item in the channel. So FBP is equivalent to FBP with immutable state, and we can reason about FBP programs more easily.

Sam

[Paul Morrison]

Hi Sam,

"So FBP is equivalent to FBP with immutable state, and we can reason about FBP programs more easily."

I'm guessing that one of the FBPs should be FP...?!

I also suspect that the FBP unique ownership rule makes mutability more palatable...?

Bert, I would think you will want to weigh in...

Regards,

Paul M

Sam

--
You received this message because you are subscribed to the Google Groups "Flow Based Programming" group.

To unsubscribe from this group and stop receiving emails from it, send an email to flow-based-programming+unsub...@googlegroups.com.

[Humberto Madeira]

Hi Paul and Sam

On Wednesday, 27 September 2017 13:00:01 UTC-4, Paul Morrison wrote:

Hi Sam,

"So FBP is equivalent to FBP with immutable state, and we can reason about FBP programs more easily."

I'm guessing that one of the FBPs should be FP...?!

I'm going to suggest that the intent was for the first FBP to be FP.  It makes more sense (to me) than the other two possibilities.

 

I also suspect that the FBP unique ownership rule makes mutability more palatable...?

Bert, I would think you will want to weigh in...

Ok.  I'll bite.  :-)

In Chicle (my own variant of FBP) an IP transiting between components must be immutable.

There are some edge issues for why that is (that Paul and I have discussed previously), 

but in the general case, going from using mutable IPs with ownership transfer to using immutable IPs 

doesn't really force any material change to the behaviour of an FBP network.

On the down side, there could potentially be some (IMO minor) performance loss (due to data copying).

On the up side, there are some definite benefits, which I definitely do use ...

... but I'm going to play nice, and avoid dragging the discussion into that direction :-)

Based on my understanding, Classic FBP was designed to work with any kind of object as an IP, and over various languages, including C++.

A unique ownership rule for IPs is necessary to make FBP work and if an IP so happens to be immutable, so much the better.  

But enforcing immutability is not really an option under its design conditions.

Chicle OTOH, is not intended to use arbitrary objects as IPs.  

It is designed to work with immutable (and serializable) IPs, and strictly enforces that.

(this is actually a more extreme difference wrt Classic FBP than it sounds on the surface)

One of the several side effects of this choice, is that a Chicle implementation is limited to languages that actually support immutability.

For example, there will never be a C++ implementation of Chicle.

But an FBP variant with immutable IPs is still not really FP.

Inside an FBP component, unlike inside an FP function, there is no need for immutability or statelessness.

FBP components can wrap collections of state internally (like collections) or externally (like data stores)

They can even act like a finite state machine.

For an FP function to do something equivalent, it needs to use something like a monad.

So you could potentially say that FBP components are roughly analogous to FP functions with monads.

YMMV, but personally I prefer to wrap my head around the concept of FBP components rather than the concept of FP monads.

Best Regards,

--Bert

[Ged Byrne]

Hi Sam,

The big problem I have with functional programming is that is lies to the programmer.  Consider your Pixmap example.  As you say, implementations for Clojure will have a more efficient alogirthm underneath the covers.  As a programmer I think I am creating a new pixmap every time but in reality a single pixmap is being modified, unless there is some reason to make a copy.  There is no direct relationship between what my code says and what the machine does.

This means that I, as a programmer, do not know what is happening with the machine.  The machine has been placed within a black box.  My instructions are taken as a declaration of intent to be interpreted and not a direct instruction to be obeyed.

For the mathematician dealing with proofs and logic, this is a wonderful thing.  For the engineering who is primarily concerned with the engine this obstruction is unwanted.  

This obfuscation is fundamental to LISP.  The programming statement is based on a fundamental lie - the machine is physically incapable of dealing with the levels of recursion required.  If anything non-trivial is to be implemented then tail call recursion is essential if a stack overflow is to be avoided.  Right at the start the language designers encountered the fundamental problem with their language: it was based on nesting synchronous calls instead of streaming parallel asynchronous sequential processes.  Did they stop and rethink their hypothesis, to find something simpler and more elegant?  No, instead they made their interpreter deceitful, dropping the left over stack entries.

This was software's fundamental error.  The belief that any problem can be solved by adding another level of indirection.  Don't go back and solve the real problem; just add another layer.  With each new level the distance between the software engineer and the engine itself is widened.

Martin Thompson has popularized a great phrase: Mechanical Sympathy.  It was originally coined by a British racing driver called Jackie Stuart:

"The term Mechanical Sympathy was coined by racing driver Jackie Stewart and applied to software by Martin Thompson. Jackie Stewart said, “You don’t have to be an engineer to be be a racing driver, but you do have to have Mechanical Sympathy.” He meant that understanding how a car works makes you a better driver. This is just as true for writing code. You don’t need to be a hardware engineer, but you do need to understand how the hardware works and take that into consideration when you design software."  https://dzone.com/articles/mechanical-sympathy 

This is the wonderful thing about FBP.  When I write an FBP processI have complete freedom.  I could write the process in machine code if I wanted.  As long as I follow a few simple principles about how I communicate with the outside world I know that the component can be used to construct much bigger, more complex systems.  The critical thing is that I do not communicate through a stack.  Each process has it's own internal stack that is not shared.  All inter-process communication is through asynchronous queues.  So simple, but it allows the developer low level freedom and the opportunity to develop mechanical sympathy.  At the same time these units can be visually compbied to construct complex systems that can be comprehended by a non-technical audience.   This is all achieved with a simple, efficient engine: https://github.com/jpaulm/javafbp/tree/master/src/main/java/com/jpaulmorrison/fbp/core/engine 

Genius.

Regards, 

Ged

On Wed, 27 Sep 2017 at 17:43 Sam Watkins <s...@nipl.net> wrote:

[...]

Imagine poking pixels to a pixmap one by one, it is simply ludicrous to try to do this in functional programming where the pixmap cannot be modified unless you copy it.  Yeah, they try to make it more efficient with trees and other data structures, which could be good in some cases, but actually for the simple case we need to allow the thing to be modified, and take a copy only if the unmodified pixmap is still needed for something.  Count references, it's not hard.  So you can have semantically functional programming with efficient mutable state also.

[...]

[Paul Morrison]

Thanks, guys!  Great insights!  I really think we are getting to the core of what FBP is all about!  FBP must be good as so many bright people have espoused it!

Ged, I like the term "mechanical sympathy" - I think it is like a phrase I would use in talking about FBP:  the quality of a "tool that fits the hand"...  Yes?  I Googled that, and got a bunch of references to "ergonomic" - that feels like a subset of our (as yet unnamed) quality. 

Then there is "Zen and the Art of Motorcycle Maintenance", by Robert M. Pirsig, which is all about something undefinable that he calls "Quality", similar to what the Ancient Greeks called "Ἀρετή" (areté, virtue, excellence), that he seems to feel underlies reality.  "He makes a case that originally the Greeks did not distinguish between "Quality" and "Truth" – they were one and the same, areté – and that the divorce was, in fact, artificial (though needed at the time) and is now a source of much frustration and unhappiness in the world, particularly overall dissatisfaction with modern life." (Wikipedia).  Maybe this dichotomy is at the root of the FBP vs. conventional programming split.  🤔  Actually it may be a cause of our present problems globally - there are so many interwingled issues, all coming to head at the same time, that the old, mechanistic approach isn't working any more.

Bert, surely serializability has to be based on some convention...?   In FBP I only require that two connected processes have to agree on what they are sending and receiving.  I agree that serializability allows you to ship an IP from one machine to another more easily, but surely the languages at both ends have to be in agreement.  Suppose we have an FBP implementation where all IPs have to have Hashmaps as their contents (or be extended Hashmaps), then the target process also has to be one that understands HashMaps, even if the IPs go through various transforms on the way...

I suppose I could define a linear format which would be easier to ship around, but then there will be limits on what can be resented this way.  For instance, what if you want to work with 4-dimensional arrays?  (Don't laugh, I once worked with a guy who turned everything into 4-dimensional arrays!)   You will still have to have specialized components that encode and decode these things...

And you're right - we really, really don't want to add Monads into FBP - ugghh!

And Sam,  I love this:

     FBP is the industrial revolution waiting to happen to software development.

Best regards,

Paul M

--

You received this message because you are subscribed to the Google Groups "Flow Based Programming" group.

To unsubscribe from this group and stop receiving emails from it, send an email to flow-based-programming+unsubscri...@googlegroups.com.

[Ged Byrne]

Hi Paul,

I’ve thought of a more succinct way to sum up my argument.

The foundations of Functional Programming are turtles, all the way down.

The foundations of Flow Based Programming are simple design and electrical engineering.

Regards,


Ged

To unsubscribe from this group and stop receiving emails from it, send an email to flow-based-progra...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "Flow Based Programming" group.

To unsubscribe from this group and stop receiving emails from it, send an email to flow-based-progra...@googlegroups.com.

[Sam Watkins]

On Wed, Sep 27, 2017 at 09:48:50PM -0700, Humberto Madeira wrote:
> I'm going to suggest that the intent was for the first FBP to be FP. It
> makes more sense (to me) than the other two possibilities.

No, it was meant to be:

So FBP is equivalent to FP with immutable state,

and we can reason about FBP programs more easily.

It's difficult to understand what I was trying to express, I guess.

The universe appears to be "mutable".

You might also imagine that we are moving through an immutable 4d universe that
already exists, or a continually branching "tree" of timelines in an universe
of all possibilities, which already exists (that's the "spirit" of the universe).

Something like that for the FBP channels too, only simpler and less awesome!

I think I just made it harder to understand, not easier... whatever!


Sam

[ern0]

> There is no direct relationship between what my code says and what the
> machine does.
>
> This means that I, as a programmer, do not know what is happening with the
> machine. The machine has been placed within a black box. My instructions
> are taken as a declaration of intent to be interpreted and not a direct
> instruction to be obeyed.

Even modern C compilers work that way.

I'v been playing around with this beautiful toy: you enter a C/C++
program in the left side, and the web page passes it to compiler, then
displays the assembly output in the right side. There're several
compilers to select from, for several platforms.

https://godbolt.org/# - caution: addictive, probably you will waste
lotsa time with it!

So, I was tried it, I've written a nested loop, with some
multiplication inside (not actual code, just similar):

int main() {
int r
for (int i = 0; i < 48; i++) {
r = r * i
for (j = 4; j < 44; j++) {
r = r - (j * 2);
}
}
return r;
}

compiled it in optimized mode (option: -O3), then the result was:

mov ax,19H
ret

WUT? Oh, yeah, as I used constants in my loops, the compiler
calculated it, so the result was really 19H.

> For the mathematician dealing with proofs and logic, this is a wonderful
> thing. For the engineering who is primarily concerned with the engine this
> obstruction is unwanted.

I like that optimization. I can write dumb/natural/verbose formula,
which the compiler transforms to an optimal equivalent. I like to be
dumb.

> This was software's fundamental error. The belief that any problem can be
> solved by adding another level of indirection. Don't go back and solve the
> real problem; just add another layer. With each new level the distance
> between the software engineer and the engine itself is widened.

Anyway, I know that feeling. Sometimes we re-implement some simple
thing on a layer, because it is too far from a lower layer, where the
simple thing already exists. For example, if I have a periodic task, I
can not set up a direct interrupt for it, but I should set up a
construction with sleeps and time fetches, which, in the deep, are
driven by a timer interrupt.

[Ged Byrne]

Hi den,

Yes, the compiler optimised. When you saw the optimisation you were able to say: “Oh yeah” because the relationship between what your code and the output is still comprehensible.

Regards,


Ged

[ern0]

But the relationship is so loose that if I want to debug, I have to
re-compile it without optimization, because the optimized code is not
related to the source's construction.

Well, I know very well this problem, when I was younger (and I needed
the money), I was working with Node.js. JavaScript is typically a
language, where you're so far from the ground, and you're living in an
artificial world. In case of C++ compiler, we understand that my
double loop is equivalent with "42". But in JS, we see a simple
program, and I know, that it's so damn far from the real life, that it
hurts. I don't know how it works under the hood, but if I knew, I
gonna cry.

Thank you for listening.

--
ern0
dataflow evangelist

[Paul Morrison]

Hi Ged,

I have to ask: turtles as in the chocolates (Turtles) or the animal?

I guess they are both soft on the inside and crunchy on the outside - if that is the metaphor you had in mind...?!

Cheers,

Paul

[Paul Morrison]

Hi Ern0,

That's cute - I actually love it when I feel the computer has outsmarted me!

Cheers,

Paul M

[John Cowan]

On Sun, Oct 1, 2017 at 10:12 PM, Paul Morrison <paul.m...@rogers.com> wrote:

Hi Ged,

I have to ask: turtles as in the chocolates (Turtles) or the animal?

I guess they are both soft on the inside and crunchy on the outside - if that is the metaphor you had in mind...?!

No, it's an old joke:

A well-known scientist (some say it was Bertrand Russell) once gave a public lecture on astronomy. He described how the earth orbits around the sun and how the sun, in turn, orbits around the center of a vast collection of stars called our galaxy. At the end of the lecture, a little old lady at the back of the room got up and said: "What you have told us is rubbish. The world is really a flat plate supported on the back of a giant tortoise." The scientist gave a superior smile before replying, "What is the tortoise standing on?" "You're very clever, young man, very clever," said the old lady. "But it's turtles all the way down!"

-- 

John Cowan          http://vrici.lojban.org/~cowan        co...@ccil.org

I am expressing my opinion.  When my honorable and gallant friend is

called, he will express his opinion.  This is the process which we

call Debate.                   --Winston Churchill 

[Paul Morrison]

Thanks, John, I should have looked it up!

Paul M

--
You received this message because you are subscribed to the Google Groups "Flow Based Programming" group.

To unsubscribe from this group and stop receiving emails from it, send an email to flow-based-programming+unsub...@googlegroups.com.

[Samuel Lampa]

On Thursday, September 28, 2017 at 6:48:50 AM UTC+2, Humberto Madeira wrote:

For an FP function to do something equivalent, it needs to use something like a monad.

So you could potentially say that FBP components are roughly analogous to FP functions with monads.

YMMV, but personally I prefer to wrap my head around the concept of FBP components rather than the concept of FP monads.

Thanks for a very interesting comment!

Based on my (quite shallow) understanding of monads, I was always thinking of them as more as configurations functions, and as such quite analogous to FBP sub-networks? ... although perhaps parametrized sub-networks (where there are only place-holders for the processes, and the concrete ones to use would have to be specified at initialization time of the sub-network).

I'm basing this understanding of light reading of the wikipedia article:

Monads have also been explained with a physical metaphor as assembly lines, where a conveyor belt transports data between functional units that transform it one step at a time

and:

Purely functional programs can use monads to structure procedures that include sequenced operations like those found in structured programming 

... but very interested to know if I'm way off here!

Cheers

// Samuel 

[Paul Morrison]

Hej Samuel,

That does sound pretty innocuous - maybe even sensible!

It was the following kind of remark that scared the h**k out of me (from https://en.wikipedia.org/w/index.php?title=Monad_(functional_programming)&oldid=802629542#The_I.2FO_monad ):

"We can think of a value of type IO as an action that takes as its argument the current state of the world, and will return a new world where the state has been changed according to the function's return value." - sounds like a global run riot!

Cheers,

Paul M

[Ged Byrne]

Hi Paul,

The whole system (world) is viewed as a single immutable global.

Conceptually the input is everything and the output is the modified creation of everything.

If our system is a single graph with two processor and three IPs like this:

(IP1)-(IP2)->[OP1]->(IP3)->[OP2]

Then that graph is out input.

If OP1 is implemented as a monad then the output looks like this after OP1 has executed.

(IP1)->[OP1]-> (IP2’)-(IP3)->[OP2]

The difference between input and output is:
1. The input buffer to OP1 no longer contains IP2. IP2 has been destroyed.
2. The input buffer to OP2 now contains IP2’. Note that this is not IP2 but a new IP that was created based on IP2 when it was processed by OP1.

If everything is immutable then the only way to change anything is to destroy the whole system and build a new one.

So we can see that the monad is basically a function that takes one system and input and output a new system. In Haskell it's called a type, but it’s the same thing. In Haskell everything is a type.

In FBP our whole system is a graph.  The graph consists of nodes (the operating processors) and edges (the connections between them).   IPs are property of and edge.  Terms and conditions apply, such as an IP can only be the property of a single edge at any time. 

So our initial state looks like this:

{

Edges: [

  {id: 1, start: null, end: "OP1", ip=[IP1, IP2]},

  {id: 2, start: "OP1", end: "OP2", ip=[IP3]},

  {id: 3, start: "OP2", end: null, ip=null} ],

IPs: [ ... ]

}

One FBP monad would take care of executing an OP, taking the current graph state as input and the updated graph state as an output.  So executing the Monad associated with OP1 will give us:

{

Edges: [

  {id: 1, start: null, end: "OP1", ip=[IP1]},

  {id: 2, start: "OP1", end: "OP2", ip=[IP2', IP3]},

  {id: 3, start: "OP2", end: null, ip=null} ],

IPs: [ ... ]

}

But we don't need to take the whole graph as input.  We only need the affected sub-graph.  In the above example only Edges 1 and 2 are affected, so we could treat them as a subgraph and simplify our Monad.  So we can define a Monad that implements an operating processor and treats all connected buffers for input and output as a single, immutable sub-graph.  Note that Monads are not operating processors.  They are a mechanism we can use for implementing them.  

Lets consider a concrete example, the Maybe Mondad described here: https://wiki.haskell.org/Maybe

It allows the programmer to specify something may not be there.  

It supports code like this: 

 f::Int -> Maybe Int
 f 0 = Nothing
 f x = Just x

The monad removes the need for null checking.  In Java we have to protect ourselves from NullPointerExceptions like this

if (x != null) {

  return x.doSomething();

} else {

  null;

}

In Haskell like langauges x is defined using Maybe, so x.doSomething() will return Nothing if x is Nothing.  Otherwise x would be defined using Just, which does not allow Nothing as a value.

In FBP we would have a component with two output ports: SOMETHING_OUT and NOTHING_OUT.  On entry the IPs are tested for something.  If it passes it is processed through the sub-graph normally and eventually reach SOMETHING_OUT.  Otherwise it is send directly to the NOTHING_OUT.  

Anywhere we fear nothing slipping into the system we use the Maybe component to filter it out.  The components that follow can then safely assume that they will always get something.

The benefit of the Haskell approach is that code can be proven to be safe.  Out FBP code still has the risk of a new Nothing that is created and sent by a component that comes after our Maybe.  We cannot prove that it will never happen.  

Personally I prefer the FBP approach because it gives us two clear paths: something and nothing.

In Haskell things are less clear.  The idea is that everything that takes a Maybe as an input must also be a Maybe.  This means that should Nothing come along, it is propagated through the other Maybes.  That's all a bit obscure for me.  I'd rather see a line that shows Nothing taking an alternative route.

Of course, in Haskell it is clear that my code is 'correct' because of the guarantee provided.  It is never truly clear that my FBP is entirely bug free.

However, I may find that the code is 'correct' but that the outcome is wrong because of a flaw in the requirements.

Then I must go to the stakeholders who provide the requirements and say "your requirements are incorrect, you must correct them."

When they then ask "what is wrong"  am I in a position to tell them?

With FBP I can work through the visual graph and they can see what is failing.  What they see they understand, and with that shared understanding a better solution can be found that solves the immediate problem.

Regards,

Ged

--

You received this message because you are subscribed to the Google Groups "Flow Based Programming" group.

To unsubscribe from this group and stop receiving emails from it, send an email to flow-based-progra...@googlegroups.com.

[Paul Morrison]

Hi Ged,

I have started to do some reading on Haskell, and some light is starting to dawn...  I have been familiar with the idea that IPs could be immutable, but to me it has always seemed more trouble than it was worth (pace Bert).  Also, I like the idea that I can ride around on an IP and see the transformations as they are applied to it (although I never implemented such a tool).

I didn't realize that "lazy cons" was an integral part of Haskell.   I ran into it via Friedman and Wise's 1976 paper, and I use it in Chap. 25 of the 1st edition - http://www.jpaulmorrison.com/fbp/recurs.shtml ...  This is what lets me use FP for "infinite" streams.  Then you just need to specify the connections somehow - no change from "classical" FBP.

I sort of grasped the part of your post where you discuss immutable IPs and monads, but then I saw "IPs are property of an edge."  I think I can see that from the point of immutable IPs, but to me IPs are "things" and I see them as "owned" by the processes that they pass through - I agree that, in my implementations, connections (edges) contain lists of IPs, so the statement is correct in that context, but it seems to me that you lose a certain degree of control, as you could theoretically have IPn, IPn', IPn'', IPn''', etc. all coexisting on different edges unless you add additional safeguards to prevent this from happening.   Or am I missing something...?

I totally agree with your last sentence, so I am not convinced that monads will make FBP programs easier to reason about - in fact rather the reverse!

Best regards,

Paul M

To unsubscribe from this group and stop receiving emails from it, send an email to flow-based-programming+unsub...@googlegroups.com.

[Paul Morrison]

In case people have not found the FP logic processing the Collate output, it can be seen on p. 246 of the 2nd ed. - Fig. 24.13.  Unfortunately, ed. 1 - http://www.jpaulmorrison.com/fbp/recurs.shtml - doesn't have Figure numbers (and it's Chap. 25).  It's about 3/4 of the way down in that web page.

It's also a bit unfortunate that Haskell seems to have chosen ":" to represent "lazy cons", whereas I used it to denote a field within an IP...   Sorry!

Exercise for the reader: does this notation require an IP to be immutable?

Regards,

Paul M