Different?

[Niclas Hedhman]

Paul,

At a conceptual level, in what way does FBP differ from Enterprise Architecture Patterns? https://camel.apache.org/components/latest/eips/enterprise-integration-patterns.html

(not talking about how Apache Camel does the implementation)

If it isn't that much difference, perhaps having an EIP starting point would help a lot of people understand what you are trying to achieve as EIP are rather well understood and utilized as a conceptual framework for enterprise software.

Cheers

Niclas

[Ged Byrne]

Hi Niclas,

This is a repost from a couple of years ago:

You can think of Flow Based Programming as Messaging Systems applied at the component level rather than application level.

If you read the basic concepts in the FBP book you will be able to see these correlations easily: http://www.jpaulmorrison.com/fbp/concepts.shtml

The basic difference is that the FBP book is talking about a finer level of abstraction.  Each component is at the level of abstraction you would normally find in functions or objects.  Rather than linking together large, complex applications the Flow Graph is routing messages through components that serve a simple, single purpose. 

Traditional EIPs are applied with Message Oriented Middleware, with channels allowing messages to pass between applications.

In Flow Based Programming the same architecture is applied at a finer level of granularity.  Applications are built up from components.

It's the application of the Pipes and Filters architectural style at a component level: http://www.eaipatterns.com/PipesAndFilters.html

The components are the filters.   Each component has ports for input and output that can be linked together by bounded queues.  These are the Channels: http://www.eaipatterns.com/MessageChannel.html

Theses ports are used to transmit Information Packets (IPs) across the Flow Graph.  These IPs are the Messages: http://www.eaipatterns.com/Message.html

A component may have multiple output ports.  Sending the IPs to different output ports allows for Message Routing: http://www.eaipatterns.com/MessageRouter.html

A component may also make changes to the IP.  This allows for Message Translation: http://www.eaipatterns.com/MessageTranslator.html

Some components will provide access to external resources, such as a file or a service.  These acts as the endpoints:http://www.eaipatterns.com/MessageEndpoint.html

All of the other patterns are just elaborations on these basic concepts: http://www.eaipatterns.com/MessagingComponentsIntro.html

Regards, 

Ged

--
You received this message because you are subscribed to the Google Groups "Flow Based Programming" group.
To unsubscribe from this group and stop receiving emails from it, send an email to flow-based-progra...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/flow-based-programming/CADmm%2BKc0-hyQAOzY2FcVSoV0197bnyUGYRZ6Aj3mn6-14OkEew%40mail.gmail.com.

[Paul Morrison]

That’s brilliant, Ged!  Thanks so much for your response to Niclas’ question!  Very clearly explained!

As Wayne Stevens used to say, FBP provides a uniform mental model… from maxi to mini!  Application designers have been using data flow design for decades, but the difficulty was always in translating that to a working system.  FBP seems to provide a comfortable level of granularity 🙂

Cheers!

To view this discussion on the web visit https://groups.google.com/d/msgid/flow-based-programming/CACtVJg91BN%3D-irLpaqSWOnbNkJi2SdAL67%2BA%2BsBApKSF%3DH7_7g%40mail.gmail.com.

[Niclas Hedhman]

On Sat, Aug 14, 2021 at 8:13 PM Ged Byrne <ged....@gmail.com> wrote:

This is a repost from a couple of years ago:

You can think of Flow Based Programming as Messaging Systems applied at the component level rather than application level.

Isn't that simply a choice? If we look at the implementation details of Camel, they applied EAI at a much lower level than "applications" and what I would call components.

 

My point is a little bit like; Loads of people know about Apache Camel and similar systems, and I think largely due to the fact that they based their conceptual model to EAI and could easily point to "there is the concept, and here is how we do it". 

I came to FBP because I thought it was something else (like electronics with discrete values concurrently in the wires, and I have a system that does that), but quickly realized it isn't that.

Cheers

Niclas

[Paul Tarvydas]

"like electronics with discrete values concurrently in the wires, and I have a system that does that"

I would be interested in hearing more about this.

pt

[Niclas Hedhman]

On Sun, Aug 15, 2021 at 1:37 PM Paul Tarvydas <paulta...@gmail.com> wrote:

"like electronics with discrete values concurrently in the wires, and I have a system that does that"

I would be interested in hearing more about this.

It is still work-in-progress and the target markets are industrial/building/home automation, energy management and possibly the larger IoT market.

   * Predefined blocks have N inputs and outputs. Only numeric/bool values.

   * Block performs some function and updates on each change on input.

   * The noflow-spec protocol is used, from web frontend to backend, so the system is always live, no edit vs running modes.

   * Websockets used as transport, and commands (say "Create Block") are sent to backend, and all changes are Events (say "Block Created") going back to all connected clients. So multi-browser updates are possible.

   * A group of blocks can be made into a new "higher level" block, with a different set of inputs/outputs.

I chose Pony Language to implement it, because it has an actor model in the language, guarantees to be free of race conditions, compiles to native code (quite performant) without a huge runtime, a language syntax that is "reasonable". Things I got (but didn't understand before digging in) was also a nicer message abstraction, i.e. sending a message and calling a function looks the same from the "caller" except it can't have a return value, but internally the message is placed on queue per actor. The actors (at language level) are unaware of queues, hence can't peek at messages or skip/prioritize. And also, the Reference Capabilities are incredibly interesting, more so than Rust owner/borrow concept. Not enough space to explain it here; but it is steep-learning-curve-warning on that. https://bluishcoder.co.nz/2017/07/31/reference_capabilities_consume_recover_in_pony.html

Anyway; The status is basically that the backend is mostly done, and the frontend is done with GoJS which is proprietary licensed and I want to replace that with something open source, as well as it only has the programming interface and lacking the scaffolding around it, i.e. management of projects, subsystems, devices, application.

I found an old screenshot

[Ged Byrne]

Hi Niclas,

It’s true that Camel does allow you to wire together simple beans, at which point it is no longer doing EIP.  EIP are patterns for integrating applications within the Enterprise.  

I’m saying EIP because these are the patterns that form the foundation for Camel.  You are using EAI, with makes the Enterprise Application Integration aspect even clearer:  

https://camel.apache.org/components/latest/eips/enterprise-integration-patterns.html

Given that this is an approach to _Integrating_ application it doesn’t make much sense to point to that as the starting point for a much older method that us used for _building_ applications. 

Camel does more than EIP now.  The developers created a framework for wiring graphs in order to implement the patterns and then extended further.  

Take, for example, Reactive patterns.  These are not EIP but they are fully supported by Camel. 

  * http://www.reactive-streams.org/

  * https://camel.apache.org/components/2.x/reactive-streams-component.html

The reactive streams component introduces a concept that is essential to FBP: back pressure.  

This is not supported by Camel’s EIP component, which require the explicit introduction of throttling to avoid overwhelming the network: 

https://camel.apache.org/components/latest/eips/throttle-eip.html

There are other aspects to FBP that are not found in either EIP or Reactive, such as the lifetime of an Information Packet that ensures that no information is ever dropped or neglected: 

“ More importantly, the ways data is viewed in FBP vs. conventional programming (as well as many FBP-inspired systems) are completely different: in FBP, data is managed in packets (IPs), which have a well-defined lifetime, from creation to destruction, and can only be owned by one process at a time, or be in transit between processes”

http://www.jpaulmorrison.com/fbp/noflo.html

I think that this is probably where FBP diverges from your approach because FBP has object like information packets in the channels rather than discrete values in the wires.  These IPs are entities with an individual identity and a well defined lifetime that can span many components.  

Regards,

Ged

--

You received this message because you are subscribed to the Google Groups "Flow Based Programming" group.
To unsubscribe from this group and stop receiving emails from it, send an email to flow-based-progra...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/flow-based-programming/CADmm%2BKdiqSEHaVAe%2BSLpVcm5u%2BdXjCKwq-3DSRb49hpp9OuW-g%40mail.gmail.com.

[Paul Morrison]

Thanks, Ged!  Very well explained!  Yes, Niclas, I view the IP "lifetime" concept as one of the key aspects of FBP - and this of course relates to the "granularity" level of FBP - yes, you can use FBP-like charts at a finer level, e.g. representing a * b as a block with two inputs and one output, but IMO this does not result in any improvement in productivity.  Graphical approaches to computing - at that level - actually predate FBP, e.g. Nassi-Shneiderman or Chapin charts - see p. 17 of my book (2nd ed.), but the programming paradigm would still have been control-flow.  The FBP paradigm, on the other hand, is more of an industrial, assembly-line, sort of mind set.

BTW Ged, since we are trying to get through to WASM, and one of the main routes is via LLVM, maybe Pony is worth looking into...?!

[Niclas Hedhman]

On Sun, Aug 15, 2021 at 5:42 PM Paul Morrison <paul.m...@rogers.com> wrote:

BTW Ged, since we are trying to get through to WASM, and one of the main routes is via LLVM, maybe Pony is worth looking into...?!

I am not sure I would recommend Pony, if you have been fighting the Rust compiler. Pony's compiler is incredibly strict, and there are too often times when the brain can't see far enough of potential race conditions. So, one needs to be prepared for steep-learning and trust that there will be relief on the other end. The upside is; If the compiler is happy, the application won't have race conditions. And that concurrency is built-in and nothing one thinks about. AND the language is small. Community of ~1000 people (as signed up to the Zulip Chat server), not sponsored by big corp and marketing dollars.

That said;

Tutorial; https://tutorial.ponylang.io/

Video; Sylvan Clebsch's introduction - https://www.youtube.com/watch?v=HGDSnOZaU7Y

And there is a "playground" available for doing small tests without worrying about installing compiler and such; https://playground.ponylang.io

Sample code (nothing particular, just a snippet that was laying in my browser cache); https://playground.ponylang.io/?gist=e0404a46b0ad755bf77abb21d4c1604c

HTH

Niclas

[Niclas Hedhman]

Maybe start with this one; https://www.youtube.com/watch?v=fNdnr1MUXp8

[Ged Byrne]

I don’t think Pony is the best option because, again, it implements it’s own solution to the problem.  It’s based on the agent model with a mailbox instead of ports.  For a taste read the documentation for the backpressure package: 

https://stdlib.ponylang.io/backpressure--index/

Once again you will find yourself fighting the solution imposed but the runtime when you try to implement FBP. 

I think the best option is going to be C/++.  Compiling CppFBP using emscripten looks to require much less than a week’s effort. 

https://emscripten.org/docs/compiling/Building-Projects.html#building-projects

Regards,

Ged

On Sun, 15 Aug 2021 at 16:42, Paul Morrison <paul.m...@rogers.com> wrote:

<snip>

To view this discussion on the web visit https://groups.google.com/d/msgid/flow-based-programming/7dfcf14f-b649-49c6-af6b-85526fbbe233n%40googlegroups.com.

[Paul Morrison]

Thanks, Ged!  Makes sense... I was thinking about that...  Not sure what to do about the Boost interface though!  Or we convert the CppFBP code and use something else for multiprocessing...?

Regards,

Paul

[Paul Tarvydas]

@niclas (and anyone else interested) I've been using the EE model (discrete events) for decades and would love to share what I've learned, although, I don't want to necessarily pollute the FBP group.  Get in touch with me.  paul tarvydas at-sign gmail.com.

pt

[Paul Tarvydas]

I see FBP as more like /bin/sh (and bash and zsh, etc, etc) than any popular programming language.

I *think* that Boost is just a library skin over raw UNIX calls (POSIX calls?).

I implemented what I needed in less than 10 operations in grash.c (https://github.com/guitarvydas/vsh/tree/master/grash) (grouped with my vsh (visual shell) prototype https://github.com/guitarvydas/vsh).

Using UNIX fork (process spawn, but easier), you get multi-core for free.

Using UNIX FDs (file descriptors), you get input and output ports for free.

Using UNIX I/O, you get bounded buffers for free (albeit, the last time I looked, the buffer size was not easy to change).

I'd be happy to explain / help further.

pt

ps. Using sh/bash/zsh/etc. you get 1 input port and 2 output ports for free.  The rest of the ports are available as /dev/fd/NN (where NN is some integer, probably there is a limit on the actual number).

pps.  IMO, one can skip Boost and go with raw sockets to get most of the way there.

ppps.  I remember reading about webworkers and thinking they were perfect for FBP (and friends).

pppps. Have you considered GO?

[Paul Morrison]

Thanks, Niclas!  Where does your sample put   ./main.ll  ?

Looks like Pony is one of those indentation-significant, no semicolon languages!  🙁

Paul

--

You received this message because you are subscribed to the Google Groups "Flow Based Programming" group.
To unsubscribe from this group and stop receiving emails from it, send an email to flow-based-progra...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/flow-based-programming/CADmm%2BKf5uVGZAB%3DtuTN_Af8d24B_qV29H0tPgUPvdMQ84dH5fA%40mail.gmail.com.

[Tom Young]

If you are considering Go language, please have a look at StreamWork:  https://github.com/tyoung3/sw. 

Go has many FBP friendly features including simple asynchronous operation with goroutines and channels;  and comprehensive modular packaging with version control. 

There seems to be no limit on the number of goroutines and channels (Linux  limits the number of file descriptors[the fd in /dev/fd/NNN] to fairly reasonably large and extendable values ).  

You can generate an entire, working project,  including a 

graph of the data flow,  from a textual network definition with StreamWork.   If you like, send me a network definition (in any text format) and I will generate a Golang project tree.

StreamWork, at the moment, is fairly Linux specific, but the project code developed is 

platform agnostic.  

Cheers, 

Tom Young
47 MITCHELL ST. 
STAMFORD, CT  06902

When bad men combine, the good must associate; ...
  -Edmund Burke 'Thoughts on the cause of the present discontents' , 1770

--
You received this message because you are subscribed to the Google Groups "Flow Based Programming" group.
To unsubscribe from this group and stop receiving emails from it, send an email to flow-based-progra...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/flow-based-programming/fe44e709-9d95-43a7-8160-9604816f9bd4n%40googlegroups.com.