how to implement dynamic workflows with JavaFBP?

[Ricardo Pino]

I want to know how to use javaFBP to create a software that allows you to dynamically change and add components to the workflow to run later

[Paul Morrison]

Sorry, it's not in the spec!  I have done some experimental work which I called "TabulaRasa", but I suspect we'd have to cripple much of the annotations checking. 

The code for JavaFBP is on Github if you want to play with it, and I could put up TabulaRasa as well if you're interested (and I believe you will run into the annotations problem, unless you modify the Network and/or Component classes).

Best regards,

Paul M.

On Sat, Jan 17, 2015 at 1:26 PM, Ricardo Pino <rwpin...@gmail.com> wrote:

I want to know how to use javaFBP to create a software that allows you to dynamically change and add components to the workflow to run later

--
You received this message because you are subscribed to the Google Groups "Flow Based Programming" group.
To unsubscribe from this group and stop receiving emails from it, send an email to flow-based-progra...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jon Nordby]

Is it acceptable to restart the network in order to activate component changes?.
If so, javafbp-runtime allows to do the basics, over the FBP protocol. https://github.com/jonnor/javafbp-runtime

[Francisco León]

I think that is necessary defining  an established procedure for changing connections between components but without suspending their internal working execution (especially if the execution of components is performed in a multi-threading environment). 

I'm currently developing a FBP framework in c++,  which delegates an special task for dispatching events between components exclusively, while other components are still running in multiple threads. Components only care about  processing the IP in their insolated space, while the main dispatcher transport the IP objects between components and handles their memory management in the queues (somehow different from the original JP design which delegates the management of the queues to the components) .  This design removes the necessity to put blocking methods (mutex) at component for accessing their queues. This allows me to building a total lock-free architecture specially when is needed to handle common resources for objects in shared memory environments. Even better, It also has the capability of sending and sharing the same IP (packets) to multiple receivers when producing packets at the same output port (one output produces common objects which are shared in multiple input queues), and still having a safe memory handling for those objects without inconsistencies; their memory are released by the main dispatcher task. The only requirement is that packets must be immutable objects.

So I think that it would be possible to create some components whose have the duty for changing and maintaining the creation and connectivity for other components in the network. These special components receive commands from other components requesting a reconfiguration of their connectivity.

[Paul Morrison]

This idea of immutable objects has come up several times on this forum.  Presumably it entails much more destruction and creation of IPs than in "classical" FBP, but I can see some advantages.  So whether you go this way depends on how much of a performance hit that will be.  However, I would still counsel against allowing multiple processes to access the same IP - I think it will involve a much more complex mental model.

IMO having immutable IPs also makes it difficult to implement IP "tree" structures, so you may have to sacrifice that function...  OTOH substreams can do pretty much anything that IP trees can do, except that you may have to specify rather large connection capacities to accommodate complex trees.

Regards,

Paul M. 

--

[Francisco León]

Hi Paul. When you mention IP "tree" structures ... did you mean collection objects like TreeMap<K,V> or HashMap<K,V> from Java? I really don't know if java has native functions for blocking those objects as const immutable objects. But In c++ is possible by writing especial object datatypes that wrap dynamic objects and handle them with a const immutable interface . 

[Paul Morrison]

IP "tree" structures are described in Chap. 12 of my book "Flow-Based Programming" (2nd ed.).  That's basically the same material as described in http://www.jpaulmorrison.com/fbp/tree.shtml .  This also explains why, in some cases, trees can be more appropriate than substreams (although of course they can be mapped into each other)..

Regards,

Paul

On Mon, Jan 19, 2015 at 12:17 PM, Francisco León <super...@gmail.com> wrote:

Hi Paul. When you mention IP "tree" structures ... did you mean collection objects like TreeMap<K,V> or HashMap<K,V> from Java? I really don't know if java has native functions for blocking those objects as const immutable objects. But In c++ is possible by writing especial object datatypes that wrap dynamic objects and handle them with a const immutable interface . 

[John Cowan]

Francisco León scripsit:

> Hi Paul. When you mention IP "tree" structures ... did you mean collection
> objects like TreeMap<K,V> or HashMap<K,V> from Java? I really don't know if
> java has native functions for blocking those objects as const immutable
> objects.

No. The Packet class has, in addition to the payload and a type indication
(payload, open bracket, close bracket, end of stream) also has left-sibling
and first-child pointers that allow one to construct an arbitrary tree
(or dag, but typically tree) of packets. Then it's the root that gets
passed through connections, carrying an arbitrary packet structure with it.
This is just a convenience, not a logical necessity.

This tree structure is logically outside the payload, and so it's usually
not taken into consideration when determining if payloads are immutable or not.

--
John Cowan http://www.ccil.org/~cowan co...@ccil.org
If I have seen farther than others, it is because I am surrounded by dwarves.
--Murray Gell-Mann

[Paul Morrison]

On Mon, Jan 19, 2015 at 1:44 PM, John Cowan <co...@mercury.ccil.org> wrote:

No.  The Packet class has, in addition to the payload and a type indication
(payload, open bracket, close bracket, end of stream) also has left-sibling
and first-child pointers that allow one to construct an arbitrary tree
(or dag, but typically tree) of packets.  Then it's the root that gets
passed through connections, carrying an arbitrary packet structure with it.
This is just a convenience, not a logical necessity.

Hi John, I checked the existing "classical" FBP implementations, and JavaFBP uses the alternating Chain and Packet structure described in my book.  The comment on the "attach" function says:

"A Packet may have multiple named Chains attached to it, accessed via a HashMap.
   * Since Packets are attached to Chains, as well as Chains to Packets,
   * this results in an alternation of Packets and Chains, creating a tree structure."

It doesn't look as though I implemented "attach" for either of the other two (C#FBP or CppFBP).  Blush!