Explicit or implicit blocking calls to send and receive?

153 views
Skip to first unread message

ComFreek

unread,
Mar 8, 2015, 11:28:51 AM3/8/15
to flow-based-...@googlegroups.com
Some languages (or language additions) allow one to use the following pseudo code:

function componentFunction() {
  inport
= /* ... */
  outport
= /* ... */

 
while (ip = inport.receive()) {
   
// do something and send
    outport
.send(ip);
 
}
}

Note that the act of receiving an IP or sending one could be possibly blocking (no IP arrived yet and the connection of the Outport has reached its capacity, respectively).
The point is that this information is not strictly conveyed in the code above, whereas some languages even force you to use some other syntax in order to allow for blocking calls:

// JavaScript
function *componentFunction() {
 
var inport = /* ... */;
 
var outport = /* ... */;

 
var ip;
 
while ((ip = yield inport.receive()) !== null) {
   
// do something
   
yield outport.send(ip);
 
}
}


Notice how one has to use (a) a generator function (indicated by '*' in front of the function name) and (b) the 'yield' keyword to yield execution while waiting for an IP to arrive.

There is a way to develop a component without these elements, namely the usage of Fibers as in Paul's JSFBP implementation. However, this is strictly bound to Node.js - there is unfortunately no way to run this in browsers.

Especially when considering a JavaScript implementations of FBP, would you say that a syntactic element like 'yield' is preferable?
Some argue that asynchronous function calls without callbacks/promises/generators, i.e. with Fiber-like methods, do not constitute idiomatic JavaScript. On the other hand one might say that the usage of 'yield' is not idiomatic FBP.

Eventually, the whole question also boils down to: "Is a non-leaky abstraction possible (in JS)?"

Paul Tarvydas

unread,
Mar 8, 2015, 11:39:59 AM3/8/15
to flow-based-...@googlegroups.com
On 15-03-08 11:28 AM, ComFreek wrote:
> - there is unfortunately no way to run this in browsers.

I may be misinterpreting what you meant, but I've shown that this can be
done in raw C without processes / threads / yields / etc. JS is
less-stunted (or equally stunted) than C, hence, it can be done in JS in
a browser. Doing it this way is even more explicit and painful, but it
can be done (this has no syntactic sugar to hide what's really going on) :

https://github.com/guitarvydas/collate-fbp-classic

pt

Tom Robinson

unread,
Mar 8, 2015, 3:01:48 PM3/8/15
to flow-based-...@googlegroups.com
More context here: https://github.com/jpaulm/jsfbp/issues/14

The main issue I have is that fibers are not part of JavaScript and are unlikely to EVER be added to JavaScript (because generators make them largely redundant, and generators ARE in JavaScript), and therefore unlikely to ever be available in browsers.

We can argue about whether fibers or generators are "better", but the fact is if we want JSFBP to run in browsers (do we?) we need to stop using fibers.

In cased you missed it, my prototype using generators is here: https://github.com/tlrobinson/sbp

--
You received this message because you are subscribed to the Google Groups "Flow Based Programming" group.
To unsubscribe from this group and stop receiving emails from it, send an email to flow-based-progra...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Tom Robinson

unread,
Mar 8, 2015, 3:13:34 PM3/8/15
to flow-based-...@googlegroups.com
Paul, I don't totally understand how this works, but it looks like you're manually writing a state machine, which is exactly how generator functions work internally. Take, for example, "copier" from my prototype and run it through the Babel "transpiler", which can compile ES6 generators to plain old ES5 (the version currently available in most browsers):


In fact, one of the benefits of the generator function "function *" and "yield" syntax (besides being "safer" than fibers: http://howtonode.org/generators-vs-fibers) is that it can be compiled to versions of JavaScript that don't natively support generators. 

--
You received this message because you are subscribed to the Google Groups "Flow Based Programming" group.
To unsubscribe from this group and stop receiving emails from it, send an email to flow-based-programming+unsub...@googlegroups.com.

Paul Morrison

unread,
Mar 8, 2015, 4:47:11 PM3/8/15
to flow-based-...@googlegroups.com

Yes, but do generators support multiple stacks?

The other Paul

To unsubscribe from this group and stop receiving emails from it, send an email to flow-based-progra...@googlegroups.com.

Tom Robinson

unread,
Mar 8, 2015, 5:09:42 PM3/8/15
to flow-based-...@googlegroups.com
I believe so, but can you explain exactly what you mean by "multiple stacks"? Does this code adequately demonstrate multiple stacks?

    import Promise from "bluebird";

    let delayPrintLoop = Promise.coroutine(function *(delay, name) {
      while (true) {
        yield Promise.delay(delay);
        console.log(name);
      }
    });

    delayPrintLoop(500, "foo");
    delayPrintLoop(1500, "bar");

And running it:

    $ babel-node multiple-stacks.es6
    foo
    foo
    bar
    foo
    foo
    foo
    bar

...

Or how about this?

    import Promise from "bluebird";

    let outer = Promise.coroutine(function *() {
      console.log("outer started");
      yield Promise.delay(1000);
      yield inner();
      console.log("outer ended");
    });

    let inner = Promise.coroutine(function *(delay, name) {
      console.log("inner started");
      yield Promise.delay(1000);
      console.log("inner ended");
    });

    outer();

Running it:

    $ babel-node multiple-stacks.es6
    outer started
    inner started
    inner ended
    outer ended

I prefer just writing code as proof. I will happily attempt to implement any cFBP program you throw at me, without using fibers. I've already done collate (https://github.com/tlrobinson/sbp/blob/master/components/collate.es6). What else do you need to demonstrate a system is considered "classical FBP"?

-tom

John Cowan

unread,
Mar 8, 2015, 5:17:13 PM3/8/15
to flow-based-...@googlegroups.com
Tom Robinson scripsit:

> I believe so, but can you explain exactly what you mean by "multiple
> stacks"? Does this code adequately demonstrate multiple stacks?

By multiple stacks is meant the ability to yield from within arbitrarily
deeply nested subroutines/methods. Python generators, for example,
cannot do this: you cannot invoke a generator bar which invokes a
function baz which then yields such that the caller of bar gains control.
Instead, baz must be itself rewritten as a generator.

This is equivalent to having general semicoroutines; see the Lua papers.

--
John Cowan http://www.ccil.org/~cowan co...@ccil.org
A few times, I did some exuberant stomping about, like a hippo auditioning
for Riverdance, though I stopped when I thought I heard something at
the far side of the room falling over in rhythm with my feet. --Joseph Zitt

Paul Morrison

unread,
Mar 8, 2015, 6:45:53 PM3/8/15
to flow-based-...@googlegroups.com


On Mar 8, 2015 5:16 PM, "Tom Robinson" <t...@tlrobinson.net> wrote:
>
> What else do you need to demonstrate a system is considered "classical FBP"?
>

I think I already suggested a test: why not take my fbptest01, modify the three components involved as per your proposals, modify the network to process a few million IPs;  now run your network, and compare its performance with the same network using the present version of JSFBP.

I would be very interested in the results.

Tom Robinson

unread,
Mar 8, 2015, 7:46:09 PM3/8/15
to flow-based-...@googlegroups.com
Does the definition of "classical FBP" include some arbitrary performance metric?

--

Paul Morrison

unread,
Mar 8, 2015, 8:44:51 PM3/8/15
to flow-based-...@googlegroups.com


On Mar 8, 2015 7:46 PM, "Tom Robinson" <t...@tlrobinson.net> wrote:
>
> Does the definition of "classical FBP" include some arbitrary performance metric?

I am rather a pragmatic type, so I just thought: let's try Tom's solution with a few million IPs.  Did you run the test - it should have taken less time than it took you to write your note!

Last time I ran my test, it averaged just under 1 microsec per send/receive pair on my desktop, so 1,000,000 IPS going through 2 send/receive pairs  should take around 2 secs. This is in the project Readme.

So I thought it would be interesting to see if yours runs slower or faster than JSFBP!  Looking forward to hearing your results.

Regards,

Paul

Tom Robinson

unread,
Mar 9, 2015, 12:58:37 AM3/9/15
to flow-based-...@googlegroups.com
Ok, but I'd still like to know if there are other reasons why my prototype wouldn't be considered "classical FBP", in particular in relation to "multiple stacks".

My prototype is a little less than twice as slow as the current version of JSFBP (about 10 seconds, vs 5 seconds, for 1 million IPs, on my machine), but that definitely doesn't doesn't mean generators are twice as slow as fibers. It uses Streams, which adds some overhead, as well as Babel to compile ES6 generators to ES5, as opposed to native generators.

If you did a straight port of JSFBP to generators, and used native generators (available in iojs and latest versions of node) I suspect the performance would be a lot closer to JSFBP. I could try that but I'm not sure when I'll get around to it.

Anyway, is this micro-benchmark even representative of real FBP programs? It seems like any sort of non-trivial processing will far outweigh this slight difference.


--
You received this message because you are subscribed to the Google Groups "Flow Based Programming" group.
To unsubscribe from this group and stop receiving emails from it, send an email to flow-based-progra...@googlegroups.com.

Paul Morrison

unread,
Mar 9, 2015, 10:26:04 AM3/9/15
to flow-based-...@googlegroups.com

That performance is pretty impressive, and is exactly what I wanted to know.

Just to clarify, the prototype you refer to is sbp, right?

In the discussion on Github, I said that we have been using back pressure, selective receive, and IPs with well-defined lifetime and owners, as a litmus test for classical FBP.  We could also consider banning one-to-many connections, although IMO an implementation that did automatic cloning in this case would still be classical - it just wouldn't make much sense!

I think array ports are pretty important - I assume you support these, as they are in JSFBP.

At another level, the implementation should conform to the two constraints described in my book - "order-preserving constraint " and "flow constraint" - Chap. 3.  I can't imagine your implementation breaking these!

So, assuming all of the above, it certainly sounds like a "classical" implementation!  The multiple stack technique was the only way I had found to implement classical FBP, but it sounds like you can do something similar with whatever  magic you use!  As John pointed out,  multiple stacks actually allow you to do sends and receives from any level in a process stack - do you also allow this?

Couple of questions : a) could you document how to write a component for people who aren't familiar with all these new facilities?

b) can your components call ordinary JS routines without special syntax?

Looks like a great job!

Comfreek, Matt, Ken, others?

Regards,

Paul

Paul Morrison

unread,
Mar 9, 2015, 10:34:12 AM3/9/15
to flow-based-...@googlegroups.com

Thanks for starting this topic, Comfreek!

On Mar 8, 2015 11:28 AM, "ComFreek" <info.c...@gmail.com> wrote:
>
> Some languages (or language additions) allow one to use the following pseudo code:
>
> function componentFunction() {
>   inport = /* ... */
>   outport = /* ... */
>
>   while (ip = inport.receive()) {
>     // do something and send
>     outport.send(ip);
>   }
> }
>
> Note that the act of receiving an IP or sending one could be possibly blocking (no IP arrived yet and the connection of the Outport has reached its capacity, respectively).
> The point is that this information is not strictly conveyed in the code above, whereas some languages even force you to use some other syntax in order to allow for blocking calls:

My point is that the component and network developers do not need to know this.  FBP processes are like Dijkstra's sequential processes - they don't care if they are stretched in time!  A send is a send, whether instantaneous or stretched in time!

>
> Eventually, the whole question also boils down to: "Is a non-leaky abstraction possible (in JS)?"
>

Sorry, Comfreek, what does this mean?  :-(

Best regards,

Paul

co...@ccil.org

unread,
Mar 9, 2015, 11:14:19 AM3/9/15
to flow-based-...@googlegroups.com
J. Paul Morrison scripsit:

> In the discussion on Github, I said that we have been using back pressure,
> selective receive, and IPs with well-defined lifetime and owners, as a
> litmus test for classical FBP.

I think that the ability to send and receive from within nested calls
is also essential, which is why Python generators don't qualify.
But see below.

OTOH, I think explicit disposition of packets is *not* essential; a
purely garbage-collected system is fine.

> We could also consider banning one-to-many
> connections, although IMO an implementation that did automatic cloning in
> this case would still be classical - it just wouldn't make much sense!

I don't think packet mutability is a requirement of classical FBP, and
a system with immutable packets might well do one-to-many connections.

> I think array ports are pretty important - I assume you support these, as
> they are in JSFBP.

Array ports are syntactic sugar: there's no difference between a single
array port of size 10 and ten separate systematically named ports.

> At another level, the implementation should conform to the two constraints
> described in my book - "order-preserving constraint " and "flow
> constraint"

+1. Systems that violate the flow constraint also violate causality!

> So, assuming all of the above, it certainly sounds like a "classical"
> implementation! The multiple stack technique was the only way I had found
> to implement classical FBP, but it sounds like you can do something
> similar
> with whatever magic you use! As John pointed out, multiple stacks
> actually allow you to do sends and receives from any level in a process
> stack - do you also allow this?

It's a mechanical transformation provided that there is a maximum
recursion depth: that's the trick of Jackson Structured Programming.
Some Scheme compilers do it automatically.
You know, you haven't stopped talking since I came here. You must
have been vaccinated with a phonograph needle.
--Rufus T. Firefly


ComFreek

unread,
Mar 9, 2015, 12:35:58 PM3/9/15
to flow-based-...@googlegroups.com
@Paul
A leaky abstraction is an alleged abstraction layer which unwillingly reveals some implementation details.

A Fiber implementation introduces a seamless integration of send() and receive(). A non-Fiber implementation will always make use of generator functions and yield or similar statements (see below for an alternative).
What I actually wanted to say is that we cannot create a compatbility layer between Fiber and non-Fiber implementations - this is at least impossible when assuming a reasonable amount of time. In theory the Fiber implementation could adopt generator functions and yield (as in Fiber.yield()) when a yield statement (as in a yield inside a generator function) occurs. However, this would only complicate things.

@Tom
I have been thinking about an alternative for generator functions and yield: async functions and await. These two syntax elements will be introduced in EcmaScript Standard 7. I have also been experimenting with them and they worked when I replaced generators and yield with them in my former stream-based promise/generators implementation.
async and await can be compiled down to ES 5 with Traceur. Maybe Babel supports them as well. In fact, they are not too different from generator functions as Traceur has the ability to implement/transpile them as such.

In my opinion async and await have a bit more (linguistic) expressiveness than generators/yield.

Tom Robinson

unread,
Mar 9, 2015, 1:10:51 PM3/9/15
to flow-based-...@googlegroups.com
@ComFreek I agree, but aren't generators/yield + Bluebird's "coroutine" [1], Q's "async" [2], or "co" [3] essentially the same thing as "async/await"? You'd basically replace "yield" with "async" and "Promise.coroutine(function * foo() {})" with "async function foo() {}". Are there any other differences?

This is how "sbp" currently works: https://github.com/tlrobinson/sbp/blob/master/core/Process.es6#L39

The important thing is that it's all based on Promises, and therefore should be interoperable with a generator wrapper (e.x. Bluebird.coroutine), async/away, or just using raw promises.


--

ComFreek

unread,
Mar 9, 2015, 3:43:00 PM3/9/15
to flow-based-...@googlegroups.com, t...@tlrobinson.net
I don't think there are other differences apart from the semantic/expressiveness aspect.

Indeed, if they are already interoperable, I would personally opt for async/await in components. That's only a stylsitic descision, though. "I await the next IP" instead of "I yield to receive the next IP".
To unsubscribe from this group and stop receiving emails from it, send an email to flow-based-programming+unsub...@googlegroups.com.

Paul Morrison

unread,
Mar 10, 2015, 9:33:26 AM3/10/15
to flow-based-...@googlegroups.com


On Mar 9, 2015 11:14 AM, <co...@ccil.org> wrote:
>
> J. Paul Morrison scripsit:
>
> > In the discussion on Github, I said that we have been using back pressure,
> > selective receive, and IPs with well-defined lifetime and owners, as a
> > litmus test for classical FBP.
>
> I think that the ability to send and receive from within nested calls
> is also essential, which is why Python generators don't qualify.
> But see below.

See where?


>
> OTOH, I think explicit disposition of packets is *not* essential; a
> purely garbage-collected system is fine.

Well, in my implementations I keep track of the number of IPs that a process owns, and I check to make sure this number has returned to zero before the process deactivates.  I wouldn't want to give this facility up!


>
> > We could also consider banning one-to-many
> > connections, although IMO an implementation that did automatic cloning in
> > this case would still be classical - it just wouldn't make much sense!
>
> I don't think packet mutability is a requirement of classical FBP, and
> a system with immutable packets might well do one-to-many connections.

A bottle travelling through a bottling factory is "mutable" - given the ownership rules of FBP, isn't immutability just an extra safeguard -  at some cost in overhead? 

And one-to-many still implies automatic cloning - which could get veeery expensive, a s e.g. in the case of IP trees.


>
> > I think array ports are pretty important - I assume you support these, as
> > they are in JSFBP.
>
> Array ports are syntactic sugar: there's no difference between a single
> array port of size 10 and ten separate systematically named ports.

I agree, with the stress on 'systematically'!


>
> > At another level, the implementation should conform to the two constraints
> > described in my book - "order-preserving constraint " and "flow
> > constraint"
>
> +1.  Systems that violate the flow constraint also violate causality!
>
> > So, assuming all of the above, it certainly sounds like a "classical"
> > implementation!  The multiple stack technique was the only way I had found
> > to implement classical FBP, but it sounds like you can do something
> > similar
> > with whatever  magic you use!  As John pointed out,  multiple stacks
> > actually allow you to do sends and receives from any level in a process
> > stack - do you also allow this?
>
> It's a mechanical transformation provided that there is a maximum
> recursion depth: that's the trick of Jackson Structured Programming.
> Some Scheme compilers do it automatically.
>

Sorry, John, I don't see how recursion fits in - what question are you answering?

Regards,

Paul

co...@ccil.org

unread,
Mar 10, 2015, 9:49:43 AM3/10/15
to flow-based-...@googlegroups.com
J. Paul Morrison scripsit:

>> But see below.
>
> See where?

Sorry -- the bottom of this message.

> Well, in my implementations I keep track of the number of IPs that a
> process owns, and I check to make sure this number has returned to zero
> before the process deactivates. I wouldn't want to give this facility up!

Yes, it's a good debugging/self-checking facility, but not essential.

>> > As John pointed out, multiple stacks
>> > actually allow you to do sends and receives from any level in a
>> > process stack - do you also allow this?
>>
>> It's a mechanical transformation provided that there is a maximum
>> recursion depth: that's the trick of Jackson Structured Programming.
>> Some Scheme compilers do it automatically.
>>
> Sorry, John, I don't see how recursion fits in - what question are you
> answering?

I was pointing out that any program can be mechanically transformed
into a state machine style, provided the amount of recursion/looping in
it is not dependent on the data. Suppose you have an FBP component with an
arbitrary number of subroutines, such that there is no recursion
(that is, no subroutine calls itself directly or indirectly) and no
loops whose length depends on the input data. (This isn't as bad as
it sounds: a loop or recursion that doesn't contain a send or receive
doesn't matter in this context.) A loop can always be
reduced to a recursive function call, so we can speak of just
recursion without loss of generality.

Then all the routines can be open-coded into one big routine.
It's easy to transform this routine so that every time it needs
to yield (to send or receive) it exits, expecting to be re-entered
when input is available or output is acceptable, as the case may be.
But if there is recursion, this transformation won't work in general
unless the recursion stack is made explicit.
Take two turkeys, one goose, four cabbages, but no duck, and mix them
together. After one taste, you'll duck soup the rest of your life.
--Groucho


Paul Tarvydas

unread,
Mar 10, 2015, 10:23:08 AM3/10/15
to flow-based-...@googlegroups.com
On 15-03-08 03:13 PM, Tom Robinson wrote:
> Paul, I don't totally understand how this works, but it looks like
> you're manually writing a state machine


Stack == state.

Process + Stack == state machine.



We're faking non-von-Neumannism on top of a von-Neumann CPU(s).

This is the only way to do it.

This is what's going on under the hood, regardless of what syntactic
sugar (incl. faking of "multiple stacks") is applied.

pt

Matthew Lai

unread,
Mar 10, 2015, 11:24:39 AM3/10/15
to flow-based-...@googlegroups.com, t...@tlrobinson.net


On Sunday, 8 March 2015 17:09:42 UTC-4, Tom Robinson wrote:

I prefer just writing code as proof. I will happily attempt to implement any cFBP program you throw at me, without using fibers. I've already done collate (https://github.com/tlrobinson/sbp/blob/master/components/collate.es6). What else do you need to demonstrate a system is considered "classical FBP"?


Hey Tom buddy can we test this one:

node A with 1 outport
node B with 1 outport
node C with 2 inports

node A outport ---> node C inport-1  ; bounded FIFO size set to 50
node B outport ---> node C inport-2 ; bounded FIFO size set to 50

Both node A and B runs in infinite loops, sending out IPs to node C. The IP can contain something as trivial as a 32-bit integer.

Node C performs the selective port read from inport 1 and 2, alternating every 5 seconds:

while ()
{
    while (not 5 seconds yet)
    {
        read from port 1
    }
    while (not 5 seconds yet)
    {
        read from port 2
    }
}

This sounds like a good test for the back pressure thing as while node C is reading port 1 the
node B ---> node C bounded FIFO will fill up quite fast and node B would be blocked from sending
more IPs.

Thanks!

Yours,

Matt

 

Paul Morrison

unread,
Mar 11, 2015, 9:45:38 AM3/11/15
to flow-based-...@googlegroups.com


On Mar 10, 2015 9:49 AM, <co...@ccil.org> wrote:
>

and no
> loops whose length depends on the input data.  (This isn't as bad as
> it sounds: a loop or recursion that doesn't contain a send or receive
> doesn't matter in this context.)  A loop can always be
> reduced to a recursive function call, so we can speak of just
> recursion without loss of generality.

Sorry, John, still lost!  If the "canonical" FBP component is Copier, surely this is a "loop with length dependent on its input"...?    Which brings me back to Collate, which caused so much trouble for NoFlo - throw looping, selective receive and back pressure into the mix, and I think it's pretty hard to do without multiple stacks.

That said, in a green thread implementation, each segment between yield and return is pretty much independent, and could theoretically use the same stack... if it weren't for subroutine local storage.   So what if we allocated that from the heap, using a dictionary to relate the subroutine (plus its entire call hierarchy) to the subroutine's local storage...  Not pretty, though!

It's ironic that we have to go to so much trouble to accommodate von Neumann machines, when non-von is so much more natural!  :-)   Some 55 years ago there was work going on on "streaming machines", but the von Neumann machine won conclusively!  Of course in those days there was no real reason to try something different!  Little did we know!

Regards,

Paul

Paul Tarvydas

unread,
Mar 11, 2015, 10:18:15 AM3/11/15
to flow-based-...@googlegroups.com
I think that I agree that you need "stacks" if you allow recursion within components.

Languages that support closures already support this notion out of the box. 

Languages like C don't support this (and don't give you access to the stack pointer).


What you guys are effectively saying is "recursion brings nothing but headaches" :-).


So, we might step back and ask the question "why allow recursion?".  (Mandate the use of bounded buffers, but allow for infinite stacks?).

Surely, FBP doesn't allow for recursive boxes (at the box level)?

In my variant of FBP, I think of boxes as subroutines and I explicitly disallow recursion (with a "busy" bit).  This hasn't bit me in the rear for 25+ years.

As for unrolling and flattening code - who cares?  A box is "syntax".  I don't look at the assembler emitted by gcc every time I use it.  The collate code I put on my github isn't pretty, but it is consistent and can be spit out by a compiler.  (And, I believe that it is more efficient than even green threads).

pt
--

Tom Robinson

unread,
Mar 11, 2015, 3:05:34 PM3/11/15
to flow-based-...@googlegroups.com


On Tue, Mar 10, 2015 at 8:24 AM Matthew Lai <mm...@sympatico.ca> wrote:

node A with 1 outport
node B with 1 outport
node C with 2 inports

node A outport ---> node C inport-1  ; bounded FIFO size set to 50
node B outport ---> node C inport-2 ; bounded FIFO size set to 50

Both node A and B runs in infinite loops, sending out IPs to node C. The IP can contain something as trivial as a 32-bit integer.

Node C performs the selective port read from inport 1 and 2, alternating every 5 seconds:

while ()
{
    while (not 5 seconds yet)
    {
        read from port 1
    }
    while (not 5 seconds yet)
    {
        read from port 2
    }
}

This sounds like a good test for the back pressure thing as while node C is reading port 1 the
node B ---> node C bounded FIFO will fill up quite fast and node B would be blocked from sending
more IPs.


My back pressure implementation using Streams is not 100% correct (see discussion at https://github.com/jpaulm/jsfbp/issues/25), but it seems like it mostly works, with the caveat that it buffers a couple extra IPs (I think it's essentially 1 for each process between the producer and consumer).

I modified your suggestion to reduce the amount of output (pause reader 100ms between each read, FIFO size = 10), but I think it demonstrates what you want:


This brings up another question I've had: is immediate/precise control of back pressure a requirement for correctness in classical FBP, or is it acceptable to have to buffer a few extra IPs?

i.e. how do you think of ports?
* are they "push" streams which push IPs downstream until they are told to pause
* are they "pull" streams that explicitly request an IP from an upstream process?

Here's a summary of the problem with Node Streams:

"This is a problem in node streams, pause is only transfered on write, so on a long chain (a.pipe(b).pipe(c)), if c pauses, b will have to write to it to pause, and then a will have to write to b to pause. If b only transforms a's output, then a will have to write to b twice to find out that c is paused."


If it is a problem we could switch to a different type of stream (like pull-stream), but it would be nice to stick with a standard stream primitive like Node's, and benefit from the ecosystem of modules with Stream interfaces.

-tom

Matthew Lai

unread,
Mar 11, 2015, 3:51:02 PM3/11/15
to flow-based-...@googlegroups.com, t...@tlrobinson.net

On Wednesday, 11 March 2015 15:05:34 UTC-4, Tom Robinson wrote:

I modified your suggestion to reduce the amount of output (pause reader 100ms between each read, FIFO size = 10), but I think it demonstrates what you want:

Looking great! I assume the system works fine without the 100ms delay correct? 


This brings up another question I've had: is immediate/precise control of back pressure a requirement for correctness in classical FBP, or is it acceptable to have to buffer a few extra IPs?

i.e. how do you think of ports?
* are they "push" streams which push IPs downstream until they are told to pause
* are they "pull" streams that explicitly request an IP from an upstream process?

I also buffer 1 extra IP for each bounded FIFO buffer, and find the push style IP delivery easier to understand and implement :)

We'll see what Mr. Morrison has to say here.


Here's a summary of the problem with Node Streams:

"This is a problem in node streams, pause is only transfered on write, so on a long chain (a.pipe(b).pipe(c)), if c pauses, b will have to write to it to pause, and then a will have to write to b to pause. If b only transforms a's output, then a will have to write to b twice to find out that c is paused."


Would a loop back connection pose a problem for the streams that you use here, for example with the attached fbp graph?
And the different streams can merge into the same inport correct?

Yours,

Matt

Tom Robinson

unread,
Mar 11, 2015, 4:17:27 PM3/11/15
to flow-based-...@googlegroups.com
I haven't tested it, but loops and merging should be fine.

And just because of the way streams work (you can pipe to multiple destinations) splitting will also "work", but that's obviously a bad idea because I haven't implemented immutable IPs or copying.

--

Tom Young

unread,
Mar 12, 2015, 5:56:02 PM3/12/15
to Flow Based Programming
I think that any realizable digital hardware architecture can be emulated on von Neuman architecture.    The challenges have to do with concurrent resource sharing, the associated language awkwardness's for dealing properly with concurrent threads, and the extreme difficulty of proving the correctness of multi-threaded programs;  rather than with the underlying hardware.

The 'Two Generals' problem comes to mind, for some reason, I can't quite put my finger on.

Tom Young
47 MITCHELL ST.
STAMFORD, CT  06902


When bad men combine, the good must associate; ...
  -Edmund Burke 'Thoughts on the cause of the present discontents' , 1770



--

Raoul Duke

unread,
Mar 12, 2015, 5:58:02 PM3/12/15
to flow-based-...@googlegroups.com
> I think that any realizable digital hardware architecture can be emulated on
> von Neuman architecture

(note that turning completeness doesn't mean that it is fun or
efficient or profitable, just possible. :-)

Paul Morrison

unread,
Mar 12, 2015, 6:58:16 PM3/12/15
to flow-based-...@googlegroups.com


On Mar 11, 2015 3:51 PM, "Matthew Lai" <mm...@sympatico.ca> wrote:
>

>
> I also buffer 1 extra IP for each bounded FIFO buffer, and find the push style IP delivery easier to understand and implement :)
>
> We'll see what Mr. Morrison has to say here.
>

Sorry, Matt and Tom, I don't quite understand what is meant by "1 extra IP".  All my implementations so far push IPs onto a connection, but the receiving process will obtain them immediately - it doesn't wait until the connection is full. 

Or are you saying that a send is not suspended, but simply reports that a connection is full?  But then surely it becomes the responsibility of the process to decide whether to send or not...  whereas in my mind a send is a send - whether "stretched in time" or not.

Matthew Lai

unread,
Mar 12, 2015, 9:17:09 PM3/12/15
to flow-based-...@googlegroups.com


On Thursday, 12 March 2015 18:58:16 UTC-4, Paul Morrison wrote:

Sorry, Matt and Tom, I don't quite understand what is meant by "1 extra IP". 

Oh, I mean when I set the capacity of the bounded FIFO to be 50, it can really stores 51 IPs :)
 

All my implementations so far push IPs onto a connection, but the receiving process will obtain them immediately - it doesn't wait until the connection is full. 

The same behaviour with my implementation :)

Or are you saying that a send is not suspended, but simply reports that a connection is full? 

Nope. Definitely a blocking send in my implementation.

Yours,

Matt

Paul Morrison

unread,
Mar 12, 2015, 10:27:25 PM3/12/15
to flow-based-...@googlegroups.com


On Mar 12, 2015 9:23 PM, "Matthew Lai" <mm...@sympatico.ca> wrote:
>
>
>
> On Thursday, 12 March 2015 18:58:16 UTC-4, Paul Morrison wrote:
>>
>> Sorry, Matt and Tom, I don't quite understand what is meant by "1 extra IP". 
>
> Oh, I mean when I set the capacity of the bounded FIFO to be 50, it can really stores 51 IPs :)
>  

Is there some advantage to this?  Just curious!

>>
>> All my implementations so far push IPs onto a connection, but the receiving process will obtain them immediately - it doesn't wait until the connection is full. 
>
> The same behaviour with my implementation :)
>>
>> Or are you saying that a send is not suspended, but simply reports that a connection is full? 
>
> Nope. Definitely a blocking send in my implementation.

Sounds good!   Tom, do you do the same?
>
> Yours,
>
> Matt
>

Tom Young

unread,
Mar 13, 2015, 10:31:17 AM3/13/15
to Flow Based Programming
The one extra IP buffer is interesting.  In my implementation, DFD, the DFD dynamically creates one extra IP when it detects a deadlock.  Each output port can have just one buffer added this way.    Otherwise connections are generally not buffered, just space for one IP per port plus possibly this one extra space per output port.  

This allows a component to send IPs to itself without deadlock, so long as it consumes each output IP immediately.

If one to many connections were permitted, this would become complex, so that is not allowed, instead a generic Split component replicates IPs, if necessary.   It is theoretically possible to automatically insert Split at run time, though.

---The other Tom

"...the shell syntax required to initiate a coprocess and connect its input and output to other processes is quite contorted..."
W. Richard Stevens [Advanced Programming in the Unix Environment, p441]

--

Paul Morrison

unread,
Mar 13, 2015, 1:24:44 PM3/13/15
to flow-based-...@googlegroups.com


On Mar 13, 2015 10:42 AM, "Tom Young" <f...@twyoung.com> wrote:
>
> The one extra IP buffer is interesting.  In my implementation, DFD, the DFD dynamically creates one extra IP when it detects a deadlock.  Each output port can have just one buffer added this way.    Otherwise connections are generally not buffered, just space for one IP per port plus possibly this one extra space per output port.  
>

Presumably you only do this once...?  I assume you are not talking about automatic "ballooning", although it turns out that in certain "big data" handling situations, a sophisticated type of ballooning may be necessary - see Chap. 16 of the 2nd ed., "Parallelism Issues",  contributed by Mike Beckerle.

> This allows a component to send IPs to itself without deadlock, so long as it consumes each output IP immediately.

I don't support zero-length connections, so all my implementations support this.


>
> If one to many connections were permitted, this would become complex, so that is not allowed, instead a generic Split component replicates IPs, if necessary.   It is theoretically possible to automatically insert Split at run time, though.

Agree... or you can have special-purpose splitters, if you are working with IP trees, for instance.
>
> ---The other Tom
>

Tom Young

unread,
Mar 13, 2015, 4:43:20 PM3/13/15
to Flow Based Programming

Presumably you only do this once...? 

At most once per output port. As it turns out: at most once per cycle in the complete network graph( redundant phrase?). 
 

I assume you are not talking about automatic "ballooning", although it turns out that in certain "big data" handling situations, a sophisticated type of ballooning may be necessary - see Chap. 16 of the 2nd ed., "Parallelism Issues",  contributed by Mike Beckerle.

Yes, there is no sophisticated buffering, total memory requirements are easily estimated, and avoidable deadlocks are avoided, at some possible performance cost with fine-grain IPs.  
 
 

Kenneth Kan

unread,
Mar 15, 2015, 2:21:11 PM3/15/15
to flow-based-...@googlegroups.com

> This allows a component to send IPs to itself without deadlock, so long as it consumes each output IP immediately.

I don't support zero-length connections, so all my implementations support this.

This is a little digression. I thought that components are not allowed to send IPs to themselves in FBP. On chapter 19, page 190, it reads, "except that a process’s output port cannot be connected back to an input port, so we have to go via a PASSTHRU component."

More generally, this seems to be slightly tied to the recursion debate earlier in this thread. The idea that I'm getting is that we would *want* to (but not necessarily need to) allow "recursion" (I noticed that there are multiple contexts in this discussion here) at the network level, but not the component level. So, my question is, what is the reason behind not allowing component-level recursion and/or sending IPs to self?

Kenneth Kan

unread,
Mar 15, 2015, 3:44:05 PM3/15/15
to flow-based-...@googlegroups.com, t...@tlrobinson.net
Continuing the conversation over on issue #14 on GitHub:

I do agree with Tom that it's extremely confusing to correct JS to use synchronous style, not because it's technically unreasonable, but because it would certainly cause massive amount of confusion among JS programmers that whatever JS implementation in such style would just be dismissed as "it doesn't work".

On GitHub:

Generators also break that assumption, but you have to explicitly opt-in by declaring a generator function (with the "function*" syntax) and use the "yield" keyword, so there's no chance of someone else's code deep in some library you're using yielding execution without your knowledge.

This continues the multiple stack issue raised earlier. Using generators does require "infecting downward" that in order for a component to be fully FBP-compliant you need all functions to be generators. I believe you need to use the `yield*` syntax in ES6 to achieve what it needs to work as an FBP component.

In short, I think ES6 generators are basically just Python-style generators. And I'm curious how we can resolve the issues that John raised about Python generators lacking multiple stacks as I don't know of a solution right now.

Paul Morrison

unread,
Mar 15, 2015, 3:50:38 PM3/15/15
to flow-based-...@googlegroups.com

You're right!   I was probably being over-protective!   I just think that allowing a process to send to itself is just very error-prone.  And its very synchronous... The current implementations support this, but DrawFBP doesn't.   Just being pragmatic, I guess!  But really why would you need to do this?  If it's some kind of simulation of recursion, IMHO it just feels alien to FBP!

From snother point of view, I don't really see the need for recursion in the FBP context, but I don't think FBP cares...  (does it or should it?)

Regards,

Paul

Kenneth Kan

unread,
Mar 16, 2015, 12:00:04 AM3/16/15
to flow-based-...@googlegroups.com
I don't see the point of recursion either. That thought came to my mind simply because it was explicitly stated in the book that it wasn't allowed without an expressed reason as to why. So curiosity struck.

Tom Young

unread,
Mar 16, 2015, 5:11:22 PM3/16/15
to Flow Based Programming
I don't see recursion as being equivalent to a component sending itself IPs. 

Consider the network, A->B->C->D and suppose it is decided to combine components B and C; then you could have A->BC->D; BC->BC (port names omitted), without much difficulty.  If this were disallowed, you would need to change the component code, thereby obscuring the dataflow and making future use of the original IP require changing the BC component yet again.   

(You would, of course, need to ensure that BC issues the IPs before attempting to receive them.)
 

Tom Young
47 MITCHELL ST.
STAMFORD, CT  06902


When bad men combine, the good must associate; ...
  -Edmund Burke 'Thoughts on the cause of the present discontents' , 1770



--

Ged Byrne

unread,
Mar 16, 2015, 5:17:23 PM3/16/15
to Flow Based Programming
Agreed. A component doesn't preserve state when it sends. A component calling itself is just looping.

John Cowan

unread,
Mar 16, 2015, 8:14:55 PM3/16/15
to flow-based-...@googlegroups.com
Ged Byrne scripsit:

> Agreed. A component doesn't preserve state when it sends. A component
> calling itself is just looping.

Some components definitely do preserve state when they send; a packet
counter is perhaps the simplest example.
"The exception proves the rule." Dimbulbs think: "Your counterexample proves
my theory." Latin students think "'Probat' means 'tests': the exception puts
the rule to the proof." But legal historians know it means "Evidence for an
exception is evidence of the existence of a rule in cases not excepted from."

Raoul Duke

unread,
Mar 16, 2015, 8:23:31 PM3/16/15
to flow-based-...@googlegroups.com
Or is there a requirement that the state have to be only in the packets?

John Cowan

unread,
Mar 16, 2015, 8:24:34 PM3/16/15
to flow-based-...@googlegroups.com
Raoul Duke scripsit:

> Or is there a requirement that the state have to be only in the packets?

Not at all. Components can be stateless or stateful internally; the
framework does not care.
By Elbereth and Luthien the Fair, you shall have neither the Ring nor me!
--Frodo

Kenneth Kan

unread,
Mar 16, 2015, 11:18:37 PM3/16/15
to flow-based-...@googlegroups.com, t...@twyoung.com
I didn't mean to imply recursion being equivalent to sending IPs to self. The two are obviously different, hence my original statement: "...not allowing component-level recursion **and/or** sending IPs to self."

While I see no reason why it should be prohibited, I still don't see a viable use case for it either. In your example, if B and C were to be combined, wouldn't you just wrap the two in a composite component with a SUBIN/SUBOUT? Otherwise, if the two were to be combined as a non-composite component, then there's no reason to send an IP to itself; as Ged called it out, it "is just looping." Why not just do that in code? In fact, if ease of development (or should I say refactoring in this case) is the objective, why even bother combining the two into a non-composite component in the first place?

Sam Watkins

unread,
Mar 16, 2015, 11:53:57 PM3/16/15
to flow-based-...@googlegroups.com
On Mon, Mar 16, 2015 at 08:18:37PM -0700, Kenneth Kan wrote:
> ... sending IPs to self ...
>
> While I see no reason why it should be prohibited, I still don't see a
> viable use case for it either.

It can be useful to send a component's state to itself in an IP, rather
than concealing it internally, so that in each step it behaves as a pure
function of its inputs rather than a black box with hidden state.

IPs can be implemented as references / pointers, so there is no need to
copy the state at each step.

Indeed, rather than prohibiting self-directed IPs, it might be good to
prohibit "internal state" (other than temporary state), and express all
state in self-directed IPs. These state loops would then be very common.

This can ease analysis, e.g. proof of correctness, optimization.
We can see the entire state of a process just by looking at the IPs.
We can "suspend" a process by writing the IPs (in their channels) out to
a file. We can move running components to a different machine.

It also can ease debugging, you can insert a component to trace the
changing state of another component. You can watch the state IP for a
certain condition, or modify it if that would be useful. For example,
we might want to "reset" a component part way through the process,
or to start a random number generator with a certain seed.

A looping state IP can be received on the same port that receives an
"IIP" or initial info packet. The IIP contains initial state including
settings, and this is sent around in a loop at each step with any
changes to the state. It seems a good idea to allow state to be
included in the IIP.

These state loops could optionally be hidden from a flow diagram to keep
it simple, perhaps giving the component a bold border to indicate that
it has a hidden state IP loop.


Sam

Ged Byrne

unread,
Mar 17, 2015, 1:52:13 AM3/17/15
to flow-based-...@googlegroups.com, Sam Watkins
Hi John,

The program counter shared state, not preserved.

In recursion a function passes control to another instance of itself and continues when it returns, it's state preserved.

FBP looks like this;

P -> P

We move forward, never looking back.

Recitation looks like this:

PPPPPPPPP
v ^
PPPPP

The first instance of P persists and we return to it later.

regards,


Ged


--
You received this message because you are subscribed to the Google Groups "Flow Based Programming" group.
To unsubscribe from this group and stop receiving emails from it, send an email to flow-based-programming+unsub...@googlegroups.com.

Ged Byrne

unread,
Mar 17, 2015, 2:08:57 AM3/17/15
to flow-based-...@googlegroups.com, Sam Watkins
To be more precise FBP looks like this:

PPPPPPPPPP
v ^
IP

There is just one instance of P, with its own continuous state.
To unsubscribe from this group and stop receiving emails from it, send an email to flow-based-programming+unsubscri...@googlegroups.com.

Ged Byrne

unread,
Mar 17, 2015, 4:30:57 AM3/17/15
to flow-based-...@googlegroups.com, Sam Watkins
Please accept my apologies for all the strange autocorrect errors. The only time I have to post is on the bus during my commute.

John Cowan

unread,
Mar 17, 2015, 7:52:47 AM3/17/15
to flow-based-...@googlegroups.com
Sam Watkins scripsit:

> It can be useful to send a component's state to itself in an IP, rather
> than concealing it internally, so that in each step it behaves as a pure
> function of its inputs rather than a black box with hidden state.

The trouble with that is that all "external" state is internal to some
component or set of components, from the point of view of the framework.
You don't want to keep your whole filesystem or database reverberating
in the IP network. So this works for small amounts of state, but not
large amounts.
The Imperials are decadent, 300 pound free-range chickens (except they have
teeth, arms instead of wings, and dinosaurlike tails). --Elyse Grasso

Tom Young

unread,
Mar 17, 2015, 10:38:50 AM3/17/15
to Flow Based Programming
I should have said 'related' instead of 'equivalent'.  My apologies.  The issues of state, recursion, and self directed IPs should be treated separately, in my opinion, even though it may be possible to, for example, avoid saving state within components by embedding state within IPs. 

I am not familiar with SUBIN/SUBOUT.  In any case, I would not make avoidable component changes.  

Sam's response below answers the use case question for self-directed IPs very well, I think.

Another use case for self-directed IPs, might be converting a monolith to use FBP.   It might be useful to introduce IPs prior to splitting the monolith into components.

"Self-directed IP" is somewhat of a misnomer(but useful, anyway), since IPs don't direct and components don't decide where IPs go to or come from.

To me, FBP is an increasingly important concept.  In just one particular, FBP saves the effort involved in creating the fragile, hierarchical sofware structures of the past.   The fewer artificial restrictions and demands placed on the concept, the more widely it can be applied.



Tom Young
47 MITCHELL ST.
STAMFORD, CT  06902


When bad men combine, the good must associate; ...
  -Edmund Burke 'Thoughts on the cause of the present discontents' , 1770



On Mon, Mar 16, 2015 at 11:18 PM, Kenneth Kan <ken...@gmail.com> wrote:
I didn't mean to imply recursion being equivalent to sending IPs to self. The two are obviously different, hence my original statement: "...not allowing component-level recursion **and/or** sending IPs to self."

While I see no reason why it should be prohibited, I still don't see a viable use case for it either. In your example, if B and C were to be combined, wouldn't you just wrap the two in a composite component with a SUBIN/SUBOUT? Otherwise, if the two were to be combined as a non-composite component, then there's no reason to send an IP to itself; as Ged called it out, it "is just looping." Why not just do that in code? In fact, if ease of development (or should I say refactoring in this case) is the objective, why even bother combining the two into a non-composite component in the first place?

--

Tom Robinson

unread,
Mar 17, 2015, 4:46:10 PM3/17/15
to flow-based-...@googlegroups.com
On Mon, Mar 9, 2015 at 7:26 AM, Paul Morrison <jpau...@gmail.com> wrote:

Just to clarify, the prototype you refer to is sbp, right?

Yeah, "sbp" (temporary name, stands for "stream-based programming", but I now think it can be considered FBP).

In the discussion on Github, I said that we have been using back pressure, selective receive, and IPs with well-defined lifetime and owners, as a litmus test for classical FBP.  We could also consider banning one-to-many connections, although IMO an implementation that did automatic cloning in this case would still be classical - it just wouldn't make much sense!

It currently doesn't prevent one-to-many connections, and doesn't do automatic cloning (or immutability), so that's something that should be fixed.

I think array ports are pretty important - I assume you support these, as they are in JSFBP.

 Yes, it has array ports (it can actually have *nested* array ports, though John Cowan pointed out array ports are basically just syntactic sugar anyway)

At another level, the implementation should conform to the two constraints described in my book - "order-preserving constraint " and "flow constraint" - Chap. 3.  I can't imagine your implementation breaking these!

Yes those should be fine.

So, assuming all of the above, it certainly sounds like a "classical" implementation!  The multiple stack technique was the only way I had found to implement classical FBP, but it sounds like you can do something similar with whatever  magic you use!  As John pointed out,  multiple stacks actually allow you to do sends and receives from any level in a process stack - do you also allow this?

I think the "magic" is Promises, which are mostly hidden by the use of generators. "send" and "receive" both return promises, and when you "yield" a promise inside a component it suspends the execution of the component until the promise is (asynchronously) resolved, at which point the component resumes, with the "yield" keyword returning the value of the resolved promise.

"send" and "receive" can be done at any level in the stack as long as each function in the stack returns a promise and that promise is eventually yielded. You can nest generators, however while SBP automatically wraps the "top" level function of the component with one of the previously mentioned "coroutine" adapters (from Bluebird, Q, co, etc) it is up to you to wrap any others, e.x. a contrived example:

let anotherFunc = Promise.coroutine(function*(proc) {
  let ip = yield proc.input("IN").receive();
  yield proc.output("OUT").send(ip);
);

function* myComponent(proc) {
  let result = yield anotherFunc(proc);
}

ES7's async/await syntax will replace the need for the "Promise.coroutine" with the "async" keyword

Couple of questions : a) could you document how to write a component for people who aren't familiar with all these new facilities?

Yes I'll try to do that soon.

b) can your components call ordinary JS routines without special syntax?

Basically yes.

* If routine is not asynchronous then yes, nothing special needs to be done.
* If it is asynchronous via returning a promise then a component can "yield" the result (the promise) just like with send and receive.
* If it is asynchronous via a callback, you'd need to adapt it using Promises manually, or wrap it with a helper library (Bluebird, Q, etc). e.x. https://github.com/petkaantonov/bluebird/blob/master/API.md#promisefromnodefunction-resolver---promise

It seems like a lot of projects are moving to Promises these days, and with ES7's async/await even more will.

Looks like a great job!

Thanks!

Tom Robinson

unread,
Mar 17, 2015, 4:46:44 PM3/17/15
to flow-based-...@googlegroups.com
BTW, for kicks (and because I want to learn Go) I ported sbp to about 80 lines of Go using Goroutines and Channels: https://gist.github.com/tlrobinson/8ea774a0e014d7b9d667#file-sbp-go

Pretty neat little language. It's about 5x faster than jsfbp and 10x faster than the JS version of sbp, though it doesn't yet have IP ownership, "real" array ports, or buffered connections (which should be easy to add: http://openmymind.net/Introduction-To-Go-Buffered-Channels/ )

One neat thing you could do with Goroutines is essentially layer on "reactive" FBP: https://gist.github.com/tlrobinson/8ea774a0e014d7b9d667#file-reactive-example-go

Tom Robinson

unread,
Mar 17, 2015, 6:24:18 PM3/17/15
to flow-based-...@googlegroups.com
Also here's a good talk on promises, ES6 generators and iterators, and ES7 async/await and observables: https://www.youtube.com/watch?v=DqMFX91ToLw&feature=youtu.be&t=10m27s

And a lot more of the theory here https://github.com/kriskowal/gtor

Sam Watkins

unread,
Mar 17, 2015, 11:48:11 PM3/17/15
to flow-based-...@googlegroups.com
On Tue, Mar 17, 2015 at 07:52:42AM -0400, John Cowan wrote:
> Sam Watkins scripsit:
>
> > It can be useful to send a component's state to itself in an IP, rather
> > than concealing it internally, so that in each step it behaves as a pure
> > function of its inputs rather than a black box with hidden state.
>
> The trouble with that is that all "external" state is internal to some
> component or set of components, from the point of view of the framework.
> You don't want to keep your whole filesystem or database reverberating
> in the IP network. So this works for small amounts of state, but not
> large amounts.

I think that is a very small "trouble" as apposed to a large benefit.

The filesystem and database can be considered external to the network.
If the process is suspended for example it is necessary to keep it in a
similar context to its original context, i.e. don't make incompatible
changes to the files it is using or the database rows it is using.

IO to users, hardware devices, and remote servers also needs to be
considered, again it's necessary to resume the process in a similar
context.

It might also be possible to consider files and database resources as a
special type of IP object. It's not necessary to convert them to a flat
object in memory, just have IPs with references to resources,
representing those resources.

A complex component implemented as a sub-graph would have its own
internal state, consisting of the IPs in its internal channels.

It would be possible to represent this state as an IP of sorts, it is a
set of named channels each containing a list (or queue) of IPs. This
would be realized as a data structure with references to the channels
(and to the IPs). This complex IP could be flattened and serialized if
necessary, so a complex component could be transferred to another server
while it is running, for example.

I'm not sure if it would be useful to do this or not, we could also
consider complex components to be "transparent", and only worry about
externalizing state of the primitive components.

Reply all
Reply to author
Forward
0 new messages