function componentFunction() {
inport = /* ... */
outport = /* ... */
while (ip = inport.receive()) {
// do something and send
outport.send(ip);
}
}
// JavaScript
function *componentFunction() {
var inport = /* ... */;
var outport = /* ... */;
var ip;
while ((ip = yield inport.receive()) !== null) {
// do something
yield outport.send(ip);
}
}
--
You received this message because you are subscribed to the Google Groups "Flow Based Programming" group.
To unsubscribe from this group and stop receiving emails from it, send an email to flow-based-progra...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "Flow Based Programming" group.
To unsubscribe from this group and stop receiving emails from it, send an email to flow-based-programming+unsub...@googlegroups.com.
Yes, but do generators support multiple stacks?
The other Paul
On Mar 8, 2015 5:16 PM, "Tom Robinson" <t...@tlrobinson.net> wrote:
>
> What else do you need to demonstrate a system is considered "classical FBP"?
>
I think I already suggested a test: why not take my fbptest01, modify the three components involved as per your proposals, modify the network to process a few million IPs; now run your network, and compare its performance with the same network using the present version of JSFBP.
I would be very interested in the results.
--
On Mar 8, 2015 7:46 PM, "Tom Robinson" <t...@tlrobinson.net> wrote:
>
> Does the definition of "classical FBP" include some arbitrary performance metric?
I am rather a pragmatic type, so I just thought: let's try Tom's solution with a few million IPs. Did you run the test - it should have taken less time than it took you to write your note!
Last time I ran my test, it averaged just under 1 microsec per send/receive pair on my desktop, so 1,000,000 IPS going through 2 send/receive pairs should take around 2 secs. This is in the project Readme.
So I thought it would be interesting to see if yours runs slower or faster than JSFBP! Looking forward to hearing your results.
Regards,
Paul
--
You received this message because you are subscribed to the Google Groups "Flow Based Programming" group.
To unsubscribe from this group and stop receiving emails from it, send an email to flow-based-progra...@googlegroups.com.
That performance is pretty impressive, and is exactly what I wanted to know.
Just to clarify, the prototype you refer to is sbp, right?
In the discussion on Github, I said that we have been using back pressure, selective receive, and IPs with well-defined lifetime and owners, as a litmus test for classical FBP. We could also consider banning one-to-many connections, although IMO an implementation that did automatic cloning in this case would still be classical - it just wouldn't make much sense!
I think array ports are pretty important - I assume you support these, as they are in JSFBP.
At another level, the implementation should conform to the two constraints described in my book - "order-preserving constraint " and "flow constraint" - Chap. 3. I can't imagine your implementation breaking these!
So, assuming all of the above, it certainly sounds like a "classical" implementation! The multiple stack technique was the only way I had found to implement classical FBP, but it sounds like you can do something similar with whatever magic you use! As John pointed out, multiple stacks actually allow you to do sends and receives from any level in a process stack - do you also allow this?
Couple of questions : a) could you document how to write a component for people who aren't familiar with all these new facilities?
b) can your components call ordinary JS routines without special syntax?
Looks like a great job!
Comfreek, Matt, Ken, others?
Regards,
Paul
Thanks for starting this topic, Comfreek!
On Mar 8, 2015 11:28 AM, "ComFreek" <info.c...@gmail.com> wrote:
>
> Some languages (or language additions) allow one to use the following pseudo code:
>
> function componentFunction() {
> inport = /* ... */
> outport = /* ... */
>
> while (ip = inport.receive()) {
> // do something and send
> outport.send(ip);
> }
> }
>
> Note that the act of receiving an IP or sending one could be possibly blocking (no IP arrived yet and the connection of the Outport has reached its capacity, respectively).
> The point is that this information is not strictly conveyed in the code above, whereas some languages even force you to use some other syntax in order to allow for blocking calls:
My point is that the component and network developers do not need to know this. FBP processes are like Dijkstra's sequential processes - they don't care if they are stretched in time! A send is a send, whether instantaneous or stretched in time!
>
> Eventually, the whole question also boils down to: "Is a non-leaky abstraction possible (in JS)?"
>
Sorry, Comfreek, what does this mean? :-(
Best regards,
Paul
--
To unsubscribe from this group and stop receiving emails from it, send an email to flow-based-programming+unsub...@googlegroups.com.
On Mar 9, 2015 11:14 AM, <co...@ccil.org> wrote:
>
> J. Paul Morrison scripsit:
>
> > In the discussion on Github, I said that we have been using back pressure,
> > selective receive, and IPs with well-defined lifetime and owners, as a
> > litmus test for classical FBP.
>
> I think that the ability to send and receive from within nested calls
> is also essential, which is why Python generators don't qualify.
> But see below.
See where?
>
> OTOH, I think explicit disposition of packets is *not* essential; a
> purely garbage-collected system is fine.
Well, in my implementations I keep track of the number of IPs that a process owns, and I check to make sure this number has returned to zero before the process deactivates. I wouldn't want to give this facility up!
>
> > We could also consider banning one-to-many
> > connections, although IMO an implementation that did automatic cloning in
> > this case would still be classical - it just wouldn't make much sense!
>
> I don't think packet mutability is a requirement of classical FBP, and
> a system with immutable packets might well do one-to-many connections.
A bottle travelling through a bottling factory is "mutable" - given the ownership rules of FBP, isn't immutability just an extra safeguard - at some cost in overhead?
And one-to-many still implies automatic cloning - which could get veeery expensive, a s e.g. in the case of IP trees.
>
> > I think array ports are pretty important - I assume you support these, as
> > they are in JSFBP.
>
> Array ports are syntactic sugar: there's no difference between a single
> array port of size 10 and ten separate systematically named ports.
I agree, with the stress on 'systematically'!
>
> > At another level, the implementation should conform to the two constraints
> > described in my book - "order-preserving constraint " and "flow
> > constraint"
>
> +1. Systems that violate the flow constraint also violate causality!
>
> > So, assuming all of the above, it certainly sounds like a "classical"
> > implementation! The multiple stack technique was the only way I had found
> > to implement classical FBP, but it sounds like you can do something
> > similar
> > with whatever magic you use! As John pointed out, multiple stacks
> > actually allow you to do sends and receives from any level in a process
> > stack - do you also allow this?
>
> It's a mechanical transformation provided that there is a maximum
> recursion depth: that's the trick of Jackson Structured Programming.
> Some Scheme compilers do it automatically.
>
Sorry, John, I don't see how recursion fits in - what question are you answering?
Regards,
Paul
I prefer just writing code as proof. I will happily attempt to implement any cFBP program you throw at me, without using fibers. I've already done collate (https://github.com/tlrobinson/sbp/blob/master/components/collate.es6). What else do you need to demonstrate a system is considered "classical FBP"?
On Mar 10, 2015 9:49 AM, <co...@ccil.org> wrote:
>
and no
> loops whose length depends on the input data. (This isn't as bad as
> it sounds: a loop or recursion that doesn't contain a send or receive
> doesn't matter in this context.) A loop can always be
> reduced to a recursive function call, so we can speak of just
> recursion without loss of generality.
Sorry, John, still lost! If the "canonical" FBP component is Copier, surely this is a "loop with length dependent on its input"...? Which brings me back to Collate, which caused so much trouble for NoFlo - throw looping, selective receive and back pressure into the mix, and I think it's pretty hard to do without multiple stacks.
That said, in a green thread implementation, each segment between yield and return is pretty much independent, and could theoretically use the same stack... if it weren't for subroutine local storage. So what if we allocated that from the heap, using a dictionary to relate the subroutine (plus its entire call hierarchy) to the subroutine's local storage... Not pretty, though!
It's ironic that we have to go to so much trouble to accommodate von Neumann machines, when non-von is so much more natural! :-) Some 55 years ago there was work going on on "streaming machines", but the von Neumann machine won conclusively! Of course in those days there was no real reason to try something different! Little did we know!
Regards,
Paul
--
node A with 1 outport
node B with 1 outport
node C with 2 inports
node A outport ---> node C inport-1 ; bounded FIFO size set to 50
node B outport ---> node C inport-2 ; bounded FIFO size set to 50
Both node A and B runs in infinite loops, sending out IPs to node C. The IP can contain something as trivial as a 32-bit integer.
Node C performs the selective port read from inport 1 and 2, alternating every 5 seconds:
while ()
{
while (not 5 seconds yet)
{
read from port 1
}
while (not 5 seconds yet)
{
read from port 2
}
}
This sounds like a good test for the back pressure thing as while node C is reading port 1 the
node B ---> node C bounded FIFO will fill up quite fast and node B would be blocked from sending
more IPs.
I modified your suggestion to reduce the amount of output (pause reader 100ms between each read, FIFO size = 10), but I think it demonstrates what you want:
This brings up another question I've had: is immediate/precise control of back pressure a requirement for correctness in classical FBP, or is it acceptable to have to buffer a few extra IPs?i.e. how do you think of ports?* are they "push" streams which push IPs downstream until they are told to pause* are they "pull" streams that explicitly request an IP from an upstream process?
Here's a summary of the problem with Node Streams:"This is a problem in node streams, pause is only transfered on write, so on a long chain (a.pipe(b).pipe(c)), if c pauses, b will have to write to it to pause, and then a will have to write to b to pause. If b only transforms a's output, then a will have to write to b twice to find out that c is paused."
--
Tom
Young
47 MITCHELL ST.
STAMFORD, CT 06902
When
bad men combine, the good must associate; ...
-Edmund
Burke 'Thoughts on the cause of the present discontents' , 1770
--
On Mar 11, 2015 3:51 PM, "Matthew Lai" <mm...@sympatico.ca> wrote:
>
>
> I also buffer 1 extra IP for each bounded FIFO buffer, and find the push style IP delivery easier to understand and implement :)
>
> We'll see what Mr. Morrison has to say here.
>
Sorry, Matt and Tom, I don't quite understand what is meant by "1 extra IP". All my implementations so far push IPs onto a connection, but the receiving process will obtain them immediately - it doesn't wait until the connection is full.
Or are you saying that a send is not suspended, but simply reports that a connection is full? But then surely it becomes the responsibility of the process to decide whether to send or not... whereas in my mind a send is a send - whether "stretched in time" or not.
Sorry, Matt and Tom, I don't quite understand what is meant by "1 extra IP".
All my implementations so far push IPs onto a connection, but the receiving process will obtain them immediately - it doesn't wait until the connection is full.
Or are you saying that a send is not suspended, but simply reports that a connection is full?
On Mar 12, 2015 9:23 PM, "Matthew Lai" <mm...@sympatico.ca> wrote:
>
>
>
> On Thursday, 12 March 2015 18:58:16 UTC-4, Paul Morrison wrote:
>>
>> Sorry, Matt and Tom, I don't quite understand what is meant by "1 extra IP".
>
> Oh, I mean when I set the capacity of the bounded FIFO to be 50, it can really stores 51 IPs :)
>
Is there some advantage to this? Just curious!
>>
>> All my implementations so far push IPs onto a connection, but the receiving process will obtain them immediately - it doesn't wait until the connection is full.
>
> The same behaviour with my implementation :)
>>
>> Or are you saying that a send is not suspended, but simply reports that a connection is full?
>
> Nope. Definitely a blocking send in my implementation.
Sounds good! Tom, do you do the same?
>
> Yours,
>
> Matt
>
--
On Mar 13, 2015 10:42 AM, "Tom Young" <f...@twyoung.com> wrote:
>
> The one extra IP buffer is interesting. In my implementation, DFD, the DFD dynamically creates one extra IP when it detects a deadlock. Each output port can have just one buffer added this way. Otherwise connections are generally not buffered, just space for one IP per port plus possibly this one extra space per output port.
>
Presumably you only do this once...? I assume you are not talking about automatic "ballooning", although it turns out that in certain "big data" handling situations, a sophisticated type of ballooning may be necessary - see Chap. 16 of the 2nd ed., "Parallelism Issues", contributed by Mike Beckerle.
> This allows a component to send IPs to itself without deadlock, so long as it consumes each output IP immediately.
I don't support zero-length connections, so all my implementations support this.
>
> If one to many connections were permitted, this would become complex, so that is not allowed, instead a generic Split component replicates IPs, if necessary. It is theoretically possible to automatically insert Split at run time, though.
Agree... or you can have special-purpose splitters, if you are working with IP trees, for instance.
>
> ---The other Tom
>
Presumably you only do this once...?
I assume you are not talking about automatic "ballooning", although it turns out that in certain "big data" handling situations, a sophisticated type of ballooning may be necessary - see Chap. 16 of the 2nd ed., "Parallelism Issues", contributed by Mike Beckerle.
Yes, there is no sophisticated buffering, total memory requirements are easily estimated, and avoidable deadlocks are avoided, at some possible performance cost with fine-grain IPs.
> This allows a component to send IPs to itself without deadlock, so long as it consumes each output IP immediately.
I don't support zero-length connections, so all my implementations support this.
Generators also break that assumption, but you have to explicitly opt-in by declaring a generator function (with the "function*" syntax) and use the "yield" keyword, so there's no chance of someone else's code deep in some library you're using yielding execution without your knowledge.
You're right! I was probably being over-protective! I just think that allowing a process to send to itself is just very error-prone. And its very synchronous... The current implementations support this, but DrawFBP doesn't. Just being pragmatic, I guess! But really why would you need to do this? If it's some kind of simulation of recursion, IMHO it just feels alien to FBP!
From snother point of view, I don't really see the need for recursion in the FBP context, but I don't think FBP cares... (does it or should it?)
Regards,
Paul
Tom
Young
47 MITCHELL ST.
STAMFORD, CT 06902
When
bad men combine, the good must associate; ...
-Edmund
Burke 'Thoughts on the cause of the present discontents' , 1770
--
--
You received this message because you are subscribed to the Google Groups "Flow Based Programming" group.
To unsubscribe from this group and stop receiving emails from it, send an email to flow-based-programming+unsub...@googlegroups.com.
To unsubscribe from this group and stop receiving emails from it, send an email to flow-based-programming+unsubscri...@googlegroups.com.
Tom
Young
47 MITCHELL ST.
STAMFORD, CT 06902
When
bad men combine, the good must associate; ...
-Edmund
Burke 'Thoughts on the cause of the present discontents' , 1770
I didn't mean to imply recursion being equivalent to sending IPs to self. The two are obviously different, hence my original statement: "...not allowing component-level recursion **and/or** sending IPs to self."While I see no reason why it should be prohibited, I still don't see a viable use case for it either. In your example, if B and C were to be combined, wouldn't you just wrap the two in a composite component with a SUBIN/SUBOUT? Otherwise, if the two were to be combined as a non-composite component, then there's no reason to send an IP to itself; as Ged called it out, it "is just looping." Why not just do that in code? In fact, if ease of development (or should I say refactoring in this case) is the objective, why even bother combining the two into a non-composite component in the first place?
--
Just to clarify, the prototype you refer to is sbp, right?
In the discussion on Github, I said that we have been using back pressure, selective receive, and IPs with well-defined lifetime and owners, as a litmus test for classical FBP. We could also consider banning one-to-many connections, although IMO an implementation that did automatic cloning in this case would still be classical - it just wouldn't make much sense!
I think array ports are pretty important - I assume you support these, as they are in JSFBP.
At another level, the implementation should conform to the two constraints described in my book - "order-preserving constraint " and "flow constraint" - Chap. 3. I can't imagine your implementation breaking these!
So, assuming all of the above, it certainly sounds like a "classical" implementation! The multiple stack technique was the only way I had found to implement classical FBP, but it sounds like you can do something similar with whatever magic you use! As John pointed out, multiple stacks actually allow you to do sends and receives from any level in a process stack - do you also allow this?
Couple of questions : a) could you document how to write a component for people who aren't familiar with all these new facilities?
b) can your components call ordinary JS routines without special syntax?
Looks like a great job!