Flows and state
Ralf Westphal
in a flow, then it´s not a single flow anymore."
I´d like to know you opinion on that. How about shared state in a FBP
network? Can there be a flow like
A -> B -> C -> D
where A and C share some state? Or would that be two flows?
Would it depend on whether the state is just read or also written to?
Aristophanes
really bad idea. Shared state is the creature from the deeps that
eats threaded programming. This is not to say that you can't the
effect. There are at least three different ways to get the effect.
The first is to include the state in the IP. The second is to have a
separate process send the state to the two processes; this works
nicely for read-only. The third is to have the two sharing processes
ship the state back and forth as it is modified. (This is known as
getting the downside of threads without the upside.)
In short, no, no, and no. The reason that the FBP works (and threads
don't) is that each process is an isolated autonomous computing
element without hidden dependencies.
Paul Morrison
anything like that quote - I don't even use the phrase "shared
state"... I have to confess I don't know exactly what it means -
perhaps someone else said it...?
And, Aristophanes, I agree with you wholeheartedly - as you say, one
of the neat things about FBP is that each process is an isolated
autonomous computing element without hidden dependencies - i.e. a
black box that can run on its own processor just as well as on a
shared processor. I think we followed all 3 of the strategies you
mention for various purposes - one example of #2 that cropped up
fairly often was that of a shared table, where one process builds the
table, and one or more read from it - as you say in read-only mode
(but not until after the table was complete). If you have read
Chapter 17 - http://www.jpaulmorrison.com/fbp/minilang.shtml - you
will remember that we used tables (especially "class code" tables)
quite a bit to facilitate what Ralf appropriately calls evolvability.
Strangely enough, Aristophanes, right after this discussion, in the
new edition there is a section, contributed by yourself, where, among
other things, you talk about shared resources - thanks again,
Aristophanes!
John Cowan
> Ralf, I have to say that I have absolutely no recollection of saying
> anything like that quote - I don't even use the phrase "shared
> state"... I have to confess I don't know exactly what it means -
> perhaps someone else said it...?
Shared state consists of variables or objects which are writable by
more than one process in such a way that the changes are visible in all
processes that examine them.
In order to manage shared state, it's necessary to regulate access to
it. The power of FBP is that it confines shared state to the buffers
inside connections, whch are accessed only in a highly disciplined
fashion. Of course, JavaFBP and C#FBP don't actually prevent the use of
shared state, but they make it unnecessary.
--
Unless it was by accident that I had John Cowan
offended someone, I never apologized. co...@ccil.org
--Quentin Crisp http://www.ccil.org/~cowan
Paul Morrison
didn't know what Ralf's quotation meant - I do know what shared state is
:-)
Ralf Westphal
small example scenario:
Imagine a questionnaire application. When started it offers a list of
questions to answer. Once all questions have been answered a score is
shown, how many questions got answered correctly. It´s a desktop
application with a small UI.
Feature 1: After startup a questionnaire is loaded from a simple file
and shown as a list of n questions each with m answer options of which
only one is the correct answer.
Feature 2: The user answers questions by selecting one of their answer
options through some interaction.
Feature 3: Once all questions have been answered a dialog box is
displayed telling the user how many correct answer she gave.
That´s a very small application. No database needed. The questionnaire
can be given as a XML file or even plain text file.
Now my question is: how would you model this app using FBP? There is
state involved: the selected answer options. They are runtime state
shared at least by Feature 2 and 3.
Looking forward to your suggestions.
Ged Byrne
Ralf Westphal
each questions?
Surely you do that in "Input Answers". But that´s the wrong place. It
´s domain logic to allow only one answer per question. So that should
not be part of the UI. It would be a lack of Separation of Concerns.
Hence the questions and their answer options need to be kept
elsewhere.
Now, how do you model that?
On 9 Feb., 11:21, Ged Byrne <ged.by...@gmail.com> wrote:
> Ralf,
>
> I must be missing somethign subtle because I'm not seeing the need for
> shared state.
>
> Here's my shot at the answer.
>
> I've shown Feature 3 as a report. That report happens to be displayed in
> a dialogue box.
>
> What am I missing?
>
> Regards,
>
> Ged
>
>
>
>
>
>
>
>
>
>
> > Now my question is: how would you model this app using FBP? There is
> > state involved: the selected answer options. They are runtime state
> > shared at least by Feature 2 and 3.
>
> > Looking forward to your suggestions.
>
>
>
> 20KAnzeigenHerunterladen
Ged Byrne
Tom Young
-Tom
"...the shell syntax required to initiate a coprocess and connect its input and output to other processes is quite contorted..."
W. Richard Stevens [Advanced Programming in the Unix Environment, p441]
Aristophanes
I agree with Tom. That said, I opine that these particular "UI
concerns" are part of the domain logic. What we are talking about is
the structure of the test and the scoring. That is domain knowledge.
How the answers actually get entered is the UI concern.
On Feb 9, 4:48 pm, Tom Young <f...@twyoung.com> wrote:
> It should be simple enough to separate the UI concerns from the domain
> logic by splitting 'Input Answers' into two components.
>
> -Tom
>
>
>
>
>
>
>
>
>
> On Thu, Feb 9, 2012 at 4:09 PM, Ged Byrne <ged.by...@gmail.com> wrote:
> > Sorry Ralf but I disagree. I think your abstracting prematurely and
> > without good reason.
>
> > I won't separate concerns until the problem domain has enough of them to
> > require separation. Until then I'll keep it simple.
>
> > If you'd like to give me some more specific requirements, then I'll update
> > the model to satisfy them.
>
> > I'm not going to introduce complexity just because somebody somewhere has
> > declared "though shalt separate the concerns."
>
> > I take these principles from XP rather than FBP. The simplest thing that
> > could possible work because you ain't gonna need it.
>
> > I'd be interested to hear how others feel.
>
> > Regards,
>
> > Ged
>
Ged Byrne
You speak of domains and concerns as if they are separate things.
They are the same. There are separate domains.
There is the Quiz domain, which is concerned with describing questions
and answers.
There is the UI domain that is concerned with describing user
interactions.
We also have the language domain that is concerned with parsing
instructions that have been encoded onto disk.
All of these domains have requirements based on specialised knowledge.
In No Silver Bullet Fred Brookes talked about the complexity of
software, and separated it into two types: the essential and
accidental.
The essential complexity comes from the requirements of the domains.
The accidental complexity comes from discussions like this. This is
why discussions like this should be avoided. The design should be as
simple as possible to satisfy the known requirements. Then it can be
shared with the domain experts so that they can give you the benefit
of their knowledge and provide the requirements for the next
iteration.
The results of accidental complexity introduced by navel gazing
developers can be seen in Conways law: "...organizations which design
systems ... are constrained to produce designs which are copies of the
communication structures of these organizations." http://en.wikipedia.org/wiki/Conways_Law
We should be aiming to avoid conways law and instead follow the
principle that form follows function: http://en.wikipedia.org/wiki/Form_follows_function
Regards,
Ged
Ged Byrne
Yes we could.
But should we?
Regards,
Ged
Paul Morrison
obvious topology would be a simple loop. If I understood the previous
discussion, it was about how to divide the answer logic between
components, and I definitely agree with the YAGNI principle. As Wayne
Stevens used to say, don't generalize until you have done the same job 3
times.
That said, however, instead of, as Ralf suggested, restricting the
problem to using small in-storage tables of questions and answers, let
us assume a database comprising the questionnaire and possible answers
sitting on the cloud, with the desktop app looking after formatting.
Now you could have the server generate HTML, or you can just store raw
data in the cloud, and use some of the processing power on the users'
desktops to do the formatting. The server side of this app might then
look something like the diagram shown on the cover of the 2nd edition of
my book - although it shows multiple back-ends, which might not be
necessary in Ralf's example. In this case, the server is servicing
multiple users, and the two sections of the attached diagram are running
on separate machines. An app which only services a single user,
running on a single machine, could of course be simpler, but you might
still want to allow for the possibility of migrating to a server
arrangement at some later date - most of the components would not need
to change...
One last thought: in light of the thread about FBP on hardware, perhaps
we should visualize FBP components running on different microprocessors,
and see what that does to our designs!
Tom Young
Tom,
Yes we could.
But should we?
Yes, in this case we should, because the point is to demonstrate a 'shared state' situation. If all functionality is confined to one component, or the UI and the logic issues are separated, then the 'shared state' issue is hidden from view.
I think the shared state issue will not be a problem in a well-designed FBP -- just keep state in the appropriate component (or IP). The object here is to show what that design might look like.
Regards,
-Tom
Regards,
Ged
On Feb 9, 10:48 pm, Tom Young <f...@twyoung.com> wrote:
> It should be simple enough to separate the UI concerns from the domain
> logic by splitting 'Input Answers' into two components.
>
> -Tom
--
Tom Young
47 MITCHELL ST.
STAMFORD, CT 06902
"When bad men combine, the good must associate; ..."
-Edmund Burke 'Thoughts on the cause of the present discontents' , 1770
This e-mail message from Tom Young is intended
only for the individual or entity to which it is addressed. This e-mail
may contain information that is privileged, confidential and exempt from
disclosure under applicable law. If you are not the intended recipient,
you are hereby notified that any dissemination, distribution or copying
of this communication is strictly prohibited. If you received this
e-mail by accident, please notify the sender immediately and destroy
this e-mail and all copies of it.
Tom Young
-Tom
Ged Byrne
Tom Young
Tom,Then the only reason we are creating shared state artificially.
I'm saying that we should first define requirements that actually introduces the problem of shared state.
Paul's suggestion of making it a multi-user system does this.
Cheers,
On 10 February 2012 17:56, Tom Young <twy...@twyoung.com> wrote:
Yes, in this case we should, because the point is to demonstrate a 'shared state' situation. If all functionality is confined to one component, or the UI and the logic issues are separated, then the 'shared state' issue is hidden from view.
I think the shared state issue will not be a problem in a well-designed FBP -- just keep state in the appropriate component (or IP). The object here is to show what that design might look like.
Regards,
Ron Lewis
Ged Byrne
The argument was that separating out the UI would expose the issue or at least demonstrate how to address the issue. How else can we address the argument than by doing the separation?
I'm saying that we should first define requirements that actually introduces the problem of shared state.Perhaps we should agree on whether we are discussing the Distributed Shared State Concurrency Problem or some other shared state problem. If the former, then I would argue that the problem does not arise unless we imagine that a FBP is intended to survive (i.e. continue to produce correct results) despite one or more component nodes failing.
Paul's suggestion of making it a multi-user system does this.Paul's diagram does not tell us how the problem is addressed. It is possible to imagine that all state is confined to the IPs and all components are stateless; or, some component(perhaps 'Display Questions' is keeping track what is going on; or something in between.
Raoul Duke
> Even if you can somehow always eliminate a shared state, there would still
> be the question of whether elminating the shared state is desirable.
seems to me the trade off is: shared state vs. inconsistency. that is,
if the state is not shared but is duplicated then there is always some
period of time after a change when the duplicated state isn't the
safe.
Ged Byrne
Raoul Duke
Ron Lewis
Paul Morrison
state involved data. All data can be held in IPs. Period. QED.
Actually, we had one fairly common shared state situation that I
described earlier in this thread: shared tables - these are still IPs,
however.
I also think there is a well known shared state (?) situation at a
"coarser-grained" level: namely shared databases. Here, IMO, the whole
sharing issue has been very thoroughly explored in the area of
traditional transaction-based systems. However, you might not even need
that complexity, as Ralf's questionnaire and its possible answers are
pretty much read-only...
What am I missing?!
Aristophanes
key point that has to be kept in mind, which I quote:
"Shared state consists of variables or objects which are writable by
more than one process in such a way that the changes are visible in
all
processes that examine them. "
processes can be initialized with IIP's or they can be passed in
IP's. These techniques don't work for objects that are writable .
Shared state issues occur naturally in simulations. For example,
suppose I am modelling the climate over time. There are a host of
global factors, e.g., the solar constant, the concentrations of
various green house gasses, volcanic activity, blah, blah, blah,
etc. Then the world is partitioned up into various regions, each
with its own particular data sets. The regions are then classified
into interacting demes. The processes describing demes must get data
from the regions and from the global factors, all of which change over
time. In simulations we can manage all of this because the simulation
occurs in small steps of time. Within a single step all of this data
is constant.
In business applications one kind of thing that occurs to me that
might involve shared state issues is the processing of a document
(e.g. mortgage application) that requires a lot of different actions
that have dependencies on each other.
Paul Morrison
this thread, there are at least three different ways to get the
have a
separate process send the state to the two processes; this works
nicely for read-only. The third is to have the two sharing processes
ship the state back and forth as it is modified. (This is known as
getting the downside of threads without the upside.)
programming, where storage read-out is non-destructive, so it is hard
to figure out whether you have read a data value already, or have
clobbered something that has not yet been read. So, if you really
need completely shared state, as described in Aristophanes' quote from
John Cowan, and none of the three strategies listed applies, FBP
doesn't help much! Of course, I may be wrong! However, as I have
said before, a conventional program is an FBP network with only one
node, so FBP adds a powerful technique to your bag of tricks - it
doesn't take anything away.
John Cowan
> [T]here are at least three different ways to get the effect. The first
> is to include the state in the IP. The second is to have a separate
> process send the state to the two processes; this works nicely for
> read-only. The third is to have the two sharing processes ship the
> state back and forth as it is modified.
There is a fourth approach, provided each change can be made atomically
under the implementation language's memory model. All components that
need read-only access to the state have direct pointers to it, but there
is a dedicated component that updates the state as it receives command
IPs from other processes.
Note that this design provides eventual consistency; that is, it's
possible for a process to see an old version of the state because the
server component has not yet updated it. If that's good enough, as it
often is, then it works cleanly.
--
A rabbi whose congregation doesn't want John Cowan
to drive him out of town isn't a rabbi, http://www.ccil.org/~cowan
and a rabbi who lets them do it co...@ccil.org
isn't a man. --Jewish saying
Ged Byrne
I appreciate the issues of shared state of been thoroughly dealt with
in traditional transaction based systems, but I'd like to be able to
explore the issues here, if others are willing.
I'd like to avoid the general discussions about shared state because
such discussions tend to favour the status quo. What I'd like to do
is explore the problems through a realistic example.
Ralf's questionnaire was a good starting point, but it doesn't force
the issues enough. We need something where the problem itself demands
that we must have shared state, rather than having to introduce it
artificially.
I will start with something abstract and general. Let's say we have a
data structure that has classes X Y Z. The classes are related in a
tree data structure.
We also have processes A B C D E F & G and two flows:
A -> B -> C -> D
E -> F -> G
Both flows update X, Y and Z. Both flows need to see the latest data
from X, Y and Z. lets says that we have an IP being processed by both
flows.
Consider this scenario: each process has a reference to a single XYZ
instance in memory. In the ABCD flow it is in the queue waiting to be
processed by B. In the EFG flow it is waiting to be processed by G.
The scheduler chooses to run the B process next. The B process
updates XYZ.
Later the scheduler choose to run G, which works on the updated
version XYZ.
For the argument, let's say that the outcome of G is directly altered
by the update in B. Let's say that B set a flag that was important to
G.
This means that there is a connection between B and G through shared
state that is not shown in the diagram.
How do we deal with such a situation?
Do we forbid it? If so then how do we enforce that rule?
Do we annotate the diagram with free text? If so, would it be better
if we could annotate it in a way that was both readable to the machine
as well as humans? What use could the framework make of such
annotation?
Here's a concrete example to help the discussion. Imagine a patient
monitoring machine in a hospital. Each bed is hooked up to a computer
that monitors the patients health and updates a central server at
least 10 times every second. That is flow ABCD.
The nurses desk has a dashboard display that shows the latest health
of the patients that must be updated at least twice every second.
That is flow EFG.
Process G raises an alarm if certain conditions defined in a rule
database are met.
This example is taken from Michael Jackson's work on Problem Frames:
http://computing.unn.ac.uk/staff/cgpv1/downloadables/CD3005/SACJrnl4.pdf
Also, please accept my apologies for getting a bit preachy last week.
Internet discussion groups should be avoided for several days after
frustrating meetings at work. Having had to bite me tongue for
several hours while everybody took a clean simple design and piled on
the unnecessary complications (if we add five optional parameters here
we can avoid a whole network call there, etc) I'm afraid I needed to
let off steam. Sorry, I'll try not to let it happen again.
Regards,
Ged
Aristophanes
Your example sorts itself out quite nicely if you accept the principle
that in FBP there is no shared state. Processes (autons) can have
state. By their nature IP's have state. How then do you handle
potential shared state? The general answer is that you put the shared
state data in its own process. Thus, in your example, we create
another process called Q that holds XYZ. The flows then could be
A -> B -> C -> D
|
E -> F -> G
multiple read-only copies of XYZ that are references under the hood.
G might be automatically updated. It might query Q. There might be
an event driver (T) that tells Q to send an update to G. And so on .
Having a process to hold XYZ means that the operations on XYZ and the
queries about XYZ are all in one place. One further point. You only
need a separate process to hold XYZ if it is being updated from more
than one place. In your hospital example B can hold the state data so
the flow diagram could be:
A -> B -> C -> D
|
E -> F -> G
handle on shared state.
Dan
If there are at least two distinct processes, A and B, that modify a shared state variable then it means there is a flow from the process to the state variables or vice-versa. If you think the processes modify that "shared state" simultaneously then it means there is no correct FBP network defined. This should be impossible by design.
First we have to clarify which part is moving and where (the process itself or the state variable - IP)? Is the "shared" state variable moving, getting through the process A and later through the process B or the process (the component) itself is moving and process the "shared" state variable in a specific order.
Both are acceptable but one thing is certain: somebody has to move, things are moving in order through pipes (connections) and something is unique at a certain moment in time and space.
When you discuss about shared state then we have to make a distinction: any "shared" state in FBP is not quite shared, i.e. there is no moment in time when a state variable is accessible simultaneously by two different processes. If this is happening there is a collision that should be impossible in FBP that promotes order through connections.
If a state variable C has to be processed by two processes A and B, you have two choices:
1) Take the C and send it through A and later through B and back to its "place of residence"
2) Take the process A and B and send them in order over C
I am sure you will ask what (2) means. All the components being pure functional, pure data or mix have input and output ports. What it is important is the dynamic connection between them after they have travelled through pipes (connections).
A pure data component (e.g. C) has also ports like functional components: at least one for write and another distinct one for read. You manipulate the "shared" state variable C only through this distinct ports. Because you can connect to "write port" only one pipe at a time then you do not share the component C for writing but you can do for reading (i.e. it's like copying / duplicating the information). Or you can send in order two values through a pipe to C input (write) port.
Into a FBP network you should have all of these aspects clearly define. The bottom line is: you do not share state for writing, the processes execute in order and order is given only by the pipes. So, in fact I should see your "problem" into the FBP diagram. In your example there are connections that you discuss about but are missing in the diagram. If you draw them in the diagram soon you will realize what the problem is.
In the end I will solve the problem of multiple processing of the same component sending in order the processes to the component. It is like sending the mechanic to a car and not the car to the mechanic.
Ged Byrne
- Read only shared state can always be eliminated by making it an IP and creating copies when necessary.
- Read-Write shared state can always be eliminated by making it a separate process.
[...] The general answer is that you put the shared
state data in its own process. Thus, in your example, we create
another process called Q that holds XYZ. The flows then could be
A -> B -> C -> D
|
T -> Q
|
E -> F -> G
[...]
Ged Byrne
I am sure you will ask what (2) means. All the components being pure functional, pure data or mix have input and output ports. What it is important is the dynamic connection between them after they have travelled through pipes (connections).
A pure data component (e.g. C) has also ports like functional components: at least one for write and another distinct one for read. You manipulate the "shared" state variable C only through this distinct ports. Because you can connect to "write port" only one pipe at a time then you do not share the component C for writing but you can do for reading (i.e. it's like copying / duplicating the information). Or you can send in order two values through a pipe to C input (write) port.
[...]
Aristophanes
It's like the proofs that an arbitrary program with gotos can be
transformed into a structured program using just a while loop. The
proofs are correct; however the have a practical flaw. There are
unstructured programs that can only be converted into basic structured
programs with either arbitrarily large costs in space or in execution
time. However if you allow more powerful flow control constructs,
such as case selection and function invocation, then all unstructured
programs can be structured without significant extra cost.
The moral is that universal transformation rules may be effective in
principle but not in practice.
On the other hand, if you start by designing FBP programs you probably
aren't going to create really messy shared state problems.
On Feb 13, 12:41 pm, Ged Byrne <ged.by...@gmail.com> wrote:
> Aristophanes,
>
> Of course, it's obvious now you point it out.
>
> So can we say the following as a general rule:
>
> creating copies when necessary.
> - Read-Write shared state can always be eliminated by making it a
Dan
In my example C was a data component that is the "shared state". In FBP I do not make much difference between pure functional components (processes) and pure data components. Each one has input and output ports as any process has.
The component C in my example was manipulated by processes A and B. I've tried to give an example when two different processes want to modify a "shared state" C. If the logic is correct (i.e. I really have such a need) then I am saying there is no shared state at all in the first place. In fact C is not shared. You have two kinds of need:
1) Write to C (using input port)
2) Read from C (using output port)
The only choice you have is to move somebody in order to make the process possible.
Solutions:
1) Move C through the processes (A and B)
2) Move the processes A and B over (to) C
In case (1) the IP is C. It means it does not stay in one place, it has to move.
In case (2) the IPs are A and B.
When I am saying "place of C" you can interpret it as a "box" (as are boxed variables in different languages). The "box" is the equivalent of the C++ pointer in FBP if you want. This is indirection i.e. you can not access C directly but through its box (wrapper).
"Place of C" (ie. the box) could be important in many cases because there are some other components that want to connect to the same C. But C is not there sometimes :) or the process is coming to C (the mechanic is coming to the car). "Box of C" could be just another wrapper component to which C dynamically connects.
In the end C is not effectively shared. The FBP program (diagram) describes the logic of interaction and the order of process. The diagram is active, it modifies itself partially at runtime. In FBP you don't need semaphores and all the usual stuff common in many languages. The object itself is the semaphore. Sometimes it is not present inside the "house" or the processes are coming through the door one by one.
Aristophanes said "The moral is that universal transformation rules may be effective in principle but not in practice."
Indeed. That's why for instance, you don't duplicate the information when some "shared" data is needed for read by other components but it's like duplicating it. Sometimes it is obligatory to duplicate e.g. when the value read should be kept even if the original changes (value snapshot).
When different components need to be kept consistent then it means there is a connection among them, there is a flow that need to be described in the FBP network.
Conclusion: In FBP there is no moment in time when two different components simultaneously access for write another target component. The logic (FBP network) should describe exactly how the interaction occurs and the order of processes.
Regards,
Dan
Aristophanes
There are issues with that fourth approach. First issue. Significant
updates take time. I don't think that in place atomic updates are all
that easy to come by, particularly if the implementation uses multiple
threads. We can (I suspect must) generate a new copy for each
update. Second issue. How do we garbage collect? Since unknown
processes can access a copy at any time, how do we know that there
aren't extant pointers to a copy. Yes we can reference count, but
that's not the golden path to reliable code.
It helps to use ports to get a copy of the state. The idea is to send
a request to the process holding the state, which sends a copy back
the requesting process. (This may be cumbersome in the original FBP
but it is an easy mod.) Since the shipped state is a read only IP we
can set up a reliable reference count - that's the beauty of requiring
an explicit discard.
Raoul Duke
> It helps to use ports to get a copy of the state. The idea is to send
> a request to the process holding the state, which sends a copy back
> the requesting process.
i don't yet see how that is a silver bullet. what if some other
process also wants the data to manipulate? how does it get merged? or
are there locks? if there are locks, how do you coordinate locks to
avoid deadlock? or zombie locks? etc.
sincerely.
Paul Morrison
suspect mostly because you use your own terminology. I went to quite
a lot of trouble to define my terms, and even provided a glossary at
the end of the book. It may be that the terms I use are obsolete and
are causing misunderstandings - for instance the term "entities",
which we used for what are now called Information Packets (IPs),
turned out to cause confusion with database theory, so we changed the
term. It would be great if you could define your terms, and perhaps
even show how they map onto the ones I have been using for the last
few decades.
For instance, I do not know what a "pure data" component is: in FBP it
is true that an FBP component has ports, but how can it be called
"pure data"? A process is an executing instance of a component, and
always has behaviour. Similarly, in the next para, how does a process
manipulate a component? Are you referring to compilers? As far as I
know, compilers are the only things that manipulate read-only
components...
Please clarify...! TIA
Re the general discussion in this thread, I am much more comfortable
when we are talking about real-life applications. I liked the one
about patient measurements, and in fact this application resembles the
one Dr. Compatangelo described in Chap. 27 of the 2nd edition, called
Independent Assisted Living. I feel we can waste a lot of effort in
trying to solve abstruse "what if" scenarios, and would rather wait
until real problems come up that we have to solve. I suppose the
reverse view is that we don't want to be blind-sided by new problems
that we have not run into before, and for which we find the existing
solutions are inadequate. I guess that's why we have engineers *and*
scientists!
On Feb 14, 4:51 am, Dan <dpolo...@gmail.com> wrote:
> ... I
Aristophanes
Cowan's fourth alternative which I quote below. That said, your
questions don't make a lot of sense to me. Let answer them as best I
can:
Q1: what if some other process also wants the data to manipulate?
A1: No other process gets "the data". Other processes get copies of
the data. In the scenario in question the processes request read-only
copies that need not be the latest version. However the same
mechanism can read-write copies. In such case a full copy must be
created.
Q2: how does it get merged?
A2: This sounds like you are envisioning a situation where two
processes get read-write copies of the data, each one separately
updates the data, the update versions are merged, and merged versions
are sent back to the process that stores the data. This can be done
but it is separate from the problem of managing state data. Basically
you need to provide update processes and a merge process.
A key point to understand is that in FBP processes hold persistent
data; any piece of persistent data has a unique home process. Copies
of persistent data can be sent to other processes within IPs; however
these copies are transient. A process that has received an IP must
either send it on to some other process or discard it.
Q3: or are there locks? if there are locks, how do you coordinate
I hope this helps.
Dan
I will try to lightly define some terms I have used.
Component: in FBP anything is a component. The term reflects the aspect that "anything" is a constituent part / ingredient of another component. The IPs are components also, the functions are components, the data structures are components. Somebody that commented in the past the phrase "everything is an object" argued that if everything is X then how can we differentiate things? Well, this is semantics. "Component" is an aspect, the aspect of being part of something else. We can call it an object but the "object" is the aspect of being "material", perceived with senses, it is identifiable, it is an individual entity. In the end that "something" remains the same in the FBP network no matter how we call it: component, object, IP etc. Word is semantics and semantics is a collection of "aspects". By the way, an "aspect" is a component, a cut in the functional space that is common in many places (used to build components). It is related to modelling (pattern-making).
IP: The IP is what you say it is. I am calling them messages or parameters. It's the same thing but with a common name. Message and parameter have a common semantic for common people. An IP could be a functional component not just "passive" data.
Pure data: is a structure of data that contains no behaviour (no functions) i.e. data is only manipulated and not interpreted by microprocessor as bytecode.
Pure function: is behaviour, is code, is bytecode, it is what the microprocessor executes. After all it is data but it is interpreted by hardware as code.
Mixture (data and functions): it is a component that contains both data and functions (behaviour).
I have said that I do not differentiate too much the "pure data" components from "pure functional components" or mix. All of them have input and output ports, so from outside all of them are the same. I can not tell (without looking inside) if a component is just data (it acts like a latch) or it is just behaviour (process something) or both. I don't have to.
Process: is the component that transforms the parameters / messages (IPs) and internal data. It maps them to the output (results). In software we know what a process is (related to the operating system) but in FBP we tend to call almost any functional component a process (node in the FBP graph). This produces confusion.
>>Similarly, in the next para, how does a process manipulate a component? Are you referring to compilers?
I will tell you what I think a compiler is for a FBP program. A compiler is a FBP program itself that transforms the FBP network described in the source code into another FBP program. The catch is that we can intertwine the compiler aspects with runtime aspects. I suppose you have encountered many cases when you have had a need to have runtime features at compile time and vice-versa, compile aspects at runtime (i.e. program modifications at runtime). A compiler is just another FBP transformation, it transforms the ingredients soup (floats, ints, elementary functions) into FBP components. Compiling is a data flow itself that refers mainly to how a FBP network is built. A compiler builds and modifies a program and is part of the initial flow of the program defined in its basic form: interconnected basic (atomic) components.
This is a long subject so I will stop here but I will address one question to you: what is the equivalent of "constructors" from the C++ language in FBP?
There are so many things to define and I have to admit that defining something is a difficult process. Besides that, English is not my native language and semantics counts a lot :).
Regards,
Dan
Paul Morrison
Vladimir Sibirov
Tom Young
Best Regards,
-Tom Young
Brand new terms would only confuse me more.
Tom Young
47 MITCHELL ST.
STAMFORD, CT 06902
"When bad men combine, the good must associate; ..."
-Edmund Burke 'Thoughts on the cause of the present discontents' , 1770
This e-mail message from Tom Young is intended
only for the individual or entity to which it is addressed. This e-mail
may contain information that is privileged, confidential and exempt from
disclosure under applicable law. If you are not the intended recipient,
you are hereby notified that any dissemination, distribution or copying
of this communication is strictly prohibited. If you received this
e-mail by accident, please notify the sender immediately and destroy
this e-mail and all copies of it.
Dan
For instance, the programmers have discussed about OOP (Object Oriented Programming) for decades by now and many still think an object in OOP paradigm resembles the real object in nature. At least us, the guys studying FBP know that it is not quite true. OOP became religious like many other paradigms. When I have discovered for the first time OOP I thought my problems were over and objects are like electronic chips that I can connect together. Nothing could be further from the truth. There was a big discrepancy between reality and concepts.
I have observed a strong tendency of people, especially those ones coming from academics, to create their own terms eluding the real meaning of words (i.e. usually defined in dictionaries). The commons sense is broken and programmers are speaking their own language using the same words but with different meanings.
Well, an object should remain an object, a component is a component, a message or parameter remain what the dictionary says they are. Also something could be a component, node in a graph, an object and a message in the same time. Nothing is wrong with this either because that particular entity has all the aspects / qualities described by these words. Sometime an object is not a message but it is a component just because it is part of something else and not passed along.
Of course in programming we have to adapt and associate some (special) phrases to our objects of work. But going far away from the common sense is a mistake.
Let's take for instance the phrase "Information Packet" (IP). Well it's good enough to tell me that it contains information but all the program (data , functions) contains information. Being a packet suggest to me that it is a container that will transport information from A to B. This is good. But why not using the common term: parameter or message that are so common and even a non programmer understand what it is? You see?
By the way, quote from Tom Young :"Among other issues: how IPs can be functional escapes me".
IPs can be functional. What is so strange about that? You can not send an object that includes behaviour to another component? You do this in OOP all the time. You can receive as parameter predicates, objects, whatever else. Maybe I have not understood your concern.
It is quite strange for me to understand why "my terms" are so special or hard to understand. I have used words that are quite common for programmers and non-programmers alike (component, functional, parameter, message, object, data etc). We don't have to invent special terms for FBP. Even FBP is object of confusion when talking about it in the context of Data Flow Programming, for instance. But the term "Data Flow Programming" was already taken :) and associated with different meanings than our desire. Besides that the flow can contain something else but just data (i.e. data without behaviour to be more clear). It can contain functional components (the wonder of Tom Young).
It seems we starve for uniqueness. I understand that we have to come up with phrases "unique" to our domain in order to differentiate from others. We think that "our domain" is special and deserves special words. I don't think so.
I don't have any need to add other special words for FBP but I do agree that each term, even if it is quite common and well understood should be put in the context of FBP and explained as such.
In the end I am curious to understand why "my terms" (that are so common after all) were so special and hard to understand.
Regards,
Dan
Aristophanes
On Feb 15, 9:49 am, Paul Morrison <paul.morri...@rogers.com> wrote:
> Does anyone else, apart from me, have a problem with Dan defining all
> his own terms differently from what is in the book, and using them in
> his explanations? Should we try for a brand new set of terms and
> definitions? Should we have some kind of vote? I am a bit confused
> about how to proceed from here... Suggestions...?
>
participants in the group with different approaches. I don't think
that we can insist upon everybody following the book. That said, the
book is a central focal point for the discussions, and people should
be encouraged to use the common nomenclature.
In Dan's case there are two problems. One is that his nomenclature
conflicts with existing nomenclature. The more important problem is
that his paradigm is substantially different from the FBP paradigm. I
haven't untangled it all yet but it is basically functional as in
functional programming and that always confuses people used to
imperative programming.
My suggestion is not to get excited.
John Cowan
> There are issues with that fourth approach. First issue. Significant
> updates take time. I don't think that in place atomic updates are all
> that easy to come by,
What counts as "significant" depends on the application. In the Java
memory model (which is the only one I'm really familiar with) a change to
a single storage location such as an instance variable, array component,
or global variable is guaranteed to happen atomically (unless it is of
type 'long' or 'double', for historical reasons).
So, for example, the update component can take command IPs from other
components, construct a new updated object on the heap, and atomically
substitute it for the old object in an exposed global variable. This is
the general case, and often not necessary: for example, if the global
state is a large matrix of floats, all the update component has to do
is set the specified (x,y) element to the new value: there is no need
to copy the whole matrix.
> Second issue. How do we garbage collect?
We garbage-collect with a garbage collector.
--
And through this revolting graveyard of the universe the muffled, maddening
beating of drums, and thin, monotonous whine of blasphemous flutes from
inconceivable, unlighted chambers beyond Time; the detestable pounding
and piping whereunto dance slowly, awkwardly, and absurdly the gigantic
tenebrous ultimate gods --the blind, voiceless, mindless gargoyles whose soul
is Nyarlathotep. (Lovecraft) John Cowan co...@ccil.org
Dan
Come on! There is not functional at all. If I pronounce the word "function" it does not mean it is about functional language. Unfortunately there are so many misunderstandings related to the functional languages as well. These ones are somewhere between imperative and declarative languages. Functional languages have a big problem dealing with state and definitely the data flow is curled as it is in the imperative languages (i.e. the results returns to the caller). That's why they have invented "arrows" and monads. It's not the case with FBP.
After all I don't understand what made you feel that my approach is functional in the first place. The fact that I said we can send functions as IPs (i.e. functional components)? The fact that I have used the word "function" or "functional"?
I really don't get it.
And yes, definitely I am not following the book word by word. Books are written by people. It is about critical thinking and the only thing that I really believe in is doubt :). I am against religion in programming, against preconceptions, against using words with a sense far away from their real meaning. Sometime we complicate our life too much for too less.
Dan
Tom Young
For instance, the programmers have discussed about OOP (Object Oriented Programming) for decades by now and many still think an object in OOP paradigm resembles the real object in nature. At least us, the guys studying FBP know that it is not quite true. OOP became religious like many other paradigms. When I have discovered for the first time OOP I thought my problems were over and objects are like electronic chips that I can connect together. Nothing could be further from the truth. There was a big discrepancy between reality and concepts.
Without commenting on OOP, I look at FBP components as similar to electronic chips that can be connected together and IPs as similar to the signals connecting the chips.
I have observed a strong tendency of people, especially those ones coming from academics, to create their own terms eluding the real meaning of words (i.e. usually defined in dictionaries). The commons sense is broken and programmers are speaking their own language using the same words but with different meanings.
Well, an object should remain an object, a component is a component, a message or parameter remain what the dictionary says they are. Also something could be a component, node in a graph, an object and a message in the same time. Nothing is wrong with this either because that particular entity has all the aspects / qualities described by these words. Sometime an object is not a message but it is a component just because it is part of something else and not passed along.
Of course in programming we have to adapt and associate some (special) phrases to our objects of work. But going far away from the common sense is a mistake.
Let's take for instance the phrase "Information Packet" (IP). Well it's good enough to tell me that it contains information but all the program (data , functions) contains information. Being a packet suggest to me that it is a container that will transport information from A to B. This is good. But why not using the common term: parameter or message that are so common and even a non programmer understand what it is? You see?
By the way, quote from Tom Young :"Among other issues: how IPs can be functional escapes me".
IPs can be functional. What is so strange about that? You can not send an object that includes behaviour to another component? You do this in OOP all the time. You can receive as parameter predicates, objects, whatever else. Maybe I have not understood your concern.
It is quite strange for me to understand why "my terms" are so special or hard to understand. I have used words that are quite common for programmers and non-programmers alike (component, functional, parameter, message, object, data etc). We don't have to invent special terms for FBP. Even FBP is object of confusion when talking about it in the context of Data Flow Programming, for instance. But the term "Data Flow Programming" was already taken :) and associated with different meanings than our desire. Besides that the flow can contain something else but just data (i.e. data without behaviour to be more clear). It can contain functional components (the wonder of Tom Young).
It seems we starve for uniqueness. I understand that we have to come up with phrases "unique" to our domain in order to differentiate from others. We think that "our domain" is special and deserves special words. I don't think so.
I don't have any need to add other special words for FBP but I do agree that each term, even if it is quite common and well understood should be put in the context of FBP and explained as such.
In the end I am curious to understand why "my terms" (that are so common after all) were so special and hard to understand.
-twy
--
"...the shell syntax required to initiate a coprocess and connect its input and output to other processes is quite contorted..."
W. Richard Stevens [Advanced Programming in the Unix Environment, p441]
John Cowan
> - Read only shared state can always be eliminated by making it an IP and
> creating copies when necessary.
If something is read-only, it's a value, not state. State by definition is
mutable.
--
You know, you haven't stopped talking John Cowan
since I came here. You must have been http://www.ccil.org/~cowan
vaccinated with a phonograph needle. co...@ccil.org
--Rufus T. Firefly
Raoul Duke
are problems with it. it seems to have been something of a pipe dream.
it seems that most "real" erlang programs end up using mnesia or some
such database -- in the end, people need to have state that is
controlled. the choice is either to hold locks so that you don't have
to merge, or to not have locks and then have to deal with merging.
(there's stm which sort of blurs things.)
(i guess i don't yet see what is novel about any of the approaches
described vs. how people have wrestled with state in e.g. distributed
or even just concurrent systems?)
sincerely.
Ralf Westphal
of days. Urgent matters required my attention...
I´ve read all postings in this thread - unfortunately they don´t
satisfy my curiosity which originally made me open the thread.
I think we should at least see, whether we agree on some basics:
#1 Wherever there are multiple flows in a non-trivial program, there
is a good chance of several flows working on the same data.
Right?
How could there be multiple flows in a program? Well, each flow could
represent what´s gonna happen in reaction to some user interaction.
One flow processes data upon a button click even, another flow
processes data upon a menu item selection, another flow processes data
when the user moves the mouse over an image etc.
#2 The SRP is paramount. Even with my original sample scenario we
should never mingle domain logic and UI code. That´s a basic no-no.
Separating concerns in that manner is necessary from the start even
though a program might be simple because nobody will ever clean up the
mess if you don´t don´t do it right away. Forget about refactoring.
Honor the Broken Window theory.
#3 Shared state (or several flows working on the same data) is not a
matter of databases. Databases are special containers for data. But a
simple calculator modelled using FBP does not need a database - but
quite inevitably needs to keep state and maybe even requires shared
state.
So where are we in the discussion on "multiple flows working on same
data" (i.e. shared state)?
It can always be avoided by lugging it around in IPs? That does not
feel right (at least as a general solution). Firstly it makes for
bloated IPs. Secondly it would require loops, since once a flow comes
to an end the IP with the state would get stuck. (Storing state in the
UI is no option for me even though the UI might initiate different
flows and thus could reintroduce the state into another flow. Violates
the SRP.)
So where´s state kept? "State-only" components? Hm... I´ve played
around with that concept - but haven´t been satisfied 100%. Sometimes
that´s a good idea... but at other times it feels a bit clumsy.
Making state orthogonal to flows is another option. Certain flow
components then depend on the same state. That works ok at times. But
in the end it does not feel right to introduce dependencies just to
handle state. Flows are such a great concept because they get rid of
dependencies.
So I´m still a bit confused...
Sam Watkins
> Paul, you are right. The terminology is of huge importance.
I think many emails on this list, including some of Dan's, Paul's,
and mine, can be difficult to understand!
Many of us have different ideas or variations on FBP.
Dan's has some different ideas which might be hard to communicate,
e.g. 'sending components as messages'. Some people will want this,
some people won't. I think it can be useful, and also confusing!
We send queries over the network to visit the database. We can't
send the database over the network to visit the query. I think the
database is like a big collection of data IPs, and the query can be
implemented as a FBP component (or encapsulated network).
So it might be a good idea for our FBP components to be portable from
machine to machine, e.g. as Java bytecode, portable source code, or a
graph definition.
I've been reading a bit about Haskell, like "these functions take other
functions as arguments, and return different functions!" I'm not sure
if this would be a good feature for FBP! It can make it hard to know
what is going on; and it can help to make general and concise programs.
Sam
Ged Byrne
"So where are we in the discussion on "multiple flows working on same data" (i.e. shared state)?"
Ged Byrne
It can always be avoided by lugging it around in IPs? That does not
feel right (at least as a general solution). Firstly it makes for
bloated IPs. Secondly it would require loops, since once a flow comes
to an end the IP with the state would get stuck.
Dan
You cannot include behavior in an IP, you can only include a description of behavior. IPs do not 'behave'.
Besides the IPs we don't have any disagreements so it seems you have picked up the fuss and not the content. Nobody was pointing out to me yet what were the special "own terms" in order to "correct" myself. So, what are they?
Ron Lewis
Sam Watkins
> I think the TOPIC of this discussion group is indeed the "content of Paul's book" and FBP as so defined.
>
> However, the digressions from the main topic are IMPORTANT and VALUABLE.
Thanks Ron, and well put!
> Dan's thoughts force us to think outside the box.
Yes, I was trying to reply something sensible about processes
and/or components as messages... but I'm still thinking!
I've been reading a book on Haskell, so my head already feels funny.
Functions, types, classes, kinds, functors, monoids, monads...
aargh, they're all blurring together!
Sam
--
"Mr. Osbourne, may I be excused? My brain is full."
http://sam.nipl.net/pix/my-brain-is-full.jpg
Paul Morrison
useful, and I also fully admit that some of my terms may not be
optimal. As Dan points out, "component" means anything that is a part
of something else, so I don't have the right to insist that it should be
restricted to the FBP (book) sense. If someone can propose a better
name, as Tom says, I would be happy to change my book (through the
wonders of Internet self-publishing) to whatever we adopt. *Every* word
in any language has connotations - you cannot avoid it. All we can do
in this business is try to define words as clearly as possible, and
hope that people will be able to push the other connotations into the
background! If I remember correctly, an early language (I think it was
IAL) tried inventing brand-new words to get around just this problem.
Another thing that I predicted is that, as FBP catches on, it will be
applied in a sort of generic sense, just like (in N. America) "bandaid"
came to mean any kind of sticking plaster - or even just a stop-gap
solution. This is where I believe we could start using Vladimir's
http://flowbased.org/ - or Tom's matrix -
https://docs.google.com/spreadsheet/ccc?key=0AqcHR01C1GJ7dDU5T0tkWHgzVURhWGw3cy1kOGJqQVE&hl=en#gid=0
, as good places to say what FBP is not, and what it is. Do we need a
new term for the "book" FBP?
I venture to say that the FBP I describe has been in continuous use at a
bank for around 40 years, evolving over time through many hardware,
software and business changes. Whatever we are going to call it, this
is the context in which these ideas were developed. So I think, at the
very least, it can serve as a solid base for future development. The
so-called "digressions" may not be very close to the book FBP as it
currently exists, but may well point the way to future extensions and
evolution - I would hope incremental, rather than us all having to start
again from scratch :-)
I also wanted to endorse Ron's comment about FBP allowing different
paradigms to coexist - this has been a key motif in my work for years,
but IMO it has become more difficult in recent years due to the
proliferation of byte codes such as JVM and CLR - hence an earlier
thread in this Group about a "universal" infrastructure - which seems to
have died...?
One other comment: has it struck people that the participants in this
conversation are located in - or hail from - at least the following that
I am sure about: Romania, Russia, Canada, UK, US,...? The Internet is
really an amazing thing! :-)
Paul Morrison
Sorry, Ralf!
Aristophanes
One of the strengths of Book FBP (to coin a term) is the component/
process/IP superstructure. It is what allows us to use flow diagrams
to put together programs using precoded elements. An important point
is that "processes" (I call them autons which is short for autonomous
computing element) are not functions in the ordinary sense of
programming. Some differences:
(1) An auton (it's my post, I can use my terminology) has indefinite
duration; it is based on suspend/resume logic rather than invoke/
return logic. In other words it is persistent in a way that function
invocations are not. Because it is persistent in can be registered in
a flow diagram as a permanent constituent.
(2) Autons are not invoked; they are activated by the presence of
content in their input queue. (Or something that looks like their
input queue, but that is another matter.)
(3) Autons have inports (input ports) and outports (output ports).
Messages arrive in the inports and are sent out in the outports. What
counts as a message is a separate matter.
(4) Flow into and out of autons is managed by an external scheduler
that is separate from the code in the auton implementations. In
contrast flow into and out of functions is embedded in the
implementation, typically on a stack or stacks. This is why the
notion of sending one auton to another is problematic.
(5) There can be multiple autons with the same executable code
persistently present in a program, potentially each with a different
state. Having persistent invocations of impure functions with
separate state is not consistent with ordinary imperative code.
The component/process distinction is important in FBP. One way to
think of it is to compare it to a shop with a supply of parts. For
example, it might be an auto repair shop with a stock of oil filters.
The shop has a parts catalog. In the catalog there are various kinds
of parts listed along with their part numbers. When I need an oil
filter I check the catalog to find the right part number and then get
it from the parts bin. That's why I am comfortable with the term
"component". However one could think of components as prototypes or
templates. Maybe template is a better term.
One of the things about information packets is that they are not
messages as such. Each IP is a unique object with a well defined
lifetime. It comes into existence either as an input from outside or
as a de novo creation. It passes from place to place until it comes
to the end of its existence. Messages, on the other hand, are not
unique objects as such.
On Feb 16, 10:30 am, Paul Morrison <paul.morri...@rogers.com> wrote:
> I agree totally, Ron and Sam! Absolutely, these "digressions" are very
> useful, and I also fully admit that some of my terms may not be
> optimal. As Dan points out, "component" means anything that is a part
> of something else, so I don't have the right to insist that it should be
> restricted to the FBP (book) sense. If someone can propose a better
> name, as Tom says, I would be happy to change my book (through the
> wonders of Internet self-publishing) to whatever we adopt. *Every* word
> in any language has connotations - you cannot avoid it. All we can do
> in this business is try to define words as clearly as possible, and
> hope that people will be able to push the other connotations into the
> background! If I remember correctly, an early language (I think it was
> IAL) tried inventing brand-new words to get around just this problem.
>
> Another thing that I predicted is that, as FBP catches on, it will be
> applied in a sort of generic sense, just like (in N. America) "bandaid"
> came to mean any kind of sticking plaster - or even just a stop-gap
Aristophanes
Perhaps the title of the thread should have been "Why isn't shared
state a problem in FBP", because apparently it doesn't seem to be.
I'm not sure that there is anything on shared state as such in the FBP
book. People who have production FBP and FBP like systems don't seem
to have a problem - at least they are not reporting them. It seems to
me that people haven't really been able to come up with good examples
of shared data issues in FBP.
One obvious reason is that FBP doesn't allow shared state - persistent
state is isolated in autonomous processes. That doesn't answer the
question; it just says that FBP can get away with not using shared
state. Here are a few thoughts.
The FBP system was developed in the context of business applications
for which a flow of process is a natural model. So to speak,
transactions enter at one end of the process, move uni-directionally
through the process, and exit at the other end. There may be
splitting and merging and updating, but there is a direction of
motion. So maybe it is the problem space that matters.
Another thought is a shared data problem often is a shared memory
problem. In imperative languages people talk about code being thread
safe. Now what this usually means is that there are multiple function
invocations using the same memory locations. In FBP this problem goes
away because the processes (autons) don't share memory.
Yet another thought is that the FBP program design process naturally
avoids creating shared data problems.
Raoul Duke
> (1) An auton (it's my post, I can use my terminology) has indefinite
http://en.wikipedia.org/wiki/Actor_model#Later_Actor_programming_languages
now, i know there are differences, and differences do matter! it is
just that it seems like this FBP thing -- the way i read the messages
mostly so far, not absolutely, just mostly -- is that it is a new
unique different never seen before never duplicated idea, which seems
odd, and also that everybody has to come up with their own vocabulary
for some reason.
perhaps one pie in the sky wunderbar solution would be a new wikipedia
page with a big table matrix on it. y axis is "your own personal term
{actor,auton,...}" and the x axis would be the features we can use to
categorize. to see the subtle distinctions.
sincerely.
Raoul Duke
> question; it just says that FBP can get away with not using shared
> state. Here are a few thoughts.
again, see the experience of the Actor model, in particular via Erlang
since that has been used extensively in real world commercial
nine-nines type software. i think you will discover that while the
rule of DAMMIT THERE'S NO SHARED STATE is a wonderful beginning to get
people out of their really bad habits, the shared state will
eventually crawl back in!
heck, look at actual hardware -- there's a fair bit of logic in cpus
to deal with cache coherency, no?
now i'm not actually an Erlang developer, i'm just a gadfly on the
wall, but my understanding is that most everybody breaks down and uses
mnesia, the Erlang database, which is really just using shared state,
ha ha. oh well. a guy can dream.
i think the main non-shared-state alternative contender these days is
the new-fangled no-sql-eventual-consistency thing.
(stm is about doing shared state better. it is not something that
would support distribution, it would fly in the face of the N
Fallacies of Distributed Computing, i believe.)
sincerely.
Aristophanes
On Feb 16, 2:08 pm, Raoul Duke <rao...@gmail.com> wrote:
> On Thu, Feb 16, 2012 at 11:17 AM, Aristophanes <c...@tiac.net> wrote:
> > (1) An auton (it's my post, I can use my terminology) has indefinite
>
> now, i know there are differences, and differences do matter! it is
> just that it seems like this FBP thing -- the way i read the messages
> mostly so far, not absolutely, just mostly -- is that it is a new
> unique different never seen before never duplicated idea, which seems
> odd, and also that everybody has to come up with their own vocabulary
> for some reason.
>
> perhaps one pie in the sky wunderbar solution would be a new wikipedia
> page with a big table matrix on it. y axis is "your own personal term
> {actor,auton,...}" and the x axis would be the features we can use to
> categorize. to see the subtle distinctions.
publish their brilliant ideas just don't dig deeply into the available
literature. Frex, the actors model and FBP were roughly concurrent in
their genesis. Academics read academic journal papers and selected
journals at that. Actually looking at industrial journals and reports
is rare at best. Similarly, people in industry seldom read academic
papers.
Aristophanes
On Feb 16, 2:12 pm, Raoul Duke <rao...@gmail.com> wrote:
> On Thu, Feb 16, 2012 at 12:08 PM, Aristophanes <c...@tiac.net> wrote:
> > question; it just says that FBP can get away with not using shared
> > state. Here are a few thoughts.
>
> again, see the experience of the Actor model, in particular via Erlang
> since that has been used extensively in real world commercial
> nine-nines type software. i think you will discover that while the
> rule of DAMMIT THERE'S NO SHARED STATE is a wonderful beginning to get
> people out of their really bad habits, the shared state will
> eventually crawl back in!
actor model even though I am using something like it myself. Actor
based (quasi actor based?) generally embed the spawning of processes/
actors/autons within the code. FBP does not. Instead the network is
created graphically, i.e., it is visible in advance versus being
constructed dynamically.
In my probably mistaken view the difference is critical. If you don't
know what you've got (which you don't in a sufficiently large dynamic
network) then you need a way to find what you need.
> heck, look at actual hardware -- there's a fair bit of logic in cpus
> to deal with cache coherency, no?
>
> now i'm not actually an Erlang developer, i'm just a gadfly on the
> wall, but my understanding is that most everybody breaks down and uses
> mnesia, the Erlang database, which is really just using shared state,
> ha ha. oh well. a guy can dream.
> i think the main non-shared-state alternative contender these days is
> the new-fangled no-sql-eventual-consistency thing.
Raoul Duke
> Well, yes, but it's not that simple. I've never been fond of the
> actor model even though I am using something like it myself. Actor
> based (quasi actor based?) generally embed the spawning of processes/
> actors/autons within the code. FBP does not. Instead the network is
> created graphically, i.e., it is visible in advance versus being
> constructed dynamically.
>
> In my probably mistaken view the difference is critical. If you don't
> know what you've got (which you don't in a sufficiently large dynamic
> network) then you need a way to find what you need.
personally i wouldn't say it is mistaken. i'm more a visual thinker
myself. and here's somebody cooler and more famous and more productive
than myself who says the same thing.
now, of course, there are a zillion ways to do something wrong, and
often visual programming has been seen as failing, but maybe like
edison some day (if fbp didn't already do it) we'll get through all
the wrong ways to use graphcial stuff for programming, and end up with
the good one(s).
sincerely.
John Cowan
> heck, look at actual hardware -- there's a fair bit of logic in cpus
> to deal with cache coherency, no?
Primarily because the original Von Neumann model didn't make distinctions
that later turned out to be useful, like between code and data.
(Admittedly, there has to be a point at which data *becomes* code.)
> now i'm not actually an Erlang developer, i'm just a gadfly on the
> wall, but my understanding is that most everybody breaks down and uses
> mnesia, the Erlang database, which is really just using shared state,
> ha ha. oh well. a guy can dream.
Yes, but it's *encapsulated* shared state. The trouble with "normal"
threaded programs in conventional languages is that everything's unsafely
shared by default, and you have to take special pains to share safely
or to avoid sharing.
--
Using RELAX NG compact syntax to John Cowan <co...@ccil.org>
develop schemas is one of the simple http://www.ccil.org/~cowan
pleasures in life....
--Jeni Tennison <co...@ccil.org>
Raoul Duke
> Yes, but it's *encapsulated* shared state. The trouble with "normal"
> threaded programs in conventional languages is that everything's unsafely
> shared by default, and you have to take special pains to share safely
> or to avoid sharing.
i don't follow. i thought RDBMS had lots of tweaky code to deal with
all the concurrency and locking and optimization and performance
issues, so i do not see how encapsulation is a silver bullet. i mean,
OO is supposedly encapsulation, and Java sucks wrt concurrency i
suspect. so there's something about the term *encapsulation* that is
worth expanding upon?
sincerely.
John Cowan
> i don't follow. i thought RDBMS had lots of tweaky code to deal with
> all the concurrency and locking and optimization and performance
> issues, so i do not see how encapsulation is a silver bullet.
Yes, it does, but all that code is encapsulated into the database engine:
it doesn't have to show up in the rest of your code.
--
John Cowan co...@ccil.org http://www.ccil.org/~cowan
O beautiful for patriot's dream that sees beyond the years
Thine alabaster cities gleam undimmed by human tears!
America! America! God mend thine every flaw,
Confirm thy soul in self-control, thy liberty in law!
--one of the verses not usually taught in U.S. schools
Raoul Duke
>> i don't follow. i thought RDBMS had lots of tweaky code to deal with
>> all the concurrency and locking and optimization and performance
>> issues, so i do not see how encapsulation is a silver bullet.
>
> Yes, it does, but all that code is encapsulated into the database engine:
> it doesn't have to show up in the rest of your code.
i guess it seems less than ideal to me when trying to figure out how
sausage is made to say "don't look at how we make sausage, just eat
it."
sincerely.
John Cowan
> Well, yes, but it's not that simple. I've never been fond of the
> actor model even though I am using something like it myself. Actor
> based (quasi actor based?) generally embed the spawning of processes/
> actors/autons within the code. FBP does not. Instead the network is
> created graphically, i.e., it is visible in advance versus being
> constructed dynamically.
>
> In my probably mistaken view the difference is critical. If you don't
> know what you've got (which you don't in a sufficiently large dynamic
> network) then you need a way to find what you need.
That's merely a special case. The configuration mini-language in
JavaFBP and C#FBP is just a stereotyped use of ordinary Java or C#
methods, and there's nothing stopping you from creating new Components
and Connections at run time, or even dynamically loading new Component
subclasses to provide novel functionality at runtime.
Like any sharp tool, dynamic network construction can cut you if misused.
Static networks can be pretty hard to understand too, if they are big
and complex enough. (I'm maintaining a static network -- a concept
rather than a processing network -- right now with about 350 nodes, and
it's hard to hold it all in my head.) However, it's common for a Unix
process to create a predecessor or successor to itself and insert them
into the pipeline, so a disciplined use of dynamic behavior can be useful.
--
If you have ever wondered if you are in hell, John Cowan
it has been said, then you are on a well-traveled http://www.ccil.org/~cowan
road of spiritual inquiry. If you are absolutely co...@ccil.org
sure you are in hell, however, then you must be
on the Cross Bronx Expressway. --Alan Feuer, NYTimes, 2002-09-20
John Cowan
> i guess it seems less than ideal to me when trying to figure out how
> sausage is made to say "don't look at how we make sausage, just eat
> it."
The point of that story is that if you looked at how it's made, you
probably wouldn't want to eat it. Similarly, it's fine to have complex
multithreaded code *somewhere* in the system -- after all, the OS kernel
is just that -- but not in *my* code!
--
"But I am the real Strider, fortunately," John Cowan
he said, looking down at them with his face co...@ccil.org
softened by a sudden smile. "I am Aragorn son http://www.ccil.org/~cowan
of Arathorn, and if by life or death I can
save you, I will." --LotR Book I Chapter 10
Raoul Duke
> The point of that story is that if you looked at how it's made, you
> probably wouldn't want to eat it. Similarly, it's fine to have complex
> multithreaded code *somewhere* in the system -- after all, the OS kernel
> is just that -- but not in *my* code!
see, i would be day dreaming smoking stuff hoping somebody would
actually consider how to do something like an OS kernel w/out having
to do pure evil Linux-kernel-have-you-seen-the-humanity-in-there style
coding. i want the future to have some theory and language and
Sufficiently Smart Compilers (i know that is a bad thing to wish for,
at least until they get over a hump that makes them not suck more than
they help) so that we aren't ever writing horrible, horrible code. :-)
Vladimir Sibirov
John Cowan
> One of the things about information packets is that they are not
> messages as such. Each IP is a unique object with a well defined
> lifetime. It comes into existence either as an input from outside or
> as a de novo creation. It passes from place to place until it comes
> to the end of its existence. Messages, on the other hand, are not
> unique objects as such.
Again, it depends. In early Smalltalk systems, messages *were* unique
objects, instances of class Message. They were created whenever code
sent a message to an object. In Smalltalk-80 (the current version),
message-sending is implemented as function call with a search for the
correct function, as in Simula-67 and all later OO languages.
However, it is still possible for the Smalltalk programmer to create
and send reified messages (at a cost in efficiency), and if there is
something wrong with a message (for example, the receiving object does
not understand it), it is reified after the fact and delivered using
the message doesNotUnderstand (which by default throws an exception,
but can be redefined)
--
John Cowan co...@ccil.org
"You need a change: try Canada" "You need a change: try China"
--fortune cookies opened by a couple that I know
John Cowan
> On shared state. From my experience of writing flow-based programs so far
> "shared state" problem occurs if there is a data resource that should be
> read/written by several processes concurrently, especially if all events in
> the system are asynchronous. But if you enclose such a resource into a
> component that manages the resource and access to it just like a DBMS does,
> accessing such a shared resource becomes as clumsy as querying a database.
Actually, querying a database is no more clumsy than reading a file from a
file-reader component. You send an IP to the database component's input
port that represents the query, and you get back a stream of IPs representing
the rows followed by an EOF IP.
--
John Cowan co...@ccil.org
"Not to know The Smiths is not to know K.X.U." --K.X.U.
Paul Morrison
did want to comment on this one, though - I really liked it. Is there
any movement to rename "components" to "templates". One additional
complexity is that a subnet can also be a component, so maybe "template"
is not quite enough... "Read-only chunk of code" (ROCOC?) - just
joking... I think!
I am really impressed with the insights you folks have into FBP! I
think Dan has made us all dig a bit more deeply!
Aristophanes
there is no confusion with C++ templates.
Vladimir Sibirov
Component has exactly the same meaning in Component-oriented programming and Integrated circuits engineering (e.g. VHDL) as in FBP. I can't see what's wrong with it if it has been used in the computer industry for say last 40 years.
18.02.2012 2:26 пользователь "Aristophanes" <c...@tiac.net> написал:
Ged Byrne
different languages, so I'd like to stress the fundamental concept
within FBP.
"Part of the reason for this is that most of today's computers
have a uniform array of pigeon-holes for storage, and this storage
behaves very differently from the way storage systems behave in real
life. In real life, paper put into a drawer remains there until
deliberately removed. It also takes up space, so that the drawer will
eventually fill up, preventing more paper from being added. Compare
this with the computer concept of storage - you can reach into a
storage slot any number of times and get the same data each time
(without being told that you have done it already), or you can put a
piece of data in on top of a previous one, and the earlier one just
disappears.... Although destructive storage is not integral to the the
von Neumann machine, it is assumed in many functions of the machine,
and this is the kind of storage which is provided on most modern
computers. Since the storage of these machines is so sensitive to
timing and because the sequencing of every instruction has to be
predefined (and humans make mistakes!), it is incredibly difficult to
get a program above a certain complexity to work properly."
- http://www.jpaulmorrison.com/fbp/concepts.shtml
Here is the thing that FBP gives us that other approaches do not. A
simple, easy to comprehend principle for dealing with data in memory.
Mutable state in Java and similar languages make software concurrency
difficult and prone to error. Users cannot understand why developers
struggle to keep their code bug free.
Immutable state in functional programming brings security but in order
to solve simple problems the programmer has to get to grips with
abstract complexities. Certainly, this may not be beyond the ability
of programmers but it makes communication with others difficult.
FBP gives us a simple compromise between the two. A set of basic
principles that allow us write simple concurrent code that is easily
understood. This is the promise given in the book's introduction:
"A meeting is called of all the people involved - not just
programmers and analysts, but users and operations personnel as well.
The essential logic of the program is put up on the wall, and the
program designers walk through the program structure with the group.
During the ensuing discussion, they realize that two new modules have
to be written and some other ones have to change places."
- http://www.jpaulmorrison.com/fbp/intro.shtml
There are lots of ideas out there: Actors, STMs, etc. They could all
be adopted but are they actually necessary? If our user's can achieve
their goals using networks of simple components that they find
comprehensible, is there any need to make things more complicated?
Regards,
Ged
On Feb 16, 10:32 pm, John Cowan <co...@mercury.ccil.org> wrote:
> Raoul Duke scripsit:
Dan
Paul Morrison
On Feb 19, 5:07 pm, Ged Byrne <ged.by...@gmail.com> wrote:
>... most of today's computers
> behaves very differently from the way storage systems behave in real
> life. In real life, paper put into a drawer remains there until
> deliberately removed. It also takes up space, so that the drawer will
> eventually fill up, preventing more paper from being added. Compare
> this with the computer concept of storage - you can reach into a
> storage slot any number of times and get the same data each time
> (without being told that you have done it already), or you can put a
> piece of data in on top of a previous one, and the earlier one just
> disappears....
a couple of quotes from my book, and it's interesting that one of the
quotes that he found significant is the one I show above. It seems to
me that a lot of the emphasis in these FBP discussions has been on the
processes, but I feel it is worth stressing that FBP's concept of data
is very different from that of the classical von Neumann machine, and
much more like storage in the real world. In the early days, we had a
hard time explaining this... until we started using terms like "desk
drawers", and then they got it! In the first FBP implementation,
AMPS, in addition to connections, we also had storage slots that could
hold a single IP, defined right in the network definition. These
could be thought of as connections with both ends connected to the
same process. Of course, in this case you couldn't suspend on a send
or receive, so send and receive were conditional (e.g. try to send,
and let me know if it didn't work). These storage facilities were
called "shunts", using a railway analogy (probably not precise!).
You can of course use local process storage for this function if the
component is a "looper", but "shunts" were "outside" the process, as
are connections, so they didn't care whether the component was a
"looper" or "non-looper". Unfortunately, programmers tended to use
shunts rather haphazardly, so in later implementations we used a stack
instead, as described in Chap. 9 (Fig. 9.4). I wanted to describe
this idea, and am wondering if anyone feels there is a useful idea
here, as we are going more visual.... or we could maybe provide
multiple stacks per process...? Comments?
Ged Byrne
Paul Morrison
On Mar 6, 2:51 am, Ged Byrne <ged.by...@gmail.com> wrote:
>
> I would say that it is the rules that restrict and control the use of
> Information Packets that makes Flow Based Programming unique.
>
> Could I go as far to say that FBP makes the end-to-end flows of the network
> deterministic rather than emergent?
>
"constraints" in Chap. 3... Two other key concepts are fixed capacity
connections, plus the ownership rule. I have never understood how
systems that allow connections to have indefinite numbers of packets
(or characters) could guarantee that every piece of data gets
processed, at least in a timely manner!
John Cowan
> For component based programming all of the focus on the individual
> components. The interfaces that connects each of them are entirely
> isolated from one another. For example, in Unix there are STDIN,
> STDOUT and STDERR. They are just byte streams.
This is true in principle, but in practice almost all the components
(programs) that participate in the pipe-and-filter architecture are
processing text line-by-line, so that each line (terminated by a newline
character) functions as an IP. Like FBP IPs, lines can be created,
passed through, modified, or dropped by any component.
(Note: Pipes are fixed-size buffers. The size varies from implementation
to implementation: it cannot be less than 512 bytes, but sizes from 8K
to 64K are more usual nowadays.)
> In Flow Based Programming the focus is on the flow, within which the
> components are just actors. The other actor is the Information Packet
> which has structure and follows rules.
Unix-style IPs usually have structure too, though some components ignore
the structure and treat the IP as a string. More typically, components
divide each IP into fields using a delimiter character (often tab
or whitespace, but settable by a parameter). In essence, lines are
dynamically rather than statically typed, but that does not mean they
are typeless.
> The combination of the two allows those reading a flow network
> to reason about the flow. It is possible to guarantee that an
> information packet that enters the network will always have a final
> destination where it will exit the network.
This is true only for the subset of networks whose components neither
create nor drop IPs. But many useful components do so. Consider a "top
ten" component, which accepts IPs containing a key such as a number,
and passes through the 10 IPs with the largest keys. (In practice,
10 would be a parameter.) Or even simpler, a component which accepts
IPs until there are no more, and then produces a single IP containing
the number of IPs read.
> Could I go as far to say that FBP makes the end-to-end flows of the
> network deterministic rather than emergent?
I don't understand what that means. If you mean that there are no race
conditions, then yes. If you mean that the fate of every packet is
statically predictable, then no.
--
How they ever reached any conclusion at all <co...@ccil.org>
is starkly unknowable to the human mind. http://www.ccil.org/~cowan
--"Backstage Lensman", Randall Garrett
Ged Byrne
A Unix component reads a stream and writes a stream. There are no mechanisms to enforce a relationship between those two streams. Lets imagine a script that has the following logic:
John Cowan
> In FBP I can answer this by answering the question will [2] or [3]
> drop any packets? If the answer is no, then I know that every packet
> received by [1] will be received by [4]. As long as [2] and [3] do
> not contain any drop statement I know it to be true.
Provided you can inspect the source code. In any case, the converse is
not true: the mere presence of calls on drop does not guarantee what, if
anything, is dropped. This is an ineluctable consequence of programming
your components in a Turing-complete language with an arbitrary amount
of local storage.
:z
> This is what I mean by the flow being deterministic. "For everything
> that happens there are conditions such that, given them, nothing else
> could happen.": http://en.wikipedia.org/wiki/Determinism
In that sense any single-threaded program is deterministic. The advantage
of all FBP-style programming is that each component is deterministic;
the necessary control of non-determinism happens inside the engine.
> In unix [2] could receive the text of shakespear on STDIN, ignore it
> and output a list of swear words to [3]. There no rules that govern
> the relationship between input and output. The output is simply a
> product of the code's behaviour.
An FBP component that does exactly that is quite possible; it drops all
its input packets and creates new output packets from scratch. Neither
one of them is particularly sensible, but similar components might be
used for mocking out network connections or databases during testing.
> If this component receives a 100 customer records and drops them all,
> replacing them with a single count value then this looks wrong to me.
:z
> Where has all the customer data gone?
>
> It tells me that if I am sending my customer records to this component
> then they should be copies, not the original.
If your customer records only exist as transient IPs within a network,
you have far worse problems than programming style. Of course they
would be copies of the persistent versions in some database.
It is quite common for me to create a network whose first few steps are
massive data reductions; they throw away almost all the input in order
to focus on what I care about at the time.
> I think more work is needed to fully realise the potential. For
> example, in the COUNT example 100 customer records could be read, 99
> dropped and one of them replaced by a single integer. That shouldn't
> be allowed. It should be necessary to drop all 100 customer records
> and then create a single count integer.
Arguably IPs should be treated as immutable. That gains most of the
performance of a move-pointer architecture while keeping the conceptual
simplicity of a copy-all-data architecture like Unix. Unfortunately this
cannot be enforced in Java.
--
As you read this, I don't want you to feel John Cowan
sorry for me, because, I believe everyone co...@ccil.org
will die someday. http://www.ccil.org/~cowan
--From a Nigerian-type scam spam
Paul Morrison
Sent from Samsung Galaxy Tab (tm) on Rogers
Ged Byrne <ged....@gmail.com> wrote:
>In FBP I can answer this by answering the question will [2] or [3] drop any
>packets? If the answer is no, then I know that every packet received by
>[1] will be received by [4]. As long as [2] and [3] do not contain any
>drop statement I know it to be true.
>
>This is what I mean by the flow being deterministic. "For everything that
>happens there are conditions such that, given them, nothing else could
>happen.": http://en.wikipedia.org/wiki/Determinism
>
>Every packet received by [4] was created by [1], [2] or [3]. They cannot
>have come from anywhere else. Every packet sent by [1] will either be
>received by [4] or dropped by [2] or [3]. These relationships across the
>flow are guaranteed by the engine, I know them to be true without having to
>look at any of the code.
>
>In unix [2] could receive the text of shakespear on STDIN, ignore it and
>output a list of swear words to [3]. There no rules that govern the
>relationship between input and output. The output is simply a product of
>the code's behaviour.
>
>This is what I mean by emergent:
>"c<http://en.wikipedia.org/wiki/Complex_system>omplex
>systems and patterns arise out of a
>m<http://en.wikipedia.org/wiki/Multiplicity_(disambiguation)>
>ultiplicity of relatively simple interactions." Each line of code is a
>simple interaction, and I have to read all of them to know what going to
>happen. That 's what software is so difficult.
>
>Considering the example of the COUNT component. It receives all IPs and
>sends emits a single count value.
>
>If this component receives a 100 customer records and drops them all,
>replacing them with a single count value then this looks wrong to me.
> Where has all the customer data gone?
>
>It tells me that if I am sending my customer records to this component then
>they should be copies, not the original. That seems untidy. Instead I
>would have two output ports from my component. One port would pass the
>records on, the other would send a single count. The principle of FBP make
>the need for this obvious.
>
>I think more work is needed to fully realise the potential. For example,
>in the COUNT example 100 customer records could be read, 99 dropped and one
>of them replaced by a single integer. That shouldn't be allowed. It
>should be necessary to drop all 100 customer records and then create a
>single count integer.
>
Ged Byrne
The problem is to create a file OUT which is a subset of another one IN, where the records to be output are those which satisfy a given criterion "c". Records which do not satisfy "c" are to be omitted from the output file. This is a pretty common requirement and is usually coded using some form of the following logic:read into a from INdo while read has not reached end of fileif c is truewrite from a to OUTendifread into a from INenddoFigure 3.2What action is applied to those records which do not satisfy our criterion? Well, they disappear rather mysteriously due to the fact that they are not written to OUT before being destroyed by the next "read". Most programmers reading this probably won't see anything strange in this code, but, if you think about it, doesn't it seem rather odd that it should be possible to drop important things like records from an output file by means of what is really a quirk of timing?
Suppose we decide instead that a record should be treated as a real thing, like a memo or a letter, which, once created, exists for a definite period of time, and must be explicitly destroyed before it can leave the system. We could expand our pseudo-language very easily to include this concept by adding a "discard" statement (of course the record has to be identified somehow). Our program might now read as follows:read record a from INdo while read has not reached end of fileif c is truewrite a to OUTelsediscard aendifread record a from INenddo
> In FBP I can answer this by answering the question will [2] or [3]
> drop any packets? If the answer is no, then I know that every packet
> received by [1] will be received by [4]. As long as [2] and [3] do
> not contain any drop statement I know it to be true.
Provided you can inspect the source code. In any case, the converse is
not true: the mere presence of calls on drop does not guarantee what, if
anything, is dropped. This is an ineluctable consequence of programming
your components in a Turing-complete language with an arbitrary amount
of local storage.
:z
[...]
John Cowan
> It also makes the component code easier to understand. Take for example
> your components that discard most of the packets to focus on what you
> are interested in. It is made explicit that packets are being dropped
> and why.
That is as much to say that manual storage reclamation is easier
to understand and less buggy than automatic storage reclamation.
All experience with garbage collectors is against that claim. Here you
are adding two lines to a trivial component; not all components will
be so trivial.
Paul Morrison scripsit:
> Most programmers reading this probably won't see anything strange in
> this code, but, if you think about it, doesn't it seem rather odd that
> it should be possible to drop important things like records from an
> output file by means of what is really a quirk of timing?
I don't understand where timing comes in. This simply assumes that
all packets belonging to a component and not otherwise disposed of are
dropped when the component terminates, or sooner if it can be proved that
they will not be needed again (which is what a garbage collector does).
Both versions of the code, with and without manual drop, have entirely
deterministic behavior.
--
John Cowan co...@ccil.org http://ccil.org/~cowan
Consider the matter of Analytic Philosophy. Dennett and Bennett are well-known.
Dennett rarely or never cites Bennett, so Bennett rarely or never cites Dennett.
There is also one Dummett. By their works shall ye know them. However, just as
no trinities have fourth persons (Zeppo Marx notwithstanding), Bummett is hardly
known by his works. Indeed, Bummett does not exist. It is part of the function
of this and other e-mail messages, therefore, to do what they can to create him.
Paul Morrison
> Paul Morrison scripsit:
>
>> > Most programmers reading this probably won't see anything strange in
>> > this code, but, if you think about it, doesn't it seem rather odd that
>> > it should be possible to drop important things like records from an
>> > output file by means of what is really a quirk of timing?
> I don't understand where timing comes in. This simply assumes that
> all packets belonging to a component and not otherwise disposed of are
> dropped when the component terminates, or sooner if it can be proved that
> they will not be needed again (which is what a garbage collector does).
> Both versions of the code, with and without manual drop, have entirely
> deterministic behavior.
coding technique: read a record into an area of storage, and then write
it out from the same area. If you assume that the record is always read
into a newly allocated object, then I think your objection holds... but
you can't force programmers to always do that - even in OO.
Paul Morrison
I believe that it is this principle, that data should just be allowed to disappear into the ether, that distinguishes Flow based from Component based.
John Cowan
> My quote above may be pre-OO, but it is (or was) a pretty standard
> coding technique: read a record into an area of storage, and then write
> it out from the same area. If you assume that the record is always read
> into a newly allocated object, then I think your objection holds... but
> you can't force programmers to always do that - even in OO.
Oh, if you are reading directly out of the OS buffer, then yes, it's
timing-dependent. I understand that pre-Unix operating systems often
did that -- just returned a pointer into the buffer, and you had to
grab your stuff before it was overwritten. In Java, what you get out
of a Reader is either a primitive value or else a String, and Strings
are immutable. Even in the C model, you can rely on the contents of
the userland buffer to remain the same (unless you change them yourself)
until you do another read, and for writes not to return until the data
has been copied into the kernel.
--
Let's face it: software is crap. Feature-laden and bloated, written under
tremendous time-pressure, often by incapable coders, using dangerous
languages and inadequate tools, trying to connect to heaps of broken or
obsolete protocols, implemented equally insufficiently, running on
unpredictable hardware -- we are all more than used to brokenness.
--Felix Winkelmann
Ged Byrne
Oops.
Ged Byrne
That is as much to say that manual storage reclamation is easierto understand and less buggy than automatic storage reclamation.All experience with garbage collectors is against that claim. Here youare adding two lines to a trivial component; not all components willbe so trivial.
Ged Byrne scripsit:
> It also makes the component code easier to understand. Take for example
> your components that discard most of the packets to focus on what you
> are interested in. It is made explicit that packets are being dropped
> and why.
John Cowan
> Many C++ developers have adopted a similar approach using RAII (Resource
> Acquisition Is Initialisation) as an effective alternative to garbage
> collection:
It isn't, really. RAII only works if the lifetime of the resource is
stack-oriented (that is, it comes into existence when a particular function
is invoked, and is disposed of when the function is about to return).
That works well *when it works*, but often produces convoluted results
or fails altogether, which is why C++ has a variety of (non-built-in)
pointer types that do reference-counting garbage collection.
> The implementation is different but the principle is the same. For any
> resource (memory, file, connection, etc) there is a handle. Responsibility
> for that handle is always held in just one place, so that it is possible to
> ensure that it is released.
Quite so. So now instead of "Who frees memory?" you have to ask "Who drops
the packet?" Not much of a gain.
> Bartosz describes resource transfer, the transfer of a handle between
> scopes, as part of his resource management:
> http://www.relisoft.com/resource/transfer.html.
I'll read up on this.
--
John Cowan <co...@ccil.org> http://www.ccil.org/~cowan
One time I called in to the central system and started working on a big
thick 'sed' and 'awk' heavy duty data bashing script. One of the geologists
came by, looked over my shoulder and said 'Oh, that happens to me too.
Try hanging up and phoning in again.' --Beverly Erlebacher
Ged Byrne
> The implementation is different but the principle is the same. For anyQuite so. So now instead of "Who frees memory?" you have to ask "Who drops
> resource (memory, file, connection, etc) there is a handle. Responsibility
> for that handle is always held in just one place, so that it is possible to
> ensure that it is released.
the packet?" Not much of a gain.
In FBP Information Packets (IP) are managed by establishing a chain of responsibility. At any one time there is a single point of responsibility for an IP, either within a processor or a queue.A queue can receive and send IPs. Every IP received (R) must be sent (S) so that (R = S)A processor can create and destroy IPs in addition to sending and receiving them. Every IP received (R) or created (C) must be either sent (S) or destroyed (D) so that (C + R = S + D).
John Cowan
> Would you agree that this is a fair summary of the rules presented by Paul
> in his book?
I would.
> Is the above a fundamental principle of FBP or just a feature of some
> implementations?
It's fundamental to the *analysis* of FBP programs.
> Would I be fair to summarise your opinion as being that the above is simply
> an implementation approach taken by some FBP that is not significant and of
> limited usefulness?
No. What I think is that explicit dropping is not a requirement on all
implementations, and implicit dropping backed by garbage collection (whether
done by reference counting, mark-sweep, stop-and-copy, or "drop everything
when the process terminates") is satisfactory and indeed preferable, as it
minimizes the amount of bookkeeping code that needs to be written.
--
Is not a patron, my Lord [Chesterfield], John Cowan
one who looks with unconcern on a man http://www.ccil.org/~cowan
struggling for life in the water, and when co...@ccil.org
he has reached ground encumbers him with help?
--Samuel Johnson
Paul Morrison
Every IP received (R) or created (C) must be either sent (S) or destroyed (D) so that (C + R = S + D).
At base, this seems to me to be a philosophical problem - OO GC seems to me to be like the old saw: "if a tree falls in the forest,...". There is a discussion about this in the FBP wiki (including a pair of limericks) - not sure who all contributed to it: http://www.jpaulmorrison.com/cgi-bin/wiki.pl?InformationPacket , plus some comments by Matthew Crewe - http://www.jpaulmorrison.com/cgi-bin/wiki.pl?MatthewCrewe .
In these FBP implementations, we also provided an IP tree mechanism, to allow fairly complex structures of IPs to travel through the network using single sends and receives. See http://www.jpaulmorrison.com/fbp/tree.htm . In this case, attaching an IP to the tree decrements the owned count, and detaching one from the tree increments the count. This implies that you cannot drop (discard) an attached IP without detaching it first! Much like real life!
Last point: in today's world, we could have all owned IPs chained into a linked list - or, in fact, any collection that supports removal of an IP from an arbitrary point in the list. This would then allow the programmer to locate undisposed-of IPs much more quickly than having to pore through the code... Comments?
John Cowan
> I think I understand where John is coming
> from, but one of the problems that we had in the early days was of
> storage not being freed up, and gradually accumulating until storage was
> full :-)
Quite so. That's why John McCarthy invented garbage collection somewhere
around 1959 for the first implementation of Lisp on the IBM 704. It's not
clear exactly when it was actually added to the implementation, somewhere
between then and 1962. McCarthy credits Daniel J. Edwards with writing
the code, which implements a classical mark-and-sweep algorithm.
> At base, this seems to me to be a philosophical problem - OO GC seems to
> me to be like the old saw: "if a tree falls in the forest,...".
Just so. The collector gains a conspectus of the entire program's state,
constructs a proof that certain parts of that state "cannot possibly
matter to any future computation" (in the words of the Scheme standard),
and frees the space they occupy for reuse.
As you can see above, GC is not tied to OO and is in fact older; the
first recognizable OO system was Simula 67.
> In these FBP implementations, we also provided an IP /tree/ mechanism,
> to allow fairly complex structures of IPs to travel through the network
> using single sends and receives.
I remember implementing that in JavaFBP. Has it been removed from the
current implementations?
> Last point: in today's world, we could have all owned IPs chained into a
> linked list - or, in fact, any collection that supports removal of an IP
> from an arbitrary point in the list. This would then allow the
> programmer to locate undisposed-of IPs much more quickly than having to
> pore through the code... Comments?
A WeakHashMap would work better, I think. (It should be a WeakSet,
but Java doesn't have those.)
--
John Cowan co...@ccil.org http://ccil.org/~cowan
If I have not seen as far as others, it is because giants were standing
on my shoulders.
--Hal Abelson