Some Statements, Questions and Observations ...

[Harsha]

# Rough Numbers

The number of Programmers ~ 20 million
The number of Programming Languages ~ 20

Number of Data Formats ~ 2000
Number of Algorithms ~ 10,000
Number of Protocols ~ 100
Number of APIs ~ 20,000
Number of Libraries / Language ~ 50

Number of lines of Code for an average application ~ 100,000
Legacy Code ~ 2 trillion

Number of Software Monopolies ~ 10
Number of Software Companies in the world ~ 50,000
Domains of Software ~ Embedded, Communication, Workflows ( Personal / Enterprise / Scientific ), Entertainment

Dominant Programming Paradigms ~ Imperative, Object Oriented followed by Functional.

# Problems with Software

Real Estate Mindset with the Architects, Investors, Brokers, Brochure Makers and Masons.
Hollywood Mindset for Startups.

Lack of Engineering Discipline, Engineering Training and Original Methods. Borrowed methods from Civil Engineering / Mechanical Engineering views dominate.

Existing Methods are Good Enough for Profits.

FBP solves these problems.

# Questions

1. How do we deal with legacy code ?
2. How do we train new programmers and existing programmers ?
3. Do we have a framework for each programming language ?
4. How do we complement existing paradigms ?
5. What is the theory of FBP ?
6. What do we do about the 50 Workflow Systems that look like FBP ? How do we stop people from re-inventing FBP ?

# Observations

On a Personal Level, I am a practical idealist at heart. I am sick and tired for Software Development and Management.

I would like to see through that Software Development be treated like Engineering rather than a profitable Italian Dish. 

FBP doesn't have a monopoly like the "Go" programming language has "Google".

Seeing how scripting languages have become successful,

1. Books / Documentation
2. Applications, Frameworks, Plugins
3. Easy to start using and deploying

For Example - PHP and Wordpress.

# and Suggestions ...

Having seen companies in the Internet-Of-Things space invent another proprietary FBP prompted me to write this.

I don't think FBP is easy to install or use.

If a tree falls in a forest and makes a sound, no one cares. 

I think Internet-Of-Things and Enterprise provide a nice space for exploring useful Applications.

Bluntly put there are no books good books for FBP, the documentation is poor.
I would like to see the following Book Titles for FBP.

1. Dealing with Legacy Code

2. Cookbook for FBP
3. How to develop Games using FBP
4. Enterprise Integration using FBP
5. Control your Robot with xxxx
6. Scaling FBP.
7. FBP for Functional Programmers
8. Develop custom flows for yyyy
9. Stop writing your Workflow Automation solution, use FBP instead !

10. Visual Patterns in Component Architectures

11. FBP for the [Beginning, Intermediate, Expert]

As you can see a lot of work needs to be done !

__END__

[Humberto Madeira]

Hi Harsha,

I am in a quirky Devil`s advocate-y kind of mood today, so I will takre the first bite.

On Saturday, 4 June 2016 11:23:14 UTC-4, Harsha wrote:

# Rough Numbers

The number of Programmers ~ 20 million
The number of Programming Languages ~ 20

More like  ~ 200 or possibly even more - the Tiobe Index tracks the top 100 programming languages and I am aware of quite a few languages not in the Tiobe Index.

Number of Data Formats ~ 2000
Number of Algorithms ~ 10,000
Number of Protocols ~ 100
Number of APIs ~ 20,000
Number of Libraries / Language ~ 50

Number of lines of Code for an average application ~ 100,000

Legacy Code ~ 2 trillion

Number of Software Monopolies ~ 10
Number of Software Companies in the world ~ 50,000
Domains of Software ~ Embedded, Communication, Workflows ( Personal / Enterprise / Scientific ), Entertainment

Dominant Programming Paradigms ~ Imperative, Object Oriented followed by Functional.

In general, I`m pretty sure your observations were picked out of mostly thin air

But more importanly, I`m not sure of their relevance to your thesis

# Problems with Software

Real Estate Mindset with the Architects, Investors, Brokers, Brochure Makers and Masons.

 

Masons?  Really?  Damn those masons for making problems with software ;-)

More seriously, you have to carve out an architectural domain so you can differentiate yourself from the rest (for legal IP purposes and for marketing claims)

Then once you have defined your architectural domaIn, you don`t want to dilute the message, so you double down on your investment.

Since quality of development is so difficult to quantify, most vendors try to differentiate themselves via the presence or absence of features (archtectural or otherwise)

This means that their architecture selection is at least partly involved in specifying their identity.

Go after that and you are going for their identity (do expect some form of retaliation)

 

Hollywood Mindset for Startups.

You do  what you gotta do to attract investors and talent.  Think of peacock mating displays...

It looks weird if you`re not a peacock.  But it seems to work for them.

Lack of Engineering Discipline, Engineering Training and Original Methods. Borrowed methods from Civil Engineering / Mechanical Engineering views dominate.

Mostly it`s because Computer Science courses are taught out of the Mathematics department (an accident of history) instead of the Engineering department.

Mathematicians don`t have Engineering discipline - they have Mathematical rigor instead :-)

Personally, I believe an Engineering approach is the way to go, but then, I was taught as an Engineer, so I suppose I would naturally think that way.

 

Existing Methods are Good Enough for Profits.

I prefer to think of it as - There`s Always Room for More Profits Specially if it Doesn`t Cost Too Much or Force me to Think

FBP solves these problems.

Not that I disagree, but I would enjoin you to elucidate on why you think so.

 

# Questions

1. How do we deal with legacy code ?

 

Wrap it, then Zap it - see the Strangler pattern

 

2. How do we train new programmers and existing programmers ?

Be successful - programmers will flock to you and train themselves.

Do expect that they won`t really change their thinking patterns.

But their eventual replacements might be better.

 

3. Do we have a framework for each programming language ?

There are hundreds of languages - we only need to pick the top few, or even just one.

Each of the top languages has millions of adherents.

 

4. How do we complement existing paradigms ?

We wrap them (FBP is a kind of orchestration paradigm) and we expose our engine as per the service paradigm.

 

5. What is the theory of FBP ?

Depends on what you mean by FBP.  

Paul Morrison is the source for all things that are classic FPB.  Just go read his books or ask him on this forum.

Non-classic FBP is less well standardized (I would go so far as to say there are multiple camps - not sure what they all are)

It is difficult to determine whether a particular implementation is classic-FBP or even to determine what distinguishing features it has. 

A survey might be in order.

 

6. What do we do about the 50 Workflow Systems that look like FBP ? How do we stop people from re-inventing FBP ?

BTW, I am one of those re-inventing people that you seem to want stopped.

As for stopping me...

Well, as long as I believe I have something that I can do even slightly better - I will keep working on it.  I will not be stopped. 

I hope that answered your question sufficiently.

# Observations

On a Personal Level, I am a practical idealist at heart. I am sick and tired for Software Development and Management.

I would like to see through that Software Development be treated like Engineering rather than a profitable Italian Dish. 

If you want to discourage profitable Italian cuisine, stop working for companies that treat Software Development in that way.

You have feet, I presume.  Go ahead and use them to vote for what you want.

FBP doesn't have a monopoly like the "Go" programming language has "Google".

Seeing how scripting languages have become successful,

1. Books / Documentation
2. Applications, Frameworks, Plugins
3. Easy to start using and deploying

For Example - PHP and Wordpress.

# and Suggestions ...

Having seen companies in the Internet-Of-Things space invent another proprietary FBP prompted me to write this.

I don't think FBP is easy to install or use.

So go make something that is.  

I find it somewhat incongruent that you want to stop re-invention efforts, yet you still complain about the existing implementation.

Paul Morrison is one person.  He has done a great job of taking things this far and he has made his code open source

But he maybe could use some help.  Code something and propose it to him to add to his implementation.

Or code it yoursef and offer it to the world.

 

If a tree falls in a forest and makes a sound, no one cares. 

Depends where the tree falls - if it falls onto a campfire a lot of someones might care.

Software ventures generally start off small, but then a few of them can catch interest and suddenly they take off.

 

I think Internet-Of-Things and Enterprise provide a nice space for exploring useful Applications.

Most programmers nowadays wouldn`t know their way around the wet end of a soldering iron without having to go Google it.

They are not so useful for the hardware development side.

Alternatively, you can use one of the pre-packaged development kits, but that tends to add cost.

And margins tend to be razor-thin (which is why Asia produces most electronics now).   

In any case, a lot of hardware startups just simply fail, so they don`t attract investors very well.

Which is why you see so many of them on Kickstarter.

But if you can get the money somehow and can last until sales start taking off... then it might be worth a shot.

You can`t win if you don`t place a bet.

 

Bluntly put there are no books good books for FBP, the documentation is poor.
I would like to see the following Book Titles for FBP.

1. Dealing with Legacy Code

2. Cookbook for FBP
3. How to develop Games using FBP
4. Enterprise Integration using FBP
5. Control your Robot with xxxx
6. Scaling FBP.
7. FBP for Functional Programmers
8. Develop custom flows for yyyy
9. Stop writing your Workflow Automation solution, use FBP instead !

10. Visual Patterns in Component Architectures

11. FBP for the [Beginning, Intermediate, Expert]

In the mid-90`s books were how programmers learned things.  At least, I did.

Nowadays, I`m not so sure.  I think programming books are too slow.  And too costly.  And too rigid.

Web sites are cheaper all around and can stay more up-to-date.

 

As you can see a lot of work needs to be done !

On that, we agree.

Best Regards,

--Bert

 

[Harsha]

4. How do we complement existing paradigms ?

We wrap them (FBP is a kind of orchestration paradigm) and we expose our engine as per the service paradigm.

 

Agreed.

 

5. What is the theory of FBP ?

Depends on what you mean by FBP.  

Paul Morrison is the source for all things that are classic FPB.  Just go read his books or ask him on this forum.

Non-classic FBP is less well standardized (I would go so far as to say there are multiple camps - not sure what they all are)

It is difficult to determine whether a particular implementation is classic-FBP or even to determine what distinguishing features it has. 

A survey might be in order.

 

6. What do we do about the 50 Workflow Systems that look like FBP ? How do we stop people from re-inventing FBP ?

BTW, I am one of those re-inventing people that you seem to want stopped.

As for stopping me...

Well, as long as I believe I have something that I can do even slightly better - I will keep working on it.  I will not be stopped. 

I hope that answered your question sufficiently.

 

I would consider Actor Model as a theory.

Smalltalk to a lesser degree, Erlang to a greater degree are implementations of that theory.

From the discussion here, 

https://groups.google.com/d/msg/flow-based-programming/4YAHdihBwSg/_i-nn683eCIJ 

and https://en.wikipedia.org/wiki/Kahn_process_networks

FBP is very close to KPN.

 

A stronger version way to rephrase the question is, Can FBP formalilized like the Actor Model or KPN ?

I think a lot of us got here by doing our own implementations :)

I just see it as wasted collective efforts.

 

Having seen companies in the Internet-Of-Things space invent another proprietary FBP prompted me to write this.
I don't think FBP is easy to install or use.

So go make something that is.  
I find it somewhat incongruent that you want to stop re-invention efforts, yet you still complain about the existing implementation.
Paul Morrison is one person.  He has done a great job of taking things this far and he has made his code open source
But he maybe could use some help.  Code something and propose it to him to add to his implementation.

That is a fair point.

Paul Morrison or anyone from the community feel free to list down any tasks / places you want to receive help in.

Simply put in the long run, I don't want to duplicate my efforts.

 

Bluntly put there are no books good books for FBP, the documentation is poor.

I would like to see the following Book Titles for FBP.

1. Dealing with Legacy Code

2. Cookbook for FBP
3. How to develop Games using FBP
4. Enterprise Integration using FBP
5. Control your Robot with xxxx
6. Scaling FBP.
7. FBP for Functional Programmers
8. Develop custom flows for yyyy
9. Stop writing your Workflow Automation solution, use FBP instead !

10. Visual Patterns in Component Architectures

11. FBP for the [Beginning, Intermediate, Expert]

In the mid-90`s books were how programmers learned things.  At least, I did.

You got me. I do think there is dire need for more books / information. I have read Paul's book.

It is a good starting point but it is neither a Reference nor an Good cookbook.

There are possibly 100,000 developers out there willing to use FBP right away.

A good book is definitely stopping me.

[Matthew Lai]

On Saturday, 4 June 2016 15:53:10 UTC-4, Harsha wrote:

A good book is definitely stopping me.

And if the "good" book does not come to your rescue in the next 10 years...

I'll suggest it is your mindset stopping you, not the book.

I needed a FBP frontend that would handle both the graph construction and display runtime status report (cpu utilization and number of IP in the FIFO)

back 4 years ago and there was nothing like that back then so I simply developed one for myself and put it to good practical use

(financial data web scraping). There is always, always a way, if you want something badly.

Write that book to "unstop" yourself.

Yours,

Matt

[Paul Morrison]

Great discussion! 

Matt, I totally agree - I believe Harsha has all the information he needs to start designing in classical FBP!  And currently 4 languages to choose from - plus yours - I'm sure you would be happy to share!  He just needs to start designing! 

And Bert, thanks for the kind words! I _would_ like some help - in almost all the areas Harsha lists.  However  I would also like to express my appreciation for all the people over the years who _have_ contributed ideas, codes, PR, etc. Too many to count, but I think there's still plenty of room in the tent!

--
You received this message because you are subscribed to the Google Groups "Flow Based Programming" group.
To unsubscribe from this group and stop receiving emails from it, send an email to flow-based-progra...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Humberto Madeira]

Hi Harsha,

On Saturday, 4 June 2016 15:53:10 UTC-4, Harsha wrote:

I would consider Actor Model as a theory.

Smalltalk to a lesser degree, Erlang to a greater degree are implementations of that theory.

From the discussion here, 

https://groups.google.com/d/msg/flow-based-programming/4YAHdihBwSg/_i-nn683eCIJ 

and https://en.wikipedia.org/wiki/Kahn_process_networks

FBP is very close to KPN.

 

KPN sounds like something vaguely in-between my approach and that of classical FBP.

However my particular flow engine approach is actually closer to that of SEDA (if you squint quite a bit). 

Or possibly it could just be considered vaguely as an outgrouth of  Message passing (the asynchronous kind)

 

It just goes to show that there are a lot of subtly-different approaches to the way you can pass data between black box components.

Others may hold a different opinion, but my personal take on it is that the mechanism for passing data between components is not that important, as long as the effects are the same.

In other words - the components ideally shouldn`t be able to tell whether they are in one engine or another.

I don`t believe that is quite true for classic FBP, since the classic FBP design relies on certain assumptions about how data is delivered, 

such as ordered arrival (implicit ordering).

Paul Morrison believes that this is one of the strengths of FBP, and it certainly makes things less complicated, which is definitely a good thing.

But there are also certain advantages to doing it differently, and I have been exploring one of those paths less-travelled.

One unfortunate side-effect of the variation in expectations is that the systems are incompatible (the components are not interchangeable).  

I have been studying classic FBP  to see if I can reduce that incompatibility through emulation of the classic FBP feature set.

But at this point, I suspect that the effort will likely not be sufficiently successful, but I also believe that the effort 

will still be useful nonetheless (if you`re going to steal ideas, steal from the best)

But even so, all of this is just regarding the processing engine.

A platform framework is more than just a processing engine.  It has to have support for UI and data modeling as well. 

UI`s can be composed from smaller pieces, and so can models and domains.

My framework has always included these items from the get go.  Classic FBP considers them external and leaves them up to the programmer.  

There are benefits to both packaging approaches.  But I believe they address entirely different audiences.

 

A stronger version way to rephrase the question is, Can FBP formalilized like the Actor Model or KPN ?

Personally, (although I don`t believe there has ever been a consensus on this) I think the important distinguishing features of generic FBP are the (stateful) black box processes, 

the encapsulation of the objects passed between them, and to a lesser extent, the asynchrony of the connections between them.

I also believe that an FBP implementation should be able to support the concept of visual (graphical) programming and ideally should provide such a component.

FBP`s greatest strengths are in its loose coupling and in its ability to stitch together (orchestrate) components that were developed independently.

Unfortunately, I don`t think this is enough to allow for formalisms.

Perhaps others in this group will add additional distinguishing features to narrow things down, 

but the result will be to exclude some implementations that have previously been mentioned in this group.

Add in enough distinguishing features, and you will be left with only classic FBP (which can certainly be formalized).

In any case, however you narrow things down, this won`t make the other implementations go away.

Maybe they will surface to form competing incommunicado communities.  -  I`m not sure that this would be net a benefit.

 

I think a lot of us got here by doing our own implementations :)

I just see it as wasted collective efforts.

Personally, I see diversity as good.  It allows for evolution.

Evolution allows for the filling of niches, and for improved adaptation (it seems to work really well in biological systems)

Many things die along the way, but evolution is not about individual survival.

Not so long ago, I used to program in Z80 assembler with DJNZ commands for looping.

Nowadays, not so much :-)

Most of  that is because of many many individual inventors that wanted to do something slightly differently.

That were then replaced by something else also slightly different.

Those now-fallen systems were the so-called giant shoulders we stood on to get where we are today.

IMHO they were hardly wasted effort.  Newer designs are informed by the results of older designs.

And heck, this is software, older designs can even incorporate ideas from newer designs.

Or do you believe that somehow we are at the end of evolution, with no further path to follow? no more ideas to add?

IMHO, the evidence strongly suggests otherwise.

And BTW, one of the primary purposes of FBP`s loose coupling is to allow programs to evolve quickly.

There are possibly 100,000 developers out there willing to use FBP right away.

My 3rd year thermodynamics professor (an emeritus dean of Engineeriring at the time) put it in this way.

There are 10,000 people for every engineer in this world.  Your education, in part, represents their investment in you.

 

What have you done for your 10,000 today?

Motivation indeed.  But also responsibility.

Those 100,000 developers will need guidance and support.  They, and those that fund them will want assurances their investment will not be wasted.

I believe it is up to those of us already in the know to provide them these things - in one way or another.

And yes, there is a lot of work to be done.

Best Regards,

--Bert

 

 

[Humberto Madeira]

Hi Matt,

I would just like to add that I also have seen your UI demo and it was very cool, very impressive indeed.

Perhaps you might weant to consider putting it up on youtube?  or perhaps you`ve already done something similar?

Regards,

--Bert

[Humberto Madeira]

Hi Paul,

On Sunday, 5 June 2016 13:41:38 UTC-4, Paul Morrison wrote:

And Bert, thanks for the kind words! I _would_ like some help - in almost all the areas Harsha lists. 

I don`t have a lot of time to give. What with doing my own implementation and all.

But if you are OK with it, I can download JavaFBP and see if I can do something to assist.

It may not necessarily be on the programming side (some of my non-classic ideas might be too much cognitive dissonance for classic FBP), 

but I like to think I am a developer of many parts :-)

Regards,

--Bert 

[Harsha]

On Monday, June 6, 2016 at 12:35:31 AM UTC+5:30, Humberto Madeira wrote:

Hi Harsha,

There are possibly 100,000 developers out there willing to use FBP right away.

My 3rd year thermodynamics professor (an emeritus dean of Engineering at the time) put it in this way.

There are 10,000 people for every engineer in this world.  Your education, in part, represents their investment in you.

 

What have you done for your 10,000 today?

Motivation indeed.  But also responsibility.

Humbling words. Thanks !

[Paul Morrison]

Hi Bert, that's a very kind offer!  I actually think that the help needed will not be so much on the programming side, as in spreading the word about FBP in general, explaining, presenting, finding out about useful conferences, writing papers, selling concepts to VPs, universities, etc. - all areas that I'm not much good at!  Actually FBP should be part of every CS curriculum... I wonder if it is being included, anywhere...?

Of course I'm sure the JavaFBP code can be improved - but we started with John Cowan's architecture, and a number of people have improved it since - the more important point IMHO is to get it (and FBP in general) used!  Hopefully there won't be so much distance between our concepts that we spend too much time analyzing differences, and not enough time finding commonalities!

I am convinced you both are right about the problem with CS's historical position as a branch of Maths - application development is definitely a branch of engineering (I'm an anthropologist myself!).  A lot of us old-guard FBPers have been saying that for a long time.  Some people accuse me of thinking like an engineer - I'll take that a a compliment!

Regards,

Paul

--

[Paul Morrison]

One of the grand old men of computing, Nate Edwards, is fond of saying, "Scientists try to find out what is; engineers build what never was."