NoFlo 0.4.1 released

479 views
Skip to first unread message

Henri Bergius

unread,
Sep 26, 2013, 4:34:40 AM9/26/13
to flow-based-...@googlegroups.com
Hi,

I pushed yesterday the NoFlo 0.4.1 release out.

Apart from some performance improvements, the main change is support
for stopping either individual processes or an entire network.

Prior to this, the NoFlo network would keep going until all packets
have been processed through, but now you can stop the execution at any
time.

This is especially useful for the NoFlo Development Environment
(https://github.com/noflo/noflo-ui) which can utilize the NoFlo
Network Protocol to control execution (modify graphs, start and stop
networks).

This doesn't require changes to most components, but components that
keep listening to some external input (for example, a web server
listening to requests) or run their own timers (like RunInterval) need
to implement the "shutdown" method to stop those things from
transmitting new packets.

https://npmjs.org/package/noflo
https://github.com/noflo/noflo/blob/master/CHANGES.md

--
Henri Bergius
Decoupling software, one piece at a time.
http://bergie.iki.fi/
@bergie

Matthew Lai

unread,
Sep 26, 2013, 8:41:06 AM9/26/13
to flow-based-...@googlegroups.com
For stopping the entire network:

How about those IPs that are "in transit"? Do you allow a "drain" period to let the
downstream components to finish processing the IPs after the upstream components
are shut down?

How about those components executing cpu bound job eg. if I have a component doing
a long Fast Fourier Transform that may take a minute, will that component get some kind
of a kill action, like the "kill -9" in *nix for example?

Matt

Henri Bergius

unread,
Sep 26, 2013, 9:10:32 AM9/26/13
to flow-based-...@googlegroups.com
Hi,

On Thu, Sep 26, 2013 at 2:41 PM, Matthew Lai <mm...@sympatico.ca> wrote:
> How about those IPs that are "in transit"? Do you allow a "drain" period to
> let the downstream components to finish processing the IPs after the upstream
> components are shut down?

Yes, the shutdown will only tell components to shut down, and will
disconnect (signal EOT) to open connections between components.
Everything that is still in transit (say, in the input buffer for
AsyncComponents) will still happen.

> How about those components executing cpu bound job eg. if I have a component
> doing a long Fast Fourier Transform that may take a minute, will that component
> get some kind of a kill action, like the "kill -9" in *nix for example?

The component will receive a 'shutdown' call, and it is up to the
component to decide what to do. For example, an interval timer
component should kill the timer. And I/O operation should probably let
the operation still complete.

Components that have real-world side effects are very interesting in
this regard. Over lunch we had a discussion on what the AR.Drone
components (see http://bergie.iki.fi/blog/noflo-ardrone/) should do on
shutdown. Should the movement commands just send a zero movement value
(stopping motion into that direction), should they stop-and-hover the
drone, or should they command it to land?

> Matt

/Henri

Giant Elk

unread,
Oct 1, 2013, 8:58:00 PM10/1/13
to flow-based-...@googlegroups.com
Hi Henri,

I was impressed with NoFlo when I first heard of the project. My claim to fame is I wrote a C pretty printer back in school and used Data Flow diagrams to make sense of how to build it, so NoFlo just rang a bell in my head. I can relate to the concepts, and playing around with Ruby on Rails is not allowing me to think about the app the way I envisioned. It makes me focus too much on implementation VS design.  

Are you planning on starting a Google+ page for NoFlo? Google+ is great for quick Q & A's and sharing best practices in short bursts.

Can you post some screen shot, examples of the debugger output, i.e. $ ./node_modules/.bin/noflo --debug graphs/ShowContents.fbp

Any plans for some easy to follow tutorials and examples along the lines of: - simple "Hello World". - a web site "Hello World". - a nice Twitter or Blog style tutorial like in the Ruby on Rails site. i.e. post a blog entry, and allow comments against the blog. Single user for simplicity. - a basic 2D game, like pong or space invaders, look at Codea on the iPad for some good examples, this is revolutionary (just clunky as it's tied to the iPad): http://twolivesleft.com/Codea
- mobile phone / tablet app: how can I get a NoFlo project to work on Andriod or iOS? I'm guessing a stand alone app wouldn't work, that Node.js would need to be on the server?

The success of opensource projects is based on many factors, but 1 common thread of success is great tutorials and documentation (making it simple and easy for people to use it in real world projects). For guys like me that are working 10 hours a day at a day job, making a decision how to spend my spare time using all the various opensource projects is overwhelming, and not enough hours to try them all out. So we rely on guys like yourself leading the way with good examples/tutorials. AngularJS is a great example, their home page has a few simple examples and you get it right away just how cool that project is.

Also, a screen cast would go a long way, showing off some of the things you can do. The YouTube videos of Ryan Dahl talking and demo'ing Node and the 10 min blog video the Rails creator made really helped casual developers get a handle and build huge momentum for Node and Rails. If you can do something similar it would be greatly appreciated. 

I'd be happy to help contribute to some NoFlo tutorials, as it would be a great way for me to learn more about how to use NoFlo in my personal projects, but need some guidance to get pointed in the right direction.

btw - as you can guess from the examples I'd like to see above, that I'm working on a web app & mobile app, that uses a database (like most web apps), something along the lines of LinkedIn. And various mobile apps from stand alone games and information type reader apps that are also stand alone. Right now I'm dropping Rails and starting up with Node/Express/Mongo. But I still want to see how for I can get with NoFlo, I think in data flows.


Thanks,
Giant Elk

Henri Bergius

unread,
Oct 2, 2013, 5:06:15 AM10/2/13
to flow-based-...@googlegroups.com
Hi,

On Wed, Oct 2, 2013 at 2:58 AM, Giant Elk <flyinghor...@gmail.com> wrote:
> Are you planning on starting a Google+ page for NoFlo? Google+ is great for
> quick Q & A's and sharing best practices in short bursts.

We actually have one already:
https://plus.google.com/112372998187205178398/posts

And then of course there is the FBP G+ community.

> Can you post some screen shot, examples of the debugger output, i.e. $
> ./node_modules/.bin/noflo --debug graphs/ShowContents.fbp

http://i.imgur.com/0nltOP2.png

This graph (from the NoFlo getting started page
http://noflojs.org/documentation/) is pretty much as simple as it can
get... there are two nodes (read file and print output) that are
connected, and a filename to read gets sent as an IIP to the file
reader.

The debug output shows the sequence of packet events inside the NoFlo network:
* At startup the IIP ("DATA") gets sent to the IN port of the Read
node. This happens in three stages:
- CONNECT ("I'm going to send you data")
- DATA ("Here is one information packet")
- DISCONNECT ("That is all for now")
* The file reader receives the IP, and does a CONNECT to its outport
("I'm going to send you data soon")
* The the file reader reads the file asynchronously
* Once the file has been read, the file reader sends a begin
group/opening bracket containing the filename ("packets from now are
from this file")
* The file contents get sent as DATA
* The bracket/group is closed, and there is a DISCONNECT ("I've sent
all I have for now")

The console output is shown in debug before the DATA event as the
component receives that faster than the debugger does :-)

For comparison, here is the same simple graph in the NoFlo Development
Environment where you can also see the data passing through
connections using the edge inspector:
http://i.imgur.com/6z5vrcI.png

> Any plans for some easy to follow tutorials and examples along the lines of:
> - simple "Hello World". - a web site "Hello World". - a nice Twitter or Blog
> style tutorial like in the Ruby on Rails site. i.e. post a blog entry, and
> allow comments against the blog. Single user for simplicity. - a basic 2D
> game, like pong or space invaders, look at Codea on the iPad for some good
> examples, this is revolutionary (just clunky as it's tied to the iPad):
> http://twolivesleft.com/Codea

Yeah, we're certainly interested in making more examples, especially
now that the latest iteration of NoFlo Development Environment makes
it easy to publish and load those as Gists.

Here is one quite recent one, a battery gauge for the AR.Drone:
http://noflojs.org/noflo-ui/#example/6733886
(a client-side NoFlo graph that talks to a server-side NoFlo graph
over WebSockets, that then talks to the AR.Drone using UDP)

> - mobile phone / tablet app: how can I get a NoFlo project to work on
> Andriod or iOS? I'm guessing a stand alone app wouldn't work, that Node.js
> would need to be on the server?

Right now NoFlo is constrained to JavaScript, which means that the
easiest way to build mobile apps with it is to do so using PhoneGap
(http://phonegap.com/).

Actually, the NoFlo Development Environment is a client-side NoFlo app
(most of the logic is handled by various NoFlo graphs), and we've
already packaged it as an Android app using this technique.

Of course FBP is in no way limited to JavaScript and NoFlo. In the
long run there will probably be Java and ObjC FBP frameworks that are
compatible with NoFlo's network protocol so that you can use the NoFlo
Development Environment to build native mobile apps.

> The success of opensource projects is based on many factors, but 1 common
> thread of success is great tutorials and documentation (making it simple and
> easy for people to use it in real world projects).
<snip>
> Also, a screen cast would go a long way, showing off some of the things you
> can do. The YouTube videos of Ryan Dahl talking and demo'ing Node and the 10
> min blog video the Rails creator made really helped casual developers get a
> handle and build huge momentum for Node and Rails. If you can do something
> similar it would be greatly appreciated.

We're intending to make a lot more tutorials and other documentation.
However, I've been hesitant to do that so far because the tools are
still changing so rapidly. Once we have a more stable NoFlo
Development Environment it will make more sense to build examples.

> I'd be happy to help contribute to some NoFlo tutorials, as it would be a
> great way for me to learn more about how to use NoFlo in my personal
> projects, but need some guidance to get pointed in the right direction.

That would be awesome! Just let us know what you need :-)

> btw - as you can guess from the examples I'd like to see above, that I'm
> working on a web app & mobile app, that uses a database (like most web
> apps), something along the lines of LinkedIn. And various mobile apps from
> stand alone games and information type reader apps that are also stand
> alone. Right now I'm dropping Rails and starting up with Node/Express/Mongo.
> But I still want to see how for I can get with NoFlo, I think in data flows.

I think Kenneth recently made some examples on how to build REST
servers with NoFlo. I hope he'll publish those soon so you can have a
starting point.

> Giant Elk

/Henri

Giant Elk

unread,
Oct 2, 2013, 10:28:23 PM10/2/13
to flow-based-...@googlegroups.com
Thanks,

I didn't think to search for NoFlo on the Pages in Google+, however what I really meant is a Google+ Community for NoFlo. The Community is where the most discussion occurs.

Thanks for the other clarifications. I understand the file Read to Display example. This would be good to show the output on the web site in a screen shot, and use a file that has simple "Hello World" text. It's the example below I can't figure out, how to show "Hello World" without a starting file and display in a web browser:

I played with the GUI a bit, I realize it's still pre-release, but couldn't get simple text or HTML showing in the Preview window, or Chrome's Console or Local Storage. Not sure if you can see this, or if it's just in my browser cache:
- I pasted the Graph JSON below

btw - what is IIP, is that the same as I2P:

I couldn't make heads or tails of this, other than a nice coloured bar:
> Environment where you can also see the data passing through
connections using the edge inspector:
http://i.imgur.com/6z5vrcI.png

JSON Graph:

  "properties": {
    "environment": {
      "runtime": "html",
      "src": "./preview/iframe.html",
      "width": 300,
      "height": 300
    },
    "name": "testGE"
  },
  "exports": [],
  "processes": {
    "core/Output_3ogtl": {
      "component": "core/Output",
      "metadata": {
        "x": 757.4583435058594,
        "y": 162.4340362548828,
        "label": "core/Output"
      }
    },
    "dom/WriteHtml_9z6xj": {
      "component": "dom/WriteHtml",
      "metadata": {
        "x": 452.4583435058594,
        "y": 206.65626525878906,
        "label": "dom/WriteHtml"
      }
    },
    "strings/SendString_lh9a4": {
      "component": "strings/SendString",
      "metadata": {
        "x": 447.4583435058594,
        "y": 63.18403625488281,
        "label": "strings/SendString"
      }
    },
    "localstorage/SetItem_phlql": {
      "component": "localstorage/SetItem",
      "metadata": {
        "x": 464.4583435058594,
        "y": 373.9618148803711,
        "label": "localstorage/SetItem"
      }
    },
    "core/Kick_9gclq": {
      "component": "core/Kick",
      "metadata": {
        "x": 204.45834350585938,
        "y": 218.1007080078125,
        "label": "core/Kick"
      }
    },
    "core/MakeFunction_t4atw": {
      "component": "core/MakeFunction",
      "metadata": {
        "x": 213.45834350585938,
        "y": 330.4340362548828,
        "label": "core/MakeFunction"
      }
    }
  },
  "connections": [
    {
      "src": {
        "process": "dom/WriteHtml_9z6xj",
        "port": "container"
      },
      "tgt": {
        "process": "core/Output_3ogtl",
        "port": "in"
      },
      "metadata": {
        "route": 0
      }
    },
    {
      "src": {
        "process": "strings/SendString_lh9a4",
        "port": "out"
      },
      "tgt": {
        "process": "core/Output_3ogtl",
        "port": "in"
      },
      "metadata": {
        "route": 0
      }
    },
    {
      "src": {
        "process": "localstorage/SetItem_phlql",
        "port": "item"
      },
      "tgt": {
        "process": "core/Output_3ogtl",
        "port": "in"
      },
      "metadata": {
        "route": 0
      }
    },
    {
      "src": {
        "process": "core/Kick_9gclq",
        "port": "out"
      },
      "tgt": {
        "process": "strings/SendString_lh9a4",
        "port": "in"
      },
      "metadata": {
        "route": 0
      }
    },
    {
      "src": {
        "process": "core/MakeFunction_t4atw",
        "port": "function"
      },
      "tgt": {
        "process": "core/Output_3ogtl",
        "port": "in"
      },
      "metadata": {
        "route": 0
      }
    },
    {
      "data": "hey hey",
      "tgt": {
        "process": "dom/WriteHtml_9z6xj",
        "port": "html"
      }
    },
    {
      "data": "Birds",
      "tgt": {
        "process": "strings/SendString_lh9a4",
        "port": "string"
      }
    },
    {
      "data": "wow",
      "tgt": {
        "process": "localstorage/SetItem_phlql",
        "port": "key"
      }
    },
    {
      "data": "wee",
      "tgt": {
        "process": "localstorage/SetItem_phlql",
        "port": "value"
      }
    },
    {
      "data": "4",
      "tgt": {
        "process": "core/MakeFunction_t4atw",
        "port": "in"
      }
    },
    {
      "data": "4*4",
      "tgt": {
        "process": "core/MakeFunction_t4atw",
        "port": "function"
      }
    }
  ]
}

- Thanks!

John Cowan

unread,
Oct 2, 2013, 11:24:50 PM10/2/13
to flow-based-...@googlegroups.com
Giant Elk scripsit:

> btw - what is IIP, is that the same as I2P:

No, it stands for Initialization Information Packet, a packet that's
created directly by the framework in order to initialize a component.
When a component starts up, it reads from input ports connected to IIPs
in order to parameterize itself.

--
John Cowan co...@ccil.org http://ccil.org/~cowan
The whole of Gaul is quartered into three halves.
--Julius Caesar

Paul Morrison

unread,
Oct 3, 2013, 1:53:50 PM10/3/13
to flow-based-...@googlegroups.com
On 02/10/2013 10:28 PM, Giant Elk wrote:
> Not sure if you can see this, or if it's just in my browser cache:
> http://noflojs.org/noflo-ui/#graph/testGE

On my machine this is another black-on-black diagram - at least I assume
there is something there... There are 3 lines of white text - the rest
is black. I have also had that problem with some flow diagrams (on
Github?). Does something have to be parametrized to make things
visible? Thanks.

Giant Elk

unread,
Oct 3, 2013, 2:19:44 PM10/3/13
to flow-based-...@googlegroups.com
Hi Paul,
That GUI editor stores the graph JSON in the browsers local storage. I woke up this morning and realized its likely not running off NoFlo in the cloud like the public demos. And I don't have Node or NoFlo installed on the machine I was using.

If you copy the JSON code above into a new Config window it may show you what I was attempting. 

I'll give it another try tonight. I'm also going to give Node RED a go, see how they do things. One of their shirt tutorials on the docs gives an example of hooking RED into an Express project, it parses data from the UK gov site on power usage. Neat example. Take a look if you have a moment. 

 


--

Cheers,
Flying Horse Dancing

Kenneth Kan

unread,
Oct 3, 2013, 4:18:38 PM10/3/13
to flow-based-...@googlegroups.com

Any plans for some easy to follow tutorials and examples along the lines of: - simple "Hello World". - a web site "Hello World". - a nice Twitter or Blog style tutorial like in the Ruby on Rails site. i.e. post a blog entry, and allow comments against the blog. Single user for simplicity. - a basic 2D game, like pong or space invaders, look at Codea on the iPad for some good examples, this is revolutionary (just clunky as it's tied to the iPad): http://twolivesleft.com/Codea
- mobile phone / tablet app: how can I get a NoFlo project to work on Andriod or iOS? I'm guessing a stand alone app wouldn't work, that Node.js would need to be on the server?

The success of opensource projects is based on many factors, but 1 common thread of success is great tutorials and documentation (making it simple and easy for people to use it in real world projects). For guys like me that are working 10 hours a day at a day job, making a decision how to spend my spare time using all the various opensource projects is overwhelming, and not enough hours to try them all out. So we rely on guys like yourself leading the way with good examples/tutorials. AngularJS is a great example, their home page has a few simple examples and you get it right away just how cool that project is.

Also, a screen cast would go a long way, showing off some of the things you can do. The YouTube videos of Ryan Dahl talking and demo'ing Node and the 10 min blog video the Rails creator made really helped casual developers get a handle and build huge momentum for Node and Rails. If you can do something similar it would be greatly appreciated. 

Hello Giant! These are really good ideas! Like Henri said, I was indeed making a screencast but I caught a nasty cold half way done and now I have to catch up with my day job obligations first. Hopefully by next week I could get back to the game.

So the screencast that I'm making is a simple todo API + an incomplete todo webapp hosted on Heroku with PostgreSQL as the backend. I can take that apart into several installments as I kind of feel like it's too bloated for starter. Is there anything that a newcomer of NoFlo would absolutely love to see? Like: how NoFlo really works, or the amazing benefits of programming in FBP mindset, or actually seeing something that you can replicate easily? I know it's ideal to do all that but given the constraint of time and people's attention span I figure I'd solicit some opinion on how I should structure this thing.
 
I'd be happy to help contribute to some NoFlo tutorials, as it would be a great way for me to learn more about how to use NoFlo in my personal projects, but need some guidance to get pointed in the right direction.

btw - as you can guess from the examples I'd like to see above, that I'm working on a web app & mobile app, that uses a database (like most web apps), something along the lines of LinkedIn. And various mobile apps from stand alone games and information type reader apps that are also stand alone. Right now I'm dropping Rails and starting up with Node/Express/Mongo. But I still want to see how for I can get with NoFlo, I think in data flows.

Yea... Rails is way too bloated. That's why I dropped it long ago too.

I think there's still a big question in the NoFlo community on where NoFlo should go. Right now it has two parts: one on the browser running on Component.js and one on Node.js. I wonder if it's a good idea to set a long-term vision of making NoFlo work like Meteor.js, which has garnered quite some attention lately because of the code-in-one-platform-and-have-all-of-the-cake value prop, though I'm told repeatedly that it's extremely slow, rendering it a "toy platform". Given the strengths of NoFlo, I can't see why we don't want to pursue the client-server-as-one-environment direction. What do you guys think?
Reply all
Reply to author
Forward
0 new messages