Yahoo Pipes

[Matt Youell]

If you haven't seen Yahoo Pipes yet, do yourself a favor and take 30
mins to play around with it.

http://pipes.yahoo.com/pipes/

There is a demo video on the site but nothing substitutes for getting
your hands dirty with it. I thought it was fun to play with and it got
my creative juices going.

At some point I heard someone float the idea of an open source version
of Pipes. If that happened I would already be using it in a few
different projects.

--
-/matt/-
http://youell.com/matt

[M. Edward (Ed) Borasky]

1. There is a whole local group devoted to this and similar subjects! It's called the Portland Data Plumbers User Group (pdpug) and we meet once a month. The Google Group is

http://groups.google.com/group/portland-data-plumbing

and the next meeting is next Tuesday

http://upcoming.yahoo.com/event/2123569

2. I wasn't very impressed with Yahoo! Pipes, but a lot of people are, so it's a good thing to know how to do. :)

3. There is someone in pdpug working on an open source equivalent. It's a DSL written in Ruby, with plans for a GUI eventually. I'll hunt down whatever links there are and post them here.

--
M. Edward (Ed) Borasky
http://www.linkedin.com/in/edborasky

I've never met a happy clam. In fact, most of them were pretty steamed.

[M. Edward (Ed) Borasky]

Dawn Foster (who started PDPUG) is doing a course on Yahoo! Pipes next month:

http://fastwonderblog.com/training/

[M. Edward (Ed) Borasky]

Here's the stuff. The project is called Tuttle, the developer is Jerry Hilts (@b3gl). I think it's on GitHub but I haven't been able to get a search to return it. I haven't checked RubyForge but it's definitely in Ruby.

[Matt Youell]

On Apr 24, 7:31 am, "M. Edward (Ed) Borasky" <zzn...@gmail.com> wrote:
> 1. There is a whole local group devoted to this and similar subjects! It's

I wasn't sure if the Data Plumbers was still active so I didn't bother
to mention it. Sorry!

> 2. I wasn't very impressed with Yahoo! Pipes, but a lot of people are, so
> it's a good thing to know how to do. :)

What impressed me about it the most was the UI that they were able to
pull off in a web browser. But aside from that the idea of a schematic
data flow is a fun one. Every now and then you'll see someone trying
to make some tool that does programming via schematic. No one ever
pulls it off, but this works because they narrowed the scope down
enough.

My clients tend to be somewhat technically savvy. I like the idea that
I could use this as a component in a system and trust that they could
make tweaks on their own.

The flip side - which is my main gripe - is that because it isn't
something I can install on my own server or a client's box, the
utility is greatly diminished.

> 3. There is someone in pdpug working on an open source equivalent. It's a
> DSL written in Ruby, with plans for a GUI eventually. I'll hunt down
> whatever links there are and post them here.

"with plans for a GUI eventually" Ugh. Lost me right there. The GUI is
the thing! :)

Thanks Ed!

--
-/matt/-
http://youell.com/matt

[Thomas Lockney]

On Fri, Apr 24, 2009 at 9:35 AM, Matt Youell <softb...@gmail.com> wrote:
> What impressed me about it the most was the UI that they were able to
> pull off in a web browser. But aside from that the idea of a schematic
> data flow is a fun one. Every now and then you'll see someone trying
> to make some tool that does programming via schematic. No one ever
> pulls it off, but this works because they narrowed the scope down
> enough.

Actually, there's some really cool and very successful work out there
in the field of dataflow programming --
http://en.wikipedia.org/wiki/Dataflow_programming. Music and video
processing software have used this paradigm quite a bit: PureData,
Max/MSP, vvvv, etc.

PureData and Max/MSP both come from the same family tree that started
in the mid-80s, so this is not really new territory by any means.

[M. Edward (Ed) Borasky]

1. I found the UI unusable. First of all, I couldn't figure out how to change stuff, and second it was slow because it was chattering back and forth across the Internet.

2.  PureData is open source, MAX/MSP is proprietary.  Thanks for reminding me -- I need to download PureData and start playing with it again. :)

3. Don't get me started on Data Flow languages, Single Assignment Languages, Functional Programming, formal semantics of programming languages, CSP, the Pi-calculus and all the other wonderful breakthroughs in computer science that were supposed to make things wonderful for hundreds of perfectly rational reasons. We're still doing serial imperative programming in C, C++, Java, Perl and PHP for the most part, using C libraries to do concurrency, vectorization, parallel programming, etc. rather than using programming language features, still managing data with relational database management systems, still throwing hardware at scalability problems, still writing tests instead of proving our programs correct, and in general, making a living at it. As far as I'm concerned, the only advance in the art of programming beyond serial imperative programming that's actually taken hold is the spreadsheet. :)

But the good news is that it's now possible to write high-performance computing software in C and the compiler does the heavy lifting. That property used to belong exclusively to FORTRAN.

#getoffmylawn :)

[Matt Youell]

On Apr 24, 10:14 am, Thomas Lockney <tlock...@gmail.com> wrote:
> Actually, there's some really cool and very successful work out there

> in the field of dataflow programming --http://en.wikipedia.org/wiki/Dataflow_programming. Music and video

> processing software have used this paradigm quite a bit: PureData,
> Max/MSP, vvvv, etc.

Yup. I saw PD at Dorkbot and I liked it. Yahoo Pipes succeeds for the
same reason. They are both specialized to a domain; a visual DSL. But
just as no one programs in English (well, no one except Markus), these
tools aren't general programming tools which was my (very minor)
point.

> PureData and Max/MSP both come from the same family tree that started
> in the mid-80s, so this is not really new territory by any means.

The novelty here isn't the new idea, it's the implementation. I've
probably not expressed myself well, but that probably isn't too
important either. Suffice it to say that the mechanics of Yahoo Pipes
isn't novel compared with Unix systems either. In fact to me that is a
huge part of the appeal.

[Thomas Lockney]

On Fri, Apr 24, 2009 at 11:11 AM, Matt Youell <softb...@gmail.com> wrote:
>
> On Apr 24, 10:14 am, Thomas Lockney <tlock...@gmail.com> wrote:
>> Actually, there's some really cool and very successful work out there
>> in the field of dataflow programming --http://en.wikipedia.org/wiki/Dataflow_programming. Music and video
>> processing software have used this paradigm quite a bit: PureData,
>> Max/MSP, vvvv, etc.
>
> Yup. I saw PD at Dorkbot and I liked it. Yahoo Pipes succeeds for the
> same reason. They are both specialized to a domain; a visual DSL. But
> just as no one programs in English (well, no one except Markus), these
> tools aren't general programming tools which was my (very minor)
> point.

Well, you didn't say "general" programming earlier. Notice how far
they both are able to stray outside their supossed-domain, though.

[Thomas Lockney]

On Fri, Apr 24, 2009 at 10:56 AM, M. Edward (Ed) Borasky
<zzn...@gmail.com> wrote:
> 3. Don't get me started on Data Flow languages, Single Assignment Languages,
> Functional Programming, formal semantics of programming languages, CSP, the
> Pi-calculus and all the other wonderful breakthroughs in computer science
> that were supposed to make things wonderful for hundreds of perfectly
> rational reasons. We're still doing serial imperative programming in C, C++,
> Java, Perl and PHP for the most part, using C libraries to do concurrency,
> vectorization, parallel programming, etc. rather than using programming
> language features, still managing data with relational database management
> systems, still throwing hardware at scalability problems, still writing
> tests instead of proving our programs correct, and in general, making a
> living at it. As far as I'm concerned, the only advance in the art of
> programming beyond serial imperative programming that's actually taken hold
> is the spreadsheet. :)

But a large part of the point of this group is *to* get you started.
So rant on..., please! ;~)