Account Options

  1. Sign in
The old Google Groups will be going away soon, but your browser is incompatible with the new version.
Google Groups Home
« Groups Home
promise-stream: Turn a stream into a commonJS promise
There are currently too many topics in this group that display first. To make this topic appear first, remove this option from another topic.
There was an error processing your request. Please try again.
flag
  6 messages - Collapse all  -  Translate all to Translated (View all originals)
The group you are posting to is a Usenet group. Messages posted to this group will make your email address visible to anyone on the Internet.
Your reply message has not been sent.
Your post was successful
 
From:
To:
Cc:
Followup To:
Add Cc | Add Followup-to | Edit Subject
Subject:
Validation:
For verification purposes please type the characters you see in the picture below or the numbers you hear by clicking the accessibility icon. Listen and type the numbers you hear
 
Raynos  
View profile  
 More options Oct 3 2012, 12:40 am
From: Raynos <rayn...@gmail.com>
Date: Tue, 2 Oct 2012 21:40:02 -0700 (PDT)
Local: Wed, Oct 3 2012 12:40 am
Subject: [ANN] promise-stream: Turn a stream into a commonJS promise

https://github.com/Raynos/promise-stream

If you want to turn a stream into a promise you can.

This mainly illustrates how streams and promises are similar for those that
like promises.

It also shows a comparison between how you flow async code through promises
and streams


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Mariusz Nowak  
View profile  
 More options Oct 3 2012, 4:11 am
From: Mariusz Nowak <mari...@medikoo.com>
Date: Wed, 3 Oct 2012 01:11:56 -0700 (PDT)
Local: Wed, Oct 3 2012 4:11 am
Subject: Re: [ANN] promise-stream: Turn a stream into a commonJS promise

Raynos, I have a feeling that you are comparing apples and oranges. Streams
are awesome, Promises are awesome but it's not the same, and not every use
case can be addressed on equal level by both of them.

From my experience there can be promises that are also a streams, but among
them are also promises that fulfill before stream ends. There can also be
promises that are not streams, of course you may represent them as stream
but if they bring no value and just emit 'end', it doesn't fit well stream
concept, does it?

Same way you can just use callbacks instead of promises, but the point is
that promises gives you better abstraction, so it's easier to construct
complicated flow with it and maintain it. See this:
https://github.com/medikoo/deferred#promises-approach How would you write
it with streams instead? Would it be more friendly than callbacks
version: https://github.com/medikoo/deferred#plain-nodejs ?


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Discussion subject changed to "[ANN] promise-stream: Turn a stream into a commonJS promise" by Jake Verbaten
Jake Verbaten  
View profile  
 More options Oct 3 2012, 12:40 pm
From: Jake Verbaten <rayn...@gmail.com>
Date: Wed, 3 Oct 2012 09:39:52 -0700
Local: Wed, Oct 3 2012 12:39 pm
Subject: Re: [nodejs] Re: [ANN] promise-stream: Turn a stream into a commonJS promise

Streams are the better abstraction for that.

Notice that with the streams version you can actually start writing to the
disk once you've successfully loaded a single file.

```
var chain = require("chain-stream")
    , fs = require("fs")
    , DirStream = require("dir-stream") // to be written
    , d = require("domain").create()

d.run(function () {
    chain(DirStream(__dirname))
        .filter(function (fileName) {
            return (file.slice(-3) === ".js" && (file !== "lib.js"))
        })
        .map(readFile)
        .pipe(fs.createWriteStream(__dirname + "/lib.js"))

})

d.on("error", function ignoreIt() {})
```


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Jake Verbaten  
View profile  
 More options Oct 3 2012, 1:11 pm
From: Jake Verbaten <rayn...@gmail.com>
Date: Wed, 3 Oct 2012 10:11:04 -0700
Local: Wed, Oct 3 2012 1:11 pm
Subject: Re: [nodejs] Re: [ANN] promise-stream: Turn a stream into a commonJS promise

The other feature is that this is all in node core. Isaacs 0.9 streams will
have the notion of a transform stream.

We can also petition node core to make fs.createReadStream(dirname) work
because a directory is just a fd with a list of files so it should work.

Then the program becomes

```
var fs = require("fs")
    , d = require("domain").create()
    , Transform = require("stream").Transform

d.run(function () {
    var onlyJsFiles = new Transform()
        , mapToFiles = new Transform()
        , addNewLine = new Transform()

    onlyJsFiles._transform = function(fileName, output, end) {
        if (fileName.slice(-3) === ".js" && (fileName !== "lib.js")) {
            output(fileName)
        }
        end()
    }

    mapToFiles._transform = function(fileName, output, end) {
        fs.readFile(fileName, end)
    }

    addNewLine._transform = function (file, output, end) {
        end(null, file + "\n")
    }

    fs.createReadStream(__dirname)
        .pipe(onlyJsFiles)
        .pipe(mapToFiles)
        .pipe(addNewLine)
        .pipe(fs.createWriteStream(__dirname + "/lib.js"))

})

d.on("error", function ignoreIt() {})
```


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Mariusz Nowak  
View profile  
 More options Oct 3 2012, 1:38 pm
From: Mariusz Nowak <mari...@medikoo.com>
Date: Wed, 3 Oct 2012 10:38:38 -0700 (PDT)
Local: Wed, Oct 3 2012 1:38 pm
Subject: Re: [nodejs] Re: [ANN] promise-stream: Turn a stream into a commonJS promise

Raynos, Streams are very powerful and I agree with that, but as your
examples shown you just cannot take asynchronous functions and configure
them with streams directly, so they're not that helpful when dealing
with asynchronicity that's not stream based.

You're point is not against promises but rather against typical
asynchronous CPS. Question is whether you can (or whether it makes sense)
to address all asynchronicity use cases with Streams, I doubt whether it
make sense.

You made a one good point about writing to file while, having already some
files ready, this is where indeed streams win, I was thinking about it
recently. Currently in deferred implementation promise is an event-emitter,
with 0.7 I want to push it forward so it can also be stream (both readable
and writable). I've got use cases where it's beneficial to have promises
that are also streams.

Anyway I find a lot of great stuff in your examples. Thanks for sharing
that!


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Jake Verbaten  
View profile  
 More options Oct 3 2012, 2:59 pm
From: Jake Verbaten <rayn...@gmail.com>
Date: Wed, 3 Oct 2012 11:59:09 -0700
Local: Wed, Oct 3 2012 2:59 pm
Subject: Re: [nodejs] Re: [ANN] promise-stream: Turn a stream into a commonJS promise

promises only work if everything is a promise. Streams only work if
everything is a stream.

Making everything a stream is cool because streams represent the endless
flow of data through your entire program. It's basically doing reactive
programming.

Making everything a stream instead of a promise has the advantage of most
things in node core are already streams. Domains work cleanly with streams,
etc.

Oh and streams are composable! You write a stream I can use it with any of
my streams. You write a promise, I have to use promises everywhere or
ignore all your work.


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
End of messages
« Back to Discussions « Newer topic     Older topic »