Why there's no callback for opening stream?

[Alexey Petrushin]

Tried to copy files using streams and pipe and got funny results. Let's consider following code (highlighted version https://gist.github.com/3850106 ):

    var copy = function(from, to, cb){  

      var fromStream = fs.createReadStream(from)

      fromStream.on('error', cb)

      var toStream = fs.createWriteStream(to)

      toStream.on('error', cb)

      fromStream.on('end', cb)

    }

    copy('non existing file a', 'non existing dir/file b', function(err){

      console.log(err)

    })

There are errors in both streams (non existing source file and no parent directory for destination file) - so, both of them will emit

'error' event, and the callback will be called twice with both errors.

    // Error will be reported twice:

    //

    // { [Error: ENOENT, open 'non existing file a'] errno: 34, 

    //    code: 'ENOENT', path: 'non existing file a' }

    // { [Error: ENOENT, open 'non existing dir/file b'] errno: 34, 

    //    code: 'ENOENT', path: 'non existing dir/file b' }

I found the solution by looking at the source of `util.pump` - it does it by wrapping callback into function that calls callback only once - for the fist error and ignoring others.

But maybe it would be nice to have a callback for `createXxxStream` or something like 'ready' or 'success' events to know that it created successfully?

[Jake Verbaten]

Use domains

var copy = function (from, to, cb) {

    return fs.createReadStream(from)
        .pipe(fs.createWriteStream(to))
        .on("end", cb)
}

domain.run(function () {
    copy("a", "b", next)
})

--
Job Board: http://jobs.nodejs.org/
Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nod...@googlegroups.com
To unsubscribe from this group, send email to
nodejs+un...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

[Weltschmerz]

Domains seems to be a good idea. Regarding your question, you could listen for the 'open' event.

https://gist.github.com/88b5877dcb8983be4aa6

[Martin Cooper]

On Sun, Oct 7, 2012 at 5:54 PM, Alexey Petrushin
<alexey.p...@gmail.com> wrote:

I'm missing how this would help. If there was such an event, either
stream could still emit an error before it (e.g. open error) or after
it (e.g. read or write error), so you'll still need to attach the two
error handlers, and you'll still need to handle the possibility of
multiple notifications. What would be the benefit of having another
event to listen for?

--
Martin Cooper

[Alexey Petrushin]

> I'm missing how this would help ...

With callbacks it may be something like code below:

> so you'll still need to attach the two error handlers, and you'll still need 

> to handle the possibility of multiple notifications.

Not sure about it but maybe `stream.destroy()` on first event should handle such cases.

    var copy = function(from, to, cb){    

      var fromStream = null

      var toStream = null

  

      var finish = function(err){

        fromStream.destroy()

        toStream.destroy()

        cb(err)

      }

      fromStream = fs.createReadStream(from, function(err){

        if(err) return cb(err)    

        toStream = fs.createWriteStream(to, function(err){

          if(err) return cb(err)

      

          fromStream.on('error', finish)

          toStream.on('error', finish)

          fromStream.on('end', finish)

          fromStream.pipe(toStream)

        })    

      })  

[Adam Crabtree]

Stream.pipe,

http://nodejs.org/docs/latest/api/all.html#all_stream_pipe_destination_options

will handle this for you in exactly the way you want:

    var copy = function(from, to, cb){  

      var stream = fs.createReadStream(from)

        .pipe(fs.createWriteStream(to))

      stream.on('error', cb)

      stream.on('end', cb)

    }

Take a look at the source for Stream.pipe and you'll see what I'm talking about; It wraps the 'error' event callback and emits only once, then does its own cleanup (you'll have to call destroy yourself, although I think maybe pipe should do this in its cleanup...).

From https://github.com/joyent/node/blob/master/lib/stream.js: 

  source.on('error', onerror);

  dest.on('error', onerror);


  // remove all the event listeners that were added.

  function cleanup() {

    source.removeListener('data', ondata);

    dest.removeListener('drain', ondrain);


    source.removeListener('end', onend);

    source.removeListener('close', onclose);


    source.removeListener('error', onerror);

    dest.removeListener('error', onerror);


    source.removeListener('end', cleanup);

    source.removeListener('close', cleanup);


    dest.removeListener('end', cleanup);

    dest.removeListener('close', cleanup);

  }

Cheers,

Adam Crabtree

--
Better a little with righteousness
       than much gain with injustice.
Proverbs 16:8

[Adam Crabtree]

`util.pump` is deprecated in favor of Stream.pipe. 

From the docs, http://nodejs.org/docs/latest/api/all.html#all_util_pump_readablestream_writablestream_callback:

util.pump(readableStream, writableStream, [callback])#

Stability: 0 - Deprecated: Use readableStream.pipe(writableStream)

Additionally, so long as you do the pipe in the same tick as the stream creation, you'll get the single 'error' event and thus the single call to callback. As a general rule of thumb, when piping, don't bother with the origin stream's events unless you have an explicit reason to do so. In this case it doesn't seem you do, in which case you would only handle the 'error' event once on the destination stream.

Cheers,

Adam Crabtree