Detecting stream.Writable#end call

[Liam]

I had the impression that _write() was the only method required in a stream.Writable subclass; is an end() implementation required to detect calls to end()?

This code shows that implementing _write() in a stream.Writable implementation won't detect a call to end()

https://gist.github.com/mtibeica/5389437

[Isaac Schlueter]

You do not need to implement end(), no. In fact, you probably shouldn't.

What are you trying to do?

Note that Writables don't get an 'end' event, but rather a 'finish' event.

> --
> --
> Job Board: http://jobs.nodejs.org/
> Posting guidelines:
> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> You received this message because you are subscribed to the Google
> Groups "nodejs" group.
> To post to this group, send email to nod...@googlegroups.com
> To unsubscribe from this group, send email to
> nodejs+un...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/nodejs?hl=en?hl=en
>
> ---
> You received this message because you are subscribed to the Google Groups
> "nodejs" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to nodejs+un...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[Liam]

We need to make an async call to clean up our writable stream implementation, before emitting finish.

[Isaac Schlueter]

Usually "clean up" means "close" rather than "finish". The "finish"
event simply means that all the data has been written, not necessarily
that any underlying resources have been cleaned up.

You could listen on "finish" and do it then.

If it's a Transform stream, and you need to make sure that you flush
something to the output after the last chunk is processed, you can
implement a `_flush(callback)` method.

[Nathan Rajlich]

FWIW I volleyed for _flush() to be part of the Writable API directly, rather than Transform, before streams2 were solidified. I had the use-case at the time and apparently Liam does as well. C'est la vie.

[Isaac Schlueter]

We could move this into Writable pretty easily. It's an API change, but certainly not unreasonable for 0.12, since it wouldn't be a breaking change. 

[Liam Breck]

So _flush would be called when .end() is called by the client of the writable stream?

That seems like an essential feature of a writable stream.

[Liam Breck]

Issue filed: https://github.com/joyent/node/issues/5315

[Floby]

It is not essential in the sense that you can already do something similar by listening to the 'finish' event. However observing oneself is a strange pattern. but one implentation of _flush could easily be:

this.on('finish', this._flush);

is that correct ?

[Liam Breck]

Only if the 'finish' event could be suppressed by _flush() for other listeners, which I don't see a way to do...

Alternatively the subclass can violate the defined API, and emit 'really_finished' from _flush()

[Liam Breck]

[responding on list]

We're talking about flushing left-over data in a scenario where data is written in blocks, for example. The last write callback hasn't fired when end() is called; or write callbacks might simply fire on nextTick unless we paused writing out blocks.

Flushing on 'finish' works only if there's no one else listening on 'finish'. And then our stream subclass doesn't conform to the Writable API.


On Wed, Apr 17, 2013 at 11:40 AM, Isaac Schlueter <i...@izs.me> wrote:

Well, you shouldn't be calling the write callback until that chunk is "flushed" anyway, right? Seems like listening to your own finish event is probably fine. Maybe I'm missing something still?