[nodejs] Flushing buffers before exiting

[Brian Takita]

Hello, I found that when using process.exit and redirecting the output
causes the program to end before the sys.puts buffers are flushed.

Is it possible to flush the buffers before exiting?

I tried process.stdin.flush() and process.stdin.end(), before calling
process.exit(), however I cannot the output is still not flushed.

Here are the steps to reproduce:

1. Create a file

var sys = require('sys')
for (var i=0; i < 2; i++) {
sys.puts(i);
}
process.stdout.flush()
process.stdout.end()
process.exit()

2. Call `node file.js`. Notice that 1 & 2 are printed.
3. Now try `node file.js > output; cat output`. Notice that some
output is missing.

The only workaround I could come up with was to add a setTimeout
around the exit.

setTimeout(function() {
process.exit()
}, 100)

However, this strikes me as kludgy and non-obvious to beginners (such
as me). Is there a better way or am I missing something?

Thank you,
Brian

--
You received this message because you are subscribed to the Google Groups "nodejs" group.
To post to this group, send email to nod...@googlegroups.com.
To unsubscribe from this group, send email to nodejs+un...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/nodejs?hl=en.

[Debacker]

You are not expected to call process.exit() that way.

That function is a kind of emergency handbrake, call it, and the program quits immediately, it's the standard UNIX syscall.

Node will automatically exists when no more callback are registered in the main event loop, and when the main process is completed. So no need to call exit.

Laurent Debacker.

[Brian Takita]

Thanks for the tip Laurent. The only reason why I ask is I may want to
have a non-zero exit code and would want to guarantee that the buffers
are flushed in that case. Also, it's typical to use exit when forking
processes (though node does not support forking but may in the
future).

Note that exceptions in the main event loop also cause this behavior.
Ryah, mentioned in the ticket that stderr should be flushed in that
case, which I agree with. I'd also argue that stdout should also be
flushed. I couldn't find any hooks (other than setTimeout) in the
event loop to accomplish this.

[Brian Takita]

A non-zero exit with flushing can be achieved by deregistering all callbacks in the main event loop and registering a callback on the application exit event which calls exit with the exit code.

An exitWithBufferFlush method shouldn't be too difficult to write.

Laurant, I really appreciate the idiomatic direction on this.

- Brian

On May 13, 2010 9:43 AM, "Brian Takita" <brian....@gmail.com> wrote:

Thanks for the tip Laurent. The only reason why I ask is I may want to
have a non-zero exit code and would want to guarantee that the buffers
are flushed in that case. Also, it's typical to use exit when forking
processes (though node does not support forking but may in the
future).

Note that exceptions in the main event loop also cause this behavior.
Ryah, mentioned in the ticket that stderr should be flushed in that
case, which I agree with. I'd also argue that stdout should also be
flushed. I couldn't find any hooks (other than setTimeout) in the
event loop to accomplish this.

On Thu, May 13, 2010 at 9:18 AM, Debacker <deba...@gmail.com> wrote: > You are not expected to cal...

[r...@tinyclouds.org]

On Fri, May 14, 2010 at 9:28 AM, Brian Takita <brian....@gmail.com> wrote:
> A non-zero exit with flushing can be achieved by deregistering all callbacks
> in the main event loop and registering a callback on the application exit
> event which calls exit with the exit code.
>
> An exitWithBufferFlush method shouldn't be too difficult to write.
>
> Laurant, I really appreciate the idiomatic direction on this.
>
> - Brian

I'd rather this be done on the stream itself. stderr is special and
should be flushed completely - in fact - I'm not sure why it isn't
right now. So maybe something like

for (var i=0; i < 10; i++) {
process.stdout.write("hello\n", function () {
if (i == 10) process.exit(42);
});
}

Of course there is also the 'drain' event. But I feel that's difficult
to mix into your example.

[Brian Takita]

Hey Ryah,

The implication of using an event on the stdout stream is every
sys.puts, stdout.write, etc. will need to do this. Not very DRY.
Also, it's would be rare for the stdout.write that initiates the exit.
Effectively, I would need to keep state that the program wants to
exit, but will not until every stdout.write is completed, which also
seems complicated.

From a purity POV, I see how stdout is yet another stream. However, in
a practical sense, it is sort of this special global stream that I
want to make sure is flushed.
The same could be said about a logger, service event, etc.

I still like the idea of it being attached to an exit (with blah,
blah) method since the cleanup can be centrally managed and the logic
would be fairly simple.

Thanks,
- Brian

[Brian Takita]

Going off of your idea, another approach could be reference counting
the number of open buffers.

Maybe something like (I know it will not fully work, but hopefully
gets the point across)

Stream.prototype.openWriteCount = 0
originalWrite = Stream.prototype.write
Stream.prototype.write = function(message, callback) {
var self = this
this.openWriteCount += 1
originalWrite(message, function() {
self.openWriteCount -= 1
callback()
})
}

process.exitWhenStreamFlushed = function(exitCode) {
var streams = arguments.slice(1)
// argh, need to block here until all the streams openWriteCount is
0, which I don't know how to do. I don't think a while loop will work.
exit(exitCode)

[Brian Takita]

Hmm, my approach has other issues.

process.addListener("exit", function() {
process.exit(42)
})

Will not cause an exit code of 42. Instead, I get an exit code of 0.

It also seems like the following does not work because the exit code is 0:

setTimeout(100, function() {
process.exit(42)
})

It seems like exit only works from within the main event loop.

I would be happy enough if there was some way of setting the exit code
without having to call exit. Maybe something like:

process.setExitCode(42)
// I can do more stuff here
...