Best way to detect file has been written to disk with fs.createWriteStream

[Andris Reinman]

What would be the best way to detect a file has been written to disk? Once I have passed all the data to a fs.WriteStream, the data is at first buffered by node and written to the disk asynchronously. There is no "end" or "close" event to listen to (or at least these don't trigger on my system (node v0.8.9)) when all bytes are actually written to the disk.

The best I have come up to is to listen to the "drain" event and then compare writablestream.bytesWritten and the length of the data I sent to the stream.

var len = 0;
myReadStream.on("data", function(chunk){
    myWriteStream.write(chunk);
    len += chunk.length;
});

myWriteStream.on("drain", function(){
    if(myWriteStream.bytesWritten == len){
        console.log("All bytes written to disk");
    }
});

But this doesn't seem very future proof, what if the conditions for the  'drain' event with file write change one day? Is there a better way for this?

[mscdex]

On Sep 23, 12:52 pm, Andris Reinman <andris.rein...@gmail.com> wrote:
> What would be the best way to detect a file has been written to disk? Once
> I have passed all the data to a fs.WriteStream, the data is at first
> buffered by node and written to the disk asynchronously. There is no "end"
> or "close" event to listen to (or at least these don't trigger on my system
> (node v0.8.9)) when all bytes are actually written to the disk.

fs.WriteStreams emit a 'close' event when the underlying file
descriptor is closed. Are you sure .end() is being called on the
stream?

[Mikeal Rogers]

there was a "close" event that happened after end() was called and it was flushed to disc. did we lose that at some point?

--
Job Board: http://jobs.nodejs.org/
Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nod...@googlegroups.com
To unsubscribe from this group, send email to
nodejs+un...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

[Ben Noordhuis]

Doesn't myReadStream.pipe(myWriteStream) work?

[Andris Reinman]

Sorry, my bad. The .end() was missing from my test script where I tested this out and in the original code where the problem occured, I had used "end" instead of "close" for listening.

Sadly piping was not an option - I wanted to stream attachment files received by e-mail to S3 but the length of the files were not known when reading from the e-mail parse stream and as S3 API works over HTTP, the actual file size was needed for content-length.

[Mikeal Rogers]

I *think* he wants to wait until the data is flushed to disc before returning a 201 or some such thing to an API client.

[Nuno Job]

Really curious about this. Also fsync or not? Is this controllable via api?

Sent from my iPhone

[Mikeal Rogers]

"close" was an event after the file handler was closed, after the write was sent. not fsync but I believe most operating systems won't close the file handler until pending data is fsync'd. of course, depending on the OS, most file system operations are lies ;)

-- 
Mikeal Rogers

Sent with Sparrow