Is fs.write() atomic (up to PIPE_BUF)? Can you call it without waiting for cb?

[Jaka Jančar]

I'm writing a node.js server that uses the cluster module. Each worker
does *lots* of logging to a shared log file (it must be just 1 file).

- Assuming log lines are smaller than PIPE_BUF (4k), is it safe to do
fs.write() and rely on it being atomic, or does Node.js break this
promise of write()?

- What's with "Note that it is unsafe to use fs.write multiple times
on the same file without waiting for the callback."?

- If writes are not atomic or not waiting for callbacks really is a
problem, what's an *efficient* alternative?

Thanks,
Jaka

[Mark Hahn]

Writes are atomic just like calling them from any other language.  

The warning is just pointing out that due to the asynchronous nature of javascript you must not assume any IO has happened until you get the callback.  So if you don't wait for callbacks the order of the writes is not guaranteed.  In your case of writing to a file from different places in your app, you wouldn't expect the order of writes to be guaranteed anyway.

--
Job Board: http://jobs.nodejs.org/
Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nod...@googlegroups.com
To unsubscribe from this group, send email to
nodejs+un...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

[Jaka Jančar]

I was thinking more along the lines:

If I do:

fs.write(fd, 'first write of two kilobytes');

fs.write(fd, 'second write three kilobytes');

is there a userspace buffer involved somewhere that could cause e.g. 4 kilobytes to be written (2k of first and 2k of second write), then write the remaining kilobyte sometime later, thus manging the record?

nodejs+unsubscribe@​googlegroups.com

[Jorge]

When you have several fs.write()s going on at once, they may end up being executed in parallel in a bunch of (up to 4 IIRC) background (libeio's) threads, so the write()s may actually happen in ~ any order. If you instead wait for the callback before issuing the next write(), then you're serializing them.
--
Jorge.

[Jaka Jančar]

That is besides the point. The question is, will the fs.write()'s map 1:1 to write()'s?

[Mark Hahn]

If you are doing append writes it is always atomic.

--

Job Board: http://jobs.nodejs.org/
Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nod...@googlegroups.com
To unsubscribe from this group, send email to

nodejs+un...@googlegroups.com

[Jorge]

The answer is *no*.

[Jorge]

And WRT "That is besides the point": you've asked "What's with "Note that it is unsafe to use fs.write multiple times on the same file without waiting for the callback.?"

On Mar 24, 2012, at 11:19 PM, Jaka Jančar wrote:

[Micheil Smith]

I think you may actually be able to use a WriteStream instead, as far as I recall,
those queued data up in an array, then they write it out as soon as possible,
which should guarantee ordering.

Alternatively, since this is a log file, you could use something like the winston
by Charlie Robbins.

– Micheil