[Filesystem] Return value of return? Partial writing?

6 views
Skip to first unread message

Daniel Friesen

unread,
Aug 28, 2009, 1:20:56 AM8/28/09
to comm...@googlegroups.com
There is one discussion that seams to be missing. I don't remember it
from the Filesystem API thread, and it's not noted on the wiki.

What does write return? Does write block? Partial writes or something else?

This seams a little inconsistent in other systems as well:

There appear to be two different ways it's handled api wise.

Partially writing if you can't write everything at once and returning
either the length you've read or what data is remaining.
or
Blocking until all data can be written or returning and passing that
task off to a underlying layer.

NSPR seams to handle this the first way returning the length that wasn't
written.
Java uses the later and simply takes the data.
Ruby seams to have a write and write_nonblock which handle the later and
former behaviors.
Python's .write seams to take the latter behavior.


Api side the only thing we really care about is:
Do we do partial writes and return the remaining data or length of read.
Or do we just take the data and guarantee in some way the system will
attempt to write all of that data rather than only writing part of it.

Either way would probably work in a technical sense:
In C/C++ layers that do partials:
If we do partials, they are simple wrappers.
If we do full write, writing will either have to block or use some other
layer to do writing in the background.

In Java and layers that just write:
If we do partials, we'll just full write and return no data or 0 length
to imitate a partial write that really writes everything.
If we do full write, they are simple wrappers.

--
~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://daniel.friesen.name]

Wes Garland

unread,
Aug 28, 2009, 9:55:28 AM8/28/09
to comm...@googlegroups.com
This probably hasn't been dealt with because it's not really relevant for files, only streams backed by ptys, sockets and pipes.

Anyhow, the right way to handle this IMHO is to return the number of bytes written.  You can implement "write_full" out of that, but not the other way around.

Passing data off to an underlying layer is not acceptable for systems programming and related tasks.  (And again, can be implemented in userland).

Wes
--
Wesley W. Garland
Director, Product Development
PageMail, Inc.
+1 613 542 2787 x 102

Joshaven Potter

unread,
Aug 28, 2009, 11:10:25 AM8/28/09
to comm...@googlegroups.com
I really like the idea of async writing, however I think most app developers would expect blocked writing.  

I would therefor like to see synchronous by default with an option to write asynchronously.

.write('Hello World', {asynchronous:true})

I prefer options over multiple methods like .write() and .write_nonblock()

I also understand that this may be a feature that should be in middle-ware not necessarily in the Common JS API.  However, I think that a smaller API is better unless it is more confusing.  Looking at multiple methods for something such as writing is not as easy for me to wade through as multiple options to a method because the relationship between the method its options are more apparent then the relationship between multiple methods.



On Fri, Aug 28, 2009 at 1:20 AM, Daniel Friesen <nadir.s...@gmail.com> wrote:



--
Sincerely,
Joshaven Potter

"No man making a profession of faith ought to sin, nor one possessed of love to hate his brother. For He that said, “Thou shalt love the Lord thy God,”  said also, “and thy neighbor as thyself.”  Those that profess themselves to be Christ’s are known not only by what they say, but by what they practice. “For the tree is known by its fruit.”" -- Ignatius

Kris Kowal

unread,
Aug 28, 2009, 4:26:12 PM8/28/09
to comm...@googlegroups.com
On Fri, Aug 28, 2009 at 8:10 AM, Joshaven Potter<your...@gmail.com> wrote:
> I really like the idea of async writing, however I think most app developers
> would expect blocked writing.

I expect blocked writing by default. We do need an async version though.

This is probably a good time to mention that we've been playing with
the idea of having functions like "write" and "flush" that do not
generally return anything meaningful, return the context object, the
stream itself in this case. This lets us do chaining.

I also generally prefer to functions to consistently return the same
type of thing regardless of their arguments and options. Even open,
although it does not always return the same type necessarily, always
returns an object with a common API. For that reason, I would prefer
"write" and "async write" to have different names.

My present notion is to have write(content) -> this and
writeFrom(buffer, begin, end) -> actual:Number, analogous to read() ->
content, read(max:Number) -> content, and readInto(buffer, begin, end)
-> actual:Number, like the new Python IO API.

Kris Kowal

Donny Viszneki

unread,
Aug 28, 2009, 5:37:07 PM8/28/09
to comm...@googlegroups.com
On Fri, Aug 28, 2009 at 11:10 AM, Joshaven Potter<your...@gmail.com> wrote:
> I really like the idea of async writing, however I think most app developers
> would expect blocked writing.
> I would therefor like to see synchronous by default with an option to write
> asynchronously.
> .write('Hello World', {asynchronous:true})

All of my experience with POSIX and Windows APIs tells me that whether
a read or write operation happens "asynchronously" (returns before
having read or written the fully requested amount) is a property of
the descriptor, and not of the operation.

> I prefer options over multiple methods like .write() and .write_nonblock()

I think both are bad ideas.

Here's some background:

http://www.ia.pw.edu.pl/~wujek/dokumentacja/gnu/libc/libc_8.html

--
http://codebad.com/

Daniel Friesen

unread,
Aug 28, 2009, 7:35:41 PM8/28/09
to comm...@googlegroups.com
Ok, looks like the group is split on this.

I started a show of hands on the topic:
https://wiki.mozilla.org/ServerJS/Filesystem_API/Show_of_hands

Marked Wes as A.2 (partial writes, return length) and me and Kris as B
(blocking).
I'm unsure what Donny thinks.

Joshaven Potter

unread,
Aug 28, 2009, 7:48:32 PM8/28/09
to comm...@googlegroups.com
you have an excellent point on returning the same type of return.



Daniel Friesen

unread,
Aug 28, 2009, 11:13:53 PM8/28/09
to comm...@googlegroups.com
Heh, I guess I agree and break that at the same time.

I don't like a async option. But I did do alt return type when I added
compat with w3c's File API to my File draft.

f.getAsText(function(text, error) { ... }, encoding); // undefined?

Because server-side things don't always make sense asynchronously I
extended the API to also work synchronously:

f.getAsText(encoding); // text, or throw error

~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://daniel.friesen.name]

Reply all
Reply to author
Forward
0 new messages