I/O proposal first draft
Hannes Wallnoefer
https://wiki.mozilla.org/ServerJS/IO/A
It is roughly based on the Filesystem API streams section, with some
inspiration taken from Python's new io module.
The biggest departure from both is the interface for seekable streams,
where I propose two read/write properties called length and position
to replace the seek()/tell()/truncate() model. Setting the length
property will truncate or extend the stream, and setting the position
will set the read/write pointer. My rationale is that this is both
better fitting for JavaScript in general and specifically for the
in-memory streams I'm defining (called ByteBuffer and StringBuffer),
which can easily double as streams _and_ convenient buffer objects
(adding append(), insert() and remove() methods on top of the stream
API). Also, seekable streems are rarely used in day to day
programming, so people won't have to relearn a lot.
Let me know what you think.
Hannes
Daniel Friesen
- I don't like the .read .readAll separation in this case.
- I don't exactly like the way .write has the writeFrom counterpart to
readInto folded into it.
- I too prefer duck typing .read, .write, ??? to indicate what is open
and possible.
- I sorta liked how Filesystem A had a OpaqueCookie which could be used
with .seek
To me it feels like this proposal attempts to make things more
javascripty in one way (which I'm not so sure about myself) but departs
from a lot of clean javascripty methods at the same time.
~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://daniel.friesen.name]
Kris Kowal
Friesen<nadir.s...@gmail.com> wrote:
> - I don't like the .read .readAll separation in this case.
The present rules in Narwhal are:
.read(Number) ByteString
is the same as specified here
.read(null) ByteString
reads up to one "block" from the underlying device.
the size of the block may be determined in platform-specific manners.
.read() / .read(undefined) ByteString
reads all
I'm not married to it, but if there's going to be a show of hands, I
think this option should be on the list.
> - I don't exactly like the way .write has the writeFrom counterpart to
> readInto folded into it.
I also prefer the separation of blocking .write and non-blocking .writeFrom.
> - I too prefer duck typing .read, .write, ??? to indicate what is open
> and possible.
I think this is reasonable.
> - I sorta liked how Filesystem A had a OpaqueCookie which could be used
> with .seek
Perhaps the .position object can be an opaque cookie instead of a
number for Text I/O. That would be the analogy anyway. The reason
for it is that finding the numeric position in a mixed-width encoded
stream is fairly intense, but an opaque cookie can close on the
corresponding byte position.
Other Stuff
=========
I think we need .forEach on readable I/O streams. .forEach on raw I/O
would use block reading. .forEach on Text I/O would use line reading.
In both cases, they could be implemented in terms of .next(), if we
add .next() and .iterator() to Raw I/O using block read too.
.writeLine is subsumable by the one-argument case of .print. The only
advantage of having .writeLine, as far as I can tell, is that it can
be passed to .forEach(stream.writeLine.bind(stream)) and it would only
print the .forEach block's first argument.
And, foremost Hannes, thank you for driving.
Kris Kowal
Hannes Wallnoefer
> - I don't like the .read .readAll separation in this case.
If more people don't like it I'm fine with dropping readAll.
> - I don't exactly like the way .write has the writeFrom counterpart to
> readInto folded into it.
creates a new String or Binary with each invocation, and readInto is a
way avoid that by reusing the same buffer, thereby increasing
efficiency. write doesn't have this problem, because you decide what
you pass into it. It's just a coincidence that readInto and write have
a similar signature, I think.
> - I too prefer duck typing .read, .write, ??? to indicate what is open
> and possible.
if (typeof s.read === "function") ...
to that?
if (s.readable()) ...
I definitely prefer the latter. It's just a more accommodating form of
duck typing.
> - I sorta liked how Filesystem A had a OpaqueCookie which could be used
> with .seek
want. It just allows you to go back to some place you've been before,
kind of like the mark()/reset() protocol of Java's InputStream, but
with the possibility to set multiple marks.
Actually, Filesystem/A uses numbers for seek()/tell() in raw streams,
and OpaqueCookie for text streams. The rationale behind this is
probably that it may be impossible to standardize the seek unit for
text streams. If you have a random access file underneath, it's hard
to use anything than bytes, while if you're using a char buffer, it's
easy to calculate in character units.
> To me it feels like this proposal attempts to make things more
> javascripty in one way (which I'm not so sure about myself) but departs
> from a lot of clean javascripty methods at the same time.
Hannes
Daniel Friesen
> On Tue, Sep 8, 2009 at 10:13 AM, Daniel
> Friesen<nadir.s...@gmail.com> wrote:
>
>> - I don't like the .read .readAll separation in this case.
>>
>
> The present rules in Narwhal are:
>
> .read(Number) ByteString
> is the same as specified here
> .read(null) ByteString
> reads up to one "block" from the underlying device.
> the size of the block may be determined in platform-specific manners.
> .read() / .read(undefined) ByteString
> reads all
>
> I'm not married to it, but if there's going to be a show of hands, I
> think this option should be on the list.
>
>> - I don't exactly like the way .write has the writeFrom counterpart to
>> readInto folded into it.
>>
>
> I also prefer the separation of blocking .write and non-blocking .writeFrom.
>
.readInto, a memcopy version of .read/.write
Hmm, ok .writeFrom as non-blocking makes sense. I was using
.writePartial() for non-blocking my draft.
> Kris Kowal
Daniel Friesen
>> and possible.
>>
>
> You mean you prefer this:
>
> if (typeof s.read === "function") ...
>
> to that?
>
> if (s.readable()) ...
>
> I definitely prefer the latter. It's just a more accommodating form of
> duck typing.
>
The function test is superfluous, and .readable() as a function is ugly
as well when it could be a property...
>> - I sorta liked how Filesystem A had a OpaqueCookie which could be used
>> with .seek
>>
>
> But an opaque cookie doesn't provide random access, which is what we
> want. It just allows you to go back to some place you've been before,
> kind of like the mark()/reset() protocol of Java's InputStream, but
> with the possibility to set multiple marks.
>
> Actually, Filesystem/A uses numbers for seek()/tell() in raw streams,
> and OpaqueCookie for text streams. The rationale behind this is
> probably that it may be impossible to standardize the seek unit for
> text streams. If you have a random access file underneath, it's hard
> to use anything than bytes, while if you're using a char buffer, it's
> easy to calculate in character units.
>
>
>> To me it feels like this proposal attempts to make things more
>> javascripty in one way (which I'm not so sure about myself) but departs
>> from a lot of clean javascripty methods at the same time.
>>
>
> Not sure I understand what you mean. Could you explain a bit?
>
> Hannes
>
unnecessary separation of .read and .readAll,
readable()/writable()/closed() as functions when they don't need to be.
Hannes Wallnoefer
>>> - I don't exactly like the way .write has the writeFrom counterpart to
>>> readInto folded into it.
>>>
>>
>> I also prefer the separation of blocking .write and non-blocking .writeFrom.
>>
> .writeFrom is non-blocking? I thought .writeFrom was the equiv of
> .readInto, a memcopy version of .read/.write
> Hmm, ok .writeFrom as non-blocking makes sense. I was using
> .writePartial() for non-blocking my draft.
think the writeFrom() was meant as a write() that takes additional
begin and end arguments to only write a part of the buffer or string -
somthing that can easily be implemented in write() IMO.
Hannes
Kris Kowal
>> .writeFrom is non-blocking? I thought .writeFrom was the equiv of
>> .readInto, a memcopy version of .read/.write
>> Hmm, ok .writeFrom as non-blocking makes sense. I was using
>> .writePartial() for non-blocking my draft.
>
> Actually, at this time the spec doesn't contain non-blocking I/O. I
> think the writeFrom() was meant as a write() that takes additional
> begin and end arguments to only write a part of the buffer or string -
> somthing that can easily be implemented in write() IMO.
Perhaps my intention was not so much non-blocking, but that write()
would be required to consume the entire content, and writeFrom() would
have the option of returning a partial write, as Daniel explicates
with writePartial(). write() would return the Stream, and writeFrom()
would return a number. Both write() and writeFrom() would be
candidates for the same (content, [begin, [end]]) argument pattern.
writeFrom() would not so much be expressly "non-blocking", but that
write() would not be possible on a non-blocking, non-buffered stream.
So the intent is not so much to denote non-blocking I/O but to permit
it, without inconveniencing the convenient pattern for high-level
write calls.
Kris Kowal
Wes Garland
Note: I recognize fully that my definition of "normal" is skewed by the behaviour of POSIX.
Incidentally, have any of you guys looked at the File API in ActionScript?
If not, why not?
If so, why isn't CommonJS taking hints from the only popular, deployed JS variant that has a File API?
http://livedocs.adobe.com/fms/2/docs/wwhelp/wwhimpl/common/html/wwhelp.htm?context=LiveDocs_Parts&file=00000715.html
Wes
Wesley W. Garland
Director, Product Development
PageMail, Inc.
+1 613 542 2787 x 102
Kris Kowal
> Rather than re-jigging the normal expection from write(), and adding the
> foreign "writeFrom" (which really sounds like writeSome() to me) -- why not
> leave write "normal" and add a writeAll() method instead?
>
> Note: I recognize fully that my definition of "normal" is skewed by the
> behaviour of POSIX.
I'm not clear about what you consider normal. Coming from my
background, it would be normal if write(content) guarantees that the
content will eventually be sent or stored, regardless of its length,
buffering, or whether the stream is non-blocking. If I'm not
mistaken, the POSIX interpretation is lower level, copies from a
buffer, can put the process to sleep to slow it down, and has to be
called in a loop to guarantee that the content is consumed entirely.
I'm suggesting that we use "write" for the former, and because of the
buffer parallel and symmetry with readInto, "writeFrom" for the
latter.
> Incidentally, have any of you guys looked at the File API in ActionScript?
> If not, why not?
> If so, why isn't CommonJS taking hints from the only popular, deployed JS
> variant that has a File API?
>
> http://livedocs.adobe.com/fms/2/docs/wwhelp/wwhimpl/common/html/wwhelp.htm?context=LiveDocs_Parts&file=00000715.html
stream.writeAll(arrayOfStrings) appears to be a
->| shortcut for |<-
arrayOfStrings.forEach(stream.print).
Kris Kowal
Kris Kowal
Looks like "write(content) Number" for both Raw and Text. The
distinction is that in raw mode, the return may be less than the
length of the content, and for buffered I/O it is guaranteed to be the
length of the content. This would be consistent with the lower-level
write. I still think it's a good idea to separate low and high level
into two functions, rather than have the behavior vary with the
underlying stream parameters.
Kris Kowal
Daniel Friesen
> On Tue, Sep 8, 2009 at 12:53 PM, Wes Garland<w...@page.ca> wrote:
>
>> Rather than re-jigging the normal expection from write(), and adding the
>> foreign "writeFrom" (which really sounds like writeSome() to me) -- why not
>> leave write "normal" and add a writeAll() method instead?
>>
>> Note: I recognize fully that my definition of "normal" is skewed by the
>> behaviour of POSIX.
>>
>
> I'm not clear about what you consider normal. Coming from my
> background, it would be normal if write(content) guarantees that the
> content will eventually be sent or stored, regardless of its length,
> buffering, or whether the stream is non-blocking. If I'm not
> mistaken, the POSIX interpretation is lower level, copies from a
> buffer, can put the process to sleep to slow it down, and has to be
> called in a loop to guarantee that the content is consumed entirely.
> I'm suggesting that we use "write" for the former, and because of the
> buffer parallel and symmetry with readInto, "writeFrom" for the
> latter.
>
I'd prefer .write to write, and have a more explicit partial writing
method (.writePartial/.writeFrom, whatever).
Ruby matches up with that using it's .write and .write_nonblock...
course we'd use cleaner names.
>> Incidentally, have any of you guys looked at the File API in ActionScript?
>> If not, why not?
>> If so, why isn't CommonJS taking hints from the only popular, deployed JS
>> variant that has a File API?
>>
>> http://livedocs.adobe.com/fms/2/docs/wwhelp/wwhimpl/common/html/wwhelp.htm?context=LiveDocs_Parts&file=00000715.html
>>
>
> stream.writeAll(arrayOfStrings) appears to be a
> ->| shortcut for |<-
> arrayOfStrings.forEach(stream.print).
>
> Kris Kowal
>
I've never seen that api. I've been working with AIR for awhile... Adobe
AIR scares me... at least in terms of the API... Now I can't believe
that Adobe has two insanely different apis to do the same thing...
Side note... Serer-side ActionScript is popular and widely deployed?
I've never heard of it... (I say widely deployed, cause there are plenty
of other ssjs systems that are usable in some way)