Simplifying core filesystem methods

[Hannes Wallnoefer]

I raised this topic on IRC yesterday - I think the open(), read(), and
write() methods in the filesystem proposal provide way too many ways
of doing things.

https://wiki.mozilla.org/ServerJS/Filesystem_API/A

I think the following function signature is going to scare the living
daylight out of potential API users:

open(path (String)|options Object, [mode (String|Array|Object)],
[options Object|mode (String|Array)])

I propose that simplify this to

open(path, [options])

where path is a string and the optional options argument is an object
like {read: Boolean, write: Boolean, append: Boolean, update: Boolean,
binary: Boolean, charset: String, delimiter: String} plus any other
flags/options that may make sense but _not_ the file path (i.e. this
is about the function signature, not what options we should support).
Calling open without the options argument would return a text reader
stream.

What do you think?

Hannes

[Ondrej Zara]

I propose that simplify this to

open(path, [options])

where path is a string and the optional options argument is an object
like {read: Boolean, write: Boolean, append: Boolean, update: Boolean,
binary: Boolean, charset: String, delimiter: String} plus any other
flags/options that may make sense but _not_ the file path (i.e. this
is about the function signature, not what options we should support).
Calling open without the options argument would return a text reader
stream.

What do you think?

+1


O.

 

Hannes

[Wes Garland]

Hannes:

I see your point, and generally like clarity. That said, I tend to pick "fopen" over "open" for trivial apps because i hate the extremely verbose syntax for such a simple methods.

open("filename", O_RDWR | O_CREAT | O_TRUNC, 0666)

is actually less friendly than

fopen("filename", "w+");

...and programmers haven't really had a hard time dealing with this.  Do you know what Python does?  I also found the syntax of the Mozilla File Object to be incredibly annoying from a programmer's point of view.  I could never remember the argument order, which words to use, and the diagnostics were poor.

If the function signature is really that scary, perhaps we should look at changing the way we write function signatures, kinda K&R style.  (Actually, writing this post and trying to parse that mess, I agree: it IS too scary)

When called as a method on an instance of Path:
open(options, mode);

When called as a method on the Path constructor:
open(path, mode, options);

Where:

• path is the path to the file, as resolved by the operating system
  
• mode is a String containing, in any order, mode characters: "rwa+bxc", ["read", "write", "append", "update", "binary", "exclusive", "canonical"], {read, write, append, update, binary, exclusive, canonical}
• options is a JavaScript object
  
  • path String
  • mode Object
  • charset String
  • newLine String
  • delimiter String

aaaaaaaaaaaaaaaaargh!  Path in options??? What hell?

I'm having a hard time Doc'ing this. Hannes, I'm not fond of your API but your point is 1000000% on the money. TOO COMPLEX!

SMELLY CODE ALERT

http://blog.urth.org/2009/04/docs-smell-your-stinky-code.html

(How do you insert blink tags in GMail?)

Wes

--
Wesley W. Garland
Director, Product Development
PageMail, Inc.
+1 613 542 2787 x 102

[Hannes Wallnoefer]

2009/5/29 Wes Garland <w...@page.ca>:

> Hannes:
>
> I see your point, and generally like clarity. That said, I tend to pick
> "fopen" over "open" for trivial apps because i hate the extremely verbose
> syntax for such a simple methods.
>
> open("filename", O_RDWR | O_CREAT | O_TRUNC, 0666)
>
> is actually less friendly than
>
> fopen("filename", "w+");

Agreed this is simpler. I proposed the options object over the
"rwa+bxc" mode string (which is also in the current proposal) because
we also have things like charset or delimiter which can't be easily
packed into a mode string. Also, I think in 90% of cases you'll be
calling open without the options argument, or with a one or two
properties in the options object.

open("filename");
open("filename", {write: true});
open("filename", {append: true, binary: true});

that's all not too verbose, is it? If it is, you can still use 1
instead of true to save three characters :-)

Hannes

[Daniel Friesen]

Hannes Wallnoefer wrote:
> 2009/5/29 Wes Garland <w...@page.ca>:
>
>> Hannes:
>>
>> I see your point, and generally like clarity. That said, I tend to pick
>> "fopen" over "open" for trivial apps because i hate the extremely verbose
>> syntax for such a simple methods.
>>
>> open("filename", O_RDWR | O_CREAT | O_TRUNC, 0666)
>>
>> is actually less friendly than
>>
>> fopen("filename", "w+");
>>
>
> Agreed this is simpler. I proposed the options object over the
> "rwa+bxc" mode string (which is also in the current proposal) because
> we also have things like charset or delimiter which can't be easily
> packed into a mode string. Also, I think in 90% of cases you'll be
> calling open without the options argument, or with a one or two
> properties in the options object.
>
> open("filename");
> open("filename", {write: true});
> open("filename", {append: true, binary: true});
>
> that's all not too verbose, is it? If it is, you can still use 1
> instead of true to save three characters :-)
>
> Hannes
>

And x is also ambiguous.

ServerJS: x: exclusive lock
PHP: x: ... If the file already exists, the *fopen()* call will fail by
returning *FALSE* and generating an error of level *E_WARNING*. ...
This is equivalent to specifying /O_EXCL|O_CREAT/ flags for the
underlying /open(2)/ system call.

To be frank. http://draft.monkeyscript.org/api/io/File.html#open the
MonkeyScript treats x like PHP does, not like ServerJS does. Why? NSPR
supports the |PR_CREATE_FILE | ||PR_EXCL| but not locking right on open,
and ServerJS's omits |PR_CREATE_FILE | ||PR_EXCL.|

~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://daniel.friesen.name]

[Kris Kowal]

On Sat, May 30, 2009 at 9:27 AM, Daniel Friesen
<nadir.s...@gmail.com> wrote:
> And x is also ambiguous.
>
> ServerJS: x: exclusive lock
> PHP: x: ... If the file already exists, the *fopen()* call will fail by
> returning *FALSE* and generating an error of level *E_WARNING*. ...
> This is equivalent to specifying /O_EXCL|O_CREAT/ flags for the
> underlying /open(2)/ system call.
>
> To be frank. http://draft.monkeyscript.org/api/io/File.html#open the
> MonkeyScript treats x like PHP does, not like ServerJS does. Why? NSPR
> supports the |PR_CREATE_FILE | ||PR_EXCL| but not locking right on open,
> and ServerJS's omits |PR_CREATE_FILE | ||PR_EXCL.|

You might want to re-read that portion of the ServerJS spec. It was
changed to reflect the C, and therefore PHP, behavior as Wes Garland
clarified. The original verbiage was a mistake on my part in
understanding the intent of the flag and knowledge of the underlying
C.

Kris Kowal

[Kris Kowal]

On Fri, May 29, 2009 at 3:16 AM, Hannes Wallnoefer <han...@gmail.com> wrote:
> I raised this topic on IRC yesterday - I think the open(), read(), and
> write() methods in the filesystem proposal provide way too many ways
> of doing things.
>
> https://wiki.mozilla.org/ServerJS/Filesystem_API/A
>
> I think the following function signature is going to scare the living
> daylight out of potential API users:
>
> open(path (String)|options Object, [mode (String|Array|Object)],
> [options Object|mode (String|Array)])

Yes, it certainly would.

> I propose that simplify this to
>
> open(path, [options])
>
> where path is a string and the optional options argument is an object
> like {read: Boolean, write: Boolean, append: Boolean, update: Boolean,
> binary: Boolean, charset: String, delimiter: String} plus any other
> flags/options that may make sense but _not_ the file path (i.e. this
> is about the function signature, not what options we should support).
> Calling open without the options argument would return a text reader
> stream.
>
> What do you think?

+1, but I still want to be able to do the following:

open(path)
open(path, "w")
read(path, "b")
write(path, content, "b")

To that end, would it be sufferable to permit the "options" to be
coerced from an fopen-like String?

And on the very edge of desirability:

open(path, "w", {charset: "UTF-8"})
write(path, content, "b", {charset: "UTF-8"})

To this end, would it be permissible for open, read, and write, to
accept a variadic list of options objects, any of which coercible from
an fopen-like String, with right-to-left precedence, that is, updated
from left to right?

open(path, [options, …])
read(path, [options, …])
write(path, content, [options …])

?

Kris Kowal

[Kris Kowal]

> write(path, content, "b", {charset: "UTF-8"})

Sorry, this example didn't make sense. Imagine:

> write(path, content, "x", {charset: "UTF-8"})

Kris Kowal

[Ash Berlin]

This strikes me as needlessly complex. Think about how you would
document this, and it starts to get hairy. If you want to specify a
charset and a mode, then pass it in the options dict/hash. Simple, and
easy to remember:

open(path, [mode]);
open(path, options);

No special cases. No complexity, almost no extra work on part of the
API user.

-ash

[Kris Kowal]

On Sat, May 30, 2009 at 3:43 PM, Ash Berlin <ash_g...@firemirror.com> wrote:
> open(path, [mode]);
> open(path, options);

+1.

[Daniel Friesen]

Ok, I'll support this by adding the options object into my spec for the
MonkeyScript File object.

(flags=mode, mode=access; This get's confusing when some people use mode
for open mode and in other cases mode refers to permissions)
Though my own open without the options object will still be a full
File.open(path, flags, [encoding=File.systemEncoding], [mode], [block]);

ie: While the API for my File object is different I aim for a level of
compatibility with ServerJS in things like what .read() returns so that
the ServerJS stuff in my environment (ie: banana('serverjs');) can be
implemented as simple wrappers around the MonkeyScript stuff.

[Hannes Wallnoefer]

2009/5/31 Ash Berlin <ash_g...@firemirror.com>:

Just to be sure, that means that the second argument can either be a
mode string, or an options object?

If we support open mode strings, should we also support them within
the options object, e.g.:

open(path, {mode: "r", charset: "UTF-8"});

Hannes

[Ash Berlin]

On 2 Jun 2009, at 10:42, Hannes Wallnoefer wrote:

>
> 2009/5/31 Ash Berlin <ash_g...@firemirror.com>:

>>
>>
>> This strikes me as needlessly complex. Think about how you would
>> document this, and it starts to get hairy. If you want to specify a
>> charset and a mode, then pass it in the options dict/hash. Simple,
>> and
>> easy to remember:
>>
>> open(path, [mode]);
>> open(path, options);
>
> Just to be sure, that means that the second argument can either be a
> mode string, or an options object?

Yes, that was my intent. Path is always required. 2nd arg is an
optional mode (a string) or a options dictionary.

>
> If we support open mode strings, should we also support them within
> the options object, e.g.:
>
> open(path, {mode: "r", charset: "UTF-8"});
>
> Hannes

If you like. I've not thought about if it would be easier to have it
spelt out in terms of X different flags or not. I'll leave that up to
you (and to list which mode strings we accept)

[Kris Kowal]

On Tue, Jun 2, 2009 at 6:22 AM, Ash Berlin <ash_g...@firemirror.com> wrote:
> On 2 Jun 2009, at 10:42, Hannes Wallnoefer wrote:
>> 2009/5/31 Ash Berlin <ash_g...@firemirror.com>:

>>> open(path, [mode]);
>>> open(path, options);

> Path is always required. 2nd arg is an
> optional mode (a string) or a options dictionary.
>> If we support open mode strings, should we also support them within
>> the options object, e.g.:
>> open(path, {mode: "r", charset: "UTF-8"});

I've updated the wiki to reflect these ideas:

https://wiki.mozilla.org/ServerJS/Filesystem_API/A#Files

; open(path String, mode|options)
: returns a stream object that supports an appropriate interface for
the given options and mode, which include reading, writing, updating,
byte, character, unbuffered, buffered, and line buffered streams.
More details follow in the [#Stream] section of this document.
* path
* mode: "rwa+bxc", or
* options
** mode String
** charset String
** read Boolean
** write Boolean
** append Boolean
** update Boolean
** binary Boolean
** exclusive Boolean
** canonical Boolean
; read(path String, [mode|options])
: opens, reads, and closes a file, returning its content.
; write(path String, content String|Binary, [mode|options])
: opens, writes, flushes, and closes a file with the given content.
If the content is a ByteArray or ByteString, the binary mode is
implied.

Please continue if there are outstanding issues. For example, whether
we require "canonical" IO or not is another issue, also whether to
rectify history's wrongs and make it clear that "canonical" IO means
"non-blocking".

Kris Kowal

[Daniel Friesen]

Ugh... canonical means non-blocking?
I thought it meant something like a stat call would grab the canonical
(after symlinks are followed) path and use that for opening so the file
handle was
on the canonical file instead of going through the symlink.
Earlier on I was considering adding that in to my File class spec since
that's what I thought it was in ServerJS.

I think at the very least canonical and exclusive need some actual
explanation to avoid confusion.
exclusive feels a little ambiguous between exclusive lock and don't create.

wait, update? was that read+write or something else?

[Ash Berlin]

On 2 Jun 2009, at 21:25, Daniel Friesen wrote:

Ditto. Thats what I thought too. This absolutely needs changing. What
API does the "c" char in the mode come from? I've never come across it
before.

Also we need to specify where the precedence is when mode and any of
the boolean flags are specified. i.e.:

open(path, {mode: "rb", binary: false, write: true, read: false }) //
WTF does this do.

Thats partly why I'm not sure having mode in the options is needed.

>
> I think at the very least canonical and exclusive need some actual
> explanation to avoid confusion.
> exclusive feels a little ambiguous between exclusive lock and don't
> create.
>
> wait, update? was that read+write or something else?
>
> ~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://
> daniel.friesen.name]

Yeah - what is update?

-ash

[Daniel Friesen]

Ash Berlin wrote:
> On 2 Jun 2009, at 21:25, Daniel Friesen wrote:
>
>
>> Kris Kowal wrote:
>>

>>> ...

>>> Please continue if there are outstanding issues. For example,
>>> whether
>>> we require "canonical" IO or not is another issue, also whether to
>>> rectify history's wrongs and make it clear that "canonical" IO means
>>> "non-blocking".
>>>
>>> Kris Kowal
>>>
>>>
>> Ugh... canonical means non-blocking?
>> I thought it meant something like a stat call would grab the canonical
>> (after symlinks are followed) path and use that for opening so the
>> file
>> handle was
>> on the canonical file instead of going through the symlink.
>> Earlier on I was considering adding that in to my File class spec
>> since
>> that's what I thought it was in ServerJS.
>>
>
> Ditto. Thats what I thought too. This absolutely needs changing. What
> API does the "c" char in the mode come from? I've never come across it
> before.
>

Oh, here's another one. Is non-blocking, non-blocking IO, or is this
SYNC (wait for physical update of writes). In which case I already have
this in my spec as "s", File.SYNC, or {sync:true}.

> Also we need to specify where the precedence is when mode and any of
> the boolean flags are specified. i.e.:
>
> open(path, {mode: "rb", binary: false, write: true, read: false }) //
> WTF does this do.
>
> Thats partly why I'm not sure having mode in the options is needed.
>

I guess mode is there so someone can write:
open(path, {mode: "r+", charset: "UTF-8"});
When all they wanted was one or two extra flags and already had a
reasonable mode?
> -ash

[Wes Garland]

Canonical can, indeed, mean something to do with blocking. Sort of.  This is especially/typically in the context of terminals (serial ports). However, in that context, it also means some other special things, like do not process ERASE or KILL characters, do not fold lowercase to upper, and do block waiting for keystrokes. Yes, Canonical input mode means blocking, not non-blocking.

  Non-canonical Mode Input Processing
     In non-canonical mode input processing, input characters are
     not assembled into lines, and erase and kill processing does
     not occur. The MIN and TIME values are used to determine how
     to process the characters received.
.....

     If ICANON is set,  canonical  processing  is  enabled.  This
     enables  the erase and kill edit functions, and the assembly
     of input characters into lines delimited by NL-c, EOF,  EOL,

SunOS 5.10          Last change: 14 Sep 2005                   24

Ioctl Requests                                         termio(7I)

     and EOL . If  ICANON is not set, read requests are satisfied
     directly from the input queue. A read is not satisfied until
     at  least  MIN characters have been  received or the timeout
     value TIME has expired between characters. This allows  fast
     bursts  of input to be read efficiently while still allowing
     single character input. The time value represents tenths  of
     seconds.

     If  XCASE is set and ICANON is set, an upper case letter  is
     accepted  on input if preceded by a backslash (\) character,
     and is output preceded by a  backslash  (\)  character.   In
     this  mode,  the following escape sequences are generated on
     output and accepted on input:
....


That said, if "canonical" has anything to with Paths, it should have to do with resolving ./../link/blah to a hard, rooted path (like realpath or resolvepath C functions).

If it has to with Streams, it should have to do with terminal line discipline, which I think is [currently] out of scope.

If it's supposed to mean simply blocking/non-blocking: nuh-uh

[Kris Kowal]

Wes's interpretation of canonical is the one intended. Please forget
my statement about non-blocking.

[Ash Berlin]

On 2 Jun 2009, at 23:41, Kris Kowal wrote:

>
> Wes's interpretation of canonical is the one intended. Please forget
> my statement about non-blocking.

Ah that one. It's been a while since I've done stuff at that low a
level.

Okay so does canonical make sense in text mode, or should it only be
possible with binary?

Also is it uses often enough that it is needed in the mode string?

And no one has yet suggested how we behave when the mode strings and
the flags contradict each other - which one takes effect?

-ash

[Wes Garland]

On Wed, Jun 3, 2009 at 4:48 AM, Ash Berlin <ash_g...@firemirror.com> wrote:

Ah that one. It's been a while since I've done stuff at that low a
level.

Me too. I'm thinking 1997, I wrote a system for implementing IVRs with voice modems.
 

Okay so does canonical make sense in text mode, or should it only be
possible with binary?

If supported, it must be possible in both. If you're getting input from a terminal, chances are very good that you need to process vt-100 escape sequences and so forth.

Also is it uses often enough that it is needed in the mode string?

I don't think so.  Not that I object to it as a mode string, but in and of itself, setting canonical mode on a terminal isn't that useful. It's the default. Setting *non-canonical* will allow character-at-a-time processing.  However, if you're setting canonical/non-canonical, I think it's also reasonable that you might want to set raw mode, modify stop bits, word length, parity, baud rates, modify the behaviour of ^C to not SIGINT, etc.   In that case, I would be more interested in a termios property on a Stream, or a termios module which takes Streams as parameters and knows how to tweak their underlying file descripors. Termios is the POSIX terminal line discipline interface. Or maybe call it stty, like the UNIX command that does that.

If the intention is to allow character-at-time input from stdin, it's my feeling that -icanon won't be enough for most folks, who will probably want to try and write things like editors. They will want raw; I use curses' raw mode for my REPL:

     With the raw() and noraw() routines, the terminal is  placed
     into  or  out  of  raw mode. Raw mode is similar to cbreak()
     mode,  in  that  characters  typed  are  immediately  passed
     through to the user program. The differences are that in raw
     mode, the interrupt, quit, suspend, and flow control charac-
     ters  are  all passed through uninterpreted, instead of gen-
     erating a signal. The behavior of the BREAK key  depends  on
     other bits in the tty driver that are not set by curses.

Or, at a lower level (stty):
   raw (-raw or cooked)
           Enable (disable) raw input and  output.  Raw  mode  is
           equivalent to setting:

           stty cs8 -icanon min 1 time 0 -isig -xcase \
               -inpck -opost

And no one has yet suggested how we behave when the mode strings and
the flags contradict each other - which one takes effect?

I'm happy leaving this as "undefined".  If we have to specify a behaviour, it should be something like "the mode string is processed as if by modifying the flags object, which is then used to set the file's flags".

Wes

[Mark Porter]

OK, I think my head just exploded.

Is it the intension of the ServerJS project to support every IO option
available in the POSIX kernel API? Or should developers be expected to
drop down to the underling environment (Java, C, etc) for low level
tasks? I ask because I got into interpreted languages specifically to
avoid this kind of low level complexity. Beyond issues of philosophy
it is going to be hard to abstract this stuff cross platform. Java
(AFAIK) just doesn't go this low level and even C isn't entirely
consistent between *NIX systems.

I'm new so I apologize if y'all already been over this. I'm just
trying to get a feel for the goals of ServerJS

Thanks,
Mark

On Jun 3, 6:20 am, Wes Garland <w...@page.ca> wrote:

[Wes Garland]

On Wed, Jun 3, 2009 at 12:02 PM, Mark Porter <dm4...@gmail.com> wrote:

 Java
(AFAIK) just doesn't go this low level and even C isn't entirely
consistent between *NIX systems.

I don't know anything about Java.

What does Java do when it wants single-character input from a terminal? Or is it only able to do line-at-a-time? I think eclipse is written in Java, which suggests that single-character input must be supported.

Wes

[Daniel Friesen]

Well, NSPR doesn't even support canonical. std(in/out/err) is separate
from normal file handles.
So I can say that even if ServerJS supports canonical I won't be
supporting it in MonkeyScript, even in the ServerJS wrapper.

~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://daniel.friesen.name]

[Ash Berlin]

On Wed, 3 Jun 2009 12:06:00 -0400, Wes Garland <w...@page.ca> wrote:
> On Wed, Jun 3, 2009 at 12:02 PM, Mark Porter <dm4...@gmail.com> wrote:
>
>> Java
>> (AFAIK) just doesn't go this low level and even C isn't entirely
>> consistent between *NIX systems.
>>
>
> I don't know anything about Java.
>
> What does Java do when it wants single-character input from a terminal?
Or
> is it only able to do line-at-a-time? I think eclipse is written in Java,
> which suggests that single-character input must be supported.
>
> Wes

Last time I looked, unbuffered stdin was not possible to get hold of in
Java.

Eclipse is different - key presses etc come is as Events through which ever
windowing platform (Swing, or what ever it is) rather than as a stdin
stream.

I think Mark Porter might have a good case - this isn't possibly to do
portably, and also isn't something that people care about very often. If
they need this, then it should probably be an extra C/'native' module that
sets this.

Thoughts?

[Wes Garland]

On Wed, Jun 3, 2009 at 12:13 PM, Daniel Friesen <nadir.s...@gmail.com> wrote:

Well, NSPR doesn't even support canonical. std(in/out/err) is separate
from normal file handles.

 
What does that have to do with anything?

So I can say that even if ServerJS supports canonical I won't be
supporting it in MonkeyScript, even in the ServerJS wrapper.

Yes, we all know you have no interest in interoperability, you've made that point loud and clear at least a dozen times.

Wes

[Wes Garland]

> Thoughts?

My feeling is that exposing terminal line discipline is not necessary, but it may be advantageous to expose a single keystroke interface which is implemented with the equivalent of noecho() raw() nonl() curses calls.

Exposing that functionality would allow applications such a vi or a simple REPL with history and line editting (like GNU readline) to be written entirely in serverjs.

[Kris Kowal]

On Wed, Jun 3, 2009 at 9:21 AM, Wes Garland <w...@page.ca> wrote:
>> Thoughts?

I'm thinking that we should drop the "canonical" mode. It was
ill-advised on my part. CANON is usually set with another function
(ioctl?) anyway, and it's by far the more common case to have a tty
stream that you want to set to canonical mode than it is to directly
open one, like open("/dev/tty", "ubc"). I think we should at least
postpone the issue, as it can be tacked on later.

Kris Kowal

[Daniel Friesen]

Wes Garland wrote:
> On Wed, Jun 3, 2009 at 12:13 PM, Daniel Friesen
> <nadir.s...@gmail.com <mailto:nadir.s...@gmail.com>> wrote:
>
>
> Well, NSPR doesn't even support canonical. std(in/out/err) is separate
> from normal file handles.
>
>
> What does that have to do with anything?

NSPR is a platform abstraction library. And a perfectly reasonable one
for using toe build an IO module with.
If it doesn't support canonical handling of a stream then should we
really be forcing the programmer of an implementation to write a pile of
extra code taking platform differences into account, for a feature that
only a few libraries are likely to even use?
I might be misunderstanding the level of code needed, but from what I
understand the code just to implement canonical mode would be roughly
the same size as the code to implement the rest of read/write written
using NSPR.

>
> So I can say that even if ServerJS supports canonical I won't be
> supporting it in MonkeyScript, even in the ServerJS wrapper.
>
>
> Yes, we all know you have no interest in interoperability, you've made
> that point loud and clear at least a dozen times.

I'm plenty interested in interoperability. There's a good portion of my
own spec that is compatible enough with ServerJS specs that both a
ServerJS version and a banana use the same back to them, just with a
different flavor of api for the application programmer that wants it.
And it's just becoming more and more interoperable. Even things like the
use of with are negotiable if a better pattern can be found. You should
have seen the original Banana plan, the exporting moved from "this",
"self", I might have even had "module" in at one point. Right now it's
to a point where someone can write a library and there is little
stopping them from running it in ServerJS with require, using the
serverjs::require inside MonkeyScript, or using it as a banana with the
added versioning benefits.
All the differences between it and ServerJS are rooted on a reason or
more for it. If I find a compromise to that reason or an alternative it
just gets closer.

I made that statement because as far as I know, I don't even see any
current implementations that support canonical, and from the looks of it
getting someone to write support for that wouldn't fit cost to benefit.
Ya, some random C/C++ programmer could throw a patch the way of the
project, but the price to ask someone to write that chunk of code
doesn't seam worth it.

> Wes
>
> --
> Wesley W. Garland
> Director, Product Development
> PageMail, Inc.
> +1 613 542 2787 x 102

[Wes Garland]

NSPR is a platform abstraction library. And a perfectly reasonable one
for using toe build an IO module with.
If it doesn't support canonical handling of a stream then should we
really be forcing the programmer of an implementation to write a pile of
extra code taking platform differences into account, for a feature that
only a few libraries are likely to even use?

NSPR supports terminal streams in canonical mode.  I still maintain that whether it does or not is completely orthogonal to this discussion.

I made that statement because as far as I know, I don't even see any
current implementations that support canonical, and from the looks of it
getting someone to write support for that wouldn't fit cost to benefit.

I challenge you to find one implementation which supports stdin but does not support stdin in canonical mode.

Ya, some random C/C++ programmer could throw a patch the way of the
project, but the price to ask someone to write that chunk of code
doesn't seam worth it.

Changing between canonical and raw streams in a curses environment requires three bytes of code. Well, 6 if you count the brackets and the semi-colon.  Different environments require different code.  In a plain vanilla POSIX/SUS3 environment without curses, switching from canonical to raw input requires approximately 5 lines of code each way.

Wes
 

[Daniel Friesen]

Wes Garland wrote:
>
> NSPR is a platform abstraction library. And a perfectly reasonable one
> for using toe build an IO module with.
> If it doesn't support canonical handling of a stream then should we
> really be forcing the programmer of an implementation to write a
> pile of
> extra code taking platform differences into account, for a feature
> that
> only a few libraries are likely to even use?
>
>
> NSPR supports terminal streams in canonical mode. I still maintain
> that whether it does or not is completely orthogonal to this discussion.

NSPR, Boost, a Java backend, etc...
Pick your abstraction. I'm just saying, if it something can't be done
easily (in comparison to other features) in most abstraction layers or
without needing to maintain a number of different ways of doing
something for each platform you're on, then we should think twice about
making it a standard available feature.
I was originally under the impression that this was one of those.
NSPR is just the one I'm most familiar with, and I haven't seen the
notes on canonical support.

>
> I made that statement because as far as I know, I don't even see any
> current implementations that support canonical, and from the looks
> of it
> getting someone to write support for that wouldn't fit cost to
> benefit.
>
>
> I challenge you to find one implementation which supports stdin but
> does not support stdin in canonical mode.
>
> Ya, some random C/C++ programmer could throw a patch the way of the
> project, but the price to ask someone to write that chunk of code
> doesn't seam worth it.
>
>
> Changing between canonical and raw streams in a curses environment
> requires three bytes of code. Well, 6 if you count the brackets and
> the semi-colon. Different environments require different code. In a
> plain vanilla POSIX/SUS3 environment without curses, switching from
> canonical to raw input requires approximately 5 lines of code each way.
>
> Wes

Ok, if it is supported easily in platform abstraction then I'll support
the same as ServerJS.

Question, canonical on file.open;
If we're putting canonical in the mode, what are we expecting it to work on?
What abstraction layers are able to support canonical mode, where?
Is this something that belongs generically in open, or is this something
which only applies to stdin (in which case, didn't we put stdin at
system.stdin rather than file.open?).

[Wes Garland]

> Question, canonical on file.open;
> If we're putting canonical in the mode, what are we expecting it to work on?

I would expect it to work on terminals (in particular stdin). It is the only place where it has any real meaning, given that we already have byte-at-a-time and line-at-a-time read methods.

> What abstraction layers are able to support canonical mode, where?

Virtually every application I have ever seen for UNIX and work-alike operating systems which work on consoles uses canonical mode, with the exception of applications built around TUIs or REPLs.   So "grep" uses canonical mode, where as "vim" does not.

> Is this something that belongs generically in open, or is this something
> which only applies to stdin (in which case, didn't we put stdin at
> system.stdin rather than file.open?).

Adding a canonical flag to open is superfluous, unless we decree that all terminal-backed Streams are non-canonical. This would be a silly decree, as the stdio and stdout streams are historically canonical.

You could, on the other hand, convince me that a way mark a stream as both raw and non-blocking would be useful.  That would override the default and let you write applications like "vim" with serverjs.

I am ambivalent as to whether this arrives via open flags or an ioctl/termios/etc-style module.  Doing it via open flags would be more portable, but Windows really isn't on my horizon (nor resume).

[Daniel Friesen]

Wes Garland wrote:
> > Is this something that belongs generically in open, or is this something
> > which only applies to stdin (in which case, didn't we put stdin at
> > system.stdin rather than file.open?).
>
> Adding a canonical flag to open is superfluous, unless we decree that
> all terminal-backed Streams are non-canonical. This would be a silly
> decree, as the stdio and stdout streams are historically canonical.
>
> You could, on the other hand, convince me that a way mark a stream as
> both raw and non-blocking would be useful. That would override the
> default and let you write applications like "vim" with serverjs.
>
> I am ambivalent as to whether this arrives via open flags or an
> ioctl/termios/etc-style module. Doing it via open flags would be more
> portable, but Windows really isn't on my horizon (nor resume).
>
> Wes

I agree that canonical doesn't fit well in file.open;

Since this is stdin stuff, I'd vote for:
system.stdin.canonical;
Or whatever property names you want to come up with for the options.

As for portability and Windows. Would closing the stdin and re-opening
it in the other mode in the background if it is changed suffice?
If so I think that would probably be the best option.

[Kris Kowal]

On Wed, Jun 3, 2009 at 2:14 PM, Daniel Friesen
<nadir.s...@gmail.com> wrote:
> As for portability and Windows. Would closing the stdin and re-opening
> it in the other mode in the background if it is changed suffice?
> If so I think that would probably be the best option.

In the long term, raw IO can be emulated using lower-level calls like
_kbhit(), as I recall.

Again, I'm advising that we defer discussion of
canonical/non-canonical IO, and removing the "c" mode character from
the specification. Canonical IO is not generally supported by flags
to "open", and is relegated to the obscurata of other functions.

Kris Kowal

[Daniel Friesen]

Deferring it till we actually get a spec that can do a reasonable amount
of normal things does sound reasonable.

[Hannes Wallnoefer]

2009/6/3 Kris Kowal <cowber...@gmail.com>:

+1 on dropping canonical, I don't think this could be implemented in
java without resorting to native code. For the purpose of implementing
line editing features, the best approach IMO is to use an existing
library such as readline or jline.

2009/6/2 Ash Berlin <ash_g...@firemirror.com>:

>
> Also we need to specify where the precedence is when mode and any of
> the boolean flags are specified. i.e.:
>
> open(path, {mode: "rb", binary: false, write: true, read: false }) //
> WTF does this do.
>
> Thats partly why I'm not sure having mode in the options is needed.
>

Yes, I also think that having both a "mode" and
"read"|"write"|"update"... properties in the options object is
redundant and potentially dangerous, so I'm for not supporting
options.mode.

Regarding the question how to deal with conflicting mode/option flags,
I suggest to demand throwing a TypeError, like ES5 defines for
conflicting property descriptors.

>>
>> wait, update? was that read+write or something else?
>>
>> ~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://
>> daniel.friesen.name]
>
> Yeah - what is update?
>

It returns an Updater object that provides read + write + seek:

https://wiki.mozilla.org/ServerJS/Filesystem_API/A#.2AReader.2C_.2AUpdater

Hannes

[Daniel Friesen]

I second a TypeError. No reason not to throw one if the programmer is
throwing stupid arguments. That's what I had planned if someone tried to
say {binary:true,text:true}.

>>> wait, update? was that read+write or something else?
>>>
>>> ~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://
>>> daniel.friesen.name]
>>>
>> Yeah - what is update?
>>
>>
>
> It returns an Updater object that provides read + write + seek:
>
> https://wiki.mozilla.org/ServerJS/Filesystem_API/A#.2AReader.2C_.2AUpdater
>
> Hannes
>

Thought so. So update is basically the same thing as my
{read:true,write:true} or "rw".

[Mark Porter]

I only use Rhino and I'm not very familiar with the other engines. In
Rhino I can access all the Java I/O and streaming classes directly. Is
this the case for SpiderMonkey as well? What about the others?

On Jun 3, 10:18 am, Ash Berlin <ash_goo...@firemirror.com> wrote:
> On Wed, 3 Jun 2009 12:06:00 -0400, Wes Garland <w...@page.ca> wrote:

[Daniel Friesen]

From JavaScript?

For native stuff one would need either Mark Finkle js-ctypes or Soubok's
jsffi.
I don't know how ready either one of them are.
Normally we just write a wrapper in C/C++

~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://daniel.friesen.name]