load (load! ?)

[Estevo]

The R-1RK lists load in section 15.2.2 as a library feature of the ports module.  No description is given of the applicative.  Although we all have a general idea of what load is meant to do, the following questions remain (more or less) open:

1 - Should the signature be (multiple-choice):

    * (load filename)

    * (load filename env)

2 - Should (load filename) evaluate its expressions in
    * the dynamic environment of the caller?
    * in a new child of the above?
    * in a fresh standard environment (like get-module)?

3 - If load can modify a preexisting environment (either by working on the dynamic environment of the caller by default, or by taking a --perhaps optional-- env argument), should it be called load! instead, for consistency with $define!, $set!, $import!, $provide!, etc.?  Although I believe this should be the case, I'll keep calling it just 'load' in the rest of this post, to avoid confusion.

4 - Should load return

    * #inert ?

    * the value of the last expression evaluated (like $sequence) ?

    * the environment it evaluated in (if new) ?

5 - Should load try and find the files to load in (multiple-choice)

    * the current working directory (whatever that means in the target OS) ?

    * the directory of the source file containing the call to load ?
    * a list of paths available for programmatic editing by the Kernel program (like Python's sys.path).  And in this case, should it be exposed as

        a dynamic keyed variable?

        a static keyed variable?

        something else?

    * a list of paths set in some environment variable (e.g. KERNELPATH, similarly to Klisp's KLISP_PATH, Python's PYTHONPATH, Java's CLASSPATH)?

(5b - Should (5) be even the responsibility of load at all, or should we have a different function that converts a 'relative' to an 'absolute' path, so load can just take the latter?  This question came up to mind when reading Klisp's docs on this matter -- see below.)

6 - Should load prevent reentry into its dynamic extent, to avoid attempts to read/close an already closed file?  Should, instead, evaluation be defined to happen logically after the physical reading of the file?  Or should this just be left out of the standard entirely?

7 - Should load work as if reading immutable evaluation structures?

Here is a description of (my possibly wrong reading of) how SINK implements load:

The filename is resolved and opened by the host Scheme environment (that is, no explicit attempt is made to search the file in the directory of the source file containing the call to load-- more on this later) and the expressions from the file are read as immutable evaluation structures and evaluated, as they are read, in the dynamic environment of the caller.  In addition, an entry guard is installed to prevent reentry into the dynamic extent of the load call.  This probably makes sense, to prevent attempts to read/close the file after it has been already closed.  But maybe it wouldn't be necessary if all expressions were read in one pass, and then all of them were evaluated.  So maybe the spec shouldn't be so implementation-specific as to mandate this guard?

The rationale for the ports module has this sentence:

    The port-based i/o tools in R5RS Scheme are already well suited to this design goal,
    and so Kernel adopts Scheme’s port tools substantially intact.

Although "this design goal" here refers to the safe handling of dead port objects, in general it seems the intention is indeed to use R5RS except where explicitly noted.

The R-1RK also has this passing remark in the description of applicative get-module:

    If all source files of a Kernel program were processed in a
    single environment, as via applicative load (§15.2.2)

which suggests that load is meant to evaluate the expressions in the source file in the dynamic environment of the caller.

Here's how R5RS describes load:

    (load filename)   optional procedure

    Filename should be a string naming an existing file con-
    taining Scheme source code. The load procedure reads ex-
    pressions and definitions from the file and evaluates them
    sequentially. It is unspecified whether the results of the
    expressions are printed. The load procedure does not
    affect the values returned by current-input-port and
    current-output-port. Load returns an unspecified value.
    Rationale: For portability, load must operate on source files.
    Its operation on other kinds of files necessarily varies among
    implementations.

R7RS makes this procedure mandatory and adds some interesting options:

    (load filename)   load library procedure
    (load filename environment-specifier)  load library procedure

    It is an error if filename is not a string.
    An implementation-dependent operation is used to trans-
    form filename into the name of an existing file con-
    taining Scheme source code. The load procedure reads
    expressions and definitions from the file and evalu-
    ates them sequentially in the environment specified by
    environment-specifier . If environment-specifier is omitted,
    (interaction-environment) is assumed.
    It is unspecified whether the results of the expres-
    sions are printed. The load procedure does not af-
    fect the values returned by current-input-port and
    current-output-port. It returns an unspecified value.
 
    Rationale: For portability, load must operate on source files.
    Its operation on other kinds of files necessarily varies among
    implementations.

This is Klisp's documentation on load

Applicative: load (load string)

    Applicative load opens the file named string for textual input;
    reads immutable objects from the file until the end of the file
    is reached; evaluates those objects consecutively in the
    created environment. The result from applicative load is inert.

    Notice that if string is a relative path it is looked in the current
    directory (whatever that means in your OS, normally the directory
    from which the interpreted was run or the directory where the
    interpreter executable lives). klisp doesn't track the directory
    from which the current code was read, so there's in principle no
    way to load a file in the same directory as the currently
    executing code with a relative path. See find-required-filename
    for a way to look for a file in a number of directories.

    SOURCE NOTE: load is enumerated in the Kernel report, but the
    description is not there yet. This seems like a sane way to define
    it, taking the description of get-module that there is in the report.
    The one detail that I think is still open, is whether to return #inert
    (as is the case with klisp currently) or rather return the value of
    the last evaluation.

Other implementations:

Since Bronze Age lisp claims compatibility with Klisp, and since I think Oto is likely to read this and correct me if I'm wrong, I'm going to be lazy and, with apologies, assume that what is true with Klisp is true for Bronze Age.

In a quick search, I couldn't find docs or source for Klink's implementation of load, or even whether this function is implemented.

I don't comment (much) on icbink since it's only now that I'm trying to bring it up to standards compliance, so not much should be read into however things are working in it currently.

So please don't expect an extensive survey of the state of Kernel implementations; I just reviewed what I could easily get my hands on.

Preliminary personal discussion:

1 - R7RS adds an optional 'environment' argument to load that looks like it could be convenient in Kernel.  Then again, the functionality can be simulated in Kernel with (eval (list load filename) env), so this question is not terribly important. Since no implementations that I know of support this and no Kernel-specific resources mention it, let's leave this as something to maybe consider for R(>1)RK?  This additional parameter should definitely not be included unless the next question is resolved as I suggest, because then load could act either as a mutating function or not, depending on how you call it, and then there's no clearly good answer to question 3.

2 - It seems clear from the wording of get-module's rationale that get-module is intended as a more hygienic alternative to load, which in turn is meant to evaluate the loaded expressions in the dynamic environment of its call.  Both R5RS and SINK do that; Klisp creates a new environment, explicitly to follow get-module's semantics.  I'm inclined to follow SINK here.

3 - Although the R-1RK, R*RS, and all Kernel implementations that I know of spell the name as load, if the response to 2 is as proposed above, I strongly believe that this environment mutating applicative should then be called load!

4 - Both Klisp and SINK return #inert (although Andrés seems not totally decided on this).  That's also the most straightforward translation to Kernel of the R*RS declaration of the return value as 'undefined'.  I definitely think that load should return #inert if it evaluates the source expressions in the dynamic environment of its call.  Now, if load is to create a new environment instead, it should probably return that environment, right?  Otherwise, it's hard to get access to whatever newly created bindings.

5 - Neither Klisp nor SINK try and look for the file in the directory of the source file containing the call to load.  I've just added this to icbink before realizing this, but I'll take it out if it would encourage incompatible code.  For implementors of modules spanning several source files, it would be handy to be able to use relative, local paths in load.  Now, what I think would be critical for a practical development system would be customizing the search path so source files don't need to specify absolute paths.  Klisp has KLISP_PATH, and there seem to be plans for extending the search path dynamically.  KERNELPATH looks like a reasonable name if this is made a (even de facto) standard across implementations.

5b - Incidentally, Klisp also factors out the file search itself into a find-required-filename applicative, which is a nice idea that hadn't crossed my mind.  If we had that in the standard (maybe with a less specific name?), load could be discharged of that responsibility, and just take an absolute filename.  That's good orthogonality and might help with accident prevention too (e.g. accidentally loading a file in the source directory, when you meant to load one in the working directory).

6 - I don't know how to word it so it's not too vague nor implementation-specific, but accident avoidance suggests that (at least) in robust implementations, file operations shouldn't happen in a dynamic extent that can be reentered.

7 - SINK sets a precedent for replying yes to this question, and I can think of no good argument for replying no.  Klisp specifies that load reads the forms as immutable.

[Oto Havle]

On 05/21/2014 05:05 PM, Estevo wrote:
> [...]

> Other implementations:
>
> Since Bronze Age lisp claims compatibility with Klisp, and since I think
> Oto is likely to read this and correct me if I'm wrong, I'm going to be
> lazy and, with apologies, assume that what is true with Klisp is true
> for Bronze Age.
>

I agree that load should return #inert and the loaded object should
be immutable. Bronze Age's violates both principles.

Concerning the return value, Bronze Age Lisp is not "robust." The
Kernel Report discusses somewhat similar problem of $and?. In analogy
to $and?, non-robust implementation of load (and also non-standard
$when and $unless) can (I think) evaluate the last form in the tail
context.

Concerning the immutability, Bronze Age Lisp does not make immutable
copy of $vau and $lambda source for performance reasons. That's why
I did not make immutable copy in load.

> 3 - Although the R-1RK, R*RS, and all Kernel implementations that I know

> of spell the name as /*load*/, /if /the response to 2 is as proposed

> above, I strongly believe that this environment mutating applicative

> should then be called /load*!*/
>

I don't think that load is an environment mutating applicative. The
effect depends on what is in the file.

[...]

Best wishes,

Oto Havle.

[Estevo]

On Fri, May 23, 2014 at 8:45 PM, Oto Havle <havl...@gmail.com> wrote:

On 05/21/2014 05:05 PM, Estevo wrote:

[...]

Other implementations:

Since Bronze Age lisp claims compatibility with Klisp, and since I think
Oto is likely to read this and correct me if I'm wrong, I'm going to be
lazy and, with apologies, assume that what is true with Klisp is true
for Bronze Age.

I agree that load should return #inert and the loaded object should
be immutable. Bronze Age's violates both principles.

  Concerning the return value, Bronze Age Lisp is not "robust." The
Kernel Report discusses somewhat similar problem of $and?. In analogy
to $and?, non-robust implementation of load (and also non-standard
$when and $unless) can (I think) evaluate the last form in the tail
context.

  Concerning the immutability, Bronze Age Lisp does not make immutable
copy of $vau and $lambda source for performance reasons. That's why
I did not make immutable copy in load.

Makes sense.  As I said, I bypass that problem by not implementing pair mutation at all.  In my case it's not (only) a simplification for performance; I actually prefer the language like that.  But that's a topic for other time.
 

3 - Although the R-1RK, R*RS, and all Kernel implementations that I know

of spell the name as /*load*/, /if /the response to 2 is as proposed

above, I strongly believe that this environment mutating applicative

should then be called /load*!*/

I don't think that load is an environment mutating applicative. The
effect depends on what is in the file.

Good point.  This might be why there's no '!' in R5RS's /load/ either.

Thanks for your reply, and (again) for kplts.  I'll hopefully have some time to bring icbink closer to compliance this weekend.

Happy kerneling,

Estevo.