Syntax definitions and the order of expansion in R7RS-large

[Marc Nieper-Wißkirchen]

R7RS-small modified R6RS in the following regard:

In R7RS-small, "uses of a syntax form [definition] cannot appear before its definition [in the program text]; the even/odd example of R6RS is not allowed". (R7RS, p.80).

This modification was certainly well-intentioned, but not thought out.  What has apparently been missed by the R7RS-small authors is that variable references are just another type of syntax and that it doesn't make sense to arbitrarily restrict forward references to some forms of syntax.

The reason why I bring it up here is that this modification must be dropped and reverted to the R6RS state of affairs in R7RS-large if the various syntactic abstractions that have been discussed and that build on what's on the current, Yellow ballot, are to fit in seamlessly.

For example, keyword systems with no runtime overhead have been discussed where procedure definitions become, in fact, macro definitions and identifier macros are used to resolve to actual procedures.  Let `define/keywords' be a hypothetical definition keyword for such procedures.  Now, as much as I can write

(let* ()

  (define (even ...) ... (odd ...) ...)

  (define (odd ...) ... (even ...) ...))

I would then want to be able to write

(let* ()

  (define/keywords (even ...) ... (odd ...) ...)

  (define/keywords (odd ...) ... (even ...) ...))

For that to work, however, we mustn't make the artificial difference between variable references and other types of syntax.

Thus, it should be clear that the R7RS-small limitation has to be dropped in R7RS-large if any of the more sophisticated syntactic abstraction facilities are added to it.

Luckily, dropping the R7RS-small limitation is still backward compatible to R7RS-small and, moreover, does no harm (but actually simplifies the language).  As a positive side effect, compatibility with R6RS is also increased.

Marc

[Daphne Preston-Kendal]

On 15 Nov 2021, at 13:15, Marc Nieper-Wißkirchen wrote:

> R7RS-small modified R6RS in the following regard:
>
> In R7RS-small, "uses of a syntax form [definition] cannot appear before its
> definition [in the program text]; the even/odd example of R6RS is not
> allowed". (R7RS, p.80).
>
> This modification was certainly well-intentioned, but not thought out.

What was the intention? Was it about cleaning up the fact that R6RS doesn’t have any defined REPL semantics?

If so, I would suggest a solution: in the Scheme implementation I’m currently planning (but am unlikely to get to doing much work on until summer), the intention is to have two kinds of environments: static and interactive. As the name implies, compiling a library with its top-level environment in static mode allows the compiler to make optimizations by assuming that top-level bindings which aren’t mutated within the library code itself are entirely immutable, gather all definitions (including, especially, syntax definitions) before doing any other evaluation, etc. An interactive environment, on the other hand, has roughly the REPL semantics allowed by R7RS small.

Could it make sense to adopt this distinction in the standard? Or, perhaps better said, to deepen the distinction between what R7RS simply calls ‘the REPL’, in which using syntax before defining it is disallowed, and code which appears within a define-library, which could have R6RS semantics?


Daphne

[Marc Nieper-Wißkirchen]

Program and library semantics are well-defined, static, and can be
agreed upon. On the other hand, REPL semantics have never been
specified as strictly, they are not important when it comes to
portability of programs, and, in any case, the "top-level is
hopeless".

That said (about the REPL), R7RS has mutable environments (first and
foremost the interaction environment) and the behavior of `define` and
`define-syntax` has to be specified for mutable environments, but I
think this is all that has to be done (REPL implementations are then
encouraged to evaluate each command as if by `(eval command
(interaction-environment))'.

The semantics can be defined as follows: Each identifier x possesses
(conceptually) a shadow version, say $x. Each shadow version $x is a
variable (bound to a location). `(define x <expr>)` is then expanded
to `(begin (alias x $x) (set! x <expr>))`. Initially, each identifier
is aliased to its shadowed version. Definitions are executed one after
the other (as if we have R6RS contours with just one definition); if a
definition occurs, the environment is updated first and then the
right-hand side evaluated. Identifiers can be redefined.

But all of this does not make the top-level less hopeless, of course.

And: due to the different requirements, some macros written to be used
in program/library top-level or procedure bodies will yield different
results when expanded in mutable environments.

Am Di., 15. Feb. 2022 um 10:51 Uhr schrieb Daphne Preston-Kendal
<d...@nonceword.org>:

> --
> You received this message because you are subscribed to the Google Groups "scheme-reports-wg2" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to scheme-reports-...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/scheme-reports-wg2/48BD87D9-DF5F-49C7-BE65-EE9B53453D6D%40nonceword.org.

[Dr. Arne Babenhauserheide]

Marc Nieper-Wißkirchen <marc....@gmail.com> writes:
> And: due to the different requirements, some macros written to be used
> in program/library top-level or procedure bodies will yield different
> results when expanded in mutable environments.

I’d like to warn about increasing differences between REPL and
code-in-files.

In Python that creates annoying problems (i.e. with multiple empty lines
within function bodies which suddenly cannot be tested easily in the
shell), and being able to paste something into the REPL for testing is a
big advantage.

Even different optimization can create problems, because that can make
it much harder to debug, but it’s hard to avoid that while keeping
interactive semantics — running a full compile step of all dependent
forms after every definition in the REPL might make interactive
incremental improvement infeasible.

Best wishes,
Arne
--
Unpolitisch sein
heißt politisch sein,
ohne es zu merken.
draketo.de

[John Cowan]

On Mon, Nov 15, 2021 at 7:15 AM Marc Nieper-Wißkirchen <marc....@gmail.com> wrote:

(let* ()

  (define (even ...) ... (odd ...) ...)

  (define (odd ...) ... (even ...) ...))

I would then want to be able to write

(let* ()

  (define/keywords (even ...) ... (odd ...) ...)

  (define/keywords (odd ...) ... (even ...) ...))

I agree that this is a fine example, and indeed the first compelling example I have seen.  But it inherits its compelling nature from the close resemblance of such macro definitions to procedure definitions, a very special case.

[John Cowan]

On Tue, Feb 15, 2022 at 4:51 AM Daphne Preston-Kendal <d...@nonceword.org> wrote:

 

If so, I would suggest a solution: in the Scheme implementation I’m currently planning (but am unlikely to get to doing much work on until summer), the intention is to have two kinds of environments: static and interactive. As the name implies, compiling a library with its top-level environment in static mode allows the compiler to make optimizations by assuming that top-level bindings which aren’t mutated within the library  code itself are entirely immutable, gather all definitions (including, especially, syntax definitions) before doing any other evaluation, etc. An interactive environment, on the other hand, has roughly the REPL semantics allowed by R7RS small.

R7RS allows for this in the last paragraph of the section "The REPL", which allows a mode in which the REPL (or a REPL) reads its input from a file.  Note that such a script file (a term not used in R7RS-small) is not interactive as such, but nevertheless has REPL semantics, not whole-program semantics.

[Marc Nieper-Wißkirchen]

What you call its compelling nature is just an instance of why it is a
good idea not to treat the scope of an identifier differently
depending on to which entity it is bound.

[John Cowan]

On Wed, Feb 16, 2022 at 9:26 AM Marc Nieper-Wißkirchen <marc....@gmail.com> wrote:

 

What you call its compelling nature is just an instance of why it is a
good idea not to treat the scope of an identifier differently
depending on to which entity it is bound.

I think it *is* a good idea in most cases, for the reasons given at <https://www.mail-archive.com/r6rs-d...@lists.r6rs.org/msg04199.html> in allegorical form.

[Marc Nieper-Wißkirchen]

Some discrepancies between the program/library/procedure bodies and
the REPL are unavoidable. If you don't like this, you will probably
want a language with fexprs.

Just do give an example, take a look at the following macro:

(define-syntax m
(syntax-rules ()
((m f)
(begin (define (f) x)
(define x 1)))))

The expansion of (m f) yields:

(begin
(define (f) x*)
(define x* 1))

where * denotes renaming/marking of an identifier. (As it is
irrelevant, I ignore the marks on begin and define.)

When the evaluation happens at the beginning of a procedure body (or
in an R6RS program or library body), a new variable named x* will be
bound to 1 in the environment, and invoking (f) would return 1. When
the evaluation happens at the REPL,

(define (f) x*)

will be executed first. For that, x* will be looked up in the
interactive environment but there will be no such x* being bound. So
the mark is stripped and the resulting identifier x is looked up in
the environment of m. Let us assume this is the interactive
environment as well. x will then refer to the (implicitly) bound cell
of the variable x at the REPL. Thus invoking (f) will return the value
of the global variable x in the interactive environment.

The next definition

(define x* 1)

won't change this, as it will bind a new marked (!) variable x* to 1
with no influence on (f).

I currently don't remember exactly why, but I think that the
alternative, namely that `begin` at the REPL evaluates the contained
definitions not one by one but using the R6RS algorithm for library
bodies, leads to other inconsistencies.


Am Mi., 16. Feb. 2022 um 11:20 Uhr schrieb Dr. Arne Babenhauserheide
<arne...@web.de>:

[John Cowan]

On Tue, Feb 15, 2022 at 5:36 AM Marc Nieper-Wißkirchen <marc....@gmail.com> wrote:

Program and library semantics are well-defined, static, and can be
agreed upon.

In R6 they are; not so in R7.

On the other hand, REPL semantics have never been
specified as strictly, they are not important when it comes to
portability of programs,

That's true only tautologically: we say that a REPL script is not a program and therefore it is not important to program portability.

and, in any case, the "top-level is
hopeless".

In my opinion, Flatt said that because like many mathematicians he is obsessed with Ausnahmslosigkeit, and feels that because there is no complete theory of something, there can be no worthwhile theory of it at all.  (Daphne and I know that while Ausnahmslosigkeit is the right assumption to start with, real data is riddled with exceptions.)

 

And: due to the different requirements, some macros written to be used
in program/library top-level or procedure bodies will yield different
results when expanded in mutable environments.

So they will.

[Marc Nieper-Wißkirchen]

This is why I wrote "well-intentioned".

It surely makes sense to write a program in a way so that it can be
mostly read linearly, but we cannot enforce it through the language.
The example by which I started this thread shows why restricting
forward references just to variables is not sound, and we can't remove
forward references for variables, which are needed for recursion.
Moreover, as one can turn many macros into equivalent procedures
(possibly taking thunks as arguments), I don't see how restricting
forward references just to variables would solve any assumed problem
in principle. In other words, even in R4RS Scheme, I could write
programs upside-down if I wanted to.

[Daphne Preston-Kendal]

I’m not impressed by this analogy.

First, as Marc points out, the restriction does not merely rule out defining on page 60 notation used on page 6, but also defining together on page 6 two forms with a dependency relationship between them. Your analogy works only assuming the distance between definition and use is intractable, assuming they’re in the ‘wrong’ order.

Second, computers aren’t humans. Humans may find it annoying to see a use of syntax on line 10 which isn’t defined until line 1000, but computers can handle it just fine. You’re proposing using a rule of the language semantics to enforce what ought to be nothing more than a coding convention.

Indeed, Dr Hardcase might well have relegated explanations of his notation to an appendix and focussed on the actual meat of his proof in the body of the paper — such things are not uncommon in academic writing. I might well wish to do similar in code, using a comment such as ‘;; In order to make this more efficient, we use a comprehend-quickly macro which is defined at the end of this file’ on line 10 when comprehend-quickly is on line 1000. Comprehend-quickly may have nothing to do with the specific purpose of the module, while also being sufficiently specific as not to warrant moving out into a distinct library. (Also, when possible I use meta-point to jump around and find definitions when I can and need to, which also doesn’t care what order or even what file things are in.)

As it is apparently only ‘an error’ to use syntax before defining it in small, we can require it to be possible in Large. I support making this change (though with a loosening of the R6RS restriction on the order of definitions and other expressions within program/library bodies, which Marc has suggested for R6.1RS, and is implied by the requirement for R7RS Large to be compatible with R7RS small).


Daphne

[Marc Nieper-Wißkirchen]

Am Mi., 16. Feb. 2022 um 15:55 Uhr schrieb John Cowan <co...@ccil.org>:
>
>
>
> On Tue, Feb 15, 2022 at 5:36 AM Marc Nieper-Wißkirchen <marc....@gmail.com> wrote:
>
>> Program and library semantics are well-defined, static, and can be
>> agreed upon.
>
>
> In R6 they are; not so in R7.

Right; I should have said that we have a model (stricter than the R7
model) that has well-defined semantics and works well with all
syntactic extensions we have been discussing.

>>
>> On the other hand, REPL semantics have never been
>> specified as strictly, they are not important when it comes to
>> portability of programs,
>
>
> That's true only tautologically: we say that a REPL script is not a program and therefore it is not important to program portability.

My premise here was that one reason for the R[67]RS notion of
libraries and top-level programs was to make these the objects that
are relevant for code exchange between the Scheme systems.

So, replace "portability of programs" with "a generally agreed-upon
mechanism to share Scheme code".

>>
>> and, in any case, the "top-level is
>> hopeless".
>
>
> In my opinion, Flatt said that because like many mathematicians he is obsessed with Ausnahmslosigkeit, and feels that because there is no complete theory of something, there can be no worthwhile theory of it at all. (Daphne and I know that while Ausnahmslosigkeit is the right assumption to start with, real data is riddled with exceptions.)

Under this premise: Taking a look at Racket seems to imply that this
"obsession" can lead very far. In any case, Scheme wouldn't be the
language we love if there hadn't been people like him and
mathematicians or computer scientists "obsessed" with the idea of
creating a theoretically compelling language.

I actually don't understand the analogy of "real data is riddled with
exceptions".

[Dr. Arne Babenhauserheide]

Marc Nieper-Wißkirchen <marc....@gmail.com> writes:

> Some discrepancies between the program/library/procedure bodies and
> the REPL are unavoidable.

That’s why I wrote to avoid *increasing* the inconsistency. Currently it
is mostly consistent (you get issues if you use stuff you have not
defined yet, which is obvious), but even small increases in
inconsistency can make experimentation in the REPL risky. Which would
for example break the geiser-workflow.

In Guile

(define-syntax m
(syntax-rules ()
((m f)
(begin (define (f) x)
(define x 1)))))

(m f)
(f)
=> 1

[Dr. Arne Babenhauserheide]

I still remember that when I started to learn programming, it confused
me to no end that I could not just write the use of a form before its
definition.

While some inconsistency is unavoidable (you cannot use what’s not yet
written), it would be nice if programs would at least behave the same in
the REPL and a file after the *whole* text has been pasted into the
REPL.

[Marc Nieper-Wißkirchen]

Am Do., 17. Feb. 2022 um 07:35 Uhr schrieb Dr. Arne Babenhauserheide
<arne...@web.de>:
>
>

> Marc Nieper-Wißkirchen <marc....@gmail.com> writes:
>
> > Some discrepancies between the program/library/procedure bodies and
> > the REPL are unavoidable.
>
> That’s why I wrote to avoid *increasing* the inconsistency. Currently it
> is mostly consistent (you get issues if you use stuff you have not
> defined yet, which is obvious), but even small increases in
> inconsistency can make experimentation in the REPL risky. Which would
> for example break the geiser-workflow.

None of what has been discussed in this thread is about increasing the
already existing inconsistencies, I think.

In any case, by citing that "the top-level is hopeless", I didn't mean
that one should not try to specify at least the meaning of `(eval
command (interaction-environment))` but that it would be naive to
expect to get semantics that is fully satisfying. Thus, one should get
right the static (or, rather, declarative) part of the language (the
program and library top-level) first and not do compromises there to
fix something that is ultimately not fixable.

> In Guile
>
> (define-syntax m
> (syntax-rules ()
> ((m f)
> (begin (define (f) x)
> (define x 1)))))
> (m f)
> (f)
> => 1

Guile probably interprets `(define x 1)` as `(set! x 1)` here; it just
shows that there is no agreed-upon semantics for the REPL or the
interaction environment (yet).

[Marc Nieper-Wißkirchen]

Am Do., 17. Feb. 2022 um 07:51 Uhr schrieb Dr. Arne Babenhauserheide
<arne...@web.de>:
>
>

> Marc Nieper-Wißkirchen <marc....@gmail.com> writes:
>
> > Am Mi., 16. Feb. 2022 um 15:43 Uhr schrieb John Cowan <co...@ccil.org>:
> > in principle. In other words, even in R4RS Scheme, I could write
> > programs upside-down if I wanted to.
>
> I still remember that when I started to learn programming, it confused
> me to no end that I could not just write the use of a form before its
> definition.
>
> While some inconsistency is unavoidable (you cannot use what’s not yet
> written), it would be nice if programs would at least behave the same in
> the REPL and a file after the *whole* text has been pasted into the
> REPL.

This would need some form like Chez Scheme's (top-level-program ...)
[1], but with it, it is perfectly possible, I think.

--

[1] https://cisco.github.io/ChezScheme/csug9.5/libraries.html#./libraries:h2

[Dr. Arne Babenhauserheide]

Marc Nieper-Wißkirchen <marc....@gmail.com> writes:

>> (define-syntax m
>> (syntax-rules ()
>> ((m f)
>> (begin (define (f) x)
>> (define x 1)))))
>> (m f)
>> (f)
>> => 1
>
> Guile probably interprets `(define x 1)` as `(set! x 1)` here; it just
> shows that there is no agreed-upon semantics for the REPL or the
> interaction environment (yet).

I *think* Guile woorks as in a letrec* form:
https://www.gnu.org/software/guile/manual/html_node/Internal-Definitions.html

[Marc Nieper-Wißkirchen]

Am Do., 17. Feb. 2022 um 10:30 Uhr schrieb Dr. Arne Babenhauserheide
<arne...@web.de>:
>
>

> Marc Nieper-Wißkirchen <marc....@gmail.com> writes:
>
> >> (define-syntax m
> >> (syntax-rules ()
> >> ((m f)
> >> (begin (define (f) x)
> >> (define x 1)))))
> >> (m f)
> >> (f)
> >> => 1
> >
> > Guile probably interprets `(define x 1)` as `(set! x 1)` here; it just
> > shows that there is no agreed-upon semantics for the REPL or the
> > interaction environment (yet).
>
> I *think* Guile woorks as in a letrec* form:
> https://www.gnu.org/software/guile/manual/html_node/Internal-Definitions.html

This is what R6RS prescribes. But this touches only internal
definitions; not definitions at the top level of a REPL.

[Martin Rodgers]

I *think* this is an example of how the lack of well-defined REPL
semantics clouds *all* our thinking on this subject. I'm particularly
troubled by the assumption that there *is* and always will be a REPL,
which seems implicit in discussions of language features like eval and
syntax-case.

For example, which REPL language does an implementation like Guile use
as default? Guile 3.x has command line options for R6 and R7, but the
default is R5. This is reasonable for Guile, given its age relative to
all 3 reports. So its easy for all the conceptual baggage of Guile's
R5 REPL to be inherited by the other REPLs. Its even easier to reuse
much of the same code to implement a REPL. I'm not suggesting the
Guile devs are confused or lazy, only that getting this right is
*hard*. I suspect the greatest obstacle for the Guile devs may be the
Guile community and *their* confusion.

However, I'll be delighted to be proved wrong on this. Please prove me
wrong. I usually avoid this entire issue because of the lack of any
clear definitions, so I welcome this discussion. I hope it bares
fruit.

> --
> You received this message because you are subscribed to the Google Groups
> "scheme-reports-wg2" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to scheme-reports-...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/scheme-reports-wg2/87r181hmbd.fsf%40web.de.
>

[Marc Nieper-Wißkirchen]

Thanks for chiming in, Martin.

To reduce confusion one should probably not call what we have been
seeking REPL semantics as everyone and every implementation has their
own understanding of what REPL semantics mean. There is also the
question of why we would want to actually fix semantics; my claim has
been that it is not necessary for sharing of code (which is one raison
d'être for the standards) and that it is up to the individual
implementations to provide modes that make incremental programming and
debugging available to developers. But the semantics of
(interaction-environment) certainly should and could be specified. If
not, we should deprecate it as a formal part of the standard.

I'd like to add that even when we restrict ourselves to the semantics
given for the REPL in R7RS, surprises can still happen when we vary
the implementation. Consider the following definitions entered at the
REPL:

(define (fits-width? s)
(<= 0 (string-length s) (max-width)))
(define (max-width) 80)

The intent of this piece of code seems clear, but we may get
unexpected results with some implementations. The point is that there
is no upper limit on the set of initial bindings of the interaction
environment. So an implementation would be free to initially bind
max-width to a macro so that (max-width) expands to, say, 32 or 64
depending on the underlying fixnum format. By the rules of the REPL,
the first definition would then expand to, say,

(define (fits-width? s)
(<= (string-length s) 64))

and the following (re-)definition of max-width as a procedure wouldn't
change anything anymore. Thus, if one really tried to write portable
REPL code, one would have to code something like

(define max-width #f)
(define (fits-width? s) ...)
(define (max-width) 80)

which, surely, no one would like to do. (Before anyone objects that
one could have simply moved the final definition of max-width to the
top: please note that I just gave an example and every example is
inherently limited; when I have a couple of mutually dependent
procedures, such a simple reordering wouldn't work.)

> To view this discussion on the web visit https://groups.google.com/d/msgid/scheme-reports-wg2/CAP-t%3D%2B75f9Z6GY6Jza8C_N%2B%2B-xtkozbPCXnvadm6D4FNPXaFYw%40mail.gmail.com.

[Taylan Kammer]

Guile seems to do something quite interesting here.


First, how is (begin ...) with definitions handled at the REPL?

scheme@(guile-user)> (begin (define (f) x) (define x 1))
scheme@(guile-user)> (f)
$1 = 1
scheme@(guile-user)> x
$2 = 1

They were spliced directly into the top level. The disassembly of
the procedure f (omitted for brevity) reveals that it looks up the
variable 'x' in the module (guile-user).


Second, how is a true internal definition handled?

scheme@(guile-user)> (define f (let () (define (f) x) (define x 1) f))
scheme@(guile-user)> (f)
$1 = 1
scheme@(guile-user)> ,disassemble f
Disassembly of #<procedure f ()> at #x558fd06e7428:

0 (instrument-entry 74)
2 (assert-nargs-ee/locals 1 0) ;; 1 slot (0 args)
3 (make-immediate 0 6) ;; 1
4 (handle-interrupts)
5 (return-values)

There's no variable reference; the optimizer eliminated 'x' and
the procedure just returns the constant 1.


Finally, the macro example:

scheme@(guile-user)> (define-syntax m (syntax-rules () ((m f) (begin (define (f) x) (define x 1)))))
scheme@(guile-user)> (m f)
scheme@(guile-user)> (f)
$1 = 1
scheme@(guile-user)> x
...
Unbound variable: x
...
scheme@(guile-user)> ,disassemble f
Disassembly of #<procedure f ()> at #x5567ffa60ac8:

0 (instrument-entry 122)
2 (assert-nargs-ee/locals 1 1) ;; 2 slots (0 args)
3 (static-ref 1 117) ;; #<variable 7f41e9bc4150 value: 1>
5 (immediate-tag=? 1 7 0) ;; heap-object?
7 (je 9) ;; -> L1
8 (static-ref 1 80) ;; #<directory (guile-user) 7f41ecf87c80>
10 (static-ref 0 102) ;; x-5fdc5a112ac2882
12 (call-scm<-scm-scm 1 1 0 111)
14 (static-set! 1 106) ;; #<variable 7f41e9bc4150 value: 1>
L1:
16 (scm-ref/immediate 1 1 1)
17 (reset-frame 1) ;; 1 slot
18 (handle-interrupts)
19 (return-values)


The fact that x ends up unbound in the top-level led me to believe
that Guile was handling this like internal definitions as suggested,
but then I checked the disassembly.

I haven't included the disassembly from the first experiment for
brevity but it's the same as this one, with the exception that in
the original one I saw the variable name 'x' whereas here we see
the variable name 'x-5fdc5a112ac2882' i.e. a renamed one.

Hmm, let's test one more thing within that same REPL session:

scheme@(guile-user)> x-5fdc5a112ac2882
$2 = 1

Hah! So Guile did a renaming pass on the macro body, then spliced
it into the top-level.


I have no idea if this is good or bad, just putting it out there.

--
Taylan

[Marc Nieper-Wißkirchen]

Am Do., 17. Feb. 2022 um 11:28 Uhr schrieb Taylan Kammer
<taylan...@gmail.com>:

Thanks for doing this experiment, Taylan! So Guile is exactly doing on
what my counter-example (for consistent behavior) is based.

PS: That one can access the expander-marked variable x* through a
non-marked name (x-5f...82 in your example) is obviously due to some
non-conformant Guile implementation hack here.

[Martin Rodgers]

On 17/02/2022, Marc Nieper-Wißkirchen <marc....@gmail.com> wrote:
> Thanks for chiming in, Martin.
>
> To reduce confusion one should probably not call what we have been
> seeking REPL semantics as everyone and every implementation has their
> own understanding of what REPL semantics mean.

Agreed. There should be some defined areas of ambiguity for
implementions to exploit. We can at least attempt to narrow these
areas to avoid the problems being discussed here. This is the "fruit"
which I hope this discussion will bare.

> There is also the
> question of why we would want to actually fix semantics; my claim has
> been that it is not necessary for sharing of code (which is one raison
> d'être for the standards) and that it is up to the individual
> implementations to provide modes that make incremental programming
> and debugging available to developers.

My experience with Scheme began by using SCM, back in 1994. I found
the REPL invaluable for learning the language. I suspect the same is
true for many others. I've also had that experience with other
languages over 4 decades of programming. Whenever there was anything
like a REPL, I made heavy use of it - and I wasn't alone.

Of course, the concept of a REPL is far older than this. I wasn't even
born before the first REPL was created.

There's also the compiled/interpreted distinction to consider. This is
independant of a REPL. Some compilers and interpreters have had
divergent semantics over fundental semantics like scoping. E.g.
Cambridge Lisp, a Standard Lisp implementation, which was both
compiled and interpreted. The compiler used lexical scoping while the
interpreter used dynamic scoping. I found this to be a great
inconvenience. It was also anachronistic. (See Richard Gabriel's
comment(s) on it.)

Both Scheme and Common Lisp got this right. However, SRFI 39
parameters have replaced fluid variables. Both R6 and R7 reflect this
shift.

> But the semantics of
> (interaction-environment) certainly should and could be specified. If
> not, we should deprecate it as a formal part of the standard.

I suggest treating (scheme eval), (scheme load) and (scheme repl)
optional features. My understanding is that both WG1 and WG2 have done
this, even if the support for syntax-case implies some kind of
compile-time eval. I think that's another area that could be better
defined, specifically regarding the definition of the language used by
syntax-case, but perhaps that's a discussion for another thread.

Your points regarding max-width nicely illustrates the REPL issues,
but I have no comments to make there other than my appreciation.
Thanks.

[Marc Nieper-Wißkirchen]

Am Do., 17. Feb. 2022 um 12:38 Uhr schrieb Martin Rodgers <mcro...@gmail.com>:

> > But the semantics of
> > (interaction-environment) certainly should and could be specified. If
> > not, we should deprecate it as a formal part of the standard.
>
> I suggest treating (scheme eval), (scheme load) and (scheme repl)
> optional features. My understanding is that both WG1 and WG2 have done
> this, even if the support for syntax-case implies some kind of
> compile-time eval. I think that's another area that could be better
> defined, specifically regarding the definition of the language used by
> syntax-case, but perhaps that's a discussion for another thread.

At least for R7RS-small, all three libraries are optional features.

The criteria by which you selected these three libraries are not clear
to me, though. The libraries (scheme load) and (scheme repl) are
directly tied to the dynamic evaluation issues we have been discussing
here, so I understand why these. The library (scheme eval), however,
is unproblematic. The only problematic case is mutable environments,
but they are not exported by (scheme eval).

As you write, compile-time evaluation is needed anyway for syntax-case
(or any other procedural macro system). Fortunately, as far as
syntax-case is concerned and when some kind of phasing is adopted from
R6RS everything is well-defined. There is no question about the
language used by syntax-case (transformers). But, please, open another
thread to discuss this if you think my claim is not correct or
questionable so that we can discuss it and see to iron out what's
problematic.

[Martin Rodgers]

On 17/02/2022, Marc Nieper-Wißkirchen <marc....@gmail.com> wrote:

> At least for R7RS-small, all three libraries are optional features.
>
> The criteria by which you selected these three libraries are not clear
> to me, though. The libraries (scheme load) and (scheme repl) are
> directly tied to the dynamic evaluation issues we have been discussing
> here, so I understand why these. The library (scheme eval), however,
> is unproblematic. The only problematic case is mutable environments,
> but they are not exported by (scheme eval).

How else is the compiler going to evaluate the expressiong using
syntax-case? It implies some kind of development environment beyond
the compiler itself. I.e. somewhere for the compiled code to *run*.
I.e. dynamic generation of code at the compiler's runtime.

This has some small relevance to this discussion, as it includes macro
defintions. However, the presence of the REPL "solves" the syntax-case
issue in one important detail - it provides a way to dynamically
define and run new code.

I mentioned the distinction between compiled and interpreted code in
my earlier post. Cambridge Lisp / Standard Lisp conflated the
compile-time and runtime environments, while Common Lisp directed
addressed the distinction with eval-when. This issue is independant of
expand phases, which are specific to hygienic macro languages, like we
have in Scheme. Taylan Kammer's recent post describes the renaming of
a variable, which is a correct of environment splicing.

> As you write, compile-time evaluation is needed anyway for syntax-case
> (or any other procedural macro system). Fortunately, as far as
> syntax-case is concerned and when some kind of phasing is adopted from
> R6RS everything is well-defined. There is no question about the
> language used by syntax-case (transformers). But, please, open another
> thread to discuss this if you think my claim is not correct or
> questionable so that we can discuss it and see to iron out what's
> problematic.

I intend to. The R6 library language and the module language in Oscar
Waddell's Phd thesis address an issue that seems to me as-yet
unspecified in R7's library language.

Thanks.

[Marc Nieper-Wißkirchen]

Am Do., 17. Feb. 2022 um 13:13 Uhr schrieb Martin Rodgers <mcro...@gmail.com>:
>
> On 17/02/2022, Marc Nieper-Wißkirchen <marc....@gmail.com> wrote:
>
> > At least for R7RS-small, all three libraries are optional features.
> >
> > The criteria by which you selected these three libraries are not clear
> > to me, though. The libraries (scheme load) and (scheme repl) are
> > directly tied to the dynamic evaluation issues we have been discussing
> > here, so I understand why these. The library (scheme eval), however,
> > is unproblematic. The only problematic case is mutable environments,
> > but they are not exported by (scheme eval).
>
> How else is the compiler going to evaluate the expressiong using
> syntax-case? It implies some kind of development environment beyond
> the compiler itself. I.e. somewhere for the compiled code to *run*.
> I.e. dynamic generation of code at the compiler's runtime.

Dynamic evaluation of code is orthogonal to the question of dynamic or
static environments (e.g. the question of REPL or static library body
semantics).

The environment, in which syntax-case transformers are evaluated,
won't be mutable; only expressions but not definitions are evaluated
when the right-hand sides of syntax definitions are processed.

> This has some small relevance to this discussion, as it includes macro
> defintions. However, the presence of the REPL "solves" the syntax-case
> issue in one important detail - it provides a way to dynamically
> define and run new code.

Apart from that the REPL and the syntax-case expander can share a lot
of code of the respectively underlying eval mechanisms, I don't see a
close connection.

> I mentioned the distinction between compiled and interpreted code in
> my earlier post. Cambridge Lisp / Standard Lisp conflated the
> compile-time and runtime environments, while Common Lisp directed
> addressed the distinction with eval-when. This issue is independant of
> expand phases, which are specific to hygienic macro languages, like we
> have in Scheme. Taylan Kammer's recent post describes the renaming of
> a variable, which is a correct of environment splicing.

For Scheme, at least for R6RS Scheme (I can't add R7RS-large yet,
because these things haven't been fixed there yet), I think it is
better to talk about "visit-time" and "invoke-time" because the
run-time of one library can happen during the expand-time of another
library. A way to remove the need of something like eval-when are meta
definitions as described here ([1]). These build directly on the
phasing model of R6RS.

> > As you write, compile-time evaluation is needed anyway for syntax-case
> > (or any other procedural macro system). Fortunately, as far as
> > syntax-case is concerned and when some kind of phasing is adopted from
> > R6RS everything is well-defined. There is no question about the
> > language used by syntax-case (transformers). But, please, open another
> > thread to discuss this if you think my claim is not correct or
> > questionable so that we can discuss it and see to iron out what's
> > problematic.
>
> I intend to. The R6 library language and the module language in Oscar
> Waddell's Phd thesis address an issue that seems to me as-yet
> unspecified in R7's library language.

I agree; voting for a procedural record system in R7RS-large was just
a first step. Now we have to integrate it with the rest of the
language, especially with the module system. Fortunately, R7RS's
module system is basically the same as R6RS's module system so we
could decide just to copy over the relevant parts from chapter 7 of
the R6RS verbatim. However, I think it makes a lot of sense to discuss
explicit vs. implicit phasing first. The situation with R6RS is not
optimal. Neither the benefits of implicit vs. explicit phasing are
achieved because R6RS has to be compatible with both.

[1] https://www.unsyntax.org/unsyntax/features/syntax/meta/phasing/2020/10/22/meta-definitions.html

[Dr. Arne Babenhauserheide]

Martin Rodgers <mcro...@gmail.com> writes:

> For example, which REPL language does an implementation like Guile use
> as default? Guile 3.x has command line options for R6 and R7, but the
> default is R5. This is reasonable for Guile, given its age relative to
> all 3 reports. So its easy for all the conceptual baggage of Guile's
> R5 REPL to be inherited by the other REPLs.

> However, I'll be delighted to be proved wrong on this. Please prove me
> wrong.

Could you give an example that would show the problem you expect?

[John Cowan]

On Thu, Feb 17, 2022 at 1:51 AM Dr. Arne Babenhauserheide <arne...@web.de> wrote:

 

I still remember that when I started to learn programming, it confused
me to no end that I could not just write the use of a form before its
definition.

That's the writer's perspective: write things down in the order you think of them.  I began by reading code (first Fortran, then assembler) for about two years before I got any access to a computer at all.  I also wrote down some programs and desk-checked them (a nearly obsolete term, I suppose); I'm sure they had plenty of bugs, systematic and sporadic.  But anyway, that gave me the reader's perspective, in which programmers communicate with people every bit as much as with computers, which is why I am a Simpleton.

While some inconsistency is unavoidable (you cannot use what’s not yet
written), it would be nice if programs would at least behave the same in
the REPL and a file after the *whole* text has been pasted into the
REPL.

Note the discussion in R7RS 4.7. where it is made clear that a REPL need not be interactive: it can read its input from a file instead.  Indeed, the issue is not so much what interactive programming requires or doesn't, but about how the principle "you can't use a thing before it's defined" works out in different contexts.  More on this later.