Expansion time in R7RS small and outermost level syntax redefinitions

[Daphne Preston-Kendal]

R7RS small generally (deliberately, as I understand it) leaves completely unspecified when and how often the expansion of macros actually takes place.

This raises a question about the interpretation of the following text in section 5.4:

> If the define-syntax occurs at the outermost level, then the global syntactic environment is extended by binding the ⟨keyword⟩ to the specified transformer, but previous expansions of any global binding for ⟨keyword⟩ remain unchanged.

From this, Marc Nieper-Wißkirchen reckoned that the R6RS expansion order cannot legally be applied to R7RS small programs and reported a bug in Larceny for apparently incorrectly doing this.
<https://github.com/larcenists/larceny/issues/825>

(import (scheme base)
(scheme write))

(define-syntax foo (syntax-rules () ((foo) 1)))
(define x (foo))
(define-syntax foo (syntax-rules () ((foo) 2)))
(display x)
(newline)

Under R6RS, this program is unquestionably invalid (redefinitions within the same context are not allowed); if it were valid, it would presumably produce 2 (as in Larceny) because the syntax definitions are required to be expanded and inserted into the environment from left to right before all of the right-hand sides of definitions are expanded.

Marc interprets that sentence section 5.4 as requiring an R7RS small implementation to produce 1.

Marc’s interpretation, however, assumes that the (foo) in the definition of x must be expanded as soon as the definition appears. This seems to conflict with the philosophy that the time and order of syntax expansion is not prescribed by the R7RS.

I think one can somewhat reasonably interpret R7RS as allowing what Larceny does, simply because it does not define when the ‘previous expansions’ are supposed to have taken place.

More questionable is the following variant:

(define-syntax foo (syntax-rules () ((foo) 1)))
(define x (foo))
(display x)
(newline)
(define-syntax foo (syntax-rules () ((foo) 2)))

which would also produce 2 under the Larceny-style interpretation. Since the implementation has to expand foo before it’s redefined in order to display the value of x. But again, since the order of expansion is otherwise undefined by R7RS small, maybe an implementation is allowed to apply this variant of the R6RS expansion order. (You don’t even need to go full R6RS here – any kind of optimization pass over a program body might lead to this kind of result.)

Consider the following variant, for example:

(define-syntax foo (syntax-rules () ((foo) 1)))
(define x (lambda () (foo)))
(define-syntax foo (syntax-rules () ((foo) 2)))
(display (x))
(newline)

Here the unspecifiedness is a lot clearer. Some implementations may expand macros in a procedure body only when the procedure is called; others may expand them as soon as the procedure is defined. To me it seems unspecified by R7RS small whether this program would print 1 or 2. So a program cannot depend on which expansions are ‘previous’ in some cases already.

(FWIW I think it was a mistake for R7RS to allow redefinitions within the same context at all – it should have been ‘an error’/undefined behaviour except in the REPL and in ‘REPL scripts’ (‘An implementation may provide a mode of operation in which the REPL reads its input from a file’). The utility of top-level redefinitions in well-written programs seems doubtful, even setting aside the interpretational problem and inter-report-version differences.)

Could some WG1 members comment on the problematic sentence in section 5.4 and how it is intended to be interpreted? Which of the variants listed in this email have a clear specified value and which not?


Daphne

[Alex Shinn]

This change was made as part of the decision to specify the (then de facto standard) semantics of redefinitions in the REPL:

https://small.r7rs.org/ticket/112/

though at a quick glance I can't find the discussions around that, and I don't have a clear recollection whether we considered making the REPL semantics different from non-REPL programs and libraries.

Note in R5RS, which has no modules, it's a common idiom to define top-level variables with a dummy value and set! them later, so I think it's reasonable that the same be possible with syntax.

While the _timing_ of macro expansions is deliberately unspecified, the order has restrictions.  The relevant clause here is:

  [...] it is an error for a definition to define an identifier whose binding has to be known in order to determine the meaning of the definition itself, or of any preceding definition that belongs to the same group of internal definitions.

which implies a left to right expansion order.  However, you can probably argue that in:

  (define x (foo))

the body of the definition, (foo), can't affect subsequent expansions and is therefore safe to delay.

Given that I believe Larceny's behavior is allowed in the small language, but maybe not in the presence of low-level macros where expansions can have side effects.

But I think the strict eager expansion is more in the spirit of section 5.4 since it results in the same behavior in a program and the REPL.

--

Alex

--
You received this message because you are subscribed to the Google Groups "scheme-reports-wg1" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scheme-reports-...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/scheme-reports-wg1/37A1571D-9186-41EE-9A47-3442DE94D122%40nonceword.org.

[Arthur A. Gleckler]

Program semantics and REPL semantics should be as close to the same as is practical.  It's important to be able to interact with and debug programs at the REPL.  Making the meaning of programs and the REPL diverge makes that harder, and feels contrary to the original spirit of the language.