Precise definition of er-macro-transformer

[Marc Nieper-Wißkirchen]

I have found two diverging descriptions of the er-macro-transformer facility:

MIT/GNU Scheme says:

 special form: er-macro-transformer expression

The expression is expanded in the syntactic environment of the er-macro-transformer expression, and the expanded expression is evaluated in the transformer environment to yield a macro transformer as described below. This macro transformer is bound to a macro keyword by the special form in which the transformer expression appears (for example, let-syntax).

Chicken says:

 

 The low-level macro facility that CHICKEN provides is called "explicit renaming" and allows writing hygienic or non-hygienic macros procedurally. When given a the return value of the one of the procedures er-macro-transformer or ir-macro-transformer instead of a syntax-rules form, define-syntaxevaluates the procedure in a distinct expansion environment (initially having access to the exported identifiers of the scheme module). The procedure takes an expression and two other arguments and returns a transformed expression.

Both descriptions seem to differ. Given ‘(er-macro-transformer expression)’, MIT/GNU Scheme expands the expression in the environment of the macro definition and then evaluates it in the transformer environment. I guess, transformer environment can only mean the syntactic environment of the er-macro-transformer expression: I don't see that it makes sense to expand an expression in one environment and evaluate it afterwards in another environment. Furthermore, MIT/GNU Scheme uses the same terminology when describing sc-transformers, and there it is later used as a synonym to the environment in which the transformer expression occurs.

On the other hand, Chicken's description seems to imply that the expression is expanded and evaluated in a special environment disjoint from the transformer environment itself.

Both descriptions are hardly compatible. Chibi seems to follow MIT/GNU Scheme's description. Is this on purpose and is this what is proposed for R7RS-large?

[Alex Shinn]

Chicken has a separate environment for macro expansion,
and modules are loaded into this with `import-for-syntax'.
In the r7rs egg, I believe everything imported for the runtime
also gets imported into the compilation environment so things
just work.

In Racket, on the other hand, you have a tower of environments
for different phases.

The trick to specifying an ER SRFI would not so much be in
specifying ER itself, but in specifying the semantics of different
phases in a way to make all implementations happy.

Are you interested? :)

--
Alex

> --
> You received this message because you are subscribed to the Google Groups
> "chibi-scheme" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to chibi-scheme...@googlegroups.com.
> To post to this group, send email to chibi-...@googlegroups.com.
> Visit this group at https://groups.google.com/group/chibi-scheme.
> For more options, visit https://groups.google.com/d/optout.

[Marc Nieper-Wißkirchen]

Am Montag, 11. April 2016 06:11:08 UTC+2 schrieb Alex Shinn:

Chicken has a separate environment for macro expansion,
and modules are loaded into this with `import-for-syntax'.
In the r7rs egg, I believe everything imported for the runtime
also gets imported into the compilation environment so things
just work.

I installed Chicken and tried the following program:

(define (kadr expression)

  (cadr expression))

(define-syntax foo

  (er-macro-transformer

   (lambda (expression rename compare)

     (kadr expression))))

(foo (display "Hello, World!"))

(newline)

It fails with "Error: during expansion of (foo ...) - unbound variable: kadr". The expansion process of the lambda form also does not use the runtime environment:

(define-syntax bar

  (syntax-rules ()

    ((bar . args)

     (lambda . args))))

(define-syntax foo

  (er-macro-transformer

   (bar (expression rename compare)

     (cadr expression))))

(foo (display "Hello, World!"))

(newline)

fails likewise.

The trick to specifying an ER SRFI would not so much be in
specifying ER itself, but in specifying the semantics of different
phases in a way to make all implementations happy.

Are you interested? :)

If only I knew what the right specification should be. One solution that could probably made portable is the restriction to er-macro-transformers of the form

(import eval)

...

  (er-macro-transformer

    (eval '(lambda (expr rename compare) ...) (environment '(scheme base) ...)))

But this is not very pretty. One could, however, add syntactic sugar so that this construct can be expressed as:

  (er-transformer ((scheme base) ...) (lambda (expr rename compare) ...))

--

Marc

[Alex Shinn]

On Mon, Apr 11, 2016 at 5:14 PM, Marc Nieper-Wißkirchen
<marc....@gmail.com> wrote:
>
> I installed Chicken and tried the following program:
>
> (define (kadr expression)
> (cadr expression))
>
> (define-syntax foo
> (er-macro-transformer
> (lambda (expression rename compare)
> (kadr expression))))
>
> (foo (display "Hello, World!"))
> (newline)
>
>
> It fails with "Error: during expansion of (foo ...) - unbound variable:
> kadr".

Chicken core is not R7RS, you need to (use r7rs).

Even then, I don't think this will work. Chicken only aims
for r7rs-small compatibility, so only needs to support
syntax-rules. You can maybe hack around this with

(eval-when (compile runtime)
(define (kadr expression) ...))

--
Alex

[Marc Nieper-Wißkirchen]

Chicken core is not R7RS, you need to (use r7rs).

Even then, I don't think this will work.  Chicken only aims
for r7rs-small compatibility, so only needs to support

Yes, (use r7rs) makes no difference.

 

syntax-rules.  You can maybe hack around this with

  (eval-when (compile runtime)
    (define (kadr expression) ...))

This works (without ‘runtime’). Still, Chicken's implementation remains different in behaviour to Chibi's. Of course, by saying that Chicken won't want to support R7RS-large, we can forget about Chicken's semantics and target Chibi's in a future specification for er-macro-transformer.

That said, I am wondering, however, whether Chibi's semantics is really what we should aim for:

1) It probably makes implementing R7RS-large much harder on systems that are less dynamic than Chibi.

2) What shall happen if the macro transformer modifies variables outside of the transformer. Shall the modifications be reflected in the runtime environment or shall we have two instances of the environment? Or do we want to leave that open? (And thus limiting portable code.)

3) From a mathematical point of view, there does not seem to be a reason why the environment in which a macro transformer is being defined, should be the same as the one in which the transformer is executed. Why should we then make this behaviour the default?

 

[Marc Nieper-Wißkirchen]

In trying to understand how one could specify good semantics of er-macro-transformer, I ran across the following bug in Chibi:

(let loop ((x 0))

  (when (< x 10)

    (let ()

      (define-syntax foo

(er-macro-transformer

(lambda (expression rename compare)

  `(,(rename 'begin)

    (,(rename 'display) ,x)

    (,(rename 'newline))))))

      (foo)

      (loop (+ x 1)))))

gives a core dump.

[Alex Shinn]

Yep, this is a known limitation
(https://github.com/ashinn/chibi-scheme/issues/259).

Contrary to what you've been suggesting, chibi doesn't mix runtime and
compile time bindings well :)

--
Alex

[Alex Shinn]

More specifically, you're trying to unquote a binding during the macro
expansion phase for a variable `x' which won't even exist until runtime.

MIT-scheme will tell you 2 is unbound here. To make this a more
friendly error in chibi I'd need to add annotations to lambdas to
indicate what phase they are being processed in.

--
Alex

[Marc Nieper-Wißkirchen]

Am Montag, 11. April 2016 15:41:13 UTC+2 schrieb Alex Shinn:

Yep, this is a known limitation
(https://github.com/ashinn/chibi-scheme/issues/259).

Contrary to what you've been suggesting, chibi doesn't mix runtime and
compile time bindings well :)

I see. Global bindings seem to work, though, as in:

(define x (read-line))

(define-syntax foo

  (er-macro-transformer

   (lambda (expr rename compare)

     x)))

(display (foo))

(newline)

After I type in a string on the console, it is echoed. However, a system where compile time and run time are not interwoven chronologically would have problems to mimic Chibi's behaviour. Wouldn't this make

er-macro-transformers non-portable?

[Alex Shinn]

On Mon, Apr 11, 2016 at 11:06 PM, Marc Nieper-Wißkirchen
<marc....@gmail.com> wrote:
>
> After I type in a string on the console, it is echoed. However, a system
> where compile time and run time are not interwoven chronologically would
> have problems to mimic Chibi's behaviour. Wouldn't this make
> er-macro-transformers non-portable?

Wrap the program in (let lp () ...) and it breaks again.

I could make this implicit so your example wouldn't work, but
it's slightly more convenient in some cases to mimic the repl
for top-level programs. The repl by definition is supposed to
allow mixing runtime and compile time to a certain extent.

Either way, this hardly makes ER macros more nor less portable.
Every macro system has its own extensions, and basically no two
implementations have identical macro systems. All of R7RS-small
is built around "is an error" semantics, to explicitly allow extensions.
This is one of the beauties of Scheme, that it allows agreement on
a minimal core while encouraging experimentation.

Let's agree on that core before even discussing forbidding variety.

--
Alex

[Marc Nieper-Wißkirchen]

Either way, this hardly makes ER macros more nor less portable.
Every macro system has its own extensions, and basically no two
implementations have identical macro systems.  All of R7RS-small
is built around "is an error" semantics, to explicitly allow extensions.
This is one of the beauties of Scheme, that it allows agreement on
a minimal core while encouraging experimentation.

I completely agree with you.

 

Let's agree on that core before even discussing forbidding variety.

My problem is that I haven't understood that core yet: Apparently, referencing global special forms (like ‘lambda’) in the macro transformer works well and is supposed to work. Likewise, referencing some global values (like the ‘cadr’ procedure) works well and is also supposed to work.

On the other hand, referencing other global values (like the result of the line input) is fragile.

So could a common core be based on the following?

"The argument to er-macro-transformer is expanded and evaluated in the environment defined by the import sets of the import declarations of the current module. It is an error if any of the identifiers referenced while expanding and evaluating the transformer is shadowed by local bindings in the syntactic environment of the transformer."

 

--
Alex

[Marc Nieper-Wißkirchen]

So could a common core be based on the following?

"The argument to er-macro-transformer is expanded and evaluated in the environment defined by the import sets of the import declarations of the current module. It is an error if any of the identifiers referenced while expanding and evaluating the transformer is shadowed by local bindings in the syntactic environment of the transformer."

One should probably add the following:

"The behaviour is unspecified if the transformer references bindings in its syntactic environment that haven't been imported into the current module and don't shadow them either."

(This is important when trying to match literals in syntax-rules macros used in the transformer itself. If such a literal is not being imported but bound locally, implementations may want to differ.)

 

 

 

 

--
Alex

[Alex Shinn]

Implementations may differ in phasing distinctions, but
outside of that hygiene rules are pretty clear. If an identifier
is bound locally it should match hygienically.

I think you were taking issue with the current chibi syntax-rules
definition, which for the ellipsis-override case expands to

(let ((ellipsis #t))
(er-macro-transformer
(lambda (expr rename compare) ...)))

where your expander requires syntax definitions to be
bare er-macro-transformers not wrapped in a let, even
if the binding is never used for anything other than matching.
But this could just as easily be expanded instead to
something like:

(er-macro-transformer
(lambda (expr rename compare)

`(let ((ellipsis #t))
(let-syntax ((,(rename 'tmp) ...))
(,(rename 'tmp) ,expr ,rename ,compare))))

where there is no longer any syntax binding wrapped in a let.

If you are interested in an R7RS-large ER SRFI, you should
probably start with the R6RS description of phasing. Also
read Ghuloum's paper on implicit phasing:

http://www.cs.indiana.edu/~dyb/pubs/implicit-phasing.pdf

--
Alex