Multiple stage compilation in a tower of languages

[Thomas Del Vecchio]

Hi all,

I'm working on a family of languages in Racket, and I'm running into a conceptual issue with the information flow of the project. I have a standard core language whose syntax is s-expressions, with some macros in the expander. Then, there is a surface language (compiled into s-expressions with brag) which directly translates into the core language. There will also be other surface languages in the future, forming a tower/tree of languages that compile into simpler ones, so I want to make sure I get this pattern right.

Conceptually, I would want the surface language to fully "compile" into the core language before the core language tries to do anything. As I develop the surface language, I don't think I should have to concern myself with the inner workings of the core language; I should be able to just produce core language code, and then have the core language handle it from there. I find myself running into an issue, however, where the core language starts to do computations before the translation is fully complete. This is an issue because the macros of the core language are taking in the syntax of the surface language which hasn't been translated yet, and so the core language doesn't recognize this syntax and breaks. 

As a small example, consider the following:

core-language-expander.rkt

(define-syntax (A-core stx)

  (syntax-parse stx #:datum-literals (b-core)

    [(A-core (b-core x)) #'(printf "A-core got value ~a.~n" x)]))

surface-language-expander.rkt

(define-syntax-rule (b-surface x) (b-core x))

(define-syntax (A-surface stx)

  (syntax-parse stx

    [(A-surface expr) #'(A-core expr)]))

Here, if we call `(A-surface (b-surface "hello"))`, we would want it to take the `A-surface`, and convert it into `A-core`, and then expand `(b-surface "hello")`. Since `A-core` is implemented as a macro, though, it takes `(b-surface "hello")` unexpanded, and so tries to evaluate `(A-core (b-surface "hello"))`, which fails.

Since `b-core` isn't defined as a function/macro in the core language (it is just a piece of syntax used in `A-core`), we can't just evaluate `(b-surface "hello")` as a value before passing it in. If we try using `local-expand` or something similar to handle the expansion as syntax, there isn't an obvious way to make it only expand macros defined in the surface language expander (and not those defined in the core language expander), which turns out to be an issue in my case.

Do any of you recommend a resource for understanding the strategy of taking a stack of languages and compiling down one layer at a time? It seems like the notion of a tower of languages would be fairly natural, yet the only success I have had is with some hacks with recursive local-expand that feel extremely unnatural, and also require further hacks to get around things like syntax-tainting.

Thanks so much!

Thomas Del Vecchio

[Sorawee Porncharoenwase]

local-expand supports the stop-list argument, so I think you can put all core forms there.

#lang racket

(require syntax/parse/define)

(define-syntax (A-core stx)
  (syntax-parse stx #:literals (b-core)
    [(A-core (b-core x)) #'(printf "A-core got value ~a.~n" x)]))

(define-syntax (b-core stx) (raise-syntax-error #f "b-core out of context"))

(define-simple-macro (b-surface x) (b-core x))

(define-syntax (A-surface stx)
  (syntax-parse stx
    [(A-surface expr)
     #:with sub-stx (local-expand #'expr 'expression (list #'b-core))
     #`(A-core sub-stx)]))

(A-surface (b-surface "hello"))

Note that I switch define-syntax-rule to define-simple-macro because define-syntax-rule will use syntax-protect, resulting in errors like cannot use identifier tainted by macro transformation.

--
You received this message because you are subscribed to the Google Groups "Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to racket-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/racket-users/26f44e6d-f1f8-465e-9639-6cd6afeffa31n%40googlegroups.com.

[Thomas Del Vecchio]

Thanks so much for your suggestion! Switching over to `define-simple-macro` made the code smell a lot less, and with a reframing of how I approached using macros, this has led to a much neater and cleaner expander that doesn't require recursive `local-expand`ing. I've spent the last day implementing this change, and it's looking good!