Macro behaves differently when run from different modules

[Jonathan Simpson]

I've recently been rewriting some ugly code to make it fully hygienic with syntax-parse. I have an implementation that appears to work when testing it from the module that contains the macros, but it fails when I run it from my main program/another module.

The macro in question is parse-level0, defined in expander-utils.rkt:

----

(begin-for-syntax 

  (define-syntax-class mag-lvl

    (pattern ({~datum level})))

  

  (define-syntax-class mag-line

    (pattern ({~literal line} expr ...))))

(define-syntax (parse-level0 stx)

  (printf "parse-level0 input: ")

  (display stx) (printf "~n")

  (syntax-parse stx

    [(_ ln:mag-line) #'ln]

    [(_ ln:mag-line (~seq lvl:mag-lvl ...+ ln2:mag-line) ...+)

     #:with branch-lines #'((~@ lvl ... ln2) ...)

     (printf "parse-level0 1~n")

     #`(when* ln (level 

            #,@(parse-level1 #'branch-lines)))]

    [(_ lvl:mag-lvl ...+ expr ...) 

     #'(error "no line at level 0")]

    [_

     (printf "parse-level0 error on input: ")

     (display stx) (printf "~n") 

     #'(error "syntax error at level 0")]))

----

This macro is provided to expander.rkt. It works when run from the REPL in expander-utils.rkt, but not when run from expander.rkt. Here's an example with some debug info printed:

expander-utils.rkt> (parse-level0 (line 1))

parse-level0 input: #<syntax:stdin::10915 (parse-level0 (line 1))>

'(1)

expander-utils.rkt> 

expander.rkt> (parse-level0 (line 1))

parse-level0 input: #<syntax:stdin::11004 (parse-level0 (line 1))>

parse-level0 error on input: #<syntax:stdin::11004 (parse-level0 (line 1))>

; syntax error at level 0

; Context:

;  /usr/local/racket/collects/racket/repl.rkt:11:26

The syntax objects there are printed with 'display' and are identical as far as I can tell, but I wonder if it could be related to interpreting line or level as a datum or literal as defined in the syntax classes I created. In this example I'm not using level so I guess that it wouldn't matter, but line is defined as a macro in expander.rkt. But it still works in expander-utils.rkt if I define the line macro there.

Anyone have an idea what might be happening here?

Full code available at https://github.com/jjsimpso/magic.

Thanks,

Jonathan

[Jonathan Simpson]

It works when I change my mag-line syntax class to:

(define-syntax-class mag-line

    (pattern (line expr ...)))

So removing the literal specifier on line seems to work. I'm still not sure why the two modules behave differently though. It seemed to work in expander-utils.rkt whether line was defined or not. So there must be something in the compilation environment that I'm missing. I'm still curious if anyone has a good explanation. I may hit issues with level further down the road as well.

-- Jonathan

[Ben Greenman]

On 9/28/19, Jonathan Simpson <jjsi...@gmail.com> wrote:
> It works when I change my mag-line syntax class to:
>
> (define-syntax-class mag-line
> (pattern (line expr ...)))
>
> So removing the literal specifier on line seems to work. I'm still not sure
> why the two modules behave differently though. It seemed to work in
> expander-utils.rkt whether line was defined or not. So there must be
> something in the compilation environment that I'm missing. I'm still
> curious if anyone has a good explanation. I may hit issues with level
> further down the road as well.

(~literal X) matches an identifier that is free-identifier=? to X
If X is defined in one module but not another, you'll see different results.

That new pattern (line expr ....) might match more things than you want it to.

[Jonathan Simpson]

In expander-utils.rkt the macros work if the line pattern is a datum or literal. In expander.rkt it only works if the line pattern is a datum. So I changed line to only match a datum. From my understanding, I think matching line as a datum is correct, so I'm happy with what I have now. I just don't understand why line also matches as a literal in expander-utils.rkt.

Thanks for your help.

-- Jonathan

[Michael Ballantyne]

Two identifiers that are both unbound and have the same symbol are considered free-identifier=?. Thus the match works when testing from expander-utils.rkt, where both the pattern and usage are unbound. It also works from expander-utils.rkt when you define `line` in that file, because the pattern and usage identifiers both share that binding. But when you define line in expander.rkt, the pattern identifier in expander-utils.rkt is unbound, but the usage identifier in expander.rkt is bound by the `line` definition, so they aren't free-identifier=?.

[Jonathan Simpson]

Ah, I get it now. Thanks for the explanation!

-- Jonathan