require: namespace mismatch while dealing with syntax-local-value of a syntax property

[Alex Knauth]

Hello,

I'm getting an error that only happens when I split a definition and a use into two small files, def.rkt and use.rkt. If I put them into the same file, the error goes away.

The error happens when I call `syntax-local-value` on an identifier stored in a syntax property, while compiling use.rkt. The syntax property that got attached by the `define-thing` macro. I've seen this error message before, but it has usually meant that a syntax object says its bound to something in a module, but the module hasn't been instantiated yet. But both modules have been instantiated so what's going on?

def.rkt:40:14: require: namespace mismatch;
 reference to a module that is not available
  reference phase: 0
  referenced module: 'def
  referenced phase level: 0
  in: x
  compilation context...:
   /Users/Alex/Desktop/ns-mismatch/use.rkt

;; use.rkt

#lang racket

(require "def.rkt")

(use-thing y)

;; def.rkt

#lang racket

(provide y use-thing)

(require syntax/parse/define

         (for-syntax syntax/transformer))

(define-syntax-parser define-stuff

  [(_ name:id)

   #:with name* ((make-syntax-introducer) #'name)

   #'(begin

       (define name* 'stuff)

       (define-syntax name

         (make-variable-like-transformer

          (syntax-property #'name* 'prop #'name #t))))])

(define-syntax-parser define-thing

  [(_ name:id stuff:id)

   #:with stuff* (local-expand #'stuff 'expression '())

   #'(define-syntax name

       (make-variable-like-transformer

        (syntax-property #''thing

                         'prop

                         (syntax-property #'stuff* 'prop)

                         #t)))])

(define-syntax-parser use-thing

  [(_ name:id)

   #:with name* (local-expand #'name 'expression '())

   #:with stuff (syntax-property #'name* 'prop)

   (displayln "pre syntax-local-value")

   (define value (syntax-local-value #'stuff))  ; <-- error happens here

   (displayln "post syntax-local-value")

   #'(void)])

;; ---------------------

(define-stuff x)

(define-thing y x)

I've tried variations on compiling first vs. running without compiling, and I've tried putting syntax-local-introduce both before putting things in a syntax property and after pulling things out of a syntax property.

Alex Knauth

[Alexis King]

I am not sure what could be causing this problem, but it seems related
to something I ran into about a year ago[1], though I was getting
slightly different behavior. At the time, Matthew had the following to
say on the subject:

> On Oct 25, 2016, at 4:28 PM, Matthew Flatt <mfl...@cs.utah.edu> wrote:
>
> Putting identifiers in syntax properties causes them to be hidden from
> various scope and module-path-shifting operations. Probably, you're
> seeing the effect of hiding an identifier from path shifting (when an
> expanded module is compiled or when a compiled module is declared and
> instantiated in at a given module path and phase).

(For a little more discussion, see the linked thread.)

At the same time, I also opened racket/racket#1495[2], which includes a
somewhat similar program to yours, though again with a different error.
Still, it might be useful context (and I would bet they are related).

[1]: https://groups.google.com/d/msg/racket-users/TGax2h8dVxs/2N15Qy4zAQAJ
[2]: https://github.com/racket/racket/issues/1495

> On Oct 3, 2017, at 1:14 PM, Alex Knauth <alex...@knauth.org> wrote:
>
> Hello,
>
> I'm getting an error that only happens when I split a definition and
> a use into two small files, def.rkt and use.rkt. If I put them into
> the same file, the error goes away.
>
> The error happens when I call `syntax-local-value` on an identifier
> stored in a syntax property, while compiling use.rkt. The syntax
> property that got attached by the `define-thing` macro. I've seen this
> error message before, but it has usually meant that a syntax object
> says its bound to something in a module, but the module hasn't been
> instantiated yet. But both modules have been instantiated so what's
> going on?
>

[Alexis King]

I realized I probably also ought to post my workaround, which I use in
Hackett: I just don’t use preserved syntax properties at all (only
unpreserved ones). I found that sticking syntax objects in preserved
syntax properties was unreliable, probably for the reasons Matthew
mentioned in the aforementioned email thread.

Instead of using preserved syntax properties, I just read the property
earlier, then embed its value (or an expression that produces its value)
in the expanded syntax instead of reading/transferring it “just in
time”. This avoids the need for properties to preserved in bytecode.
What do I mean by this? Well, instead of expanding to something like
this:

#'(define-syntax x
(make-variable-like-transformer
(syntax-property #'x* 'prop (syntax-property #'y 'prop)))

...where 'prop is preserved on #'y, I instead expand to something like
this:

#`(define-syntax x
(make-variable-like-transformer
(syntax-property
#'x* 'prop
(quote-syntax #,(syntax-property #'y 'prop))))

This way, the syntax property’s value is actually embedded in the fully
expanded syntax as an expression, and the invocation of
make-variable-like-transformer receives a “hardcoded” value. This means
'prop no longer needs to be preserved on #'y.

I managed to fix your example program with this technique. This changes
the definitions of define-stuff and define-thing to the following:

(define-syntax-parser define-stuff
[(_ name:id)
#:with name* ((make-syntax-introducer) #'name)
#'(begin
(define name* 'stuff)
(define-syntax name
(make-variable-like-transformer

(syntax-property #'name* 'prop #'name))))])

(define-syntax-parser define-thing
[(_ name:id stuff:id)
#:with stuff* (local-expand #'stuff 'expression '())
#`(define-syntax name
(make-variable-like-transformer
(syntax-property
#''thing
'prop

(quote-syntax #,(syntax-property #'stuff* 'prop)))))])

Now, you might argue that this technique doesn’t apply in general, since
it explicitly uses quote-syntax, which wouldn’t work for other things
that can be put into preserved syntax properties. That’s true! In fact,
I use prefab structures as my types in Hackett, so I needed to provide
a replacement for preserved properties containing prefab structures.
Fortunately, it is trivial to produce an expression that evaluates to
and value that is a valid preservable syntax property. The function
Hackett uses looks like this:

(define preservable-property->expression
(match-lambda
[(and (app prefab-struct-key (? values prefab-key))
(app struct->vector (vector _ fields ...)))
#`(make-prefab-struct
'#,prefab-key
#,@(map preservable-property->expression fields))]
[(? list? lst)
#`(list #,@(map preservable-property->expression lst))]
[(cons a b)
#`(cons #,(preservable-property->expression a)
#,(preservable-property->expression b))]
[(? syntax? stx)
#`(quote-syntax #,stx)]
[(and (or (? boolean?) (? symbol?) (? number?) (? char?)
(? string?) (? bytes?) (? regexp?))
datum)
#`(quote #,datum)]
[other
(error 'preservable-property->expression
"non-preservable syntax property\n value: ~e"
other)]))

To use this function, just replace the hardcoded use of quote-syntax to
an evaluated use of preservable-property->expression. Using this
function, define-thing looks like this:

(define-syntax-parser define-thing
[(_ name:id stuff:id)
#:with stuff* (local-expand #'stuff 'expression '())
#`(define-syntax name
(make-variable-like-transformer
(syntax-property
#''thing
'prop

#,(preservable-property->expression
(syntax-property #'stuff* 'prop)))))])

This wouldn’t work for any use of preservable syntax properties, since
it relies on the particular interplay of make-variable-like-transformer
and local-expand that Turnstile uses. However, I think it will work for
your case.

Alexis

[Alex Knauth]

Oh, thanks for the advice!

I don't have time to look at it now but later this evening or tomorrow I'll try to apply this.

Alex Knauth