"cannot reference undefined identifier" when using eval in Pollen

[Joel Dueck]

Wondering if MB/anyone can help me understand this problem I'm having. I got pretty far but I think I've hit a wall.

A bit of setup: When dealing with multiple output formats you write a lot of tag functions of the form:

; In pollen.rkt

(define (bold elems)

  (case (current-poly-target)

    [(pdf) (pdf output here)

    [else (html output here)]))

Instead of doing all this, I'd like to automatically split the implementation for each target format into separate functions, defined in separate files. (There are a couple reasons for this but making dependencies clearer for makefile purposes is a big one.) Thus:

; the new pollen.rkt

(require "tags-pdf.rkt")   ; defines function (pdf-bold attrs elems)

(require "tags-html.rkt")  ; defines function (html-bold attrs elems)

(require "polytag.rkt")    ; defines poly-branch-tag macro

 

(provide (all-from-out "tags-html.rkt"))

(provide (all-from-out "tags-pdf.rkt")) 

 

(poly-branch-tag bold)

So to that end I wrote the poly-branch-tag macro, a simplified version of which is here:

(define-for-syntax site-poly-targets '(pdf html))

(define-syntax (poly-branch-tag stx)

  (syntax-parse stx

    [(_ TAG:id)

     (with-syntax ([POLY-FUNCS (for/list ([target (in-list site-poly-targets)])

                                         (list target (format-id stx "~a-~a" target #'TAG)))])

       #'(define-tag-function (TAG attributes elems)

           (define poly-func (second (assq (current-poly-target) 'POLY-FUNCS)))

           ((eval poly-func) attributes elems)))]))

I started by using the (case) branching, but I wanted a way to be able to add output formats in the future without fiddling with the macro every time.

All of this seemed to be working fine in my single-file DrRacket test, but when I try this in an actual Pollen project I get an error (see end of this post). It complains that html-bold is undefined, but it is very clearly defined in tags-html.rkt (which also contains a (provide (all-defined-out)) directive). Indeed, I can change test.poly.pm to call html-bold directly and that works. So calling html-bold directly works, but a macro that evaluates to ((eval 'html-bold) args) doesn't. 

I feel I must be missing some nuance of the scope in which Pollen files work. E.g., I can run the following in DrRacket:

(define (html-bold attrs elems)

  `(strong ,@elems))

((eval 'html-bold) '() '("ahoy"))

But the following in test.poly.pm gives me the same error as I get when using my macro:

#lang pollen

 

◊(define (html-bold elems) `(strong ,@title))

◊((eval 'html-bold) '() '("ahoy"))

raco pollen render -t html posts/test.poly.pm

rendering posts/test.poly.pm

rendering: posts/test.poly.pm as test.html

rendering: /template.html.p as /template.html

html-bold: undefined;

 cannot reference undefined identifier

  context...:

   ...n/pollen/tag.rkt:79:10

   (submod /Users/joel/Documents/code/thenotepad/posts/test.poly.pm g1083 inner): [running body]

   (submod /Users/joel/Documents/code/thenotepad/post/test.poly.pm g1083): [traversing imports]

   /Users/joel/Documents/code/thenotepad/posts/test.poly.pm: [traversing imports]

   /Users/joel/Library/Racket/6.4/pkgs/pollen/pollen/private/cache-utils.rkt:33:0: path->hash

   /Users/joel/Library/Racket/6.4/pkgs/pollen/pollen/private/cache-utils.rkt:76:14: temp16

   /Applications/Racket v6.4/collects/file/cache.rkt:63:2: fetch-and-continue

   /Applications/Racket v6.4/collects/racket/contract/private/arrow-val-first.rkt:335:3

   /Users/joel/Library/Racket/6.4/pkgs/pollen/pollen/private/cache-utils.rkt:67:0: cache-ref!

   /Applications/Racket v6.4/collects/racket/private/more-scheme.rkt:373:13: hash-ref!

   /Users/joel/Library/Racket/6.4/pkgs/pollen/pollen/cache.rkt:24:4: cached-require-base

   /Users/joel/Library/Racket/6.4/pkgs/pollen/pollen/render.rkt:169:0: render-markup-or-markdown-source33

   render26

   render-from-source-or-output-path

   loop

   (submod /Users/joel/Library/Racket/6.4/pkgs/pollen/pollen/private/command.rkt raco): [running body]

[Matthew Butterick]

On Jul 19, 2016, at 9:42 PM, Joel Dueck <dueck...@gmail.com> wrote:

So to that end I wrote the poly-branch-tag macro, a simplified version of which is here:

(define-for-syntax site-poly-targets '(pdf html))

(define-syntax (poly-branch-tag stx)

  (syntax-parse stx

    [(_ TAG:id)

     (with-syntax ([POLY-FUNCS (for/list ([target (in-list site-poly-targets)])

                                         (list target (format-id stx "~a-~a" target #'TAG)))])

       #'(define-tag-function (TAG attributes elems)

           (define poly-func (second (assq (current-poly-target) 'POLY-FUNCS)))

           ((eval poly-func) attributes elems)))]))

The problem is with 'POLY-FUNCS in the macro template. When you put the quote mark on the front, then everything inside POLY-FUNCS gets treated as a symbol. But in fact, you only want the target name to behave as a symbol, and you want your new function-name identifier to behave as a variable (that refers to your imported function). 

The solution is to match the target and function separately in your `with-syntax` pattern, and then only put the quote mark on the first one (once this problem is fixed, notice that the `eval` becomes unnecessary; notice also the use of the `...` matching operator so we can handle the new syntax bits in the template one at a time, rather than all at once):

;;;;

#lang racket

(require rackunit (for-syntax syntax/parse racket/syntax) pollen/tag pollen/setup)

(define (html-bold . xs) "this is html-bold")

(define (pdf-bold . xs) "this is pdf-bold")

(define-for-syntax site-poly-targets '(pdf html))

(define-syntax (poly-branch-tag stx)

  (syntax-parse stx

    [(_ TAG:id)

     (with-syntax ([((POLY-TARGET POLY-FUNC) ...) (for/list ([target (in-list site-poly-targets)])

                                                            (list target (format-id stx "~a-~a" target #'TAG)))])

       #'(define-tag-function (TAG attributes elems)

           (define poly-func (second (assq (current-poly-target) (list (list 'POLY-TARGET POLY-FUNC) ...))))

           (poly-func attributes elems)))]))

(poly-branch-tag bold)

(check-equal? (bold "hi")

              "this is html-bold")

(check-equal? (parameterize ([current-poly-target 'pdf])

                (bold "hi"))

              "this is pdf-bold")

;;;;

Alternatively, you could write the `poly-branch-tag` macro using a `case` statement. No leading quote necessary for POLY-TARGET here, because `case` auto-quotes whatever is on the left side of each branch. (Notice again the use of `...` to repeat the branch with each target / function pair.)

;;;;

(define-syntax (poly-branch-tag stx)

  (syntax-parse stx

    [(_ TAG:id)

     (with-syntax ([((POLY-TARGET POLY-FUNC) ...) (for/list ([target (in-list site-poly-targets)])

                                                         (list target (format-id stx "~a-~a" target #'TAG)))])

       #'(define (TAG . xs)

           (case (current-poly-target)

             [(POLY-TARGET) (apply POLY-FUNC xs)] ...)))]))

;;;

--
You received this message because you are subscribed to the Google Groups "Pollen" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pollenpub+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Joel Dueck]

On Wednesday, July 20, 2016 at 1:56:02 AM UTC-5, Matthew Butterick wrote:

The problem is with 'POLY-FUNCS in the macro template. When you put the quote mark on the front, then everything inside POLY-FUNCS gets treated as a symbol. But in fact, you only want the target name to behave as a symbol, and you want your new function-name identifier to behave as a variable (that refers to your imported function). 

The solution is to match the target and function separately in your `with-syntax` pattern, and then only put the quote mark on the first one (once this problem is fixed, notice that the `eval` becomes unnecessary; notice also the use of the `...` matching operator so we can handle the new syntax bits in the template one at a time, rather than all at once):

This is a big help for my understanding of macros, thanks. I felt that eval was a bit of a hack, but I had not understood that with-syntax could use `...` to produce repeating patterns that aren't tied to the syntax pattern passed in by `syntax-parse`. I guess I might have been able to infer this from Hendershott's explanation that `with-syntax` can be considered as simply a rearranged `syntax-case`.

I'm still curious as to why one can't `eval` a symbol matching the name of a function in a Pollen file though, even when that function is defined in the same file. I will probably never absolutely need to use it that way, but it runs counter to my understanding of Pollen files as full-fledged Racket programs.

[Matthew Butterick]

On Jul 20, 2016, at 8:37 AM, Joel Dueck <dueck...@gmail.com> wrote:

This is a big help for my understanding of macros, thanks. I felt that eval was a bit of a hack, but I had not understood that with-syntax could use `...` to produce repeating patterns that aren't tied to the syntax pattern passed in by `syntax-parse`. I guess I might have been able to infer this from Hendershott's explanation that `with-syntax` can be considered as simply a rearranged `syntax-case`.

`syntax-case`, `with-syntax`, and `syntax-parse` all rely on syntax patterns [1] to destructure syntax objects (though `syntax-parse` supports some extra options that the others do not [2]).

You can think of syntax patterns as a mini-language for parsing syntax, similar to how regular expressions are a mini-language for parsing strings. And, like regexps, you often have more than one option for how you break down a syntax object.

In the case of a list-shaped syntax object like `(foo bar baz)` you can:

+ match the whole thing with a single-term pattern like `THE-WHOLE-LIST` 

+ match each element with a list pattern like `(FIRST-ELEMENT SECOND-ELEMENT THIRD-ELEMENT)`

+ match each element with an ellipsis pattern like `(LIST-ELEMENT ...)` or `(FIRST-ELEMENT LIST-ELEMENT ...)`

+ match the head and tail of the list with a dot pattern like `(FIRST-ELEMENT . OTHER-ELEMENTS)`

Getting the hang of the ellipsis is the most important skill, I think, because it's a very clever operator that lets you write tight syntax templates.

[1] http://docs.racket-lang.org/reference/stx-patterns.html?q=syntax%20pattern

[2] http://docs.racket-lang.org/syntax/stxparse-patterns.html?q=syntax%20pattern

I'm still curious as to why one can't `eval` a symbol matching the name of a function in a Pollen file though, even when that function is defined in the same file. I will probably never absolutely need to use it that way, but it runs counter to my understanding of Pollen files as full-fledged Racket programs

When you use `eval` in a program (as opposed to in the REPL, which allows more casual `eval`ing), you have to specify a namespace for resolving identifiers. That's why this code will fail with a "bold: unbound identifier" error:

#lang racket

(require rackunit)

(define (bold . xs) "this is bold")

(check-equal? ((eval 'bold) "hi") "this is bold")

If you just want to use the surrounding namespace, you can use a namespace anchor to capture the namespace and pass it to `eval`. That's why this code works:

#lang racket

(require rackunit)

(define (bold . xs) "this is bold")

(define-namespace-anchor nsa)

(check-equal? ((eval 'bold (namespace-anchor->namespace nsa)) "hi") "this is bold")

Though in general, when you find yourself using `eval` it's a sign that you're taking an unnecessary detour.