Is this "inside out" macro technique common?

[Ryan Kramer]

A shortcut to answering my question would be to tell me that `with-quasisyntax` in the following paste already exists: http://pasterack.org/pastes/48885

== Context ==

In my current project, I have a macro that is doing a lot of work. (It is Typed Racket, but my question applies equally to untyped Racket.) This macro takes in a struct id, field ids, and field types. It generates the struct definition along with some serialization and deserialization machinery.

Then I encounter a new requirement: I want to be able to convert at least one these structs to an association list. (This differs from serialization.) The most obvious way to me is to add that functionality to my already-large macro. But it is getting difficult to maintain, and it feels like there should be a better way.

== Solution? ==

I came up with the approach in this paste, and it seems to work great: http://pasterack.org/pastes/48885 (Of course, if I had more structs I would make certain things reusable, it just demonstrates the concept.)

Custom syntax transformers also came to mind, but they seem a bit cumbersome for something that I don't intend to expose to external code.

I'm kind of surprised that it took me so long to discover this approach. Unless there is some nasty footgun that I haven't hit yet, I expect I'll be using this approach a lot in the future. But if it feels so right, then why haven't I seen it until now?

[D. Ben Knoble]

> A shortcut to answering my question would be to tell me that `with-quasisyntax` in the following paste already exists: http://pasterack.org/pastes/48885

Without taking a detailed look, is there anything about with-quasisyntax that with-syntax (https://docs.racket-lang.org/reference/stx-patterns.html#%28form._%28%28lib._racket%2Fprivate%2Fstxcase-scheme..rkt%29._with-syntax%29%29) doesn't do?

I would also highly recommend syntax-parse and it's various abstractions.

[D. Ben Knoble]

Ah, I'm now seeing that with-quasi implicitly #`s the body; I believe with syntax-parse, #:with, and #' + template vars + #` when needed you might be ok.

[Ryan Kramer]

The name `with-quasisyntax` is not very good, because it is not simply a quasi version of `with-syntax`. The most interesting part is that it calls `eval-syntax` up front. The result feels like a "universal macro" -- it can be used to implement both foo->assoc and assoc->foo which look like they would traditionally need separate macros (or one large macro that handles both).

I am noticing that this use of `eval-syntax` can cause error messages to be less good, but I still think it's OK for "private" code.

On Fri, Aug 13, 2021 at 2:45 PM D. Ben Knoble <ben.k...@gmail.com> wrote:

Ah, I'm now seeing that with-quasi implicitly #`s the body; I believe with syntax-parse, #:with, and #' + template vars + #` when needed you might be ok.

--
You received this message because you are subscribed to a topic in the Google Groups "Racket Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/racket-users/61cQImHJfZI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to racket-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/racket-users/2ede4034-80ec-49ad-9782-8883f8d47085n%40googlegroups.com.

[Michael Ballantyne]

The essential primitive here seems to me to be:

(define-syntax (splice stx)

  (syntax-case stx ()

    [(_ e ...)

     (eval-syntax #'(begin e ...))]))

With with-quasisyntax being:

(define-syntax-rule

  (with-quasisyntax ([a b] ...) template)

  (splice (with-syntax ([a b] ...) (quasisyntax template))))

Forms like splice appear in the metaprogramming systems of other programming languages such as Template Haskell (https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/exts/template_haskell.html) and Converge (https://convergepl.org/) but I haven't seen the pattern widely used in Racket.

I think this comes from a different philosophy. Systems like Template Haskell think of metaprogramming as a way to automatically generate code. From "Template Meta-programming for Haskell" (https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/meta-haskell.pdf):

      The purpose of the extension is to allow programmers to compute some parts of their program

       rather than write them, and to do so seamlessly and conveniently

In Racket we generally think instead about creating either language extensions or new sub-languages. Code generation happens to be the way we implement the extensions or sub-languages, but the focus is on the new syntax we are defining. If we find repeated patterns of code we start by thinking about the language or language feature we wish was there, and then implement that. Within those implementations we use abstractions like syntax-parse, syntax classes, and syntax-parse template metafunctions.

Without the whole picture of the problem you're trying to solve, it's hard to evaluate how splices compare to those alternatives, though.

That said, here are two alternative implementations of splice:

(define-syntax (splice2 stx)

  (syntax-case stx ()

    [(_ e ...)

     #`(let-syntax ([m (lambda (stx) e ...)]) (m))]))

(require (for-syntax racket/syntax))

(define-syntax (splice3 stx)

  (syntax-case stx ()

    [(_ e ...)

     (syntax-local-eval #'(begin e ...))]))

Whereas eval-syntax only allows access to the module-level expander environment, these can access the local environment. That might matter if you need to use syntax-local-value to access information from syntax bindings. The following works with splice2 and splice3, but not splice:

(let-syntax ([x 'foo])

  (splice3

   (displayln (syntax-local-value #'x))

   #'(void)))

I recommend using splice3 because it avoids the intermediate expansion step of the let-syntax .

[Ryan Kramer]

Thank you! `splice` is indeed the essential primitive here, so it's nice to see it extracted and named properly.

The difference between eval-syntax and syntax-local-eval is good to know also. I think there is a bug in the documentation (or maybe in Racket) because when I try `(syntax-local-eval #'(begin e ...) #f)` I get

..\..\Program Files\Racket-8.0.0.11\collects\racket\syntax.rkt:234:0: syntax-local-bind-syntaxes: contract violation
  expected: (or/c internal-definition-context? (listof internal-definition-context?))
  given: #f

No big deal, the empty list works, but the documentation says #f should be accepted too.

> Without the whole picture of the problem you're trying to solve, it's hard to evaluate how splices compare to those alternatives, though.

When choosing how to implement a macro, my first thought is "can I make define-syntax-rule work?" And if the answer is yes, then I use it -- I won't need the documentation and probably won't make a mistake. On the other end of the spectrum is syntax-parse which is way more powerful, but I will certainly need the documentation and mistakes are more likely. When I stumbled upon my splice variant, it seemed to be only slightly more complex than define-syntax-rule while enabling a much wider range of usage patterns. Or so I thought at the time, but now I'm less sure. I'll have to see how it plays out.

[Michael Ballantyne]

>  I think there is a bug in the documentation (or maybe in Racket) because when I try `(syntax-local-eval #'(begin e ...) #f)`

Thanks for the report, I submitted a PR with a fix (https://github.com/racket/racket/pull/3964).

Referring back to your original message,

>  (Of course, if I had more structs I would make certain things reusable, it just demonstrates the concept.)

I don't understand how you would use `splice` there. It seems like you would make a macro that takes in information about the fields, etc. In the body of the macro you are already writing phase 1 code, and the output is already used as generated code. Where does `splice` come in?

In case it's helpful, here's an example of how you might use a helper macro to separate out the work related to the association list representation: http://pasterack.org/pastes/11555