Getting `template` out of experimental

[Alexis King]

It’s no secret that I love syntax/parse, but I’m a little less vocal
of my love of syntax/parse/experimental/template, mostly because
of that scary “experimental” in the name. Really, though, template
is great, and it’s to syntax what syntax-parse is to syntax-case.
Is there anything blocking making it a stable API? I’d imagine that,
at this point, it’s probably used by more than a few people.

If one of the main qualms is the use of the cryptic ?? and ?@
identifiers, may I suggest alternate names? I think it would make
sense to re-use syntax-parse’s ~or and ~seq names, since those are
the closest duals in terms of functionality. One argument against
might be that it would make it harder to produce syntax that contains
syntax-parse patterns, so it would be possible to use an alternate
sigil, instead, but I think it probably wouldn’t be too hard for
users to wrap patterns with escapes if necessary.

Otherwise, are there any other reasons for keeping template marked
experimental? It’s a wonderful tool, but I feel a bit guilty depending
on it if it’s not technically a stable API.

Alexis

[Dupéron Georges]

Le mercredi 18 janvier 2017 00:02:17 UTC+1, Alexis King a écrit :

It’s no secret that I love syntax/parse, but I’m a little less vocal
of my love of syntax/parse/experimental/template, mostly because
of that scary “experimental” in the name. Really, though, template
is great, and it’s to syntax what syntax-parse is to syntax-case.

I'm using it a lot too, and it works fine, so I'm voting to pull it out of experimental too.

I'll shamelessly point to my small PR https://github.com/racket/racket/pull/1514 which adds syntax-local-template-metafunction-introduce, and would be nice to include in the non-experimental version.

I have been wanting for a while to implement a way to escape syntax templates (for now those from syntax/parse/experimental/template), and have the escaped code be iterated over if it is under an ellipsis, and now seems a good time to do this as it would require some changes in syntax/parse/template. Here is a simplistic example:

(syntax-case #'(1 2 3)
  [(x ...)
   (template (#,(* (syntax-e #'x) 2) ...))])

;; => #'(2 4 6)

There are a couple of issues to address:

• Within the #,escaped form, syntax pattern variables should have a nesting level one less than their outer nesting level.
  
• The (template) form should be able to know which syntax pattern variables are referred to by the fully-expanded #,escaped form, to know which ones to iterate over (and to actually perform the iteration at run-time).

The problem is that these informations depend on each other:

• Within the #,escaped form, the identifier x should be shadowed with a less nested syntax pattern variable, and this needs to be done before expanding the #,escaped form, as nested uses of syntax or template need to know the correct depth
  
• The (template) form needs to fully-expand the #,escaped form in order to know which syntax pattern variables are referenced within.

I'm not sure how to solve this predicament.

• I thought about modifying syntax-mapping-depth in racket/private/sc to subtract an “outer depth” stored in a make-parameter, but then the run-time value of the variable pointed to by syntax-mapping-valvar would not be correct. Another run-time make-parameter could store the "current path", so that syntax-mapping-valvar returns that subpart, but this starts to feel like piling hacks on top of each other.
  
• Another solution is to change syntax and template to cooperate, with a similar use of make-parameter. A quick search for syntax-mapping-depth on GitHub showed only these two forms, in various forks of the official Racket repository, so I think it's reasonably safe to change those instead.
  
• A third solution would be to change syntax-case and syntax-parse to record all identifiers bound as syntax pattern variables. It would then be possible to preemptively shadow all of those (somewhat costly, but this allows many improvements and modifications of template, for example I have written a subtemplate macro which automatically calls generate-temporaries to create yᵢ ... identifiers if there is a syntax pattern variable under ellipses named xᵢ, and the current implementation relies on a boatload of heuristics to find the "original" syntax pattern variable).

I'm also concerned about the performance implications of changing such a low-level mechanism in Racket. Even if the change to syntax-mapping-depth and syntax-mapping-valvar, syntax and template, or syntax-case and syntax-parse is as simple as checking the value of a parameter, these are used everywhere.

I'm open to any suggestion!