Templates interlinked within templates interlinked

[Joel Dueck]

I was pretty happy to learn about the web-server/templates package, which allows you to include the contents of a template file and have it dropped into the lexical context of the calling site. I’ve often wanted something like this so I could write only a single copy of things like headers and footers, and partials for listing chapters/articles in different contexts, etc.

I started fiddling with it in a Pollen project and, come to realize it was working even though I hadn’t `require`d or `provide`d any functions from web-server/template. How is this possible, I think. Some digging revealed that Pollen implements its own `include-template` function—and that it is provided automatically for code inside a Pollen template.

I guess this is cool? But

• have you thought about including it in the docs? It’s pretty useful, and the docs could also help avoid confusion among those who already know about web-server/template. The docs do mention that include-template is used behind the scenes but not that it's also available in Pollen templates. With appropriate warnings about XSS concerns.
  
• might it also be worth providing or reimplementing web-server/template's `in` function? [2] (maybe it already does so, I confess I haven’t checked yet) And then, perhaps including it in the docs somewhere as well.
• are there tradeoffs between the `include-template` provided by Pollen and the one in web-server/template?

[1]: http://docs.racket-lang.org/web-server/templates.html#%28form._%28%28lib._web-server%2Ftemplates..rkt%29._include-template%29%29

[2]: http://docs.racket-lang.org/web-server/templates.html#%28form._%28%28lib._web-server%2Ftemplates..rkt%29._in%29%29

[Matthew Butterick]

On Mar 15, 2018, at 6:00 AM, Joel Dueck <dueck...@gmail.com> wrote:

I was pretty happy to learn about the web-server/templates package, which allows you to include the contents of a template file and have it dropped into the lexical context of the calling site. I’ve often wanted something like this so I could write only a single copy of things like headers and footers, and partials for listing chapters/articles in different contexts, etc.

Pollen templates can do all this too, for a not-very-surprising reason (keep reading) ...

I started fiddling with it in a Pollen project and, come to realize it was working even though I hadn’t `require`d or `provide`d any functions from web-server/template. How is this possible, I think. Some digging revealed that Pollen implements its own `include-template` function—and that it is provided automatically for code inside a Pollen template.

The `include-template` in Pollen is the same one as in `web-server/templates` [1] But I copied its source and changed it so that it could return binary files (like PDFs), which the usual one cannot.

• have you thought about including it in the docs? It’s pretty useful, and the docs could also help avoid confusion among those who already know about web-server/template. The docs do mention that include-template is used behind the scenes but not that it's also available in Pollen templates. With appropriate warnings about XSS concerns.
  

I suppose I could. I've just never been clear what could be done with `include-template` directly that isn't already possible through existing Pollen functions. When you apply a template to a markup source during a render, you're already using `include-template`.

• might it also be worth providing or reimplementing web-server/template's `in` function? [2] (maybe it already does so, I confess I haven’t checked yet) And then, perhaps including it in the docs somewhere as well.

Same — I suppose I've always thought that because Pollen has a notion of X-expressions and tag functions, they subsume `in` (which, unlike `include-template`, does no heavy lifting anyhow)

[1] https://github.com/mbutterick/pollen/blob/master/pollen/private/include-template.rkt

[Joel Dueck]

On Thursday, March 15, 2018 at 10:11:25 AM UTC-5, Matthew Butterick wrote:

Pollen templates can do all this too, for a not-very-surprising reason (keep reading) ...The `include-template` in Pollen is the same one as in `web-server/templates`

Yes. That was the realization I had come to in my first post. I just didn't know it before (it = "Pollen templates can do all this too") because `include-template` is not listed in the Pollen docs as being available to template contents.
 

[...] I've just never been clear what could be done with `include-template` directly that isn't already possible through existing Pollen functions. When you apply a template to a markup source during a render, you're already using `include-template`.

So (just to be clear) here's the kind of thing I'm interested in doing:

;; template.html.p
<html>
    ◊include-template["templates/head.html"]
<body>
<h1>Chapters</h1>

◊in[chapter (children here)]{
  ◊include-template["templates/chapter-title-excerpt.html"]
}

;;;;;;;;;;;;;templates/head.html
<head>
   <title>@(select-from-metas 'title here)</title>
   <link rel="stylesheet" href="/style.css">
   [lots of fiddly javascript, favicons, etc]
</head>

;;;;;;;;;;;;;;templates/chapter-title-excerpt.html
<article>
 <h1>@(select-from-metas 'title chapter)
 <p>@(->html (select-from-doc 'excerpt chapter))</p>
  [...]
</article>

In fine, what interests me is smaller templates reusable within multiple "main" templates, the ones that Pollen grabs and applies to the markup source. And I like that Pollen gives me include-template as an option for doing this.

Yes, I could just add a set of "HTML template" helper functions to pollen.rkt (and then maybe a set of "Latex template" helper functions, etc). But:

- I like not having to pass arguments between a template and its helper functions. The "sub-template" just knows everything that it's parent template knows.
- I like editing these as snippets of HTML more than as functions in a module
- I like that reusable template stuff can be separated from all my tag functions while still being automatically available in the templates themselves.

I freely concede that all this is sugar for my subjective biases. And fortunately for me, it's already implemented so I don’t have to make an argument for its inclusion. I just thought more people would like to know about it.

Also, even though they're subjective, these preferences above seem at least akin to those that must have led to Pollen having any "templates", period, no?

• might it also be worth providing or reimplementing web-server/template's `in` function? [2] (maybe it already does so, I confess I haven’t checked yet) And then, perhaps including it in the docs somewhere as well.

Same — I suppose I've always thought that because Pollen has a notion of X-expressions and tag functions, they subsume `in` (which, unlike `include-template`, does no heavy lifting anyhow)

Well, within a doc (tag functions, etc), no, I don't see that I would use `in`. It's mainly just shorthand for use in templates, i.e. places where you're outside a doc context and want a quick way to glue a fairly static, reusable chunk of HTML (or LaTeX or whatever) together.

The case `in` makes for itself [1] is that it shortens a template like this:

<table>

@for/list[([c clients])]{

@list{

<tr><td>@(car c), @(cdr c)</td></tr>

}

}

</table>

To this:

<table>

@in[c clients]{

<tr><td>@(car c), @(cdr c)</td></tr>

}

</table>

Which, hey, I might use that. And I could do `(require (only-in web-server/templates in))` to get it (to avoid a collision with `include-template` (maybe unnecessary?)). But since Pollen already gives me include-template, why not that one too? Although I can see possible arguments against this one.

[1]: http://docs.racket-lang.org/web-server/templates.html?q=in#%28part._.Gotchas%29

[Matthew Butterick]

On Mar 15, 2018, at 11:49 AM, Joel Dueck <dueck...@gmail.com> wrote:

In fine, what interests me is smaller templates reusable within multiple "main" templates, the ones that Pollen grabs and applies to the markup source. And I like that Pollen gives me include-template as an option for doing this.

I see what you mean, though wouldn't `file->string` suffice? Here, we turn the main template into a `pp` file so the Pollen commands therein get executed. These commands bring in subtemplates with other Pollen commands, but those commands don't get executed until "index.html.pm" is rendered with the resulting "template.html" as its template: 

;; index.html.pm

#lang pollen

◊(define-meta title "foobar")

Hi there

;; template.html.pp

#lang pollen

◊(require racket/file)

<html>

◊file->string["head.html"]

◊file->string["body.html"]

</html>

;; head.html

<head>

   <title>◊(select-from-metas 'title here)</title>

   <link rel="stylesheet" href="/style.css">

</head>

;; body.html

<body>

<h1>Chapters</h1>

◊(->html doc)

</body>

BTW1 I concede this method calls for some finesse in terms of remembering which ◊ commands get executed when. 

BTW2 Your question is fair. I've thought along similar lines. At an earlier stage in Pollen's life I implemented a special template dialect that would allow templates to have templates. Genetic residue here [1] It would've used a different command character so the two rounds of Pollen commands would stay distinct. But that also seemed possibly like playing 12-dimensional chess. Though maybe its time has come.

BTW3 All that said, `include-template` isn't wrong, but in my experience it can be slow, so if there's a sleeker solution, so much the better for everyone.

The case `in` makes for itself [1] is that it shortens a template like this:

Could you post a full working example as a gist or equiv so I can fiddle with it? 

[1] https://docs.racket-lang.org/pollen/Setup.html?q=pollen%2Fsetup#%28def._%28%28lib._pollen%2Fsetup..rkt%29._setup~3atemplate-source-ext%29%29

[Joel Dueck]

On Thursday, March 15, 2018 at 8:20:09 PM UTC-5, Matthew Butterick wrote:

I see what you mean, though wouldn't `file->string` suffice? Here, we turn the main template into a `pp` file so the Pollen commands therein get executed.

I hadn't considered that! I can't think of any reason why that method wouldn't work just as well.
 

At an earlier stage in Pollen's life I implemented a special template dialect that would allow templates to have templates. [..] It would've used a different command character so the two rounds of Pollen commands would stay distinct.

That's interesting, but the need to think in terms of "rounds of Pollen commands" is not clear to me. Whether using `include-template` or `file->string`, there is only one "round": the sub-template gets lifted into the current context and then evaluated right alongside everything else. Maybe I'm oversimplifying it though?
 

The case `in` makes for itself [1] is that it shortens a template like this:

Could you post a full working example as a gist or equiv so I can fiddle with it?

Sure, here it is: https://gist.github.com/otherjoel/cde1aef9b582392efcdb12d74d65bf7c

Predictably, GitHub doesn't allow me to change the name of the gist or the ordering of files, so the least relevant files are at the beginning and the most relevant one (template-index.html.p) is at the end.

[Matthew Butterick]

On Mar 16, 2018, at 2:30 PM, Joel Dueck <dueck...@gmail.com> wrote:

At an earlier stage in Pollen's life I implemented a special template dialect that would allow templates to have templates. [..] It would've used a different command character so the two rounds of Pollen commands would stay distinct. 

That's interesting, but the need to think in terms of "rounds of Pollen commands" is not clear to me. Whether using `include-template` or `file->string`, there is only one "round": the sub-template gets lifted into the current context and then evaluated right alongside everything else. Maybe I'm oversimplifying it though?

In my example, we end up with two rendering passes:

1. template.html.pp → template.html

2. index.html.pm + template.html → index.html

Some Pollen commands apply to the first pass; some to the second.

In the example I gave you, consider what would happen if we lifted "head.html" directly into "template.html.pp":

;; template.html.pp

#lang pollen

◊(require racket/file)

<html>

<head>

   <title>◊(select-from-metas 'title here)</title>

   <link rel="stylesheet" href="/style.css">

</head>

◊file->string["body.html"]

</html>

This wouldn't work, because in rendering pass #1, Pollen would evaluate `◊(select-from-metas 'title here)` and get an error, because `here` isn't available yet.

In my stupid template language, you could write the file like so, with the `∂` character denoting template-level commands:

;; template.html.pt

#lang pollen

∂(require racket/file)

<html>

<head>

   <title>◊(select-from-metas 'title here)</title>

   <link rel="stylesheet" href="/style.css">

</head>

∂file->string["body.html"]

</html>

Note also the different file extension. In this case, Pollen would say "aha, this is a .pt file, so we'll treat `∂` as the command character (and thereby leave the #\◊ commands alone)". On the second pass, the ◊ commands would be interpreted as usual.

 

The case `in` makes for itself [1] is that it shortens a template like this:

Could you post a full working example as a gist or equiv so I can fiddle with it? 

Sure, here it is: https://gist.github.com/otherjoel/cde1aef9b582392efcdb12d74d65bf7c

Predictably, GitHub doesn't allow me to change the name of the gist or the ordering of files, so the least relevant files are at the begin

OK thanks. I've just pushed an update with a new macro called `for/splice` that works the same way as `in` (but with better name, and better cooperation under the hood with Pollen) If it works the way you expect, I'll add it to the docs.