Semantic markup in generated X-expression?

[Daniel Sockwell]

After reading through the Pollen documentation, I'm extremely sold on the power of semantic markup. But I'm a bit confused about whether to preserve semantics in the generated X-expressions and, if so, how we would do so.

Consider the following toy example:

```
#lang pollen

◊(books (book "The Hitchhiker's Guide to the Galaxy")
(book "Catch 22")
(book "The Name of the Wind"))
◊(films (film "Star Wars: A New Hope")
(book "One Flew Over The Cuckoo's Nest")
(book "The Hitchhiker's Guide to the Galaxy"))
```

This is nicely semantic markup and, by default, produces an equally semantic X-expression:

```
'(root
(books
(book "The Hichhiker's Guide to the Galaxy")
(book "Catch 22")
(book "The Name of the Wind"))
"\n"
(films
(film "Star Wars: A New Hope")
(book "One Flew Over The Cuckoo's Nest")
(book "The Hick hiker's Guide to the Galaxy"))
"\n")
```

However, this markup does not (yet) produce valid HTML. If we want it to produce valid HTML we could define functions like this in our `pollen.rkt` file (these are toy examples that leave a lot out for simplicity):

```
#lang racket
(require pollen/tag
pollen/decode)
(provide (all-defined-out))

(define (books . elements) `(ul ,@elements))
(define (book title) `(li ,title))

(define (films . elements) `(ul ,@elements))
(define (film title) `(li ,title))
```

Now our nice semantic markup produces valid HTML, exactly as we wanted.

However, we've now lost the semantics in our generated X-expression. We're now getting

```
'(root
(ul
(li "The Hitchhiker's Guide to the Galaxy")
(li "Catch 22")
(li "The Name of the Wind"))
"\n"
(ul
(li "Star Wars: A New Hope")
(li "One Flew Over The Cuckoo's Nest")
(li "The Hitchhiker's Guide to the Galaxy"))
"\n")
```

In addition to not looking as pretty, the loss of our semantic tags in the X-expression means we can't do things like write a decoder that distinguishes between `(book "The Hitchhiker's Guide to the Galaxy")` and `(film "The Hitchhiker's Guide to the Galaxy")`. In general, my intuition is that the semantics should be preserved as long as possible and only stripped out when taking the final step of transforming out data to the output format (here, HTML). But I'm not sure if that's possible/worth the effort.

This leads me to the question I started with: is there a better approach that would preserve the semantic meaning in our generated X-expression (while still yielding valid HTML)? Or does it just not make sense to try to preserve semantics in the X-expression, given that we have semantics in the markup? I played around with `(decode)` a bit and couldn't get it to work for this. But I wasn't sure if that's because I don't yet fully understand `(decode)` or because I was trying to get it to do something it wasn't meant to do.

I'd appropriate anyone's perspective on this.

(By the way, Matthew, thanks very much for your helpful replies to my recent questions. I haven't replied just to say "thanks" out of a desire not to spam the group, but I very much appreciate your help!)

[dsoc...@gmail.com]

I just noticed an error in my earlier example:  I listed two films as `(book)`.   To be clear, all of the children of `(films)` should have been films, not books.  That error was unintentional, but underscores my point a bit: because the semantics were stripped out of the X-expression, it was much harder to catch that error!

[Matthew Butterick]

You could preserve the semantic information inside a `class` attribute, or custom attribute. These would also be reachable through CSS selectors, if you're into that.

[dsoc...@gmail.com]

You could preserve the semantic information inside a `class` attribute, or custom attribute. These would also be reachable through CSS selectors, if you're into that.

True, and I considered that.  (Something like  '(define book  (default-tag-function 'li #:class "book"))` would do the trick nicely).

But note that that's not quite what I was asking about: that preserves the semantic info all the way through to the generated HTML.  As you mentioned, that could be good if I wanted to apply CSS based on the semantics.  But it wouldn't do what I was asking about: keep the semantics in the X-expression without keeping them in the HTML.

Based on your answer, I'm assuming that there isn't a simple way to do what I was asking.  Based on that (and rereading some of the docs), I've concluded that I was thinking of the generated X-expression all wrong.  I was thinking of the X-expression as basically still a source file, albeit one that has been processed.  But I it seems like it's very nearly an output file—it's pseudo-HTML (or pseudo-text, or pseudo-LaTex, or whatever the final output format is).  From that point of view, it makes perfect sense that it'd have a structure that's limited to the semantics of HTML/text/LaTex.

  

[Matthew Butterick]

OK, then suppose you store the attributes in the X-expressions for the purposes of your internal Pollen processing. You can always strip out those private attributes before the `doc` is injected into the HTML template. Does that work?

You're right that the X-expression closely models HTML output (which makes it a convenient choice for Pollen). But it's also a vanilla Racket list structure, so you can manipulate it with the usual list functions (with the `txexpr` module sugaring over some small annoyances)