proposal for the web.py templating system (codename: templetor)
Aaron Swartz
system for web.py. I've written some basic parsing and processing code
so far, but I thought I'd write up a spec and send it around to get
your thoughts before I start implementing the details. Let me know
what you think.
[BTW: If you're wondering why I haven't replied to emails lately, I've
been on sabbatical this week.]
# web.py templating system (codename: templetor)
There are almost as many Python templating systems as there are web
frameworks (and, indeed, it seems like many templating systems are
adopting web framework-like features), so it is with some trepidation
that I work on a new one. Sadly, again I find that my requirement are
met by nothing else:
1. The templating system has to _look_ decent. No `<%#foo#%>` crud.
2. Reuse Python terms and semantics as much as possible.
3. Expressive enough to do real computation.
4. Usable for any text language, not just HTML and XML.
And requirements for the implementation as well:
4. Sandboxable so that you can let untrusted users write templates.
5. Simple and fast implementation.
So here's my entry.
## Variable substitution
Look, a $string.
Hark, an ${arbitrary + expression}.
Gawk, a $dictionary[key].function('argument').
Cool, a $(limit)ing.
Stop, \$money isn't evaluated.
We use basically the same semantics as (rejected) [PEP
215](http://www.python.org/peps/pep-0215.html). Variables can go
anywhere in a document.
## Newline suppression
If you put a backslash \
at the end of a line \
(like these) \
then there will be no newline.
renders as all one line.
## Expressions
Here are some expressions:
$for var in iterator: I like $var!
$if times > max:
Stop! In the name of love.
$else:
Keep on, you can do it.
$try:
$get(input)
$except:
Couldn't find it.
That's all, folks.
All your old Python friends are here: `if`, `while`, `for`, `try`,
`except`, `else`. `break`, `continue`, and `pass` also act as you'd
expect. (Obviously, you can't have variables named any of these.) The
Python code starts at the `$` and ends at the `:`. The `$` has to be
at the beginning of the line, but that's not such a burden because of
newline suppression (above).
Also, we're very careful about spacing -- all the lines will render
with no spaces at the beginning. (Open question: what if you want
spaces at the beginning?)
There are a couple changes from Python: `for` and `while` now take an
`else` clause that gets called if the loop is never evaluated.
(Possible feature to add: Django-style for loop variables.)
## Comments
$# Here's where we hoodwink the folks at home:
Please enter in your deets:
CC: [ ] $#this is the important one
SSN: $#Social Security Number#$ [ ]
Comments start with `$#` and go to `#$` or the end of the line,
whichever is first.
## Code
Sometimes you just need to break out the Python.
$ mapping = {
$ 'cool': ['nice', 'sweet', 'hot'],
$ 'suck': ['bad', 'evil', 'awful']
$ }
Isn't that $mapping[thought]?
That's$ del mapping $ fine with me.
$ complicatedfunc()
$ for x in bugs:
$ if bug.level == 'severe':
Ooh, this one is bad.
$ continue
And there's $x...
Code begins with a `$` and a space and goes until the next `$` or the
end of the line, whichever comes first. Nothing ever gets output if
the first character after the `$` is a space (so `complicatedfunc`
above doesn't write anything to the screen like it might without the
space).
## Definitions
$def present(name, title):
$if title == "President":
<h1>$name</h1>
$else:
<h2>$name ($title)</h2>
That's right, `def` is here too. It can be called in a variable
substitution like any other function (it returns a string, obviously).
$set what:
Eat me, fool!
(the story of my life)
`set` acts sort of like `def` except it creates a string, not a
function that returns a string.
## Python integration
A template begins with a line like this:
$def with (name, title, company='BigCo')
which declares that the template takes those arguments. (The `with`
keyword is special, like `def` or `if`.)
Inside Python, the template looks like a function that takes these
arguments. It returns a storage object with the special property that
evaluating it as a string returns the value of the body of the
template. The elements in the storage object are the results of the
`def`s and the `set`s.
Perhaps an example will make this clearer. Here's a template:
$def with (post)
$set title: $post.title
<p>$markdown(post.body)</p>
<p class="byline">by $post.author</p>
Here's another:
$def with (self)
<html><head>
<title>$self.title</title>
</head><body>
<h1>$self.title</h1>
$self
</body></html>
Now let's say we compile both from within Python, the first as `page`,
the second as `base`. Here's how we might use them:
print base(page(post))
`page` takes the argument post and returns an object whose string
value is a bit of HTML showing the post with its title in the property
`title`. `base` takes this object and places the title in the
appropriate place and displays the page itself in the body of the
page. The Python code prints out the result.
_Where did `markdown` come from? It wasn't passed as an argument._ You
can pass a list of functions and variables to the template compiler to
be made globally available to templates.
mrstone
Looks very promising. I especially like the python integration.
I'm also interested in the sandboxing parts since I will need that for
the project I'm working with.
-Sten
F.S
> ## Code
>
> Sometimes you just need to break out the Python.
>
> $ mapping = {
> $ 'cool': ['nice', 'sweet', 'hot'],
> $ 'suck': ['bad', 'evil', 'awful']
> $ }
>
> Isn't that $mapping[thought]?
> That's$ del mapping $ fine with me.
Using the same char for both variables and code could be confusing.
// FS
vinjvinj
>>1. The templating system has to _look_ decent. No `<%#foo#%>` crud.
It uses {{ }}, which in my experience is very clean
>>2. Reuse Python terms and semantics as much as possible.
Allows for, if, script
>>4. Usable for any text language, not just HTML and XML.
I use it for generating code and also generating emails
And requirements for the implementation as well:
4. Sandboxable so that you can let untrusted users write templates.
>> I'm planning to use pylint checker to check for malicious code
>>5. Simple and fast implementation.
It comes in one file and is very fast.
So here's my entry.
>> Look, a $string.
{{string}}
>> Hark, an ${arbitrary + expression}.
{{arbitrary + expression}}
>> Gawk, a $dictionary[key].function('argument').
{{dictionary[key].function('argument')}}
>> Cool, a $(limit)ing.
{{limit}}ing
>> Stop, \$money isn't evaluated.
\{{money isn't evaluated.
>> $for var in iterator: I like $var!
{{for var in iterator}}
{{endfor}}
>> $if times > max:
>> Stop! In the name of love.
>> $else:
>> Keep on, you can do it.
{{if times > max:}}
Stop! In the name of love.
{{$else:}}
Keep on, you can do it.
{{endif}}
>> $try:
>> $get(input)
>> $except:
>> Couldn't find it.
They don't have a try/except but that should not be difficult to add
That's all, folks.
You can write arbitrary python code in the template using:
{{script}}
some python code
{{endscript}}
And all of preppy comes in one source file.
Aaron Swartz
The way I see it, there's one metacharacter ($) and what it does
depends on the second character. If it's a ( or a letter, then it's a
variable, if it's a space or a keyword, then it's code.
Aaron Swartz
> >> I'm planning to use pylint checker to check for malicious code
I don't think that's going to be very safe.
peter
> my requirement are met by nothing else:
Just one question. Will you still be able to use third party template
engines?
Armin Ronacher
> 1. The templating system has to _look_ decent. No `<%#foo#%>` crud.
/ %} or <? / ?>.
> 2. Reuse Python terms and semantics as much as possible.
Not good. Template Engines shouldn't provide access to the python
syntax as such.
Templates should only be allowed to display data. Not to calculate or
implement
application logic.
> 3. Expressive enough to do real computation.
see above.
> 4. Usable for any text language, not just HTML and XML.
+1
> 4. Sandboxable so that you can let untrusted users write templates.
when you want to support the whole python syntax this will lead you
into big
troubles. But if you manage it, it is a good thing :-)
> ## Variable substitution
>
> Look, a $string.
> Hark, an ${arbitrary + expression}.
> Gawk, a $dictionary[key].function('argument').
> Cool, a $(limit)ing.
>
> Stop, \$money isn't evaluated.
A bit stupid. $string as such is good, but the idea with the directory
accessing is not
that good. I would recommend
$dictionary.key.subkey.function('argument')
> ## Newline suppression
>
> If you put a backslash \
> at the end of a line \
> (like these) \
> then there will be no newline.
>
> renders as all one line.
And what about leading whitespaces?
> ## Expressions
>
> Here are some expressions:
>
> $for var in iterator: I like $var!
>
> $if times > max:
> Stop! In the name of love.
> $else:
> Keep on, you can do it.
>
> $try:
> $get(input)
> $except:
> Couldn't find it.
>
> That's all, folks.
Ouch. Indention?? Stupid in html templates ;-) You should do something
like this::
$if times > 10:
Stop! In the name of love.
$else:
Keep on, you can do it.
$end
And $ for variables and blocks is a bit strange.
> Also, we're very careful about spacing -- all the lines will render
> with no spaces at the beginning. (Open question: what if you want
> spaces at the beginning?)
Ouch again. Spaces are very important. Especially when outputting
ASCII content.
> There are a couple changes from Python: `for` and `while` now take an
> `else` clause that gets called if the loop is never evaluated.
Very good idea :)
> (Possible feature to add: Django-style for loop variables.)
+1 again
> ## Code
>
> Sometimes you just need to break out the Python.
>
> $ mapping = {
> $ 'cool': ['nice', 'sweet', 'hot'],
> $ 'suck': ['bad', 'evil', 'awful']
> $ }
Strange. I would prefer this::
$python:
mapping = {
'cool': ['nice', 'sweet', 'hot'],
'suck': ['bad', 'evil', 'awful']
}
$end
But i wouldn't allow the execution of python code in a template :-/
> *snip*
Hm. Looks strange. Have a look at the Jinja and django template
inheritance.
I think you should provide something like that.
There are some nice ideas in it but i won't use such a template engine.
Aaron Swartz
> engines?
Yeah, of course. There's absolutely nothing in web.py that _has_ to be
used, nor is there much that requires web.py.
Aaron Swartz
> $dictionary.key.subkey.function('argument')
Cheetah has this and in actual applications, this is the biggest
source of hard to find bugs I have ever seen. It's absolutely awful.
> But i wouldn't allow the execution of python code in a template :-/
Neither do we -- it's really just for assignments.
Armin Ronacher
> > that good. I would recommend
> > $dictionary.key.subkey.function('argument')
>
> Cheetah has this and in actual applications, this is the biggest
> source of hard to find bugs I have ever seen. It's absolutely awful.
parent frames
locals as template context. Never saw such a strange behavior.
Filip Salomonsson
>
> > 1. The templating system has to _look_ decent. No `<%#foo#%>` crud.
> Idea is good, but it doesn't work. You will need delmiter chars like {%
> / %} or <? / ?>.
Why?
> Templates should only be allowed to display data. Not to calculate or
> implement application logic.
Some people keep saying this. But, again: why? Not being able to do
loops and some calculations in the templates would drive me absolutely
bonkers.
> $string as such is good, but the idea with the [dictionary]
> accessing is not that good.
Once more: why?
> I would recommend
> $dictionary.key.subkey.function('argument')
So, does $dictionary.items() mean call the 'items' method of a
dictionary, or call the function contained in dictionary[items]? And
what about dictionary['items']? Would you prefer having the template
engine try them all and, if so, in what order?
> Ouch. Indention?? Stupid in html templates ;-) You should do something
> like this::
>
> $if times > 10:
> Stop! In the name of love.
> $else:
> Keep on, you can do it.
> $end
I think I'd prefer this, too. Provided that the above actually outputs
the leading spaces. Indenting blocks is nice, but I don't think it's
worth it in a templating system until I've seen a clean solution for
outputting indented stuff.
> But i wouldn't allow the execution of python code in a template :-/
Well, not _any_ python code (definitely not import, etc.). But
execution of code that happens to look like python seems fine to me.
(I would _not_ use a templating system that only let me pass in
strings - I can do that with good old %-formatting or
string.Template.)
--
filip salomonsson
Aaron Swartz
> parent frames locals as template context. Never saw such a strange behavior.
In my experience the NameMapper stuff is worse, but both are going away.
manatlan
why not an xml templating engine to be able to write (x)html, like we
write python ... like that
html:
head:
title:
hello world
body:
div id="content":
content
should result in :
<html><head><title>hello world</title></head><body><div
id="content">content</div></body></html>
indentation build the xml structure ... and line's ending with an ":"
is a tag ...
and we could place code like that : (by creating a node starting with a
"@" (like decorator))
html:
head:
title:
hello world
body:
@for id,name in list:
div id="${id}":
${name}
i'd prototyped it in an 100 lines code (with for/if/capture) and some
eval ;-)
i found it very natural and pythonic ....
ruivaldo_
+1 * 10
I liked.
>
>
--
"Machines take me by surprise with great frequency." (Alan M. Turing)
***
ruivaldo_
http://ruivaldo.wordpress.com
Aaron Swartz
> why not an xml templating engine to be able to write (x)html, like we
> write python ... like that
I've rarely had to write XML, so I can't I'm particularly interested.
XHTML requires numerous restrictions above normal XML and I, for one,
would like my resulting HTML to look good instead of
machine-generated.
David Terrell
> indentation build the xml structure ... and line's ending with an ":"
> is a tag ...
> and we could place code like that : (by creating a node starting with a
> "@" (like decorator))
>
> html:
> head:
> title:
> hello world
> body:
> @for id,name in list:
> div id="${id}":
> ${name}
>
>
> i'd prototyped it in an 100 lines code (with for/if/capture) and some
> eval ;-)
> i found it very natural and pythonic ....
I think this is a great variation on python templating, and I would
probably use it for some things -- but I don't think it should be
the default for web.py. Lots of things besides well-formed html
(email, text files, etc) can be output by a webserver.
manatlan
and i've made a little tutorial here :
http://manatlan.online.fr/hypy.php
and it's possible to render non-xhtml too !!! see the tuto (at the end)
manatlan
http://manatlan.online.fr/hypy.php
it's now able :
- to compile template into bytecode (pyc files)
- to use "nested for"
and it's always able to help you to produce xhtml/xml strings ... or
let you produce all kinds of text files (html, txt, mail ...)
peter
this is a great feature & allows stable & unstable to co-exist.
manatlan
tutorial + download are always here http://manatlan.online.fr/hypy.php
changelog :
- inheritance (concept of masterpage and child-templates)
- "else clause" for the "if"
- compile() method to compile himself its template to bytecode pyc
manatlan
changelog :
- mainly : it works well on win32 platforms now ...
i'll stop bugging you, now ..
i will stick on my hypy for templating webpy ;-)
hypy seems usable (and is 300 lines of code)
vmaasalo
templates. The syntax is nice, beautiful and expandable. They use {{
foo }} for variables and {% for blah in blahs %} for commands.
Expandable. Template inheritance. Very fast.
Armin Ronacher
Tavis Rudd
To clarify, neither of these is going away. Rather, they are both
optional and always have been.
Aaron, I don't use code like that personally, but I don't see how it
would lead to hard to find bugs. Can you provide an example?
Armin, we have had no reports of issues with the frame locals stuff.
It's an internal implementation detail that is not exposed to the
programmer. What exactly have you encountered? If there's an issue,
we'd like to know about it and resolve it.
Cheers,
Tavis
Tavis Rudd
Cheetah, for one, meets all of your 6 stated requirements, although #1
and #6 are matters of opinion. You wrote me just last month to say how
much you liked Cheetah. What has changed since then?
peter
being able to do loops and some calculations in the templates would
I have to agree on this. The ability to do simple logic is important.
Experience is with Andy Wardleys Template Toolkit (Perl) [1] allows for
some simple yet effective looping & branching.
For example from the docs [2] to generate a number of urls for example
would be (for a moment put aside the '[% foo %]' markup and concentrate
on logic only) ....
<ul>
[% FOREACH link = webpages %]
<li><a href="[% link.url %]">[% link.title %]</a>
[% END %]
</ul>
As seen above its not your *inline "bar" language* (bad) but a very
simple to understand PDL [3] that programmers and user interface
designers alike can understand and write.
The question is HOW the logic is coded in the template? I would prefer
*PDL* rather than Python or *Pythonish" logic. [4] But thats my
personal choice. I can see the merit of logic in templates but embedded
logic has its problems.. is a pain to debug etc.
Reference
[1] Andy Wardley, 'Template Toolkit':
<http://www.template-toolkit.org/>
[Accessed Tuesday, 28 February, 2006]
[2] Template Toolkit, 'Tutorial documentation':
<http://www.template-toolkit.org/docs/plain/Tutorial/Web.html'>
[Accessed Tuesday, 28 Febraury 2006]
[3] PDL or Program Definition Language. Psuedocode logic you write
before you write you application code as outlined in P54, 4.2, PDL for
Pro`s, 'Code Complete' by Steve McConnell, MS Press, 1993.
[4] Template.py, 'See the Template.py in development on the front
page':
<http://webpy.org/>
[Accessed Tuesday, 28 Febraury 2006]
David Terrell
> As seen above its not your *inline "bar" language* (bad) but a very
> simple to understand PDL [3] that programmers and user interface
> designers alike can understand and write.
>
>
> The question is HOW the logic is coded in the template? I would prefer
> *PDL* rather than Python or *Pythonish" logic. [4] But thats my
> personal choice. I can see the merit of logic in templates but embedded
> logic has its problems.. is a pain to debug etc.
Check out clearsilver (www.clearsilver.net). Great toolkit, trac uses
it, very data driven templating.
Armin Ronacher
Regards,
Armin