Idea for pure-python templates using AST.
Paul Wray
Ive had what I think is a great idea for pure-python templates (I can almost
hear the groans, bear with me...)
For the impatient, proof of concept is at http://pastie.org/2379978
demonstrating simple substitution, balanced tags using context manager,
subtemplates, and template inheritance.
I'm posting here to get opinions on:
* the merits of the idea, (or otherwise!)
* whether there are any established/mature templating systems that use this
approach, or whether its come up before,
* ideas for improvements and advice on other aspects such as sandboxing
* of course, to share the idea in case others want to use it
Background: Ive been working on an application that recursively renders
html/xml documents from heterogenoeus trees, with separate classes for each
document component. First I built my own renderer but was dissatisfied with
the repetitive code. Then looked at Mako and Jinja, and used Jinja but was
still disatisfied, because I still needed a render method in each class to
do preparation, and also the template which was centrally managed by the
Jinja loader and environment. I found a way to call templates recursively
via Jinja filters, but was not sure that it wouldnt blow up in my face, so I
also had separate calls to render the children of each node, and then feed
the value to the parent template. I kept thinking that there must be a way
to get the brevity and clarity of declarative templates, plus the simplicity
of pure python loops, tests and function calls.
The idea:
Python syntax allows a statement to be a bare literal or identifier. These
have no effect on the program.
So the function below is legal python:
def myFunc():
'a'
x = 45
'b'; 'c'; x
So is this (within the appropriate class context of course):
def body(self, r):
'<h1>'; self.heading; '</h1>'
'<ul>'
for itm in self.items:
'<li>'; itm; '</li>'
'</ul>'
The idea is simply to use python ASTs to transform this code so that it
accumulates the values of the bare expressions.
I think this give the best of both worlds - declarative syntax for the
template literals, but in a pure python context, giving you all the power of
python control statements, classes etc.
For application contexts outside pure python code (ie user-created
templates) , it would be simple to extend the technique to creating template
functions from strings, and insert the resulting methods into a namespace
for execution.)
I think, given the technique is already working with the AST, that
sandboxing should not be too hard either - advice on this aspect would be
appreciated.
Paul Wray
aspineux
> Hello all
>
> Ive had what I think is a great idea for pure-python templates (I can almost
> hear the groans, bear with me...)
>
> demonstrating simple substitution, balanced tags using context manager,
> subtemplates, and template inheritance.
You code fail, see below for other comment
Traceback (most recent call last):
File "Download/pastie-2379978.rb", line 108, in <module>
make_template(template1)
File "Download/pastie-2379978.rb", line 60, in make_template
ast.fix_missing_locations(astFromSrc)
File "/usr/lib/python2.6/ast.py", line 133, in fix_missing_locations
_fix(node, 1, 0)
File "/usr/lib/python2.6/ast.py", line 132, in _fix
_fix(child, lineno, col_offset)
File "/usr/lib/python2.6/ast.py", line 132, in _fix
_fix(child, lineno, col_offset)
File "/usr/lib/python2.6/ast.py", line 121, in _fix
if 'lineno' in node._attributes:
AttributeError: 'arguments' object has no attribute '_attributes'
>
> I'm posting here to get opinions on:
> * the merits of the idea, (or otherwise!)
> * whether there are any established/mature templating systems that use this
> approach, or whether its come up before,
> * ideas for improvements and advice on other aspects such as sandboxing
> * of course, to share the idea in case others want to use it
This is very original ! First time I see it. I like it.
But how to debug large template ?
How to find/detect a missing </TAG> ?
This is very important. This is one big advantage of Genshi over Kid
How to report the correct error at the correct line ?
How to find/get nice editor to edit large template ?
You could call it PHP :-)
>
> I think this give the best of both worlds - declarative syntax for the
> template literals, but in a pure python context, giving you all the power of
> python control statements, classes etc.
>
> For application contexts outside pure python code (ie user-created
> templates) , it would be simple to extend the technique to creating template
> functions from strings, and insert the resulting methods into a namespace
> for execution.)
>
> I think, given the technique is already working with the AST, that
> sandboxing should not be too hard either - advice on this aspect would be
> appreciated.
Maybe a good idea.
But still a lot of work to get the level of existing libraries.
Maybe better if mixed with other tools like Genshi to create widget.
Maybe next toscawidget could use such a technique.
Thanks for sharing
>
> Paul Wray
Terry Reedy
More generally, Python has expression statements, with the result of the
expression ignored. These are usually function calls with side-effects.
"print('x')" has no effect on the program and the return value is
usually ignored.
> So the function below is legal python:
>
> def myFunc():
> 'a'
> x = 45
> 'b'; 'c'; x
>
> So is this (within the appropriate class context of course):
>
> def body(self, r):
> '<h1>'; self.heading; '</h1>'
> '<ul>'
> for itm in self.items:
> '<li>'; itm; '</li>'
> '</ul>'
>
> The idea is simply to use python ASTs to transform this code so that it
> accumulates the values of the bare expressions.
Interesting idea, though I have no experience for comparison.
--
Terry Jan Reedy
Chris Angelico
> The idea is simply to use python ASTs to transform this code so that it
> accumulates the values of the bare expressions.
That'd be similar to what the interactive loop does. Are you aware,
though, that docstrings are bare expressions? You may have a syntactic
collision there.
ChrisA
anand jeyahar
I did it. Finally managed to port mysqltuner.pl to python. Was a
real pain in the butt doing it from bottom up manually, without ever
really learing perl syntax. But i finally got it done. Now i need help
testing it. find it here.
g...@github.com:anandjeyahar/mysqlDbAdmin-python.git.
Also i never really thought about design. Just blindly/mechanically,
translated from perl to python. So criticize and let me know how i can
improve this.
Thanks and Regards,
==============================================
Anand Jeyahar
https://sites.google.com/site/anandjeyahar
==============================================
The man who is really serious,
with the urge to find out what truth is,
has no style at all. He lives only in what is.
~Bruce Lee
Love is a trade with lousy accounting policies.
~Aang Jie
Irmen de Jong
> The idea:
> Python syntax allows a statement to be a bare literal or identifier.
> These have no effect on the program.
>
> So the function below is legal python:
>
> def myFunc():
> 'a'
> x = 45
> 'b'; 'c'; x
>
> So is this (within the appropriate class context of course):
>
> def body(self, r):
> '<h1>'; self.heading; '</h1>'
> '<ul>'
> for itm in self.items:
> '<li>'; itm; '</li>'
> '</ul>'
>
Looks very similar to PTL what Quixote uses:
http://www.quixote.ca/overview/paper.html
(never used it though, and I doubt Quixote used ASTs)
Irmen
Paul Wray
> On Tue, Aug 16, 2011 at 12:33 PM, Paul Wray <paul.w...@det.nsw.edu.au> wrote:
> > The idea is simply to use python ASTs to transform this code so that it
> > accumulates the values of the bare expressions.
>
> That'd be similar to what the interactive loop does. Are you aware,
> though, that docstrings are bare expressions? You may have a syntactic
> collision there.
>
> ChrisA
Thanks yes ama aware of docstrings but did not consider.
They are easy to strip out though.
Chris Angelico
> Thanks yes ama aware of docstrings but did not consider.
> They are easy to strip out though.
>
Maybe. You'd have to take notice of what's a docstring and what's the
first element to be outputted. Or alternatively, just forbid
docstrings on those functions.
ChrisA
Tim Roberts
>
>Ive had what I think is a great idea for pure-python templates (I can almost
>hear the groans, bear with me...)
>The idea:
>Python syntax allows a statement to be a bare literal or identifier. These
>have no effect on the program.
>So is this (within the appropriate class context of course):
>
>def body(self, r):
> '<h1>'; self.heading; '</h1>'
> '<ul>'
> for itm in self.items:
> '<li>'; itm; '</li>'
> '</ul>'
This is essentially how the original CherryPy version 1 web framework
worked. In the end, I think it was decided that this represented too
much of a mix of processing and presentation, and CherryPy 2 and 3 use a
different scheme.
--
Tim Roberts, ti...@probo.com
Providenza & Boekelheide, Inc.
Paul Wray
Thanks. Yes it looks much the same, and it does use AST. This from the
ptl_compile.py module:
"""Compile a PTL template.
First template function names are mangled, noting the template type.
Next, the file is parsed into a parse tree. This tree is converted
into
a modified AST. It is during this state that the semantics are
modified
by adding extra nodes to the tree. Finally bytecode is generated
using
the compiler package.
"""
Paul Wray
>
> Traceback (most recent call last):
> File "Download/pastie-2379978.rb", line 108, in <module>
> make_template(template1)
> File "Download/pastie-2379978.rb", line 60, in make_template
> ast.fix_missing_locations(astFromSrc)
> File "/usr/lib/python2.6/ast.py", line 133, in fix_missing_locations
> _fix(node, 1, 0)
> File "/usr/lib/python2.6/ast.py", line 132, in _fix
> _fix(child, lineno, col_offset)
> File "/usr/lib/python2.6/ast.py", line 132, in _fix
> _fix(child, lineno, col_offset)
> File "/usr/lib/python2.6/ast.py", line 121, in _fix
> if 'lineno' in node._attributes:
> AttributeError: 'arguments' object has no attribute '_attributes'
>
Sorry, should have specified Python 2.7.
Also Pastie thinks its ruby code so the donwloaded file is confusingly
___.rb