Genshi -> Mako... almost there!

[Alec Flett]

Howdy.

We're evaluating switching to Mako from Genshi, primarily for
performance reasons. We run under Pylons and use Genshi in a fairly
specific way (we make extensive use of py:match in a similar way you'd
use <%def> / <%call> in Mako)

We have some 150+ templates to convert, so I wrote up an automated
script to do most of the work. See attached.

The basic approach I took was to use genshi to load the template and
process all the py: directives... then I derived from HTMLSerializer,
and use it to expand all the templates. This reaches pretty deep into
genshi and even into closures that genshi has defined, but the result
seems decent. It doesn't quite get interpreted by

Here are some of the limitations - some of which I think are worth
fixing, others..well... I plan to deal with the output manually after
running the script:

- Genshi compiles py:with statements down to an AST tree before
evaluation, but at that point I've lost all context. I've considered
trying to write an ast visitor that decompiles the AST back to python.
I'll bet for most AST nodes, it would be pretty simple.. but I thought
I'd ask around to see if anyone had tackled this before. On top of
this, Mako has no real notion of 'with' that I'm aware of, so I don't
have an easy way to scope variables just to one block

- Genshi allows '.' notation for dict member access - but not Mako. I
wish Mako had a similar mode, but I figure there's not much we can do
here other than wrap all our template-visible with some kind of
wrapper. We'll probably just deal with this by hand.

- I wasn't quite sure the equivalent of py:attrs in Mako - is there
any way to refer to caller.attributes() or anything?

- py:match conversion only works for the very specifc tag paths (i.e.
path="user-image") and not complex XPaths (i.e. path="user-
image[@id=X]")

- select() is not handled at all obviously. My hope here was to
recognize very specific patterns like "select('@*')" and "select('*|
text()')"

Please play with it! Send patches. If there's enough interest, I'll
start a google code project for this.

Alec

[Michael Bayer]

On Jul 22, 2008, at 8:03 PM, Alec Flett wrote:

>
> - Genshi compiles py:with statements down to an AST tree before
> evaluation, but at that point I've lost all context. I've considered
> trying to write an ast visitor that decompiles the AST back to python.
> I'll bet for most AST nodes, it would be pretty simple.. but I thought
> I'd ask around to see if anyone had tackled this before. On top of
> this, Mako has no real notion of 'with' that I'm aware of, so I don't
> have an easy way to scope variables just to one block

py:with is....like python 2.6 with ? or just sets up some local
variables ? (I only used genshi briefly)

> - Genshi allows '.' notation for dict member access - but not Mako. I
> wish Mako had a similar mode, but I figure there's not much we can do
> here other than wrap all our template-visible with some kind of
> wrapper. We'll probably just deal with this by hand.

hm how do you tell apart someobject.someattr versus
somedict.somekey ? the compiler doesn't know the type of the parent.

> - I wasn't quite sure the equivalent of py:attrs in Mako - is there
> any way to refer to caller.attributes() or anything?

Mako recently supports somenamespace.attr.<somekey>, such as:

<%!
foo = "bar"
%>

then in a parent template:

${self.attr.foo}

this would work with "caller" too in a component-call-with-content
(they're all namespaces).

[alecf]

>
> py:with is....like python 2.6 with ?   or just sets up some local  
> variables ?  (I only used genshi briefly)

it's more like Javascript "with" - it just sets up a local scope
(literally inserts a new scope object into the scope chain)

It looks like:

<py:with vars="x=4">
<span>x is ${x}!</span>
</py:with>

and ${x} wouldn't evaluate outside of the py:with

Regarding "." vs "[]" access:

> hm how do you tell apart   someobject.someattr versus  
> somedict.somekey ?   the compiler doesn't know the type of the parent.

In the case of genshi, I believe it tries a getattr first, then falls
back to dict access. so it's admittedly slower, but the reduced
punctuation makes the templates more readable. Again, not a huge deal,
something I'd like to see in Mako but can easily do a lot of search
and replace post-conversion.

> > - I wasn't quite sure the equivalent of py:attrs in Mako - is there
> > any way to refer to caller.attributes() or anything?
>
> Mako recently supports somenamespace.attr.<somekey>, such as:

Here's a use case in Genshi:

<py:match path="sometag">
<span py:attrs="select('@*')">This is sometag. content: $
{select('text()')}</span>
</py:match>

<sometag class="one">foo</sometag>
<sometag class="two">bar</sometag>

would generate:
<span class="one">This is sometag. content: foo</span>
<span class="two">This is sometag. content: bar</span>

Basically py:attrs takes a dictionary of name/value pairs and
generates HTML/XML attributes from the key/value pairs.

So does mako's attr allow this (assuming some function makeattr() that
turns an object into name="value" pairs)

<%def name="sometag()">
<span ${makeattrs(caller.attr)}>This is sometag: ${caller.body()}</
span>
</%def>

<%call name="sometag()" class="one">
<% class="one" %>
foo
</%call>
<%call name="sometag()">
<% class="two" %>
bar
</%call>

If so, that is fantastic, and addresses my needs perfectly!

Alec

[Michael Bayer]

On Jul 23, 2:01 pm, alecf <al...@metaweb.com> wrote:
>
> So does mako's attr allow this (assuming some function makeattr() that
> turns an object into name="value" pairs)
>
> <%def name="sometag()">
>     <span ${makeattrs(caller.attr)}>This is sometag: ${caller.body()}</
> span>
> </%def>
>
> <%call name="sometag()" class="one">
> <% class="one" %>
> foo
> </%call>
> <%call name="sometag()">
> <% class="two" %>
> bar
> </%call>
>

"namespace.attr" is a template-level thing, its basically Myghty's <
%attr> tag.

In this case, your def could accept arguments:


<%def name="sometag(attrs=None)">
</%def>

<%call expr="sometag(attrs={'class':'one'})">

.. or use additional defs from the caller:

<%def name="sometag()">
<span ${caller.attrs()}>sometag...
</%def>

<%call expr="sometag()">
<%def name="attrs()">class='one'</%def>
...body...
</%call>

I've started using preprocessors to make <%call> look nicer, i.e. like
the one in http://techspot.zzzeek.org/?p=28.

[alecf]

>
> I've started using preprocessors to make <%call> look nicer, i.e. like
> the one inhttp://techspot.zzzeek.org/?p=28.

That was one thing I was meaning to ask as well - if there was any
interest in the mako project to just build in this preprocessing, or
just start understanding these tags.

I find the syntax really cleans up the mako templates, it would be
nice not to require any special configuration to make that happen.

Alec

[Jorge Vargas]

On Tue, Jul 22, 2008 at 6:03 PM, Alec Flett <al...@metaweb.com> wrote:
> Howdy.

>
>
> - I wasn't quite sure the equivalent of py:attrs in Mako - is there
> any way to refer to caller.attributes() or anything?
>

There isn't.

Although it can be emulated, it had to be done for TW's mako support
http://toscawidgets.org/trac/tw/browser/tw/core/mako_util.py

[Roger Demetrescu]

Yeah... and here is a good place to see how mako_util.py is used:

http://toscawidgets.org/trac/tw.forms/browser/tw/forms/templates

You can do a side-by-side comparison between genshi and mako templates
for tw.forms templates.

Cheers,

Roger