Proposal for a new templatetag definition syntax

[Alex Gaynor]

Hello all,

For many years, writing templatetags has been among the most hilariously

complicated things we Django developers did. Anyone who has written parsing for

templatetags, over and over, shares this pain. Further, the current syntax

present a tremendous problem for Armin Ronacher's GSOC towards template

compilation: template tags define a ``render()`` method, which takes the

current context (a stack of dictionaries) and perform some behavior which is

opaque to the caller. This is a problem because one of the core objectives of

the template compilation infrastructure is to store the template context in

Python local variables, rather than in dictionaries. For all these reasons,

several of us (myself, Idan, Russ, Carl Meyer, Andrew Godwin, Jonas Obrist,

Chris Beaven) just sat down (we actually stood, mostly) and tried to iron out a

proposal that solves these problems, taking inspiration from the plethora of

libraries that exist today.

We ultimately created two possible solutions, one inspired primarily by

django-classy-tags and django-templatetag-sugar, the other mostly inspired by

django-ttags. We came to a fragile agreement on the first. We primarily

evaluated two cases for these, one very simple, the other more complex, and

compared the resulting implementations.

The first case we considered was a templatetag which takes two, required,

arguments. Invocation looks like ``{% mytag foo bar %}``.  The two definitions

look like:

    class MyTag(Tag):

        args = [

            Argument("foo"),

            Argument("bar"),

        ]

    class MyTag(Tag):

        foo = Argument()

        bar = Argument()

the second case we considered was a tag which has one required, positional,

argument, and two optional, keyword arguments, which can occur in any order,

followed by a final, optional keyword argument, meaning any of the following

invocations are valid:

    {% mytag src limit 1 offset 10 %}

    {% mytag src limit 1 offset 10 with foo %}

    {% mytag src limit 1 %}

    {% mytag src offset 10 limit 1 %}

    {% mytag src %}

and the two implementations were:

    class MyTag(Tag):

        args = [

            Argument("source"),

            Unordered(

                NamedArgument("limit", required=False),

                NamedArgument("offset", required=False),

            ),

            NamedArgument("as", required=False, resolve=False)

        ]

    class MyTag(Tag):

        source = Argument()

        limit = NamedArgument(required=False)

        offset = NamedArgument(required=False)

        as_ = BasicArgument(required=False)

        class Meta:

            ordering = (

                ('source',),

                Unordered('limit', 'offset'),

                ('as_',)

            )

The general consensus was that the second implementation was *very* slightly

preferable in the simple case, however it was significantly more complicated in

the second case, and thus the first implementation was preferable overall.

We notable did *not* discuss what the syntax for defining the output was, at

this stage we were only concerned with the syntax definition. This is because

the actual ``render()`` equivalent will be orthogonal to the parsing stage

equivalent.

For your consideration,

Alex

--
"I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
"The people's good is the highest law." -- Cicero

[Wim Feijen]

Hi Alex,

Probably I am thinking way too simple, but if you gave me a free
choice of how to write templatetags in my code, I would prefer:

def mytag(foo, bar):
# some code
return output

or:

class MyTag(Tag):
def __init__(self, foo, bar):
self.foo = foo
self.bar = bar

and

def mytag(src, limit=1, offset=10, ??)

or:

class MyTag(Tag):
def __init__(self, limit=1, offset=10, ??):
self.limit = limit
self.offset = offset

def render(self):
"""do some nasty stuff to limit and offset and return something."""

Just kick me if I am talking non-sense, that's ok.

Wim

[Alex Gaynor]

--
You received this message because you are subscribed to the Google Groups "Django developers" group.
To post to this group, send email to django-d...@googlegroups.com.
To unsubscribe from this group, send email to django-develop...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.

This solution sounds nice, but it simply doesn't work for many examples.  a) It fails to match the template language's understanding of what a node is (they're created once at compilation time, and reused across all invocations) and b) doesn't work with many types of arguments, for example the argument form "{% foo limit 3 %}" where limit is some significant name, passes some argument to this tag, this rather falls over in the case of things like "{% foo limit 3 as bar %}" since as is a keyword in Python, and finally it fails to express things like the sometimes ordered, sometimes unordered nature of arguments.

[Mikhail Korobov]

I gave up defining this template tag with existing template tag libraries:

{% easy_map <address> [<width> <height>] [<zoom>] [using <template_name>] %}

Can positional arguments be optional? Will it be possible to express this syntax?

[Anssi Kääriäinen]

From the department of stupid ideas: Would it be possible to support
the above syntax? I mean, the MyTag definition would be just:

class MyTag(Tag):
syntax = "mytag <address> [<width> <height>] [<zoom>] [using
<template_name>]"

That would be my preferred solution.

There is just the little problem of how to parse the syntax. I don't
know how to do that. If there is any support for this idea, I am
willing to check if this is possible.

- Anssi

[lucky]

Hello, Alex,

I agree with Wim Feijen. The ability to implement a arbitrary syntax
for tag parameters is too abstract problem and this causes an increase
in the complexity of the solution. Tag delelopers are forced to define
a grammar and deal with lexing and parsing. End users must to learn
and remember a custom syntax for each of tags. A similar problem is
solved by developers of python for CLI, and they came up with `http://
docs.python.org/library/argparse.html` library (however, in our case
it is too complicated way, imho).

The writing of templates should be an easy work, with a low threshold
to entry. It would be nice if template tags will provide us with some
common basic syntax for most of them. Without the need to develop a
grammar and to RTFM a tag parameters syntax when we are need to use
them. The description of the custom syntax have be resorted to only
when absolutely necessary.

We should not be afraid of the python-like grammar for tag
paramenters. There is not principal difference beetween custom {%
mytag src limit 1 offset 10 %} and pythonic {% mytag src limit=1
offset=10 %}. The restrict syntax makes the solution simpler.

The saving of results in context variable looks like a common task and
it should be delegated to an external tool (like a filter {% mytag src|
as:bar %}).

There are just few cases where custom syntaxis is nesessary. For
example the logic and mathematical expressions with rich grammar
rules. But simplest way in those cases is to use specialized
libraries, there problems of grammar definition, lexing and parsing
already solved. Look http://pyparsing.wikispaces.com/ for example.

Evgeny

--
I'm sorry for my silly english


On Sep 11, 2:11 am, Alex Gaynor <alex.gay...@gmail.com> wrote:

[Jonas H.]

On 09/12/2011 10:47 AM, lucky wrote:
> Hello, Alex,
>
> I agree with Wim Feijen. The ability to implement a arbitrary syntax
> for tag parameters is too abstract problem and this causes an increase
> in the complexity of the solution. Tag delelopers are forced to define
> a grammar and deal with lexing and parsing. End users must to learn

> and remember a custom syntax for each of tags. [...]

>
> We should not be afraid of the python-like grammar for tag
> paramenters. There is not principal difference beetween custom {%
> mytag src limit 1 offset 10 %} and pythonic {% mytag src limit=1
> offset=10 %}. The restrict syntax makes the solution simpler.

+1

Template tags should be forced to have a simple syntax. Users should not
have to look up the syntax each time.

[Cal Leeming [Simplicity Media Ltd]]

I may have misunderstood this thread slightly but, should templatetags follow the same structure as *args, **kwargs??

i.e.

*args 

{% create_title "hello world" 800 800 %}

**kwargs

{% create_title title="helloworld" width=800 height=800 %}

mixture

{% create_title "helloworld" width=800 height=800 %}

Apologies if I have completely missed the point of this thread.

Cal

--

You received this message because you are subscribed to the Google Groups "Django developers" group.

To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to django-developers+unsubscribe@googlegroups.com.

[Jonas H.]

On 09/12/2011 11:49 AM, Cal Leeming [Simplicity Media Ltd] wrote:
> I may have misunderstood this thread slightly but, should templatetags
> follow the same structure as *args, **kwargs??
>
> i.e.
>
> *args
> {% create_title "hello world" 800 800 %}
>
> **kwargs
> {% create_title title="helloworld" width=800 height=800 %}
>
> mixture
> {% create_title "helloworld" width=800 height=800 %}
>
> Apologies if I have completely missed the point of this thread.
>
> Cal

I guess it's not the "point of this thread" but rather Evgeny's design
proposal. Alex' intention seems to be discussion on a new syntax to
define template tags.

[Jannis Leidel]

Hi all,

> For many years, writing templatetags has been among the most hilariously
> complicated things we Django developers did. Anyone who has written parsing for
> templatetags, over and over, shares this pain. Further, the current syntax
> present a tremendous problem for Armin Ronacher's GSOC towards template
> compilation: template tags define a ``render()`` method, which takes the
> current context (a stack of dictionaries) and perform some behavior which is
> opaque to the caller. This is a problem because one of the core objectives of
> the template compilation infrastructure is to store the template context in
> Python local variables, rather than in dictionaries. For all these reasons,
> several of us (myself, Idan, Russ, Carl Meyer, Andrew Godwin, Jonas Obrist,
> Chris Beaven) just sat down (we actually stood, mostly) and tried to iron out a
> proposal that solves these problems, taking inspiration from the plethora of
> libraries that exist today.
>
> We ultimately created two possible solutions, one inspired primarily by
> django-classy-tags and django-templatetag-sugar, the other mostly inspired by
> django-ttags.

FTR, the app is called django-ttag, not django-ttags.

I'd like to express my sincere discomfort with the two example cases you used
to develop the API. At least from the experience I have with writing template
tags (with the standard API, @simple_tag, django-template-sugar,
django-classy-tags and django-tagcon/ttag) it was the simple use cases that
were more common, rather than the complex in day-to-day work. While one could
say that this has something to do with my coding style and clients, I'd have
to say that keeping the template tag API simple instead of complex has always
been a major goal. So unless there was a decision to start encouraging moving
business logic from the views to template tags (which would be indicated by
increased complexity) I don't see a reason to prefer the API that allows
complex API.

As a matter of remote contribution to the discussion at the sprint, I've just
checked if django-ttag (the example which you preferred for the simple case)
is capable of passing your complex case, too (yes):

https://github.com/jezdez/django-ttag/compare/django-tag-api

I'm happy that there was some discussion at the conference sprint though,
since it's an important topic, but I'd appreciate if you'd provide further
explanation of why you prefer the first, django-classy-tag and
django-template-sugar influenced example.

Since I've also worked on django-ttag (including when it was still called
django-tagcon a long time ago) quite a bit, I assume you mostly discussed how
to handle ordering the different kinds of arguments (positional vs. keyword
vs. "named" arguments). Luckily we already have a pretty strong convention for
that, as you can see when specifying forms or models. Which is why I'm
confused why an "args" class attribute would be needed in your preferred example.
Any elaboration as to why this verbose syntax is needed would be appreciated.

Jannis

[Donald Stufft]

I'll +1 the restriction of template tags to being arg/kwarg like. I see no reason, other then porting already written tags, to worry about the ability to do ``{% mytag foo as bar %}``. Personally I would think it would be desirable to make this match python as much as possible. Python programmers will find it more familiar then attempting to turn parameter passing into a parsing situation, and non python programmers will find it simple enough learn quickly. Being a restricted grammar, it also has the benefit to have less surprises, you don't have to worry about how this random tag works, there's one (obvious) way of doing it.

Something like

class MyTag(tags.Tag):

    width = tag.IntegerArgument()

    height = tag.IntegerArgument()

    using = tag.StringArgument(required=False)

    as = tag.StringArgument(required=False, default=None)

which would translate to:

``{% mytag 600 700 %}``

``{% mytag 600 700 as="mytag_return" %}``

``{% mytag 600 700 "template_name.html" %}``

``{% mytag 600 600 using="template_name.html" as="mytag_return" %}``

The only thing I really don't like about this is the ``as`` kwarg. We could stuff a boolean into a Meta class that is something like ``allow_assignment`` which will support something more like what we have now…

``{% mytag 600 700 as "mytag_return" %}``

``{% mytag 600 700 "template_name.html" as "mytag_return" %}``

``{% mytag 600 700 using="template_name.html" as "mytag_return" %}``

With the stipulation that the as [keyword] must appear as the last item in the tag.

--
You received this message because you are subscribed to the Google Groups "Django developers" group.

To post to this group, send email to django-d...@googlegroups.com.
To unsubscribe from this group, send email to django-develop...@googlegroups.com.

[LeeS]

Not sure if this is still an ongoing discussion, but I prefer to write
all new tags with a simple syntax that matches Python. There's no
real advantage in defining a unique, custom syntax for a template
tag.

All HTML authors are already familiar with Python's kwargs syntax,
since it exactly matches how HTML tags and attributes themselves
work. There's no great leap of difficulty going from:

<div class="something" style="something">
to
{% tag foo="something" bar="something" %}

Allowing for arbitrary custom syntax makes tags harder for HTML
coders, since they have to remember these special custom syntaxes
which they're not used to.

> > To post to this group, send email to django-d...@googlegroups.com (mailto:django-d...@googlegroups.com).
> > To unsubscribe from this group, send email to django-develop...@googlegroups.com (mailto:django-develop...@googlegroups.com).

[Julien Phalip]

On 13 October 2011 15:30, LeeS <lse...@gmail.com> wrote:

Not sure if this is still an ongoing discussion, but I prefer to write
all new tags with a simple syntax that matches Python.  There's no
real advantage in defining a unique, custom syntax for a template
tag.

All HTML authors are already familiar with Python's kwargs syntax,
since it exactly matches how HTML tags and attributes themselves
work.  There's no great leap of difficulty going from:

<div class="something" style="something">
to
{% tag foo="something" bar="something" %}

Allowing for arbitrary custom syntax makes tags harder for HTML
coders, since they have to remember these special custom syntaxes
which they're not used to.

Note that since the recent r16908 [1], this is easily achievable with simple_tag, include_tag or assignment_tag, which now replicate the Python way of handling *args and **kwargs.

The only thing that's still not possible with these is to define a complex or custom syntax, e.g.: {% mytag with myvar using "blah" %}

However, if one is OK to not have a custom syntax, then the following is now dead-easy:

{% mytag with=myvar using="blah" %}

@simple_tag

def mytag(with=None, using="Yadi Yada"):

    ...

Cheers,

Julien

[1] https://code.djangoproject.com/changeset/16908