Hi all,
regarding issue #2594 "Template system should handle whitespace
better" explaining the need for some kind of (optional) whitespace
stripping from the output of templates, I've been looking at the
proposed solutions and compared them to other template and macro
engines.
In particular, a historical but still widely used and general-purpose
macro engine is m4. Its own feature for controlled newline stripping
is the "dnl" reserved word, "Discard to Next Line." It works by
"chomping" all input until and including the next newline.
For example, to call a macro foo() without copying over the newline
that appears after the macro call in the template, a m4 programmer
would write:
foo('bar`, `whatever')dnl
An equivalent feature in Django templates would enable template
developers to strip newlines from specific lines, while keeping
backwards compatibility with existing templates.
So if the general idea is well-accepted, I propose the "{#" token. The
example from the issue would become:
<ul>
{% for item in items %}{#
<li>{{ item }}</li>
{% endfor %}{#
</ul>
It is already a reserved token in the template system, so it's not
going to break anything. The existing comment syntax {# ... #} is
already restricted to single-line comments, so there are no multi-line
comments that would break. Plus, I'd wager it could be implemented
quite efficiently.
I definitely want to see a resolution to this issue. Django templates
when white space matters (e.g. plain text emails) are painful. But, I
don't think adding {# to the end of lines is easy to understand at a
glance, it doesn't even convey a hint of meaning like "dnl" does, but
either option is pretty ugly to my eyes.
I'd personally like to see a new template tag that will switch on the
new behaviour, and a deprecation path for this to become the default.
Cheers.
Tai.
On Feb 25, 1:10 am, Tobia <tobia.confo...@gruppo4.eu> wrote:
> Hi all,
> regarding issue #2594 "Template system should handle whitespace
> better" explaining the need for some kind of (optional) whitespace
> stripping from the output of templates, I've been looking at the
> proposed solutions and compared them to other template and macro
> engines.
> In particular, a historical but still widely used and general-purpose
> macro engine is m4. Its own feature for controlled newline stripping
> is the "dnl" reserved word, "Discard to Next Line." It works by
> "chomping" all input until and including the next newline.
> For example, to call a macro foo() without copying over the newline
> that appears after the macro call in the template, a m4 programmer
> would write:
> foo('bar`, `whatever')dnl
> An equivalent feature in Django templates would enable template
> developers to strip newlines from specific lines, while keeping
> backwards compatibility with existing templates.
> So if the general idea is well-accepted, I propose the "{#" token. The
> example from the issue would become:
> <ul>
> {% for item in items %}{#
> <li>{{ item }}</li>
> {% endfor %}{#
> </ul>
> It is already a reserved token in the template system, so it's not
> going to break anything. The existing comment syntax {# ... #} is
> already restricted to single-line comments, so there are no multi-line
> comments that would break. Plus, I'd wager it could be implemented
> quite efficiently.
On Fri, Feb 24, 2012 at 2:10 PM, Tobia <tobia.confo...@gruppo4.eu> wrote: > Hi all, > regarding issue #2594 "Template system should handle whitespace > better" explaining the need for some kind of (optional) whitespace > stripping from the output of templates, I've been looking at the > proposed solutions and compared them to other template and macro > engines.
> In particular, a historical but still widely used and general-purpose > macro engine is m4. Its own feature for controlled newline stripping > is the "dnl" reserved word, "Discard to Next Line." It works by > "chomping" all input until and including the next newline.
> For example, to call a macro foo() without copying over the newline > that appears after the macro call in the template, a m4 programmer > would write:
> foo('bar`, `whatever')dnl
> An equivalent feature in Django templates would enable template > developers to strip newlines from specific lines, while keeping > backwards compatibility with existing templates.
> So if the general idea is well-accepted, I propose the "{#" token. The > example from the issue would become:
> <ul> > {% for item in items %}{# > <li>{{ item }}</li> > {% endfor %}{# > </ul>
> It is already a reserved token in the template system, so it's not > going to break anything. The existing comment syntax {# ... #} is > already restricted to single-line comments, so there are no multi-line > comments that would break. Plus, I'd wager it could be implemented > quite efficiently.
> What do you think?
> Tobia
I'd be strongly -1 on anything that makes template language look more like m4!
Having said that, I can see this being useful. We use templates to render most kinds of output. When we render text emails, the newlines and indentation are particularly important, as everything gets displayed 'as-is' by the client, and having a template that is readable and also renders correctly is tricky.
Generally where this happens is where we open a block level tag, and would then normally have a newline for readability, but cannot due to us not wanting a newline to appear. This could be addressed by having a different open/close tag for tags which chomp the preceeding/next character if it is a newline. Eg:
In your folder are: {^ for item in folder ^} {{ item }}, {^ endfor ^}
This would render as "In your folder are: item1,item2,item3,"
Changing some of the carets:
In your folder are: {% for item in folder ^} {{ item }}, {^ endfor %}
This would render as "In your folder are:\nitem1,item2,item3,\n"
On Fri, Feb 24, 2012 at 9:45 AM, Tom Evans <tevans...@googlemail.com> wrote: > In your folder are: > {^ for item in folder ^} > {{ item }}, > {^ endfor ^}
> This would render as "In your folder are: item1,item2,item3,"
> Changing some of the carets:
> In your folder are: > {% for item in folder ^} > {{ item }}, > {^ endfor %}
> This would render as "In your folder are:\nitem1,item2,item3,\n"
Fwiw, I had a different solution to this problem once, which I implemented in a project using jinja to generate configuration files:
1. Prior to processing the template, replace all blank lines with a tag (e.g. "BLANKLINE").
2. After processing the template, remove all blank lines.
3. Remove all instances of BLANKLINE, which restores the blank lines the author intended.
This works remarkably well, and the output is easy to predict by the template author. For the case you mentioned above, you would still need to join the lines, though:
Tai Lee wrote:
> I don't think adding {# to the end of lines is easy to understand at a
> glance, it doesn't even convey a hint of meaning like "dnl" does
I beg to differ. {# is already recognized by template authors as
meaning "start of comment", and they know (or should know) that it
cannot extend through more than one line. Therefore I'd think it
intuitive that it will "eat" till the end of the line and not beyond.
Look:
Here are your subscriptions:
{% for thing in things %}{#
- {{ thing.name }}
You added it on {{ thing.date }}
{% endfor %}{#
you can manage your subscriptions...
Tom Evans wrote:
> I'd be strongly -1 on anything that makes template language look more
> like m4!
I'll tell you, m4 can be quite addictive once you grasp the basics! :)
> This could be addressed by having a different open/close tag for tags
> which chomp the preceeding/next character if it is a newline. Eg:
> {^ for item in folder ^}
I don't think adding new reserved characters would make the language
simpler for template authors, nor for the the template parser, nor for
the sake of backwards compatibility. {# is already part of it.
But I can see the need to chomp whitespace before a template tag, as
well as the newline after it.
Martin J. Laubach wrote:
> For this (avoiding newlines) the currently discussed multiline tags
> would work pretty well too without adding more cruft to the template
>language:
> foo bar {#
> #}baz
If this can be accomplished without massive performance penalties, I
agree with you.
Maybe {# #} could be made multiline at first, with special code, while
waiting for a proper implementation of generic multiline tags. This
would certainly be more forward-compatible than my proposal above
and it would solve more whitespace problems, if not all.
Jinja implements whitespace control by putting a minus sign after/before the % in a tag - http://jinja.pocoo.org/docs/templates/#whitespace-control - I haven't tried it myself, but it looks like {% tag -%} is equivalent to your {% tag %}{# .
On 24 February 2012 17:37, Tobia <tobia.confo...@gruppo4.eu> wrote:
> Tai Lee wrote: > > I don't think adding {# to the end of lines is easy to understand at a > > glance, it doesn't even convey a hint of meaning like "dnl" does
> I beg to differ. {# is already recognized by template authors as > meaning "start of comment", and they know (or should know) that it > cannot extend through more than one line. Therefore I'd think it > intuitive that it will "eat" till the end of the line and not beyond.
> Look:
> Here are your subscriptions: > {% for thing in things %}{# > - {{ thing.name }} > You added it on {{ thing.date }} > {% endfor %}{# > you can manage your subscriptions...
> Tom Evans wrote: > > I'd be strongly -1 on anything that makes template language look more > > like m4!
> I'll tell you, m4 can be quite addictive once you grasp the basics! :)
> > This could be addressed by having a different open/close tag for tags > > which chomp the preceeding/next character if it is a newline. Eg: > > {^ for item in folder ^}
> I don't think adding new reserved characters would make the language > simpler for template authors, nor for the the template parser, nor for > the sake of backwards compatibility. {# is already part of it.
> But I can see the need to chomp whitespace before a template tag, as > well as the newline after it.
> Martin J. Laubach wrote: > > For this (avoiding newlines) the currently discussed multiline tags > > would work pretty well too without adding more cruft to the template > >language:
> > foo bar {# > > #}baz
> If this can be accomplished without massive performance penalties, I > agree with you.
> Maybe {# #} could be made multiline at first, with special code, while > waiting for a proper implementation of generic multiline tags. This > would certainly be more forward-compatible than my proposal above > and it would solve more whitespace problems, if not all.
> Tobia
> -- > You received this message because you are subscribed to the Google Groups > "Django developers" group. > To post to this group, send email to django-developers@googlegroups.com. > To unsubscribe from this group, send email to > django-developers+unsubscribe@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/django-developers?hl=en.
On Friday, February 24, 2012 7:48:29 PM UTC+1, ojno wrote:
> Jinja implements whitespace control by putting a minus sign after/before > the % in a tag - http://jinja.pocoo.org/docs/templates/#whitespace-control - > I haven't tried it myself, but it looks like {% tag -%} is equivalent to > your {% tag %}{# .
Yes it is, and why nicer and more readable imo, using {# #} to achieve that effect is a horrible idea imo.
On Friday, February 24, 2012 11:47:21 PM UTC+1, Florian Apolloner wrote:
> Hi,
> On Friday, February 24, 2012 7:48:29 PM UTC+1, ojno wrote:
>> Jinja implements whitespace control by putting a minus sign after/before >> the % in a tag - >> http://jinja.pocoo.org/docs/templates/#whitespace-control - I haven't >> tried it myself, but it looks like {% tag -%} is equivalent to your {% tag >> %}{# .
> Yes it is, and why nicer and more readable imo, using {# #} to achieve > that effect is a horrible idea imo.
Adding more symbols to existing tags (e.g. {^ for x in y ^} or {% for
x in y -%}), multi-line comment tags that don't actually include a
comment, and half baked comment tags (where the closing tag is
assumed) are all going to make templates uglier, and harder to read.
The comment tag based solutions don't even solve the problem
completely, because leading white space is still there, and users are
still required to carefully position comment tags all around block
tags to remove whitespace.
What's better about {% for x in y -%} or {^ for x in y ^} over {%
stripwhitespace on %} at the top of your template, followed by {% for
x in y %}? I think the latter is far more readable.
My ideal solution is not to add new ways to mark each individual
template tag that should have surrounding white space stripped, but to
simply enable the removal of lines that have only block tags and no
actual content. 99% of the time this is the right thing to do, and we
just need a deprecation path and new template tag so that template
authors can opt-in to the new behaviour now.
Cheers.
Tai.
On Feb 25, 4:37 am, Tobia <tobia.confo...@gruppo4.eu> wrote:
> Tai Lee wrote:
> > I don't think adding {# to the end of lines is easy to understand at a
> > glance, it doesn't even convey a hint of meaning like "dnl" does
> I beg to differ. {# is already recognized by template authors as
> meaning "start of comment", and they know (or should know) that it
> cannot extend through more than one line. Therefore I'd think it
> intuitive that it will "eat" till the end of the line and not beyond.
> Look:
> Here are your subscriptions:
> {% for thing in things %}{#
> - {{ thing.name }}
> You added it on {{ thing.date }}
> {% endfor %}{#
> you can manage your subscriptions...
> Tom Evans wrote:
> > I'd be strongly -1 on anything that makes template language look more
> > like m4!
> I'll tell you, m4 can be quite addictive once you grasp the basics! :)
> > This could be addressed by having a different open/close tag for tags
> > which chomp the preceeding/next character if it is a newline. Eg:
> > {^ for item in folder ^}
> I don't think adding new reserved characters would make the language
> simpler for template authors, nor for the the template parser, nor for
> the sake of backwards compatibility. {# is already part of it.
> But I can see the need to chomp whitespace before a template tag, as
> well as the newline after it.
> Martin J. Laubach wrote:
> > For this (avoiding newlines) the currently discussed multiline tags
> > would work pretty well too without adding more cruft to the template
> >language:
> > foo bar {#
> > #}baz
> If this can be accomplished without massive performance penalties, I
> agree with you.
> Maybe {# #} could be made multiline at first, with special code, while
> waiting for a proper implementation of generic multiline tags. This
> would certainly be more forward-compatible than my proposal above
> and it would solve more whitespace problems, if not all.
On Saturday, February 25, 2012 5:27:43 AM UTC+1, Tai Lee wrote:
> Adding more symbols to existing tags (e.g. {^ for x in y ^} or {% for > x in y -%}), multi-line comment tags that don't actually include a > comment, and half baked comment tags (where the closing tag is > assumed) are all going to make templates uglier, and harder to read.
"-" is only supposed to be used in some edge cases where it's really important that you don't have whitespace, usually there is no reason to care for whitespace at all -- so it shouldn't make templates that ugly if you don't use it everywhere.
{% stripwhitespace on %} at the top of your template, followed by {% for
> x in y %}? I think the latter is far more readable.
Usually I don't want to strip everything, but just at one location.
> My ideal solution is not to add new ways to mark each individual > template tag that should have surrounding white space stripped, but to > simply enable the removal of lines that have only block tags and no > actual content. 99% of the time this is the right thing to do, and we > just need a deprecation path and new template tag so that template > authors can opt-in to the new behaviour now.
And what about the other 1%, there are even cases where whitespace is important ;) Either way, why is stripping whitespace the right thing to do? Given my experience "I don't care if there is or not" is what's 99% true of the time…
> On Saturday, February 25, 2012 5:27:43 AM UTC+1, Tai Lee wrote:
> > Adding more symbols to existing tags (e.g. {^ for x in y ^} or {% for
> > x in y -%}), multi-line comment tags that don't actually include a
> > comment, and half baked comment tags (where the closing tag is
> > assumed) are all going to make templates uglier, and harder to read.
> "-" is only supposed to be used in some edge cases where it's really
> important that you don't have whitespace, usually there is no reason to
> care for whitespace at all -- so it shouldn't make templates that ugly if
> you don't use it everywhere.
> {% stripwhitespace on %} at the top of your template, followed by {% for
> > x in y %}? I think the latter is far more readable.
> Usually I don't want to strip everything, but just at one location.
> > My ideal solution is not to add new ways to mark each individual
> > template tag that should have surrounding white space stripped, but to
> > simply enable the removal of lines that have only block tags and no
> > actual content. 99% of the time this is the right thing to do, and we
> > just need a deprecation path and new template tag so that template
> > authors can opt-in to the new behaviour now.
> And what about the other 1%, there are even cases where whitespace is
> important ;) Either way, why is stripping whitespace the right thing to do?
> Given my experience "I don't care if there is or not" is what's 99% true of
> the time…
In most situations white space matters:
{{ user.lastname }} {{ user.firstname }}
On Saturday, February 25, 2012 10:04:21 AM UTC+1, Anssi Kääriäinen wrote:
> In most situations white space matters: > {{ user.lastname }} {{ user.firstname }}
Right, but
"""
{{ user.lastname }} {{ user.firstname }}
"""
would have produced exactly the same output in HTML, hence my statement that you usually don't have to care about it that much. Eg, you need to make sure that there is whitespace but it doesn't matter how much since it's collapsed anyways (and outside of tags it doesn't matter at all…)
> On Saturday, February 25, 2012 10:04:21 AM UTC+1, Anssi Kääriäinen wrote:
> > In most situations white space matters:
> > {{ user.lastname }} {{ user.firstname }}
> Right, but
> """
> {{ user.lastname }}
> {{ user.firstname }}
> """
> would have produced exactly the same output in HTML, hence my statement
> that you usually don't have to care about it that much. Eg, you need to
> make sure that there is whitespace but it doesn't matter how much since
> it's collapsed anyways (and outside of tags it doesn't matter at all…)
I was just illuminating why putting {% stripwhitespace on %} in the
beginning of the template might not be that great an idea. Even with
the line change between the first and lastname they would be
concatenated if all whitespace is stripped.
I think we agree on this issue. Maybe I should have quoted the
original poster of the idea instead...
I think this discussion is focusing on template tags, not template variables. Maybe even a subset of template tags (e.g. block level tagsif, for, block, filter, etc). Template variables and inline tags (e.g. now) shouldn't have white space stripped.
In 100% of cases I can think of I either wouldn't care (in HTML templates) or I would want lines that only contain block level tags and no actual content removed from the rendered output of the template.
The only time I would explicitly NOT want this to happen is when I have an existing template where white space matters and it has been carefully crafted to work around Django's white space issues. I think SmileyChris' solution from the ticket is the way to go, but in a backwards compatible way. Make it opt-in with a new template tag, similar to the way auto escaping is handled, and optionally a deprecation path that will make this the default eventually.
I don't think anyone has suggested stripping ALL white space around ALL template tags, or even ANY template variables.
On Saturday, 25 February 2012 21:04:26 UTC+11, Florian Apolloner wrote:
> Hi,
> On Saturday, February 25, 2012 10:04:21 AM UTC+1, Anssi Kääriäinen wrote:
>> In most situations white space matters: >> {{ user.lastname }} {{ user.firstname }}
> Right, but
> """
> {{ user.lastname }} > {{ user.firstname }}
> """
> would have produced exactly the same output in HTML, hence my statement > that you usually don't have to care about it that much. Eg, you need to > make sure that there is whitespace but it doesn't matter how much since > it's collapsed anyways (and outside of tags it doesn't matter at all…)
> Cheers,
> Florian
On Saturday, 25 February 2012 21:04:26 UTC+11, Florian Apolloner wrote:
> Hi,
> On Saturday, February 25, 2012 10:04:21 AM UTC+1, Anssi Kääriäinen wrote:
>> In most situations white space matters: >> {{ user.lastname }} {{ user.firstname }}
> Right, but
> """
> {{ user.lastname }} > {{ user.firstname }}
> """
> would have produced exactly the same output in HTML, hence my statement > that you usually don't have to care about it that much. Eg, you need to > make sure that there is whitespace but it doesn't matter how much since > it's collapsed anyways (and outside of tags it doesn't matter at all…)
> Cheers,
> Florian
On Saturday, 25 February 2012 21:04:26 UTC+11, Florian Apolloner wrote:
> Hi,
> On Saturday, February 25, 2012 10:04:21 AM UTC+1, Anssi Kääriäinen wrote:
>> In most situations white space matters: >> {{ user.lastname }} {{ user.firstname }}
> Right, but
> """
> {{ user.lastname }} > {{ user.firstname }}
> """
> would have produced exactly the same output in HTML, hence my statement > that you usually don't have to care about it that much. Eg, you need to > make sure that there is whitespace but it doesn't matter how much since > it's collapsed anyways (and outside of tags it doesn't matter at all…)
>> You just made a strong argument against multiline tags: I wouldn't want > to see them abused this way!
Ah, I fully expected to be shot down on "aesthetic" reasons. I still think it's better to have some (maybe slightly ugly) way to do things than no way and revelling in perceived beauty, and if it's as easy as the above, well, then so be it.
Let me make sure I've got this right --- The situation being discussed is not where whitespace is insignificant and can be stripped, but where whitespace is important and you want to control the exact amount of it, e.g. plain text emails. In this case, just using stripwhitespace is not enough. Right?
On 25 February 2012 04:27, Tai Lee <real.hu...@mrmachine.net> wrote:
> What's better about {% for x in y -%} or {^ for x in y ^} over {% > stripwhitespace on %} at the top of your template, followed by {% for > x in y %}? I think the latter is far more readable.
Jonathan French wrote:
> Let me make sure I've got this right --- The situation being discussed is
> not where whitespace is insignificant and can be stripped, but where
> whitespace is important and you want to control the exact amount of it,
> e.g. plain text emails. In this case, just using stripwhitespace is not
> enough. Right?
Yes, that is the main problem.
Django templates are useful (and used) for all sorts of text-based
formats. For some of these, an exact control over whitespace is needed
throughout the entire file (eg. text/plain), while for others, control
is needed in some sensitive places (eg. element attributes in html and
xml, which according to standars are not allowed to contain literal
newlines.)
The only option currently available to control whitespace is *not to
use any* for the template syntax itself, in effect writing long one-
liners. The admin site contains several examples of such:
<div class="inline-related{% if forloop.last %} empty-form last-
related{% endif %}" id="{{ inline_admin_formset.formset.prefix }}-{%
if not forloop.last %}{{ forloop.counter0 }}{% else %}empty{% endif
%}">
Yes, in this case whitespace could be added between the class="" and
id="" attributes, but the main issue remains there.
Multiline comments {# #} and/or generic multiline tags would offer a
more workable alternative than the present situation, allowing for
syntax-only whitespace in templates. But they admittedly suffer from
some serious ugliness and "denaturation" of Django templates.
After reading all the above, I can see the arguments against this:
This *will* be abused the instant it gets pushed, even if it's done on
good intentions, such as allowing for multiline {% trans %}, in effect
taking the beauty and readability out of the template language.
But the main issue is still there, with its huge one-liners and other,
even less readable workarounds. I think a new idea is needed,
something more fine-grained than stripwhitespace and less denaturating
than multiline comments. (Yes, I'm officially recognizing my original
idea as flawed. Sorry for that. :-)
Agreed. Seeing that just changed me from +1 to -0 for multiline tags.
Although the multi-line comment syntax isn't that bad, it is at least
clear what it does and why.
Now that I look at it, the main problem is that you want the <div>
appear to be on one line in the HTML. So, it should be one-liner in
the template too, or otherwise it will look strange. However, dealing
with div tags isn't the hard case, it is the email template. Django
doesn't have a good answer for that. Multi-variable include/with/
blocktrans tags are a problem too.
When reading the latest template-related threads on this list it is
pretty clear that there is no right answer for template design
philosophy questions. For example I use mainly Jinja2 for two reasons.
I need to generate largish reports, and Jinja2 beats Django's template
engine in performance by an order of magnitude. In addition, Jinja's
design philosophy fits my needs better than Django's. But, I do not
claim Jinja2 is the right answer for everybody.
We will just need to accept that it is impossible to create a template
language that fits everybody's needs perfectly. If you want something
different, use something different. I think good ideas how to make
life easier for external template engine users are welcome.
> Django templates are useful (and used) for all sorts of text-based > formats. For some of these, an exact control over whitespace is needed > throughout the entire file (eg. text/plain),
Agreed. When I started using Django templates I was very surprised at the output I was seeing.
After a while I realised that the thing that suprised me was that newlines were being output for what I considered to be just 'directives' -- that is, lines that contained tags, not content. For example, {% if %} and {% endif %} tags on lines by themselves.
Honestly, my use case then was to make my HTML look pretty, so was able to let it go pretty easily. Recently however I used the template system to produce an email with a textual pricing table, and there the lack of easy whitespace control hurt there.
Would it be feasible to add some logic, something along the lines of:
"Template lines containing just tags with no literal content do not produce a line in the output (unless of course the tag itself produces one)"
After all, new lines are easy to add, but currently painful to suppress...
The ticket is still accepted (and is now 6 years old), and the
consensus from this thread now appears to be that the "dnl" and multi-
line comment solutions put forward here are not preferred.
I think the next step is for any interested parties to look at the
latest patches on that ticket, update them for trunk and make sure
they still work, have the required tests and/or docs, and try to get
it committed.
That's... quite fantastic. A actual patch for the exactly the behaviour I had in mind.
Why was it not commited for 1.3 or 1.4? From my perspective it's a big win, and reading the bug notes, the issues -- backwards combatibility, docs, tests -- seem to have been addressed.
It looks like a lot of work looks went into this patch, why did it stall?
> It looks like a lot of work looks went into this patch, why did it stall?
Same reason any ticket stalls - it seems that nobody felt strongly enough about it to put the time into reviewing and thoroughly testing the patch and marking it Ready for Checkin. If you'd like to see it in (post 1.4 at this point, of course), feel free to do that!
> Same reason any ticket stalls - it seems that nobody felt strongly > enough about it to put the time into reviewing and thoroughly testing > the patch and marking it Ready for Checkin. If you'd like to see it in > (post 1.4 at this point, of course), feel free to do that!
Fair enough. I realised after posting that my question was particularly poorly timed, given where we are in the release cycle. I apologise for that -- I was overcome with excitement to find that somebody had already written the patch I wanted!
