Autoescaping: good time?
Michael Radziej
since the unicode waves have calmed down, wouldn't it be a nice time to
include the autoescaping patch?
I've brought most of the patches in ticket #2359 up to date. There are 3
patches, the first two deal with implementing autoescape as discussed and
based on original ideas from Simon Willison, the third rewrites the admin
interface to take use of it.
It doesn't make sense to put work in the third patch since the admin rewrite
isn't done, but the other patches could be applied independently. Actually,
I think it would be a good idea to first put in the autoescaping, see if
there are any issues overlooked, and only then make the admin interface use
it. Auto-escaping is off by default, so there shouldn't be any compatibility
issues at all.
As you can see from the history, I use this patch for quite some time in
production. Last comment from Malcolm (thanks for this great patch!), the
patch author, asked Adrian for a review.
http://code.djangoproject.com/ticket/2359
Cheers,
Michael
--
noris network AG - Deutschherrnstraße 15-19 - D-90429 Nürnberg -
Tel +49-911-9352-0 - Fax +49-911-9352-100
http://www.noris.de - The IT-Outsourcing Company
Vorstand: Ingo Kraupa (Vorsitzender), Joachim Astel, Hansjochen Klenk -
Vorsitzender des Aufsichtsrats: Stefan Schnabel - AG Nürnberg HRB 17689
Jacob Kaplan-Moss
> since the unicode waves have calmed down, wouldn't it be a nice time to
> include the autoescaping patch?
Malcolm, Simon and I talked about this at OSCON, and I'm now
convinced. I'd like to have Malcolm look over your work, Michael, and
make any changes he wants; from there lets get it done.
Jacob
Michael Radziej
Thanks for the quick feedback! During the various merges it got touched
quite a bit, so I'd consider a review essential. But it's still Malcolm's
work. I only merged and the bugs are properly mine ;-)
Tom Tobin
> issues at all.
I felt my hackles rising until I read this part. ^_^ I'm somewhere
between -0 and +0 at this point so long as auto-escaping is off by
default (and -1 otherwise, as per my objections in previous
discussions on the topic). I still don't like it, but I have the
feeling this is *never* going to go away. :P
Malcolm Tredinnick
On Wed, 2007-08-01 at 16:51 +0200, Michael Radziej wrote:
> On Wed, Aug 01, Jacob Kaplan-Moss wrote:
>
> >
> > On 8/1/07, Michael Radziej <m...@noris.de> wrote:
> > > since the unicode waves have calmed down, wouldn't it be a nice time to
> > > include the autoescaping patch?
> >
> > Malcolm, Simon and I talked about this at OSCON, and I'm now
> > convinced. I'd like to have Malcolm look over your work, Michael, and
> > make any changes he wants; from there lets get it done.
>
> Thanks for the quick feedback! During the various merges it got touched
> quite a bit, so I'd consider a review essential. But it's still Malcolm's
> work. I only merged and the bugs are properly mine ;-)
I updated the patch during OSCON and it's pretty much ready to go. I've
held off launching it this week because I'm on the road and in a hotel
that hasn't discovered the Internet yet. So connectivity is a bit
sporadic. Pending the outcome of this thread, I'll drop it in
(defaulting to off for now) when I get back to Sydney next week and can
be around to nurse people through any immediate problems.
The latest set of patches in the ticket were pretty much applicable. I
think I tweaked one thing and have read through it in light of recent
changes to trunk and updated the docs a bit. The one thing Jacob and I
agreed to change at OSCON was to move to "autoescape on" and "autoescape
off", rather than "autoescape/noautoescape" as the tags. This way, if
somebody solves the impossible and comes up with a way to escape
Javascript in a robust fashion, we can have "{% autoescape js %}" or
similar; it's just a bit more extensible and Jacob made a convincing
argument.
Regards,
Malcolm
James Bennett
> I felt my hackles rising until I read this part. ^_^ I'm somewhere
> between -0 and +0 at this point so long as auto-escaping is off by
> default (and -1 otherwise, as per my objections in previous
> discussions on the topic). I still don't like it, but I have the
> feeling this is *never* going to go away. :P
I don't like it, but I don't think we have a choice: if it's off by
default, the people who need to be using it will never turn it on. But
if it's on by default, the people who think they don't need it will
know enough to turn it off again.
--
"Bureaucrat Conrad, you are technically correct -- the best kind of correct."
Tom Tobin
>
> On 8/1/07, Tom Tobin <kor...@korpios.com> wrote:
> > I felt my hackles rising until I read this part. ^_^ I'm somewhere
> > between -0 and +0 at this point so long as auto-escaping is off by
> > default (and -1 otherwise, as per my objections in previous
> > discussions on the topic). I still don't like it, but I have the
> > feeling this is *never* going to go away. :P
>
> I don't like it, but I don't think we have a choice: if it's off by
> default, the people who need to be using it will never turn it on. But
> if it's on by default, the people who think they don't need it will
> know enough to turn it off again.
I've only had a chance to take a brief look at the latest changes on
that ticket (and I'm seeing a bunch of patches that are showing as
blank in Trac for some reason), so I'm speaking a bit from ignorance
here. I can probably live at -0 with something that lets me set one
global setting, e.g., ``AUTOESCAPE = False``; anything that forces me
to do ``{% autoescape off %}`` in every single base template is going
to make me flip out with the hardest -1 I can muster. :P
Jacob Kaplan-Moss
Unfortunately, AUTOESCAPE=False just is a bad idea; that's asking to
introduce our very own "magic_quotes" feature, and we all know how
well that worked for PHP.
Is autoescaping being on *really* that big a deal to you? How many
places do your variables actually contain HTML? And if the answer is
"many", then is it really all that big a deal to make a one-time
modification to your base templates?
Fact is there isn't a solution that's gonna make everyone happy.
Experienced developers like you and me are gonna hate this since it
feels like making Django not trust us any more.
However, there's a *huge* number of not-so-experienced developers out
there, and we *have* to keep their best needs in mind. Again with the
PHP: we've seen how well trusting developers not to write security
holes works. Anything we can do to prevent newbies from XSS attacks,
we should.
We've seen how bad XSS can be; I think we have an almost moral obligation here.
Jacob
Tom Tobin
mitigation proposal:
Set autoescaping on by default for anything ending in ``.html`` (and,
perhaps, ``.htm``), and off otherwise.
Being (at least ideally) language-neutral has precedent in Django;
we've already moved away from assuming templates end in ``.html``, and
at least part of the reasoning there was acknowledging that the
template language is used for more than just HTML. By having
autoescape off by default for non-HTML templates, one can write
one-off plain-text email templates, server configuration files, LaTeX,
or what-have-you — stuff that doesn't necessarily lend well to
hierarchical template extension — without always having to drop an {%
autoescape off %} line on top.
Is this a bit magical? Yes, of course — but certainly no more so than
having autoescaping on by default, and in fact removes a bit of magic
from non-HTML use cases.
Jacob Kaplan-Moss
> mitigation proposal:
Mmmmm.... Starbucks....
Yes, we had quite a good little argument about all this. It's a tricky
problem; it feels like the only way to get this right is to piss off
anyone not producing HTML, and that kinda blows.
So, given that, I'm pretty happy with Tom's mini-counter-proposal. It
might take some nasty hacking to "remember" where a template was
loaded from (not sure; gotta check), but not having < show up in
".txt" templates would be a Good Thing, methinks.
Malcolm -- think this'll be possible to do in a minimally ugly way?
Jacob
Deryck Hodge
> Is autoescaping being on *really* that big a deal to you? How many
> places do your variables actually contain HTML? And if the answer is
> "many", then is it really all that big a deal to make a one-time
> modification to your base templates?
>
Hi, Jacob.
May I ask a clarifying question here? Is autoescaping going to be on
by default, or no? And what exactly is it that's being added here --
a template tag for turning on/off autoescaping?
I know we've gone round and round about this, so I don't mean to cover
old ground, but I did search the archives and wiki a bit and I'm still
not clear. If it's not on by default, as James said, how does this
make life safer for anyone? Don't get me wrong, I've argued against
autoescaping before, but if it's in the framework why not turn it on
by default? And I guess I don't really see the difference in turning
on/off autoescaping globally in a base template vs. a global setting.
Cheers,
deryck
James Bennett
> May I ask a clarifying question here? Is autoescaping going to be on
> by default, or no? And what exactly is it that's being added here --
> a template tag for turning on/off autoescaping?
The proposal is:
* Autoescaping is on by default.
* A tag will be provided, which will inherit down into child
templates, to turn it off if you desire (so you could turn off
escaping in your base template, and anything inheriting from it would
also have escaping off).
* Maybe -- in Tom's counter-proposal -- this will only apply in
templates whose filenames end with '.html'.
Personally, I don't like the last bit of that, because there are use
cases where you'd be wanting to use HTML in a template that didn't
come from a file whose extension was '.html'.
Jacob Kaplan-Moss
> by default, or no? And what exactly is it that's being added here --
> a template tag for turning on/off autoescaping?
Yes, the plan is to have autoescaping on unless you explicitally turn
it off in your template (or by not using ".html" template file names
if Tom's proposal holds).
What's being added is both the autoescaping behavior -- Django
currently does no automatic escaping whatsoever -- and the mechanism
to turn it off.
I think the current debate has moved away from "should we have
autoescaping" since I think we're all pretty much agreed that *some*
form of automatic escaping is neccisary if we're gonna protect our
users from shooting themselves in the foot.
> I know we've gone round and round about this, so I don't mean to cover
> old ground, but I did search the archives and wiki a bit and I'm still
> not clear. If it's not on by default, as James said, how does this
> make life safer for anyone? Don't get me wrong, I've argued against
> autoescaping before, but if it's in the framework why not turn it on
> by default?
Yup, you said it. If we're gonna do this, we should do it right. I've
been pretty much convinced that we've got to have autoescape, and it's
got to be on by default. It's basically a tradeoff between annoyance
(of having to turn it off when you need it off) and security, and
security has to win that argument.
> And I guess I don't really see the difference in turning
> on/off autoescaping globally in a base template vs. a global setting.
The problem with a global AUTOESCAPE setting is that it makes writing
distributable Django applications impossible.
OK, that came out a bit cryptic, so let me explain:
Back in my days as a PHP developer, I tried to write a handful of
distributable libraries (including a really, really shitty ORM. You
think Django's ORM is weak? You shoulda seen this one...). It isn't
long until I ran into trouble with magic_quotes. I'll save the
heartache, but the short version is that it's actually impossible to
write code that handles magic_quotes defensively. You literally
*can't* write code that works for any input without dictating that
magic_quotes must be on or off.
From a user's point of view, it sucks even harder. So you install that
cool PHP forum everyone's been crowing about. In the process, you turn
off magic_quotes like the docs tell you too. A few months later, you
install new blog software, and start getting weird SQL errors. Oops,
that blog needs magic_quotes on, but if you do that, the forum
breaks...
The lesson here is that code interpretation *can't* be dependent on
environment. If we have a global autoescape setting, we can kiss
easily-distributable Django apps goodbye.
If, however, we turn it off in a base template, we're golden; apps can
simply distribute a base.html that does whatever's right for the
library.
Jacob
Deryck Hodge
> cool PHP forum everyone's been crowing about. In the process, you turn
> off magic_quotes like the docs tell you too. A few months later, you
> install new blog software, and start getting weird SQL errors. Oops,
> that blog needs magic_quotes on, but if you do that, the forum
> breaks...
>
> The lesson here is that code interpretation *can't* be dependent on
> environment. If we have a global autoescape setting, we can kiss
> easily-distributable Django apps goodbye.
>
> If, however, we turn it off in a base template, we're golden; apps can
> simply distribute a base.html that does whatever's right for the
> library.
I guess I was envisioning the work flow where someone sets autescape
off in their site-wide base.html, but if I as an app developer rely on
the behavior being one way and I know to just explicitly set
autoescape on/off at the app-level templates (again assuming this
really matters), I can see the difference.
So I'm cool with all this. I agree it's inevitable and just want the
implementation to be as sane as possible. This seems headed down the
right path to me.
Thanks for the detailed response and clarification!
Cheers,
deryck
Luke Plant
> Okay, post-Starbucks chat with my co-workers, here's my last-ditch
> mitigation proposal:
>
> Set autoescaping on by default for anything ending in ``.html`` (and,
> perhaps, ``.htm``), and off otherwise.
-1 on this, it's much too magic for me. If you actually look at
implementing it, it feels even worse -- you have to modify the
Template.render() method to use information it doesn't even reliably
have (the name of the template -- won't exist for templates from
strings), and then *modify* the context object it is passed in on that
basis...it's nasty.
Also consider cases where someone goes from using a template inline in a
Python file (as I have in a few places in my source code for very small
templates) to having them stored in files. Copying and pasting into a
new file certainly qualifies as a simple refactor in my mind -- I
probably wouldn't bother even testing the contents of the output. It
would be pretty evil if the template could start behaving differently
after doing this.
As for autoescaping on by default, I'm +1 -- I'm a reasonably
experienced Django developer, and sometimes I forget |escape. And
sometimes this mistake has existed in Django admin. The fact that you
can get it right 99% of the time is no argument -- the 1% is enough to
get you totally pwned. The answer to this is the same as the answer to
SQL injection -- 'Never fix a bug twice' [1].
Regards,
Luke
[1] http://www.red-sweater.com/blog/125/easy-programming -- see part way
down.
--
Women take to goodhearted men. Also from.
Luke Plant || http://lukeplant.me.uk/
Malcolm Tredinnick
[...]
> So, given that, I'm pretty happy with Tom's mini-counter-proposal. It
> might take some nasty hacking to "remember" where a template was
> loaded from (not sure; gotta check), but not having < show up in
> ".txt" templates would be a Good Thing, methinks.
>
> Malcolm -- think this'll be possible to do in a minimally ugly way?
For file based loaders, it's technically possible. For anything not
file-based, pick one. We'd better extend the file extensions to
include .xhtml and .htm as well, I guess, since they're common product
from some tools and alleged operating systems.
Artistically ugly, but I've given up caring. This is just insane. On or
off really, really doesn't matter. Off by default and requiring anybody
who cares about being vaguely professional has to type in the tag isn't
even that big a deal.
Anybody who thinks this decision is actually a really big life-changing
decision needs to re-evaluate their priorities. And perhaps pick some
bigger-item bugs to fix. This is too depressing; just pick one and move
on.
Malcolm
Tom Tobin
>
> On Wednesday 01 August 2007 19:56:05 Tom Tobin wrote:
> > Okay, post-Starbucks chat with my co-workers, here's my last-ditch
> > mitigation proposal:
> >
> > Set autoescaping on by default for anything ending in ``.html`` (and,
> > perhaps, ``.htm``), and off otherwise.
>
> -1 on this, it's much too magic for me. If you actually look at
> implementing it, it feels even worse -- you have to modify the
> Template.render() method to use information it doesn't even reliably
> have (the name of the template -- won't exist for templates from
> strings), and then *modify* the context object it is passed in on that
> basis...it's nasty.
I'm indeed assuming implementation issues can be surmounted; right now
I'm just floating the idea, and I'm more than willing to take a crack
at implementation if it gets a reasonable pass.
> Also consider cases where someone goes from using a template inline in a
> Python file (as I have in a few places in my source code for very small
> templates) to having them stored in files. Copying and pasting into a
> new file certainly qualifies as a simple refactor in my mind -- I
> probably wouldn't bother even testing the contents of the output. It
> would be pretty evil if the template could start behaving differently
> after doing this.
Let's assume autoescaping is on unless the template engine knows
otherwise; your inline templates will work as expected both before and
after extraction (again, assuming you're extracting them to a file
ending in ``.html``). Inline templates that shouldn't be escaped can
either be decorated somehow, or have the ``{% autoescape off %}`` tag
prepended. (I'd prefer the former, but my objection for this uncommon
subset of an already uncommon case is weaker than my general
file-template autoescaping objection.)
Tom Tobin
>
> Anybody who thinks this decision is actually a really big life-changing
> decision needs to re-evaluate their priorities. And perhaps pick some
> bigger-item bugs to fix. This is too depressing; just pick one and move
> on.
Translating "passionate about getting this right for Django" to
"considering this a life-changing decision" is one *hell* of a jump.
Please don't tar me with that brush; I'm sorry that you're weary of
this debate, but that's just not cool. This is a priority for me
*within Django*; we all have different parts of Django that we focus
on perhaps more than others, and this happens to be one of mine.
James Bennett
> otherwise; your inline templates will work as expected both before and
> after extraction (again, assuming you're extracting them to a file
> ending in ``.html``).
Let's just do:
1. Autoescape on by default.
2. Autoescape is turned off by the {% autoescape off %}
3. Autoescape happens irregardless of what the template's source file
or source string happened to be named.
Tom Tobin
> > Let's assume autoescaping is on unless the template engine knows
> > otherwise; your inline templates will work as expected both before and
> > after extraction (again, assuming you're extracting them to a file
> > ending in ``.html``).
>
> Let's just do:
>
> 1. Autoescape on by default.
> 2. Autoescape is turned off by the {% autoescape off %}
> 3. Autoescape happens irregardless of what the template's source file
> or source string happened to be named.
Thank you for taking this opportunity to remind me of the autoescaping
proposal. I must have forgotten it somewhere along the way here.
(Next time, you *could* just say you're -1.) :P
James Bennett
I'm just saying: let's do this.
Malcolm's right: the amount of argument this is generating is out of
proportion to how important the change is, so let's just do it and
move on.
Gary Wilson
> Let's just do:
>
> 1. Autoescape on by default.
> 2. Autoescape is turned off by the {% autoescape off %}
> 3. Autoescape happens irregardless of what the template's source file
> or source string happened to be named.
Is it done yet?
Jason Davies
wrote:
> We've seen how bad XSS can be; I think we have an almost moral obligation here.
Agreed; I'm +1 on this proposal for this reason.
Genshi [1] seems to do a similar thing: autoescape is on by default.
Jason
[1] http://genshi.edgewall.org/wiki/GenshiFaq#HowcanIincludeliteralXMLintemplateoutput
Michael Radziej
> Let's just do:
>
> 1. Autoescape on by default.
> 2. Autoescape is turned off by the {% autoescape off %}
> 3. Autoescape happens irregardless of what the template's source file
> or source string happened to be named.
I tried to keep myself out of this discussion, as I also don't care so much.
But have you thought about the admin interface? Would it work when
autoescape is on by default?
I'd still suggest a 'default off' policy at least when autoescape is
introduced, just to give users a transition period. A 'default on' policy
means that autoescape will break a lot of custom template tags and filters.
The policy could then change to 'default on' shortly before a new release.
Rob Hudson
> I'd still suggest a 'default off' policy at least when autoescape is
> introduced, just to give users a transition period. A 'default on' policy
> means that autoescape will break a lot of custom template tags and filters.
> The policy could then change to 'default on' shortly before a new release.
I kind of think it should just be done 100% in 0.97 and announced in
the release notes. Everyone should know that pre 1.0 there will be
some backward incompatible changes and should watch out for them, no?
So far Django has had great docs on transitioning between releases.
-Rob
Yuri Baburov
This is also like PHP magic_quotes!
How would it work together with template extending and including?
To the end of current block?
Would {{ block.super }} do autoescape?
These questions also should be decided.
(Maybe some of these questions are already have clean answer, but
anyway, I must ask them)
+1 if you add autoescape stack with {% autoescape pop %} or better {%
noescape %} {% endnoescape %}.
For those who told about template-wise or total disabling autoescape -
my idea is to add something like render_noautoescaped to
django.shortcuts, noescape filter and autoescape middleware.
--
Best regards, Yuri V. Baburov, ICQ# 99934676, Skype: yuri.baburov,
MSN: bu...@live.com
James Bennett
> I'm -1 for {% autoescape on %} and {% autoescape off %}.
> This is also like PHP magic_quotes!
No, it's not. PHP's magic_quotes is set at a higher level; because the
behavior will be on by default and the only mechanism to change it
will be a template tag, you don't run into the "is it on or off"
confusion that plagues PHP's magic_quotes setting -- you can safely
write a template and *know* that if you've altered the autoescape
behavior in that template, no other component can secretly "override"
you by flipping it back.
> How would it work together with template extending and including?
> To the end of current block?
> Would {{ block.super }} do autoescape?
> These questions also should be decided.
Please read the entire thread above -- autoescaping will not work at
the level of blocks; it will only work at the level of autoescaping or
not autoescaping the **entire** output of the template, meaning all
includes, all tag output and all child templates will be affected by
it. If you don't want autoescaping, simply turn it off in a base
template that all your templates inherit from, and you don't have to
worry about it.
> +1 if you add autoescape stack with {% autoescape pop %} or better {%
> noescape %} {% endnoescape %}.
I would give this a huge -1 because of the complexity it would
introduce; either the entire template should be escaped or the entire
template should not be escaped, period. Anything else is begging for
headaches as people try to guess about which parts of a template get
escaped automatically and which parts don't.
> For those who told about template-wise or total disabling autoescape -
> my idea is to add something like render_noautoescaped to
> django.shortcuts, noescape filter and autoescape middleware.
I would also be -1 on this; the sole means of control needs to be the
template system, not higher-level shortcuts, because a template author
**must** be able to assume that what's in the template is the sole and
absolute arbiter of escaping behavior. Again, anything else is begging
for headaches.
Todd O'Bryan
> Please read the entire thread above -- autoescaping will not work at
> the level of blocks; it will only work at the level of autoescaping or
> not autoescaping the **entire** output of the template, meaning all
> includes, all tag output and all child templates will be affected by
> it. If you don't want autoescaping, simply turn it off in a base
> template that all your templates inherit from, and you don't have to
> worry about it.
Can this possibly be right? If I want to include a single variable that
includes some HTML (such as {{ form.as_table }}), do I have to turn off
auto-escaping for everything else in the page?
Todd
Niels
I am firmly +1 on what is described there:
- autoescaping can be turned off (or on) at the block level
- strings can be marked to prevent them from getting escaped
- views may switch autoescape off by setting an attribute on the
context
Is this all out-of-date information?
Best regards
Niels
[1] http://code.djangoproject.com/wiki/AutoEscaping#Autoescaping
On Aug 3, 1:14 am, "James Bennett" <ubernost...@gmail.com> wrote:
James Bennett
> Can this possibly be right? If I want to include a single variable that
> includes some HTML (such as {{ form.as_table }}), do I have to turn off
> auto-escaping for everything else in the page?
From the descriptions above -- of using the autoescape toggle in a
base template to inherit through to every child template -- it seems
that the only logical thing is for it to not be something which
"expires" at the end of a block, but instead to apply to an entire
template.
If I've misread that, somebody correct me.
SmileyChris
> Can this possibly be right? If I want to include a single variable that
> includes some HTML (such as {{ form.as_table }}), do I have to turn off
> auto-escaping for everything else in the page?
I'm imagining .as_table will probably return a string already marked
as safe, but to answer your question:
{{ my_html|safe }}
Simon Willison
> Set autoescaping on by default for anything ending in ``.html`` (and,
> perhaps, ``.htm``), and off otherwise.
I've been thinking about this a bit, and it seems like it could work
well if it was done the other way round - basically, autoescaping is
on for everything EXCEPT templates where the template name is known /
and/ it ends in .txt. I don't think this would be too hacky to
implement - templates that are loaded (as opposed to constructed from
a string) already know their template name as part of the template
error handling code; all that would be needed would be a way to tell a
newly created template to default to autoescape off, and then a bit of
code in the relevant template loader to special case for template
names ending in .txt.
Generally I'm really glad to see that most people have come round to
autoescaping being on by default now. I personally don't see it as a
way of protecting newbie developers so much as it's a way of
protecting all developers from one tiny mistake blowing the security
of their application wide open.
Jacob Kaplan-Moss
> includes some HTML (such as {{ form.as_table }}), do I have to turn off
> auto-escaping for everything else in the page?
No, it's not quite right. There's an autoescape controller block that
you use to unconditionally turn autoescaping on or off::
This {{ content }} will be autoescaped
since the default template rendering
will be in autoescape mode.
{% autoescape off %}
This {{ content }} will be *not* be {{ escaped }}.
Nor will the contents of this
{% block childblock %}{% endblock %}
{% autoescape on %}
This {{ content }} will be escaped.
{% endautoescape %}
{% endautoescape %}
There are also two template filters, ``raw`` and ``escape``. They both
*always* do the right thing, reguardless of which "mode" you're in.
Double escaping won't happen, and marking something as raw when
autoescape is off is a no-op.
Finally, if you write template tags/filter that return "safe" content
-- i.e. a markdown filter, or whatever -- you can mark the returned
strings as safe and they won't be escaped in any mode.
This is all there in the patch, FYI, but we'll need to be sure to
document it extremely clearly before it drops.
Jacob
SmileyChris
wrote:
> The one thing Jacob and I
> agreed to change at OSCON was to move to "autoescape on" and "autoescape
> off", rather than "autoescape/noautoescape" as the tags. This way, if
> somebody solves the impossible and comes up with a way to escape
> Javascript in a robust fashion, we can have "{% autoescape js %}" or
> similar; it's just a bit more extensible and Jacob made a convincing
> argument.
In this case, could we change it to {% autoescape html %} to be
explicit?
And a unicode issue I came across reading through the latest patch -
should mark_safe and mark_for_escaping be updated to use force_unicode?
Tai Lee
templates, applied to all templates regardless of extension.
Michael Radziej
Yuri Baburov schrieb:
> I'm -1 for {% autoescape on %} and {% autoescape off %}.
> This is also like PHP magic_quotes!
My dear ...
I really ask you to apply the patch in the ticket (hey, that's easy!) and
play a bit with it. All the django developers have gone through the php
experience school and learned what to avoid.
You'll find out that it's been thought out well and takes care of typical
problems like how to treat html code fragments within template variables.
The docs also explain it, but playing with the patch is probably more fun ;-)
Michael
Michael Radziej
>
> On 8/2/07, Todd O'Bryan <toddo...@mac.com> wrote:
> > Can this possibly be right? If I want to include a single variable that
> > includes some HTML (such as {{ form.as_table }}), do I have to turn off
> > auto-escaping for everything else in the page?
>
> From the descriptions above -- of using the autoescape toggle in a
> base template to inherit through to every child template -- it seems
> that the only logical thing is for it to not be something which
> "expires" at the end of a block, but instead to apply to an entire
> template.
>
> If I've misread that, somebody correct me.
Well, "autoescape" is not a setting, it builds a TemplateNode.
{% autoescape %} always needs an {% endautoescape %}, generating a node
during template parsing. Everything between {% autoescape %} and {%
endautoescape %} is put in its subnode list.
If a VariableNode is below an {% autoescape on %} in the template node tree,
autoescape is in effect, if it is below an {% autoescape off %}, it won't.
(Deepest autoescape node wins, of course.)
So, it all depends on the node tree, and not at all on the template that
ultimately defines a block.
By the way, it is also possible to write templates that are completely
independent of the autoescape setting in effect without using any
{% autoescape %} filter tag:
- Add an 'escape' filter to each variable that needs escaping
(and autoescape will *not* escape it another time)
- Add a 'safe' filter to all the other variables, and it will be marked
as safe, so that autoescaping won't touch it.
This actually adds some documentation for any reviewer. 'safe' means:
"Hey, I haven't simply forgot escaping, I meant it!"
Niels
> > From the descriptions above -- of using the autoescape toggle in a
> > base template to inherit through to every child template -- it seems
> > that the only logical thing is for it to not be something which
> > "expires" at the end of a block, but instead to apply to an entire
> > template.
>
> > If I've misread that, somebody correct me.
>
> Well, "autoescape" is not a setting, it builds a TemplateNode.
>
> {% autoescape %} always needs an {% endautoescape %}, generating a node
> during template parsing. Everything between {% autoescape %} and {%
> endautoescape %} is put in its subnode list.
>
> If a VariableNode is below an {% autoescape on %} in the template node tree,
> autoescape is in effect, if it is below an {% autoescape off %}, it won't.
> (Deepest autoescape node wins, of course.)
>
> So, it all depends on the node tree, and not at all on the template that
> ultimately defines a block.
>
> By the way, it is also possible to write templates that are completely
> independent of the autoescape setting in effect without using any
> {% autoescape %} filter tag:
>
> - Add an 'escape' filter to each variable that needs escaping
> (and autoescape will *not* escape it another time)
>
> - Add a 'safe' filter to all the other variables, and it will be marked
> as safe, so that autoescaping won't touch it.
>
> This actually adds some documentation for any reviewer. 'safe' means:
> "Hey, I haven't simply forgot escaping, I meant it!"
Jacob used the word 'raw' here, which is indeed a more appropriate
name for the "don't escape me" filter.
Niels
>
> Michael
>
Michael Radziej
bikeshed?
Ivan Sagalaev
> {% autoescape %} always needs an {% endautoescape %}, generating a node
> during template parsing. Everything between {% autoescape %} and {%
> endautoescape %} is put in its subnode list.
>
> If a VariableNode is below an {% autoescape on %} in the template node tree,
> autoescape is in effect, if it is below an {% autoescape off %}, it won't.
> (Deepest autoescape node wins, of course.)
So this "deepest wins" effectively means that one can't set autoescaping
to "off" from a base template as James was suggesting, right? (I'm not
against it, just trying to clarify.)
Michael Radziej
To make this clear, I'll provide an example:
base.html:
{% autoescape off %}
<html><body>
{% block bla %}
<p>{{xyz}}</p>
{% endblock bla %}
</body></html>
{% endautoescape %}
sub.html:
{% extends "base.html" %}
{% block bla %}
{% autoescape on %}
<p>And now to something completely different: {{abc}}</p>
{% endautoescape %}
{% endblock bla %}
If you use sub.html, abc will be escaped due to the {% autoescape on %} in
the line above. That's the 'deeper' rule: There's not a global switch to
switch autoescape on or off.
Ivan Sagalaev
> To make this clear, I'll provide an example:
Thanks! That's cleared it up.
Niels
> On Fri, Aug 03, Niels wrote:
> > On Aug 3, 11:10 am, Michael Radziej <m...@noris.de> wrote:
> > > This actually adds some documentation for any reviewer. 'safe' means:
> > > "Hey, I haven't simply forgot escaping, I meant it!"
>
> > Jacob used the word 'raw' here, which is indeed a more appropriate
> > name for the "don't escape me" filter.
>
> bikeshed?
>
> Michael
>
This is totally unrelated to the color of my grandma's bike, but it
used to be black.
:-)
Tom Tobin
>
> On Aug 1, 7:56 pm, "Tom Tobin" <korp...@korpios.com> wrote:
> > Set autoescaping on by default for anything ending in ``.html`` (and,
> > perhaps, ``.htm``), and off otherwise.
>
> I've been thinking about this a bit, and it seems like it could work
> well if it was done the other way round - basically, autoescaping is
> on for everything EXCEPT templates where the template name is known /
> and/ it ends in .txt. I don't think this would be too hacky to
> implement - templates that are loaded (as opposed to constructed from
> a string) already know their template name as part of the template
> error handling code; all that would be needed would be a way to tell a
> newly created template to default to autoescape off, and then a bit of
> code in the relevant template loader to special case for template
> names ending in .txt.
Hmm ... that could work; we'd just need a reasonable whitelist. .txt
is the obvious case; I'm trying to think of others (typical
configuration files, etc.). Off the top of my head, I'd add .conf,
.ini, and the regex ^\..*rc$ — can anyone chime in with other
fairly-obvious cases?
Marty Alchin
> Hmm ... that could work; we'd just need a reasonable whitelist. .txt
> is the obvious case; I'm trying to think of others (typical
> configuration files, etc.). Off the top of my head, I'd add .conf,
> fairly-obvious cases?
I'd add .py to that list, if it gets done this way.
-Gul
Jacob Kaplan-Moss
> I'd add .py to that list, if it gets done this way.
Wait - are you using Django templates to generate *Python*?
That's... just... *sick*.
Jacob
Marty Alchin
> Wait - are you using Django templates to generate *Python*?
Actually, no. But since Django uses Django templates to generate (bits
of) Django[1] code, I thought it might be worth specifying.
> That's... just... *sick*.
Ah, but sickness must be put into perspective. You have no idea.
-Gul
[1] http://code.djangoproject.com/browser/django/trunk/django/conf/project_template
Forest Bond
Jacob Kaplan-Moss
Ha!
Right you are -- I'm the sick one here, it appears :)
Sorry for the derail; carry on.
Jacob
Nicola Larosa
> Generally I'm really glad to see that most people have come round to
> autoescaping being on by default now. I personally don't see it as a
> way of protecting newbie developers so much as it's a way of
> protecting all developers from one tiny mistake blowing the security
> of their application wide open.
Very well said, and worth emphasizing; thanks, Simon.
--
Nicola Larosa - http://www.tekNico.net/
Any word not used as an expletive is not being used
to its fullest potential. -- Fred Drake, March 2007
Adrian Holovaty
> Let's just do:
>
> 1. Autoescape on by default.
> 2. Autoescape is turned off by the {% autoescape off %}
> 3. Autoescape happens irregardless of what the template's source file
> or source string happened to be named.
I'm +1 on this approach. Let's just do it.
Adrian
--
Adrian Holovaty
holovaty.com | djangoproject.com
Tom Tobin
>
> On 8/1/07, James Bennett <ubern...@gmail.com> wrote:
> > Let's just do:
> >
> > 1. Autoescape on by default.
> > 2. Autoescape is turned off by the {% autoescape off %}
> > 3. Autoescape happens irregardless of what the template's source file
> > or source string happened to be named.
>
> I'm +1 on this approach. Let's just do it.
Do you have any particular issues with Simon's modification of my
proposal (turning autoescaping off by default only for a whitelist of
extensions such as .txt)? If I'm going to go ahead and whip up some
code, I'd like there to be *some* chance in hell of it getting
accepted. ^_^
Tom Tobin
Or hell, maybe I'm looking at this the wrong way.
A lot of my angst here comes from the collision between my desire for
Django's template system to serve as a good *general-purpose*
templating system for everything from HTML to LaTeX to what-have-you,
and Django's (obviously) web-centric standpoint. There's been quite a
bit of talk over time regarding splitting Django's template system out
into a separate library; maybe it's time to do that. It would make
perfect sense for the *library* to take a setting for defaulting
autoescaping to on or off depending on a developer's needs in various
contexts, while still keeping autoescaping on by default for Django's
use of the library.
Thoughts?