Silently capturing all UnicodeDecodeError exceptions in widget rendering
Ben Gerdemann
code, that eventually turned out to be a Unicode encoding problem. The
reason it took me so long to track down the problem is this code here
in 'template/__init__.py':
794 class VariableNode(Node):
...
801 def render(self, context):
802 try:
803 output = force_unicode(self.filter_expression.resolve
(context))
804 except UnicodeDecodeError:
805 # Unicode conversion can fail sometimes for reasons
out of our
806 # control (e.g. exception rendering). In that case,
we fail quietly.
807 return ''
Is there a good reason that the widget rendering is silently capturing
all Unicode errors? Now that I know that it's doing this, it won't
take me so long to figure out the problem if it happens again, but it
certainly wasn't expected behavior. At the very least, maybe we could
return 'UnicodeDecodeError' instead of a blank string?
Cheers,
Ben
Ludvig Ericson
I think this is a symptom of a much greater issue: the templating
system *silences* errors a lot. I think there should be some logging
mechanism (possibly using Python's logging module?) that logs errors
in templates.
I for one would *love* this, so instead::
logger = logging.getLogger("django.template")
# ...
try:
do_whatever()
except MyError:
logger.
I mean, they're errors, but the reason they're silenced is that the
show often must go on. Not that an error condition is impossible.
- Ludvig
Ludvig Ericson
>> logger = logging.getLogger("django.template")
>> # ...
>> try:
>> do_whatever()
>> except MyError:
>> logger.
>
>
> logger = logging.getLogger("django.template")
> # ...
> try:
> do_whatever()
> except MyError:
> return ''
... and then sending the reply to myself... twice.
- Ludvig
Malcolm Tredinnick
> I just spent a good part of my afternoon tracking down a bug in my
> code, that eventually turned out to be a Unicode encoding problem. The
> reason it took me so long to track down the problem is this code here
> in 'template/__init__.py':
>
> 794 class VariableNode(Node):
> ...
> 801 def render(self, context):
> 802 try:
> 803 output = force_unicode(self.filter_expression.resolve
> (context))
> 804 except UnicodeDecodeError:
> 805 # Unicode conversion can fail sometimes for reasons
> out of our
> 806 # control (e.g. exception rendering). In that case,
> we fail quietly.
> 807 return ''
>
> Is there a good reason that the widget rendering is silently capturing
> all Unicode errors?
Yes. Non-developer-controlled content (which is usually what's being
converted by filters) should not be allowed to crash template rendering.
The data coming into the template variables could be from a lot of
location and, as a final fallback, if it gets that far without being
caught, I decided not to let it blow up the rendering process.
Similarly, inserting an error string isn't the right thing to do by
default, as it's more or less entirely unhelpful to the user.
All that being said, if you want to come up with a patch to introduce
something similar to TEMPLATE_STRING_IF_INVALID that is inserted in
cases like that (including something like the "%s" support for that
setting) when developers are debugging, that would probably be a good
idea.
End of the day, though, the current (default) behaviour is not to be
distracting to the end reader of the site. You're right that it
inadvertently impacts development, so let's fix that problem as well.
Regards,
Malcolm
Malcolm Tredinnick
Well, you'll need to come up with some concrete cases, since the general
claim isn't really true (make that a different thread, though, since
it's not the same topic as this one).
We silence a particular set of errors: mostly those related to accessing
non-existent variables. Those errors aren't a sign of a problem,
necessarily, since it permits a wide range of template re-use as you
don't need the full set of variables the template uses in order to use
that template.
There's a slight side-effect that other emitters of KeyError and
AttributeError get bitten, that's true, but almost impossible to avoid.
That's why TEMPLATE_STRING_IF_INVALID exists, since those problems can
easily be detected at development time if they are happening
inadvertently.
The Unicode case is probably a situation where logging (or the addition
of a logging hook) might be useful (since it's non-developer content and
thus not necessarily detectable at development time). What other errors
do you think are being unnecessarily silenced? Specifics are important
here, as the silencing we are doing is generally deliberate and with
fairly well thought-out reasons.
Regards,
Malcolm
Ben Gerdemann
On Feb 20, 10:52 pm, Malcolm Tredinnick <malc...@pointy-stick.com>
wrote:
> converted by filters) should not be allowed to crash template rendering.
> The data coming into the template variables could be from a lot of
> location and, as a final fallback, if it gets that far without being
> caught, I decided not to let it blow up the rendering process.
>
> Similarly, inserting an error string isn't the right thing to do by
> default, as it's more or less entirely unhelpful to the user.
>
> All that being said, if you want to come up with a patch to introduce
> something similar to TEMPLATE_STRING_IF_INVALID that is inserted in
> cases like that (including something like the "%s" support for that
> setting) when developers are debugging, that would probably be a good
> idea.
>
> End of the day, though, the current (default) behaviour is not to be
> distracting to the end reader of the site. You're right that it
> inadvertently impacts development, so let's fix that problem as well.
(impossible?) to design a template that will always render without
exceptions no matter the user input. Is that true?
Anyway, I like Ludvig's idea, but I will propose instead a new
template tag like {% template_errors %} that would output a list of
all captured exceptions. The tag would only have output when
DEBUG=True so that you could throw it into your templates without
having to delete it for deployment. Thoughts?
Cheers,
Ben