Django UnicodeEncodeError in errorlist

[Andrei]

Hi,

I am having an issue with rendering Django's ErrorList if one of my error list items is unicode. When Django renders my errorlist

{{ form.non_field_errors }}

it runs the following code:

class ErrorList(list, StrAndUnicode):
    """
    A collection of errors that knows how to display itself in various formats.
    """
    def __unicode__(self):
        return self.as_ul()

    def as_ul(self):
        if not self: return u''
        return mark_safe(u'<ul class="errorlist">%s</ul>'
                % ''.join([u'<li>%s</li>' % conditional_escape(force_unicode(e)) for e in self])) 

then in force_unicode:

s = unicode(str(s), encoding, errors) 

and then translation in lazy:

def __str_cast(self):
    return str(self.__func(*self.__args, **self.__kw))

The problem is that my string contains 'å' symbol and str(u'å') raises UnicodeEncodeError. Is there a good reason why force_unicode and lazy do not use smart_str? I have to do it myself and provide error messages as str objects instead of unicode to make it work.

So I get TemplateSyntaxError Caught UnicodeEncodeError while rendering: 'ascii' codec can't encode character u'\xe5' in position 17: ordinal not in range(128). This seems telling that rendering my error list item (which is u'å') caused the first UnicodeEncodeError having unicode message 'ascii' codec can't encode character u'\xe5' and then second UnicodeEncodeError while rendering the message from the first one. Am I mistaken?

Django version: 1.3.1 (but this seems to happen in 1.4 as well)

Full traceback:https://raw.github.com/gist/2499077/ba60cb752acdb429dd6c2814ffb24272037a367a/UnicodeEncodeError.txt

http://stackoverflow.com/questions/10332674/django-unicodeencodeerror-in-errorlist

Thanks,

Andrei

[Bill Freeman]

On 4/26/12, Andrei <andrei...@gmail.com> wrote:
> Hi,
>
> I am having an issue with rendering Django's ErrorList if one of my error
> list items is unicode. When Django renders my errorlist
>
> {{ form.non_field_errors }}
>
> it runs the following

> code<https://github.com/django/django/blob/1.3.1/django/forms/util.py#L46>
> :

>
> class ErrorList(list, StrAndUnicode):
> """
> A collection of errors that knows how to display itself in various
> formats.
> """
> def __unicode__(self):
> return self.as_ul()
>
> def as_ul(self):
> if not self: return u''
> return mark_safe(u'<ul class="errorlist">%s</ul>'
> % ''.join([u'<li>%s</li>' %
> conditional_escape(force_unicode(e)) for e in self]))
>
> then in

> force_unicode<https://github.com/django/django/blob/1.3.1/django/utils/encoding.py#L74>
> :

>
> s = unicode(str(s), encoding, errors)
>
> and then translation in

> lazy<https://github.com/django/django/blob/1.3.1/django/utils/functional.py#L209>
> :

>
> def __str_cast(self):
> return str(self.__func(*self.__args, **self.__kw))
>
> The problem is that my string contains 'å' symbol and str(u'å') raises
> UnicodeEncodeError. Is there a good reason why force_unicode and lazy do
> not use smart_str? I have to do it myself and provide error messages as str
> objects
> instead of unicode to make it work.
>

> So I get TemplateSyntaxError *Caught UnicodeEncodeError while rendering:

> 'ascii' codec can't encode character u'\xe5' in position 17: ordinal not in

> range(128)*. This seems telling that rendering my error list item (which is
> u'å') caused the first UnicodeEncodeError having unicode message *'ascii'
> codec can't encode character u'\xe5'* and then second UnicodeEncodeError

> while rendering the message from the first one. Am I mistaken?
>
> Django version: 1.3.1 (but this seems to happen in 1.4 as well)
>
> Full traceback:
> https://raw.github.com/gist/2499077/ba60cb752acdb429dd6c2814ffb24272037a367a/UnicodeEncodeError.txt
> http://stackoverflow.com/questions/10332674/django-unicodeencodeerror-in-errorlist
>
> Thanks,
> Andrei

I *think* that there's only one exception here. The nested
force_unicode calls probably aren't unusual.

A possible work around for you would be to set the python
interpreter's default string encoding to utf-8. Then that call to
str() inside __str_cast() on a unicode string containing a non ASCII
character would work.

Whether or not this is a weakness in the django.utils is above my pay grade.