Django UnicodeEncodeError in errorlist

[Andrei]

Hi,

I am having an issue with rendering Django's ErrorList if one of my error list items is unicode. When Django renders my errorlist

{{ form.non_field_errors }}

it runs the following code:

class ErrorList(list, StrAndUnicode):
    """
    A collection of errors that knows how to display itself in various formats.
    """
    def __unicode__(self):
        return self.as_ul()

    def as_ul(self):
        if not self: return u''
        return mark_safe(u'<ul class="errorlist">%s</ul>'
                % ''.join([u'<li>%s</li>' % conditional_escape(force_unicode(e)) for e in self])) 

then in force_unicode:

s = unicode(str(s), encoding, errors) 

and then translation in lazy:

def __str_cast(self):
    return str(self.__func(*self.__args, **self.__kw))

The problem is that my string contains 'å' symbol and str(u'å') raises UnicodeEncodeError. Is there a good reason why force_unicode and lazy do not use smart_str? I have to do it myself and provide error messages as str objects instead of unicode to make it work.

So I get TemplateSyntaxError Caught UnicodeEncodeError while rendering: 'ascii' codec can't encode character u'\xe5' in position 17: ordinal not in range(128). This seems telling that rendering my error list item (which is u'å') caused the first UnicodeEncodeError having unicode message 'ascii' codec can't encode character u'\xe5' and then second UnicodeEncodeError while rendering the message from the first one. Am I mistaken?

Django version: 1.3.1 (but this seems to happen in 1.4 as well)

Full traceback: https://raw.github.com/gist/2499077/ba60cb752acdb429dd6c2814ffb24272037a367a/UnicodeEncodeError.txt

Thanks,

Andrei

[Thomas Guettler]

this question belongs to django-user.

You need to use u'å' not 'å' in your python code.

> Is there a good reason
> why |force_unicode| and |lazy| do not use |smart_str|?

smart_str() creates a bytestring, but in Django everything is unicode until the end. Only one of the last
steps is to encode the unicode result to (most of the times) utf8.

I have seem a lot of unicode errors in django. But most of them were fixed long ago.

The last (I know of): https://code.djangoproject.com/ticket/18063
"repr() should return only ascii, not unicode"

Thomas

Am 26.04.2012 20:41, schrieb Andrei:
> Hi,
>
> I am having an issue with rendering Django's ErrorList if one of my error list items is unicode. When Django renders my
> errorlist
>
> |{{ form.non_field_errors}}
> |
>

> it runs the following code <https://github.com/django/django/blob/1.3.1/django/forms/util.py#L46>:

>
> |class ErrorList(list, StrAndUnicode):
> """
> A collection of errors that knows how to display itself in various formats.
> """
> def __unicode__(self):
> return self.as_ul()
>
> def as_ul(self):
> if not self: return u''
> return mark_safe(u'<ul class="errorlist">%s</ul>'

> % ''.join([u'<li>%s</li>' % conditional_escape(force_unicode(e)) for ein self]))
> |
>
> then in |force_unicode| <https://github.com/django/django/blob/1.3.1/django/utils/encoding.py#L74>:
>
> |s= unicode(str(s), encoding, errors)
> |
>
> and then translation in |lazy| <https://github.com/django/django/blob/1.3.1/django/utils/functional.py#L209>:

>
> |def __str_cast(self):
> return str(self.__func(*self.__args, **self.__kw))
> |
>

> The problem is that my string contains 'å' symbol and |str(u'å')| raises |UnicodeEncodeError|. Is there a good reason
> why |force_unicode| and |lazy| do not use |smart_str|? I have to do it myself and provide error messages as |str|

> objects instead of unicode to make it work.
>

> So I get TemplateSyntaxError /Caught UnicodeEncodeError while rendering: 'ascii' codec can't encode character u'\xe5' in
> position 17: ordinal not in range(128)/. This seems telling that rendering my error list item (which is |u'å'|) caused
> the first UnicodeEncodeError having unicode message /'ascii' codec can't encode character u'\xe5'/ and then second

> UnicodeEncodeError while rendering the message from the first one. Am I mistaken?
>
> Django version: 1.3.1 (but this seems to happen in 1.4 as well)
>
> Full traceback: https://raw.github.com/gist/2499077/ba60cb752acdb429dd6c2814ffb24272037a367a/UnicodeEncodeError.txt
>
> Thanks,
>
> Andrei
>
>
>

> --
> You received this message because you are subscribed to the Google Groups "Django developers" group.
> To view this discussion on the web visit https://groups.google.com/d/msg/django-developers/-/pYaefZxhll4J.
> To post to this group, send email to django-d...@googlegroups.com.
> To unsubscribe from this group, send email to django-develop...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.

--
Thomas Guettler, http://www.thomas-guettler.de/
E-Mail: guettli (*) thomas-guettler + de

[Andrei]

Hi Thomas,

You seem read my email too quickly. I do use u'å'. In force_unicode() and lazy() Django runs str(u'å') which results in the exception(s). If it would use smart_str instead, the exception would not happen. Am I missing something?

Andrei

On Friday, April 27, 2012 9:11:19 AM UTC+2, guettli wrote:

You need to use u'å' not 'å' in your python code.
 

smart_str() creates a bytestring, but in Django everything is unicode until the end. Only one of the last

steps is to encode the unicode result to (most of the times) utf8.

> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to django-developers+unsubscribe@googlegroups.com.

> For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.

--
Thomas Guettler, http://www.thomas-guettler.de/
E-Mail: guettli (*) thomas-guettler + de

> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to django-developers+unsubscribe@googlegroups.com.

> For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.

--
Thomas Guettler, http://www.thomas-guettler.de/
E-Mail: guettli (*) thomas-guettler + de

> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to django-developers+unsubscribe@googlegroups.com.