Proposal: A better way to handle unic�de in messages, etc

[Yo-Yo Ma]

I was trying some things out, and I noticed that it isn't actually
possible (without using a char-code and converting it) to put
something like ° (degree symbol) into a "message".

Example:

this code:
messages.info(request, request.GET.get('symbol'))

with this URL:
http://127.0.0.1:8080/?symbol=%BO

will result in:
UnicodeEncodeError at /
('ascii', u'\ufffd', 0, 1, 'ordinal not in range(128)')


Perhaps this is beyond what a framework should deal with, but if
possible, it'd be handy. I built a little site for my wife using
PostgreSQL and if she puts a trademark symbol into a page title, the
success message that should come up on the next (which includes the
title) page causes an exception.

[Luke Plant]

On Wed, 2010-10-20 at 09:50 -0700, Yo-Yo Ma wrote:
> I was trying some things out, and I noticed that it isn't actually
> possible (without using a char-code and converting it) to put
> something like ° (degree symbol) into a "message".
>
> Example:
>
> this code:
> messages.info(request, request.GET.get('symbol'))
>
> with this URL:
> http://127.0.0.1:8080/?symbol=%BO
>
> will result in:
> UnicodeEncodeError at /
> ('ascii', u'\ufffd', 0, 1, 'ordinal not in range(128)')

This would definitely be classed as a bug. But I for one can't
reproduce it. Assuming your default charset is UTF-8, your URL should
actually contain '%C2%B0'. With that change, your example code works
perfectly on a page I tried. (Without it, I still don't get an
exception).

Luke

--
If you can't answer a man's arguments, all is not lost; you can
still call him vile names. (Elbert Hubbard)

Luke Plant || http://lukeplant.me.uk/

[Michael]

request.GET should convert %codes into their appropriate characters, the
problem is likely that messages.info is expecting an ascii string, not
unicode.

messages.info(request, unicode(request.GET.get('symbol'),
errors='replace'))

Will replace the degree symbol with a ?, if I remember correctly.
Otherwise you'll need to re-encode it using something else, like url
encoding, and decode it again before displaying.

--
Michael <mhal...@gmail.com>

[Russell Keith-Magee]

On Thu, Oct 21, 2010 at 12:50 AM, Yo-Yo Ma <baxters...@gmail.com> wrote:
> I was trying some things out, and I noticed that it isn't actually
> possible (without using a char-code and converting it) to put
> something like ° (degree symbol) into a "message".

Sure. Sounds like a bug to me. If you can't call a Django method with
unicode content and have it work reliably, that's a bug. Fixing a bug
doesn't require a proposal -- it needs a ticket and a patch. Open a
ticket and it will go through the normal process.

Yours,
Russ Magee %-)

[Yo-Yo Ma]

Thanks, I'll do that in the morning with a traceback, and steps to
reproduce it.

On Oct 20, 6:15 pm, Russell Keith-Magee <russ...@keith-magee.com>
wrote:

[Yo-Yo Ma]

OK, I finally created a ticket: http://code.djangoproject.com/ticket/14525