Encodings outside of utf-8

[ccahoon]

If you take a look at my branch, you will see I have a feature in
progress that takes both the Accept-Charset header (provided that the
view passes along the request) and the Content-Type (which must be
provided in the view) into account.

http://code.djangoproject.com/browser/django/branches/soc2009/http-wsgi-improvements

It uses these to determine if HttpResponse should use a different
encoding. Right now, it finds valid, non UTF-8 codecs to use, and
encodes using them. However, I do not have knowledge of these
encodings enough to test this part properly (so far).

Does anyone have a use-case they could share with me?

(This came about in reference to ticket 10190, but is part of HTTP1.1
compliance).

Thanks for any help.

[Jeremy Dunck]

On Wed, Jun 17, 2009 at 3:50 PM, ccahoon<chris....@gmail.com> wrote:
...

>
> Does anyone have a use-case they could share with me?
>

I hear Malcolm's hovercraft is full of eels.

[Malcolm Tredinnick]

For those wondering if Jeremy might have lost his mind (well, that's
another topic), I created this file:

http://www.pointy-stick.com/~malcolm/hovercraft.py

a couple of years ago as a test case for Unicode and UTF-8 handling.
It's a bunch of good test data for putting into models if you're wanting
to experiment with non-ASCII strings. See also [1] for how I used it in
practice.

However, all this data is UTF-8 encoded in the source, so it's not great
test data as it stands for different HTTP encoding. It wouldn't take a
lot of extra processing to convert strings to various local encodings,
but it's not an out-of-the-box help here.

I suspect Chris is after some real-world cases of where people are using
special Accept-Charset headers or special output encoding. The latter is
likely to occur when talking to existing systems that maybe aren't as
capable at full HTTP compliance. The former is not going to be
particularly common, although I know of some situations in the East
Asian part of the world where special encoding is beneficial, so maybe
some Asian Pacific developers have some examples. I'm also interested in
this, since I'm mentoring Chriss for SOC and want to reality check what
we're doing against... well... reality. I can use my experience in the
region a fair bit to come up with some cases I've seen in practice, but
other people have more exposure than I do.

[1] http://www.flickr.com/photos/malcolmtredinnick/494663115/

Regards,
Malcolm