Character encoding question

[tlvp]

This has more to do perhaps with MIME encoding for email than with
character encoding for HTML, but, if someone knows, I'd be happy to learn:

by what encoding scheme does an o-acute (looking like so: " ó ") get itself
encoded as " =EF=BF=BD " in email? Can that be a known Yahoo!-email-to-
Thunderbird POP3 conversion defect (err, "issue")?

Is UTF-8 the wrong [View]-[Character Encoding] option to be using in TB?

Many thanks! And cheers, -- tlvp
--
Avant de repondre, jeter la poubelle, SVP.

[Andreas Prilop]

On Mon, 6 Aug 2012, tlvp wrote:

> Content-Transfer-Encoding: 8bit

>
> This has more to do perhaps with MIME encoding for email

Quite so.

> by what encoding scheme does an o-acute (looking like so: " ó ")
> get itself encoded as " =EF=BF=BD " in email?

This is “Quoted-Printable” as opposed to “8bit”.
However, EF BF BD is not “o with acute accent” but the byte order
mark (BOM).

--
In memoriam Alan J. Flavell
http://www.alanflavell.org.uk/charset/

[Thomas 'PointedEars' Lahn]

Andreas Prilop wrote:

> On Mon, 6 Aug 2012, tlvp wrote:
>> by what encoding scheme does an o-acute (looking like so: " ó ")
>> get itself encoded as " =EF=BF=BD " in email?
>
> This is “Quoted-Printable” as opposed to “8bit”.
> However, EF BF BD is not “o with acute accent” but the byte order
> mark (BOM).

In which encoding? The UTF-8 BOM is EF _BB BF_. EF BF BD, on the other
hand, encodes U+FFFD REPLACEMENT CHARACTER in UTF-8.


PointedEars
--
Use any version of Microsoft Frontpage to create your site.
(This won't prevent people from viewing your source, but no one
will want to steal it.)
-- from <http://www.vortex-webdesign.com/help/hidesource.htm> (404-comp.)

[Andreas Prilop]

On Tue, 7 Aug 2012, Thomas 'PointedEars' Lahn wrote:

>> However, EF BF BD is not “o with acute accent” but the byte order
>> mark (BOM).
>

> The UTF-8 BOM is EF _BB BF_. EF BF BD, on the other hand,
> encodes U+FFFD REPLACEMENT CHARACTER in UTF-8.

Right!

[Thomas 'PointedEars' Lahn]

(Which indicates that something went wrong while *sending* the e-mail, not
reading it.)

I have found it out with the excellent Unicode Code Converter and UniView
Web tools by Richard Ishida (W3C, Unicode Consortium), IMHO very useful for
Web authors and programmers alike:

<http://www.rishida.net/tools/conversion/> pp.


PointedEars
--
Prototype.js was written by people who don't know javascript for people
who don't know javascript. People who don't know javascript are not
the best source of advice on designing systems that use javascript.
-- Richard Cornford, cljs, <f806at$ail$1$8300...@news.demon.co.uk>

[tlvp]

On Tue, 07 Aug 2012 17:52:05 +0200, Thomas 'PointedEars' Lahn wrote:

> Andreas Prilop wrote:
>
>> On Tue, 7 Aug 2012, Thomas 'PointedEars' Lahn wrote:
>>>> However, EF BF BD is not “o with acute accent” but the byte order
>>>> mark (BOM).
>>>
>>> The UTF-8 BOM is EF _BB BF_. EF BF BD, on the other hand,
>>> encodes U+FFFD REPLACEMENT CHARACTER in UTF-8.
>>
>> Right!
>
> (Which indicates that something went wrong while *sending* the e-mail, not
> reading it.)

That's good to know. Thank you!

> I have found it out with the excellent Unicode Code Converter and UniView
> Web tools by Richard Ishida (W3C, Unicode Consortium), IMHO very useful for
> Web authors and programmers alike:
>
> <http://www.rishida.net/tools/conversion/> pp.

Have you any advice you might suggest I pass along to the sender of that
email (the Amazon.com affiliate CreateSpace.com) so they can fix whatever
it is that "went wrong while sending the e-mail"?

> PointedEars

TIA; and cheers, -- tlvp

[Andreas Prilop]

On Tue, 7 Aug 2012, tlvp wrote:

> Have you any advice you might suggest I pass along to the sender
> of that email (the Amazon.com affiliate CreateSpace.com) so they
> can fix whatever it is that "went wrong while sending the e-mail"?

If possible, try both
Content-Transfer-Encoding: Quoted-Printable
Content-Transfer-Encoding: 8bit

If possible, try both
Charset=ISO-8859-1
Charset=UTF-8

That makes four combinations.

[Thomas 'PointedEars' Lahn]

Andreas Prilop wrote:

> On Tue, 7 Aug 2012, tlvp wrote:
>> Have you any advice you might suggest I pass along to the sender
>> of that email (the Amazon.com affiliate CreateSpace.com) so they
>> can fix whatever it is that "went wrong while sending the e-mail"?
>
> If possible, try both
> Content-Transfer-Encoding: Quoted-Printable
> Content-Transfer-Encoding: 8bit
>
> If possible, try both
> Charset=ISO-8859-1
> Charset=UTF-8
>
> That makes four combinations.

That is bad or at least unfitting advice. The server-side application (at
CreateSpace.com) needs to be fixed. For some reason, conversion to a
Unicode encoding failed partially, so the Unicode replacement character was
generated by the converter and encoded *properly* in the Internet message.

The part of the server-side application that does the conversion needs to be
fixed. Using a non-Unicode encoding for output when the underlying data is
stored in a Unicode encoding (such as UTF-8) – which I am assuming here –
should be the last resort, as Unicode is a superset of other character sets.

It is equally possible that conversion to a Unicode encoding already failed
when storing the data. In either case, the application that generates the
e-mail text, not the application that generates the e-mail, needs to be
fixed.

Since the discussion has become relevant to Web authors at last, I set F'up2
comp.infosystems.www.authoring.misc; modify as you see fit. Thank you in
advance for not posting off-topic again.


PointedEars
--
Danny Goodman's books are out of date and teach practices that are
positively harmful for cross-browser scripting.
-- Richard Cornford, cljs, <cife6q$253$1$8300...@news.demon.co.uk> (2004)