Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

Character set used in Message List pane

374 views
Skip to first unread message

Bob Henson

unread,
Apr 23, 2011, 9:08:21 AM4/23/11
to
I've been unsuccessfully looking around to try to find a setting that
will change the character set used in the message list pane (not the
font, I know how to do that). The small problem I have is that when the
characters ñ or Ñ appear in a poster's name, the characters display
correctly in the message pane, but they show up as the "Black diamond
with a question mark on it" symbol that shows up when Thunderbird can't
cope with a character.

Changing the font doesn't affect it (usually set to Verdana), so I'm
assuming it's the character set being used that's wrong. No huge deal,
but it's irritating, so if anyone has any ideas how to fix it, it would
be appreciated.


--
Dyslexia lures, KO!

Mike Easter

unread,
Apr 23, 2011, 9:48:42 AM4/23/11
to
Bob Henson wrote:
> I've been unsuccessfully looking around to try to find a setting that
> will change the character set used in the message list pane (not the
> font, I know how to do that).

That pane is called the 'thread' pane, even if some of us don't sort by
References :-) (Sort by Unthreaded)

> The small problem I have is that when the characters ñ or Ñ appear in
> a poster's name, the characters display correctly in the message
> pane, but they show up as the "Black diamond with a question mark on
> it" symbol that shows up when Thunderbird can't cope with a
> character.

I find this moz kb article very helpful
http://kb.mozillazine.org/Font_settings_in_Thunderbird

Your Tb 3.1.9 should work better than Tb 2 did (does for those still
using it).

Options/ Display/ Advanced button/ Top menu select Fonts for Other
languages (because Tb thinks UTF-8 is another language, sorta)

2nd section Allow messages to use other fonts

3rd section character encodings Incoming mail user defined; I think you
are already configured for outgoing UTF-8.


--
Mike Easter

Ralph Fox "[testing-ñÑ-ñÑ-ñÑ]"

unread,
Apr 23, 2011, 9:58:11 AM4/23/11
to
On Sat, 23 Apr 2011 14:05:08 +0100, in message
news://news.mozilla.org/a8-dnXdwlNXdUi_Q...@mozilla.org
Bob Henson wrote:

> I've been unsuccessfully looking around to try to find a setting that
> will change the character set used in the message list pane (not the
> font, I know how to do that). The small problem I have is that when the
> characters ñ or Ñ appear in a poster's name, the characters display
> correctly in the message pane, but they show up as the "Black diamond
> with a question mark on it" symbol that shows up when Thunderbird can't
> cope with a character.

AIUI Thunderbird doesn't use a character set for the message list pane.
Rather, Thunderbird tries to interpret each message's headers (such as
the sender's name) using the correct character set for that particular
message. Different messages can use different character sets for their
sender's name and other headers.

For some messages, information is available in the message to tell
Thunderbird what the header's character set is. Other times,
Thunderbird has to revert to using a default character encoding.


> Changing the font doesn't affect it (usually set to Verdana), so I'm
> assuming it's the character set being used that's wrong. No huge deal,
> but it's irritating, so if anyone has any ideas how to fix it, it would
> be appreciated.

Have you tried changing this setting?
1) Right-click on the newsgroup or folder
2) Select "Properties" from the right-click menu
3) On the "General Information" tab, adjust the
"Default Character Encoding" setting.

--
Kind regards
Ralph

Bob Henson

unread,
Apr 23, 2011, 10:15:37 AM4/23/11
to

I already have all those settings, and the page bookmarked. However,
Ralph just replied using those characters in his name and they displayed
perfectly in both panes, so there must be a problem with the one person
who uses them regularly. Thanks anyway, it looks as though it's someone
else's problem not mine.

--
"If it's a hobby to us and a job to you, why are you doing such a shoddy
job?" - Linus Torvalds to Microsoft

Bob Henson

unread,
Apr 23, 2011, 10:24:45 AM4/23/11
to

The characters displayed just fine in your message in both panes. I
think the clue lies in your second paragraph, I looked back and the
poster whose messages won't display correctly uses OE6, which does not
put a Content Type header in. Having said that, I have the particular
newsgroup set to force UTF-8, which should have covered it. However,
I'll go and try and ISO setting just to see if it makes a difference -
but for normal use I can't make the change just for one person's
messages. Thanks for the tips anyway, it certainly looks as though it's
his fault not mine.
--
"I am" is the shortest English Sentence. "I do" gets you the longest.

Mike Easter

unread,
Apr 23, 2011, 10:29:29 AM4/23/11
to
Bob Henson wrote:

> Ralph just replied using those characters in his name and they displayed
> perfectly in both panes, so there must be a problem with the one person
> who uses them regularly. Thanks anyway, it looks as though it's someone
> else's problem not mine.

Ralph's name non-ascii test characters was using MIME Q encoding/
encoded word syntax/ as you can see in his message source.

Compare your problem sender's 'method' of encoding the characters with
how Ralph's is done.


--
Mike Easter

Bob Henson

unread,
Apr 23, 2011, 10:37:36 AM4/23/11
to
On 23/04/2011 14:58, Ralph Fox "[testing-ñÑ-ñÑ-ñÑ]" wrote:

Hmm, this is quite odd, I just forced the newgroup concerned to default
to ISO-8859-1 and the characters displayed correctly - in both panes.
Whilst that is odd (why should the message pane and the thread pane
appear differently?) unless ISO-8859-1 causes problems elsewhere, it
would appear that my best bet would be to change everything to default
to that set. From my very limited knowledge, I would have thought that
UTF-8 would have covered everything. Ah well, thanks again.

--
Change is inevitable, except from a vending machine.

Tony Mechelynck

unread,
Apr 23, 2011, 11:06:21 AM4/23/11
to Bob Henson

Well, since your Thunderbird (like my SeaMonkey) was able to represent
them correctly in the headers of Ralph's message, at least we know that
they have the capability.

Now there are quite a number of encodings which may be used to represent
the (uppercase and lowercase) n-with-tilde character: OT1H any encoding
able to represent either the full set of Unicode codepoints (including
UTF-8, GB18030, and all endianness variants of UTF-16 and UTF-32) or
only the lowest 65536 codepoints (UCS-2 with BOM, UCS-2be, UCS-2le) but
also OTOH all "Western" encodings, which are specifically constructed to
meet the needs of (among others) Spanish: ISO-8859-1, ISO-8859-15,
cp850, MacRoman, Windows-1252, etc. If your "usual" correspondent's
messages use one of these but specify something else (or nothing) in
their From-line, then the receiving mail client would have to be
misconfigured the same way in order to understand it. It might be
interesting, if you still have one of those messages, to use "View
Source" (Ctrl+U) in order to see the exact coding used by the sending
mailer's headers (and be able to tell him what's wrong with them).


Best regards,
Tony.
--
Government spending? I don't know what it's all about. I don't know
any more about this thing than an economist does, and, God knows, he
doesn't know much.
-- Will Rogers

Mike Easter

unread,
Apr 23, 2011, 11:14:11 AM4/23/11
to

The Flying Nun (Flyiñg Ñuñ) in the .uk group doesn't use Q encoding or
any encoded word syntax ie b64 to handle the non-ascii characters in his
handle.

previewable bitly link to GG example http://bit.ly/gLFh0Z+


--
Mike Easter

Tony Mechelynck

unread,
Apr 23, 2011, 11:30:20 AM4/23/11
to Bob Henson

UTF-8 can represent anything, but it requires between two and four bytes
for each character not part of the 7-bit US-ASCII character set. OTOH,
ISO-8859-1, a "Western" encoding, can only represent the first 256
codepoints of Unicode, but it does it by means of exactly one byte per
character. This means that it is more economical than UTF-8 for any
Western language making a heavy use of accented letters (English, OTOH,
hardly uses any accented letters at all, and for that language UTF-8,
ISO-8859-anything, or even US-ASCII, are about equivalent).

Normally, messages using headers in any encoding other than 7-bit
US-ASCII ought to specify in the concerned headers which encoding is
used, and use the same kind of quoted-printable-like recoding as that
used in Ralph's message. Whenever a message does that, the wisest way to
treat it is not to "force" any different encoding on it. However, there
are a few messages going around using 8-bit encodings without specifying
what they use. The most frequent case in Western Europe and the Americas
is with one of ISO-8859-1, ISO-8859-15 or Windows-1252, but it might be
different in Eastern Europe or Asia. So I think you can afford to
"default" messages which don't specify an encoding to ISO-8859-1, but
IMHO you shouldn't "force" anything on those which already mention what
they use.

Best regards,
Tony.
--
Save energy: be apathetic.

Bob Henson

unread,
Apr 23, 2011, 11:49:45 AM4/23/11
to
On 23/04/2011 16:14, Mike Easter wrote:
> Mike Easter wrote:
>> Bob Henson wrote:
>>
>>> Ralph just replied using those characters in his name and they displayed
>>> perfectly in both panes, so there must be a problem with the one person
>>> who uses them regularly. Thanks anyway, it looks as though it's someone
>>> else's problem not mine.
>>
>> Ralph's name non-ascii test characters was using MIME Q encoding/
>> encoded word syntax/ as you can see in his message source.
>>
>> Compare your problem sender's 'method' of encoding the characters with
>> how Ralph's is done.
>
> The Flying Nun (Flyiñg Ñuñ) in the .uk group doesn't use Q encoding or
> any encoded word syntax ie b64 to handle the non-ascii characters in his
> handle.
>
> previewable bitly link to GG example http://bit.ly/gLFh0Z+
>
>
The very chap that I had trouble with. However, having set everything
here back to ISO-8859-1, as you will see in the above quote (perhaps?),
it didn't work in the message window in your message now.

--
Prediction is very difficult, especially about the future. - Niels Bohr

Bob Henson

unread,
Apr 23, 2011, 11:55:56 AM4/23/11
to
On 23/04/2011 16:14, Mike Easter wrote:
> Mike Easter wrote:
>> Bob Henson wrote:
>>
>>> Ralph just replied using those characters in his name and they displayed
>>> perfectly in both panes, so there must be a problem with the one person
>>> who uses them regularly. Thanks anyway, it looks as though it's someone
>>> else's problem not mine.
>>
>> Ralph's name non-ascii test characters was using MIME Q encoding/
>> encoded word syntax/ as you can see in his message source.
>>
>> Compare your problem sender's 'method' of encoding the characters with
>> how Ralph's is done.
>
> The Flying Nun (Flyińg Ńuń) in the .uk group doesn't use Q encoding or
> any encoded word syntax ie b64 to handle the non-ascii characters in his
> handle.
>
> previewable bitly link to GG example http://bit.ly/gLFh0Z+
>
>

I had to go back to UTF-8 to get your message to display his name
correctly. Could the answer to the whole thing be the Thunderbird
doesn't automatically detect the settings correctly - a bug, in other words?

Mike Easter

unread,
Apr 23, 2011, 12:52:06 PM4/23/11
to
Bob Henson wrote:
> Mike Easter wrote:

>>> Ralph's name non-ascii test characters was using MIME Q encoding/
>>> encoded word syntax/ as you can see in his message source.

>> The Flying Nun (Flyiñg Ñuñ) in the .uk group doesn't use Q encoding or

>> any encoded word syntax ie b64 to handle the non-ascii characters in his
>> handle.

> I had to go back to UTF-8 to get your message to display his name


> correctly. Could the answer to the whole thing be the Thunderbird
> doesn't automatically detect the settings correctly - a bug, in other words?

The situation with displaying the tildes in the message body just now
when my headers have announced the charset (which will be there in the
body below) as UTF-8 is a different problem/situation from
the/your/Nun's *not* encoding the tildes in the manner in which Ralph
encoded his experimental name tildes in an earlier message in this thread.

That is, you can configure your Tb3 to 'flip' itself automatically
between ISO-8859-1 and UTF-8 depending on what is required and
named/identified in the message headers for the message body -- but that
business doesn't cover what is happening in the thread pane - which is
headers.

That is why - a good reason - that the content of the headers should
play by the MIME 'rules' of Q encoding (or b64). I think the moz kb
article I cited earlier mentions the RFCs and so does this wiki article

http://en.wikipedia.org/wiki/MIME#Encoded-Word
Multipurpose Internet Mail Extensions (MIME) is an Internet standard
that extends the format of e-mail to support:
* Text in character sets other than ASCII
* Non-text attachments
* Message bodies with multiple parts
* Header information in non-ASCII character sets

--
Mike Easter

Ralph Fox

unread,
Apr 23, 2011, 12:55:42 PM4/23/11
to
On Sat, 23 Apr 2011 16:55:56 +0100, in message
news://news.mozilla.org/lo-dnf6NJtUaay_Q...@mozilla.org
Bob Henson wrote:

> I had to go back to UTF-8 to get your message to display his name
> correctly. Could the answer to the whole thing be the Thunderbird
> doesn't automatically detect the settings correctly - a bug, in other words?


Here, my Thunderbird correctly displayed Mike Easter's message,
including the name "Flyiñg Ñuñ". Thunderbird automatically picked up
the charset=UTF-8 from the Content-Type header of Mike Easter's message.
My "Default Character Encoding" is ISO-8859-1.

Make sure you do not have a check-mark in the box "Apply default to all
messages in the folder (individual message character encoding settings
and auto-detection will be ignored)".

If you did have a check-mark in this box then Thunderbird would ignore
the charset specified in the Content-Type header.


--
Kind regards
Ralph

Bob Henson

unread,
Apr 23, 2011, 1:49:47 PM4/23/11
to
On 23/04/2011 17:55, Ralph Fox wrote:
> On Sat, 23 Apr 2011 16:55:56 +0100, in message
> news://news.mozilla.org/lo-dnf6NJtUaay_Q...@mozilla.org
> Bob Henson wrote:
>
>> I had to go back to UTF-8 to get your message to display his name
>> correctly. Could the answer to the whole thing be the Thunderbird
>> doesn't automatically detect the settings correctly - a bug, in other words?
>
>
> Here, my Thunderbird correctly displayed Mike Easter's message,
> including the name "Flyińg Ńuń". Thunderbird automatically picked up
> the charset=UTF-8 from the Content-Type header of Mike Easter's message.
> My "Default Character Encoding" is ISO-8859-1.
>
> Make sure you do not have a check-mark in the box "Apply default to all
> messages in the folder (individual message character encoding settings
> and auto-detection will be ignored)".
>
> If you did have a check-mark in this box then Thunderbird would ignore
> the charset specified in the Content-Type header.
>
>
I'll go back and make sure that I didn't have the check-mark set, but I
don't think I did.

--
If all seems to be going well, you obviously have no idea what is really
happening.

Bob Henson

unread,
Apr 23, 2011, 2:17:47 PM4/23/11
to
On 23/04/2011 17:55, Ralph Fox wrote:

None of them were checked, they were all free to switch. Apparently they
aren't doing, though.

--
You know you're old when you don't mind where your partner goes out to,
so long as they don't ask you along too.

Larry Gusaas

unread,
Apr 23, 2011, 2:46:27 PM4/23/11
to
Do you have auto-detect enabled? Go to View>Character Encoding>Auto-Detect and select "Universal".


Larry
--
_____________________________________________________________________________

Larry I. Gusaas

Moose Jaw, Saskatchewan   Canada
Website:   http://larry-gusaas.com
"An artist is never ahead of his time but most people are far behind theirs." - Edgard Varese


Bob Henson

unread,
Apr 23, 2011, 3:04:01 PM4/23/11
to
On 23/04/2011 19:46, Larry Gusaas wrote:
> On 2011/04/23 12:17 PM Bob Henson wrote:
>> On 23/04/2011 17:55, Ralph Fox wrote:
>>> On Sat, 23 Apr 2011 16:55:56 +0100, in message
>>> news://news.mozilla.org/lo-dnf6NJtUaay_Q...@mozilla.org
>>> Bob Henson wrote:
>>>
>>>> I had to go back to UTF-8 to get your message to display his name
>>>> correctly. Could the answer to the whole thing be the Thunderbird
>>>> doesn't automatically detect the settings correctly - a bug, in other words?
>>>
>>> Here, my Thunderbird correctly displayed Mike Easter's message,
>>> including the name "Flyiñg Ñuñ". Thunderbird automatically picked up
>>> the charset=UTF-8 from the Content-Type header of Mike Easter's message.
>>> My "Default Character Encoding" is ISO-8859-1.
>>>
>>> Make sure you do not have a check-mark in the box "Apply default to all
>>> messages in the folder (individual message character encoding settings
>>> and auto-detection will be ignored)".
>>>
>>> If you did have a check-mark in this box then Thunderbird would ignore
>>> the charset specified in the Content-Type header.
>>>
>>>
>> None of them were checked, they were all free to switch. Apparently they
>> aren't doing, though.
>>
> Do you have auto-detect enabled? Go to View>Character Encoding>Auto-Detect and
> select "Universal".
>
>
I do, yes, but thanks for the idea.
--
No I haven't got a personality disorder - all three of us are just fine!

EE

unread,
Apr 23, 2011, 3:26:27 PM4/23/11
to
What about View > Character Encoding? If you change that to UTF-8, does
that fix it?

Bob Henson

unread,
Apr 23, 2011, 3:36:28 PM4/23/11
to

No, afraid not.

--
The secret of a successful relationship in incompatibility - he has to
have the income, she has to have the patibility.

Ralph Fox

unread,
Apr 23, 2011, 3:56:30 PM4/23/11
to
On Sat, 23 Apr 2011 19:17:47 +0100, in message
news://news.mozilla.org/FsydnSKkC4pbii7Q...@mozilla.org
Bob Henson wrote:


Then I am stumped.

All I can suggest is to try it again, but this time with Thunderbird
running in Thunderbird's "Safe Mode".


--
Kind regards
Ralph

Ron K.

unread,
Apr 23, 2011, 6:05:40 PM4/23/11
to
Ralph Fox on 4/23/2011 3:56 PM, keyboarded a reply:

An FYI, do a Config Edit and filter on "UTF" to see a short list of
related prefs. One of the defaults is network.standard-url.encode-utf8 set
to true.

My understanding is UTF-8 is the internet standard for headers and URL's.

For users on Windows, there are two defaults fonts used, depending on
detection of XP or Vista/Win7. TB3.1 began using Calibri woth Vista/Win7
since it is system font shipped with the OS. This is a UNICODE font with
extended international character support, so less likely to have an
'unknown' character substitution in a screen display.

--
Ron K.
Who is General Failure, and why is he searching my HDD?
Kernel Restore reported Major Error used BSOD to msg the enemy!

Bob Henson

unread,
Apr 24, 2011, 6:49:18 AM4/24/11
to
Ron K. wrote:

> Path: Xl.tags.giganews.com!border1.nntp.dca.giganews.com!nntp.giganews.com!local2.nntp.dca.giganews.com!nntp.mozilla.org!news.mozilla.org.POSTED!not-for-mail
> NNTP-Posting-Date: Sat, 23 Apr 2011 17:05:18 -0500
> Date: Sat, 23 Apr 2011 18:05:40 -0400
> From: "Ron K." <kil...@gisco.net>
> User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.2.15) Gecko/20110303 Thunderbird/3.1.9
> MIME-Version: 1.0
> Newsgroups: mozilla.support.thunderbird
> Subject: Re: Character set used in Message List pane
> References: <a8-dnXdwlNXdUi_Q...@mozilla.org> <-M6dnenEP-cmRS_Q...@mozilla.org> <guqdnXIABYWYQi_Q...@mozilla.org> <hZudnYrSe4jXfy_Q...@mozilla.org> <3qednRSGh9dZcS_Q...@mozilla.org> <lo-dnf6NJtUaay_Q...@mozilla.org> <jJOdne1ttosOmS7Q...@mozilla.org> <FsydnSKkC4pbii7Q...@mozilla.org> <pI-dnR4HZoVvsy7Q...@mozilla.org>
> In-Reply-To: <pI-dnR4HZoVvsy7Q...@mozilla.org>
> Content-Type: text/plain; charset=ISO-8859-15; format=flowed
> Content-Transfer-Encoding: 7bit
> Message-ID: <WtadnWcA95qD0C7Q...@mozilla.org>
> Lines: 42
> X-Usenet-Provider: http://www.giganews.com
> NNTP-Posting-Host: 69.43.221.183
> X-AuthenticatedUsername: NoAuthUser
> X-Trace: sv3-zJ1PzxZ3lNXFYFFHmW+lSq+7673kKrChktzNU9fLmiS4GMCydkFrge0oelVeD/7RWgCkb/fyokvKCWS!o0QzlAG+kp/P0hJ8htWIj3MOIp2vDzdSBxFXO/gJY3I+NHdq+ZbZVUnk1wuFTqK9pHsOryVavd9+!+gJgpAkQ3dHC
> X-Complaints-To: ab...@mozilla.org
> X-DMCA-Complaints-To: ab...@mozilla.org
> X-Abuse-and-DMCA-Info: Please be sure to forward a copy of ALL headers
> X-Abuse-and-DMCA-Info: Otherwise we will be unable to process your complaint properly
> X-Postfilter: 1.3.40
> Bytes: 3281
> X-Original-Bytes: 3220
> Xref: number.nntp.dca.giganews.com mozilla.support.thunderbird:115880

As a last resort I may try switching to that to see if it affects the
problem in any way - I use Verdana because my old eyes find it easy to
read.


--
Work is the curse of the drinking classes.

0 new messages