Encoding mess
Gilles Sadowski
I use mutt in conjunction with emacs (set editor=emacsclient).
For quite some time, I have trouble with character encodings.
Here are several instances of behaviour I don't understand:
1. I receive a mail; typing 'v', mutt indicates that the encoding is
iso-8859-1.
I type 'r' and the quoted text is sent to an emacs buffer where the
accented characters are not correctly displayed.
Strangely, with this same mail, when I type 'e', the text (together
with headers etc.) is sent to an emacs buffer, but this time the
accented characters are displayed correctly!
2. I prepare 2 test files with emacs. The caracters input is the same
for both but one I save with iso-8859-1 and the other with utf-8.
I attach both files to a mail but mutt reports the first one as being
utf-8 and the other as iso-8859-1 (although it is the other way
around)!
3. I type 'm' and write a mail in emacs that contains accented
characters.
'C-x #' to return to mutt and mutt reports the mail as utf-8. I send
it and the recipient gets garbled characters.
I write the same mail but this, as I know that the encoding is not
utf-8, I type 'C-t' in mutt and when it asks 'Convert to iso-8859-1
upon sending?' I answer 'No'. This time the message is displayed
correctly at the recipient.
This is very frustrating. Any hint on how to solve this?
Other info:
- I use mutt inside an 'uxterm' terminal, locale en_US.UTF-8 (set
charset=utf-8)
- GNU/Linux Debian (testing/unstable)
Gilles
Sven Guckes
> - I use mutt inside an 'uxterm' terminal..
"upgrade"
homework:
$ mutt -v
$ file `which uxterm`
Sven
Alain Bench
In article <2de1633d.03092...@posting.google.com>
Gilles Sadowski <gilles....@ulb.ac.be> wrote:-
> I receive a mail; typing 'v', mutt indicates that the encoding is
> iso-8859-1. I type 'r' and the quoted text is sent to an emacs buffer
> where the accented characters are not correctly displayed.
Mutt converts the Latin-1 mail to $charset (here UTF-8) before
feeding the editor: Is your Emacs configured to read UTF-8 text?
> with this same mail, when I type 'e', the text (together with headers
> etc.) is sent to an emacs buffer, but this time the accented
> characters are displayed correctly!
Here Mutt feeds editor with raw mail, unconverted. So this seems to
mean your Emacs is configured to read properly Latin-1 text. Not good in
an UTF-8 locale.
> I prepare 2 test files with emacs. The caracters input is the same for
> both but one I save with iso-8859-1 and the other with utf-8. I attach
> both files to a mail but mutt reports the first one as being utf-8 and
> the other as iso-8859-1 (although it is the other way around)!
May seem fuzzy, but it makes sense once you know the charset
displayed in compose menu is not the file's charset (Mutt always assumes
it is $charset), but the charset to which the file will be converted
upon sending (the first suitable of $send_charset).
- 1st file is real Latin-1, but because of your $charset Mutt assumes
it's UTF-8, and converting it to any of $send_charset fails. You have to
use <edit-type> (^T) to instruct Mutt it's Latin-1 and you don't want
conversion. Or use the $file_charset feature of Takashi Takizawa's
Japanese patch (a must-have).
- 2nd file is real UTF-8, Mutt chooses Latin-1 as the best adapted
first necessary and sufficient charset in $send_charset, and upon
sending converts UTF-8 to Latin-1, labelled Latin-1. All is well.
> I type 'm' and write a mail in emacs that contains accented
> characters. 'C-x #' to return to mutt and mutt reports the mail as
> utf-8. I send it and the recipient gets garbled characters.
Probably Emacs has saved your writings in Latin-1. Can't work.
Configure your editor so it saves in your current locale's charset.
> Any hint on how to solve this?
Set your editor to read/write your terminal's charset. All will
magically become not correctly garbled, whatever this means. ;-)
Bye! Alain.
--
Mutt muttrc tip to send mails in best adapted first necessary and sufficient
charset (version for East Europe Latin-2/CP-852/CP-1250 terminal users):
set send_charset="us-ascii:iso-8859-1:iso-8859-15:windows-1252:iso-8859-2:windows-1250:utf-8"
Gilles Sadowski
> Hi Gilles,
>
Hello Alain.
> In article <2de1633d.03092...@posting.google.com>
> Gilles Sadowski <gilles....@ulb.ac.be> wrote:-
>
> > I receive a mail; typing 'v', mutt indicates that the encoding is
> > iso-8859-1. I type 'r' and the quoted text is sent to an emacs buffer
> > where the accented characters are not correctly displayed.
>
> Mutt converts the Latin-1 mail to $charset (here UTF-8) before
> feeding the editor: Is your Emacs configured to read UTF-8 text?
>
No, indeed.
I came to that conclusion trying to sort out the rather elliptic post
by Sven; but yours explains everything very clearly!
I read part of emacs doc in order to configure it so that the default
encoding for new files should be UTF-8, but I still doesn't know how
to do it :-(
Even so, I'm not sure this UTF-8 default for emacs is the right thing
to do: I wonder if this would not cause emacs to improperly read files
written in other charset (?)
So what you suggests seems much better:
> > Any hint on how to solve this?
>
> Set your editor to read/write your terminal's charset. All will
> magically become not correctly garbled, whatever this means. ;-)
>
I'd be glad if you can tell me how to do it for emacs (or just point
to some URL).
Also, since I use emacsclient with an already started server, the
encoding switch should happen by testing the locale environment for
each new buffer (but is the right locale settings being passed at
buffer creation? And how to access it? Probably I'd better ask these
questions on an emacs discussion group!).
Maybe I should just give up on using emacs for writing e-mails.
I've just skimmed through the "UTF-8 and Unicode": they suggest Qemacs
for use on UTF-8 terminals.
Which editor do you use?
Thanks a lot for your help.
Gilles
Gilles Sadowski
material at
http://mail.nl.linux.org/linux-utf8/2002-12/msg00161.html
I tried it in my .emacs, but it doesn't seem to do much good (accented
characters still garbled...)