UTF-8 support

[patric aristide]

Hi

Not sure if this is actually possible with Alpine but I'm
struggling to display non English characters correctly.
ATM I've set alpine to use ISO 8859-1 which works okay but
isn't ideal. My OS doesn't support UTF-8 and I'm accessing
alpine on a remote server via terminal. The terminal itself
doesn't support UTF-8 either but uses Latin-1 instead.
Is there a way of displaying UTF-8 mail correctly or would
this rely on my terminal?
Server runs NetBSD but I'm on RISC OS here.

cheers
Patric
--
GL

[J.O. Aho]

Think you may need the iconv patch for pine.
Take a look at this post: http://marc.info/?l=pine-info&m=106323768018267&w=2

Not sure if it will work with alpine or if it would break something else.

--

//Aho

[Lucas Levrel]

Alpine uses utf-8 internally and natively converts to any display charset
(see help of M S C "Display Character Set").

If I understand Patric correctly, he's willing to display non-latin-1
chars in a latin-1 terminal. I don't think it's a pine-related question,
rather an OS one, so maybe some comp.os.* group would be more appropriate.

--
LL

[patric aristide]

On 2012-05-31, Lucas Levrel <lucas....@u-pec.fr> wrote:
> Le 31 mai 2012, J.O. Aho a ?crit :

Yeah, sounds like it can't be done then. If Alpine converts to whatever
the terminal uses I'm stuck with ISO 8859-1.

thanks
Patric
--
GL

[Andreas Prilop]

On Wed, 30 May 2012, patric aristide wrote:

> I'm struggling to display non English characters correctly.

Which characters exactly?

> My OS doesn't support UTF-8

I wonder which operating system you are using in the year 2012.
What do you get on
$ locale -a
?

> The terminal itself doesn't support UTF-8 either but
> uses Latin-1 instead.

Which terminal program? Can you select any other encodings?

[patric aristide]

On 2012-05-31, Andreas Prilop <prilo...@trashmail.net> wrote:
> On Wed, 30 May 2012, patric aristide wrote:
>
>> I'm struggling to display non English characters correctly.
>
> Which characters exactly?
>

German Umlauts, Euro symbol etc. ISO 8859-1 will display them but
Alpine complains about not using the correct encoding which would
be UTF-8

>> My OS doesn't support UTF-8
>
> I wonder which operating system you are using in the year 2012.
> What do you get on
> $ locale -a
> ?

"file not found" but that's not too surprising given RISC OS
doesn't use unix commands. I did mention the OS in the original
post, sorry if that wasn't clear though.
RISC OS uses a mixture between the so called territory module
and alphabets. Unfortunately the UTF8 alphabet, while available
is still under developement and not really usable.
Most users live in the UK so international language support is
low priority.

>
>> The terminal itself doesn't support UTF-8 either but
>> uses Latin-1 instead.
>
> Which terminal program? Can you select any other encodings?

Nettle which AFAIK uses the zap text editor's fonts but doesn't
support or even know about Unicode itself. I'm not 100% sure
about it though, contacted the developer for more info.

Really just wanted to know if Alpine would give me Unicode
without the underlying OS here supporting it (yet).
Since Alpine converts to whatever the terminal is capable of
I'm out of luck it seems.

Patric
--
GL

[unruh]

Yes. Alpine does not paint the pixels onto the screen. It hands off that
task to the operating system/terminal program, as it should.
>
> Patric

[cha...@washington.edu]

On May 30, 4:25 pm, patric aristide <t...@sdf-eu.org> wrote:

> Not sure if this is actually possible with Alpine but I'm
> struggling to display non English characters correctly.
> ATM I've set alpine to use ISO 8859-1 which works okay but
> isn't ideal. My OS doesn't support UTF-8 and I'm accessing
> alpine on a remote server via terminal. The terminal itself
> doesn't support UTF-8 either but uses Latin-1 instead.
> Is there a way of displaying UTF-8 mail correctly or would
> this rely on my terminal?

Press M S C and take a look at the options:

Display Character Set
Keyboard Character Set
Posting Character Set
Unknown Character Set

You want to set Display Character Set to ISO-8859-1, which is what
your terminal can handle. Read the help text to understand what this
option does, since the behavior you want to set is combined between
the values of these options.

I hope this helps.

--
Eduardo
http://patches.freeiz.com/alpine/

[Lucas Levrel]

Le 31 mai 2012, patric aristide a écrit :

> German Umlauts, Euro symbol etc. ISO 8859-1 will display them but
> Alpine complains about not using the correct encoding which would
> be UTF-8

> Really just wanted to know if Alpine would give me Unicode
> without the underlying OS here supporting it (yet).
> Since Alpine converts to whatever the terminal is capable of
> I'm out of luck it seems.

I don't understand exactly understand what your problem is. Your original
post seemed to indicate you're stuck with a latin-1 terminal.

Is the problem the warnings from Alpine? Then maybe set Display Character
Set to Latin 1.

Do you want to force Alpine to output utf-8 chars to your terminal? Then
maybe set Display Character Set to UTF 8. But that sounds weird, unless
you can use a UTF-8 capable terminal.

--
LL

[Andreas Prilop]

On Thu, 31 May 2012, patric aristide wrote:

>> Which characters exactly?
>
> German Umlauts, Euro symbol etc.

I asked "exactly"; you answered "etc".
You ask for help but you continue to hide necessary information.

> ISO 8859-1 will display them

ISO-8859-1 has *no* euro sign.

> Alpine complains about not using the correct encoding
> which would be UTF-8

You do not need UTF-8 for German letters and the euro sign.
You can do with Windows-1252. Set the character-set variables
in Alpine to "Windows-1252". Even if you cannot see a euro sign
in your terminal, you will be able to *quote* a euro sign
from other people correctly.

Here is a small test for you in Windows-1252:

Ä A diaeresis
ä a diaeresis
É E acute
é e acute
Ö O diaeresis
ö o diaeresis
Ü U diaeresis
ü u diaeresis
€ euro sign
¢ cent sign

[Andreas Prilop]

On Fri, 1 Jun 2012, I wrote:

> Set the character-set variables in Alpine to "Windows-1252".

And also
[ ] disable-charset-conversions
[X] downgrade-multipart-to-text
[X] pass-c1-control-characters-as-is

[patric aristide]

On 2012-06-01, Lucas Levrel <lucas....@u-pec.fr> wrote:
> Le 31 mai 2012, patric aristide a ?crit :

>
>> German Umlauts, Euro symbol etc. ISO 8859-1 will display them but
>> Alpine complains about not using the correct encoding which would
>> be UTF-8
>
>> Really just wanted to know if Alpine would give me Unicode
>> without the underlying OS here supporting it (yet).
>> Since Alpine converts to whatever the terminal is capable of
>> I'm out of luck it seems.
>
> I don't understand exactly understand what your problem is. Your original
> post seemed to indicate you're stuck with a latin-1 terminal.
>

exactly

> Is the problem the warnings from Alpine? Then maybe set Display Character
> Set to Latin 1.
>

yes and that's what I did

> Do you want to force Alpine to output utf-8 chars to your terminal? Then
> maybe set Display Character Set to UTF 8. But that sounds weird, unless
> you can use a UTF-8 capable terminal.
>

that's what I wanted to know and it's not possible, okay

Patric
--
GL

[patric aristide]

On 2012-06-01, Andreas Prilop <prilo...@trashmail.net> wrote:
> On Thu, 31 May 2012, patric aristide wrote:
>
>>> Which characters exactly?
>>
>> German Umlauts, Euro symbol etc.
>
> I asked "exactly"; you answered "etc".
> You ask for help but you continue to hide necessary information.

I don't have any control over what letters people use in their
emails. Point is, I set Alpine to display ISO 8859-1 which my
terminal can handle.
Since these mails are encoded in UTF-8 Alpine warns me that it's
not set to the correct encoding and some characters might not
be displayed correctly. Hence my question.
There isn't any more information.

>
>> ISO 8859-1 will display them
>
> ISO-8859-1 has *no* euro sign.
>

RISC OS is set to use the int'l currency symbol at 0xA4
to display the Euro symbol.

>> Alpine complains about not using the correct encoding
>> which would be UTF-8
>
> You do not need UTF-8 for German letters and the euro sign.
> You can do with Windows-1252. Set the character-set variables
> in Alpine to "Windows-1252". Even if you cannot see a euro sign
> in your terminal, you will be able to *quote* a euro sign
> from other people correctly.
>

RISC OS doesn't support Windows-1252 (why should it?)

> Here is a small test for you in Windows-1252:
>

> ? A diaeresis
> ? a diaeresis
> ? E acute
> ? e acute
> ? O diaeresis
> ? o diaeresis
> ? U diaeresis
> ? u diaeresis
> ? euro sign
> ? cent sign

patric
--
GL

[Andreas Prilop]

On Fri, 1 Jun 2012, patric aristide wrote:

> I set Alpine to display ISO 8859-1 which my terminal can handle.

Use Windows-1252 in Alpine, which is a superset of ISO-8859-1.
Set your character set variables to "Windows-1252".

> Since these mails are encoded in UTF-8 Alpine warns me that it's
> not set to the correct encoding and some characters might not
> be displayed correctly.

Set
[X] quell-charset-warning

>> ISO-8859-1 has *no* euro sign.
>
> RISC OS is set to use the int'l currency symbol at 0xA4
> to display the Euro symbol.

You confuse ISO-8859-1 and ISO-8859-15.

ISO-8859-1 has *no* euro sign.

ISO-8859-15 has the euro sign at position xA4.

> RISC OS doesn't support Windows-1252 (why should it?)

Pine and Alpine support Windows-1252.

>> Here is a small test for you in Windows-1252:
>> ? A diaeresis

You are a hopeless case.

Ä A diaeresis

[Helmut Richter]

On Fri, 1 Jun 2012, Andreas Prilop wrote:

> On Fri, 1 Jun 2012, patric aristide wrote:
>
> > I set Alpine to display ISO 8859-1 which my terminal can handle.
>
> Use Windows-1252 in Alpine, which is a superset of ISO-8859-1.
> Set your character set variables to "Windows-1252".
>
> > Since these mails are encoded in UTF-8 Alpine warns me that it's
> > not set to the correct encoding and some characters might not
> > be displayed correctly.

Pine will transliterate UTF-8 characters so that you can hopefully read
them. If the character is in ISO 8859-1 (or Windows-1252 with some more
characters if supported by OS and terminal) the transliteration is
lossless. Some other characters may be replaced by similar-looking
ISO-8859-1 characters (IIRC, some accented Latin letters by their base
letters without diacritics). Other characters, among them all in
non-Latin scripts, will be renderes as ?.

Thus, the warning is in order.

> Set
> [X] quell-charset-warning

... if you know possible problems well enough that you need no warning.

I'm now using UTF-8 but have used your setup with ISO 8859-1 for many
years. I found the warning not disturbing but helpful.

--
Helmut Richter

[patric aristide]

On 2012-06-01, Andreas Prilop <prilo...@trashmail.net> wrote:

> On Thu, 31 May 2012, patric aristide wrote:
>
>
> Here is a small test for you in Windows-1252:
>

> ? A diaeresis
> ? a diaeresis
> ? E acute
> ? e acute
> ? O diaeresis
> ? o diaeresis
> ? U diaeresis
> ? u diaeresis
> ? euro sign
> ? cent sign

Sorry but all the above just displays as
"?" Please correct me if I'm wrong but since
my terminal can't read Windows-1252 I struggle to see
any advantage in having pine display it.

patric
--
GL

[patric aristide]

On 2012-06-01, Andreas Prilop <prilo...@trashmail.net> wrote:

> On Fri, 1 Jun 2012, patric aristide wrote:
>
>> I set Alpine to display ISO 8859-1 which my terminal can handle.
>
> Use Windows-1252 in Alpine, which is a superset of ISO-8859-1.
> Set your character set variables to "Windows-1252".
>
>> Since these mails are encoded in UTF-8 Alpine warns me that it's
>> not set to the correct encoding and some characters might not
>> be displayed correctly.
>
> Set
> [X] quell-charset-warning
>
>>> ISO-8859-1 has *no* euro sign.
>>
>> RISC OS is set to use the int'l currency symbol at 0xA4
>> to display the Euro symbol.
>
> You confuse ISO-8859-1 and ISO-8859-15.
> ISO-8859-1 has *no* euro sign.
> ISO-8859-15 has the euro sign at position xA4.
>

TBH you're probably right. Documentation mentioned it doesn't
confirm to standards on my plattform.

>> RISC OS doesn't support Windows-1252 (why should it?)
>
> Pine and Alpine support Windows-1252.
>
>>> Here is a small test for you in Windows-1252:
>>> ? A diaeresis
>
> You are a hopeless case.
>

> ? A diaeresis

Tried your suggested settings with mixed results, some characters
did show up correctly but Yen, Euro, fractions, degrees and a few
others (sorry but don't know what they're called) ended up as "?"

cheers
Patric
--
GL

[patric aristide]

Scrap that, my bad

[Andreas Prilop]

On Fri, 1 Jun 2012, Helmut Richter wrote:

>> [X] quell-charset-warning

>
> I found the warning not disturbing but helpful.

You may get a warning

The following text is in the "ISO-8859-1" character set.
Your display is set for the "Windows-1252" character set.
Some characters may be displayed incorrectly.

which I don't find helpful but disturbing since Windows-1252
covers ISO-8859-1.

I have a different approach: I include "Content-Type"
under "viewer-hdrs" and therefore I always see the charset
parameter of all messages.

[Andreas Prilop]

On Fri, 1 Jun 2012, patric aristide wrote:

> Please correct me if I'm wrong

You are wrong and you are a hopeless case
because you refuse to learn anything.

[patric aristide]

Are you trying to troll me?
My question was innocent enough and has been answered
sufficiently albeit not by your post I'm afraid.

Patric
--
GL