John E. Davis, slrn posts: “ Re: Re: U-FeFF ”.
_...@jeff_relf.seattle.invalid
Using slrn, you ( Junior ) posted: “ Re: Re: U-FeFF ”.
Your “Subject:” line is shown here:
http://Google.COM/group/alt.free.newsservers/msg/2fcb5466c0e936f5?dmode=source
slrn doesn't think U-FeFF is white·space;
so, when it strips leading white·space, it leaves it in.
( <Windows.H>'s GetStringTypeW() tells me what's white·space or not )
In UTF-8, U-FeFF is known as the “UTF-8 signature”,
it should be stripped out ( as Google Groups does ).
There's nothing you can do to change slrn's behavior;
John E. Davis might want to look into it.
John E. Davis
wrote:
> Your “Subject:” line is shown here:
> http://Google.COM/group/alt.free.newsservers/msg/2fcb5466c0e936f5?dmode=source
>
> slrn doesn't think U-FeFF is white·space;
> so, when it strips leading white·space, it leaves it in.
From what I understand, the primary use of U+FEFF is as a byte-order
mark at the beginning of a byte-stream (e.g., a file) to indicate the
endianness of the stream. While UTF-8 needs no such indicater, it is
sometimes used at the beginning of the stream to indicate that Unicode
is used. It is unclear what to do with the character when it occurs
in the middle of a stream, such as is the case we are discussing here.
In this case, slrn simply passes it along.
> ( <Windows.H>'s GetStringTypeW() tells me what's white·space or not )
S-Lang makes use of the Unicode tables from unicode.org for
determining what is whitespace. In particular, this table is used:
<http://www.unicode.org/Public/5.2.0/ucd/UnicodeData.txt>
It is described here:
<http://www.unicode.org/reports/tr44/tr44-4.html>
The entry for U+FEFF is:
FEFF;ZERO WIDTH NO-BREAK SPACE;Cf;0;BN;;;;;N;BYTE ORDER MARK;;;;
The relevant field value here is the third one, "Cf", which means that
the character is a "format control character". Contrast this with the
entry for U+0020, which is the ASCII space character:
0020;SPACE;Zs;0;WS;;;;;N;;;;;
The value of the third field is "Zs", which is "a space character (of
various non-zero widths)". It is for this reason that slang does not
treat U+FEFF as whitespace. This also seems to be consistent with
java, according to:
<http://www.fileformat.info/info/unicode/char/feff/index.htm>
The character is also not listed as Unicode whitespace here:
<http://en.wikipedia.org/wiki/Whitespace_(computer_science)>
With that said, I cannot help but to wonder what purpose this
character is serving in the middle of the Subject header.
Thanks,
--John
_...@jeff_relf.seattle.invalid
If you ( John Davis ) won't cross·post, perhaps you'd
subscribe to my news·group ( Alt.Free.Newsservers ).
At A.F.N. we discuss “attachment·free” NNTP servers ( and clients ).
Note, we've been talking about replies to this post:
“ Re: U-FeFF, un·like US ASCII 32, has no width. ”
news:_@Jeff_Relf.Seattle.2009_Nov16.9.24pm.oh
In Windows Live Mail, when you reply to it,
the title becomes, “ Re: U-FeFF, un·like US ASCII 32, has no width. ”
( not “ Re: Re: U-FeFF [...] ” ).
You told me:
“ The entry for U+FEFF is:
FEFF;ZERO WIDTH NO-BREAK SPACE;Cf;0;BN;;;;;N;BYTE ORDER MARK;;;;
The relevant field value here is the third one, "Cf", which means that
the character is a "format control character". ”.
Right, it's not printable ( i.e. not “black·space”, so to speak ).
User·Data ( i.e. the title ) starts after the "Subject: ";
so that's where I put the “UTF·signature”
( to indicate that it's UTF·8, not ASCII, not Windows·1252 ).