Silly but important Unicode question

[tlvp]

With the rudimentary Win Vista or Win XP text editors Notepad or Wordpad,
one has amongst the "Save as... " options:

Encoding: Unicode | Unicode big endian | UTF-8 (in Notepad), or
Save as type: ... Unicode Text Document ... (in Wordpad).

Two fundamental questions, if I may -- and one subsidiary one:

1) How exactly do Notepad's UTF-8, Unicode, and Unicode Big-endian differ?
2) Which of those (if any) does Wordpad's "Unicode" correspond to?

Beyond that,

3) which of those will give the sort of file that, other factors not
precluding it, the site validator.w3.org will agree is deserving of the
following META tag (in the HEAD element)? -- :

<meta http-equiv=Content-Type content="text/html; charset=UTF-8">

I'd always assumed it was only Notepad's UTF-8 files, that both Notepad and
Wordpad could open and resave nondestructively. Now I'm starting to wonder:
will Wordpad's Unicode Text Documents work just as well?

Many thanks for your guidance! Cheers, -- tlvp
--
Avant de repondre, jeter la poubelle, SVP.

[Jukka K. Korpela]

2012-04-19 6:26, tlvp wrote:

> 1) How exactly do Notepad's UTF-8, Unicode, and Unicode Big-endian differ?

In Notepad, UTF-8 is UTF-8 with BOM. An easy way to check this out is to
launch Notepad, enter "foo", save as "UTF-8", and open the file in
Notepad++ (http://notepad-plus-plus.org). The File format menu in
Notepad++ then shows the encoding; the menu has separately "UTF-8
(without BOM)" and "UTF-8". I think we can thus deduce what the latter
means. For further confirmation, if you right-click on the file icon in
Windows and select Properties, you'll see that the size is 6 bytes.
That's 3 bytes for the 3 Ascii letters plus 3 bytes for the UTF-8
representation of the BOM.

ObHTML: Contrary to widespread rumors around, even on W3C pages, the BOM
(Byte Order Mark, U+FEFF, at the very start of a document) is useful in
UTF-8 encoded HTML documents rather than a problem (which it used to be
in the previous millennium). Browsers have recognized it for a long
time. It acts as a useful "signature" that tells that the data is UTF-8
encoded, even in the absence of HTTP headers, XML prologues, meta tags,
and whatever.

"Unicode" in Notepad is UTF-16, more specifically UTF-16LE (little-endian).

Unicode big-endian is UTF-16BE. The difference between UTF-16LE and
UTF16BE is the byte order in 16-bit code units.

For completeness: The fourth option in the file formats in Notepad,
named "ANSI", has nothing to do with ANSI, the national standards
institute of the United States. "ANSI" is an intentional misnomer by
Microsoft and refers to whatever 8-bit encoding that is used as the
internal 8-bit encoding, depending on Windows version. None of those
encodings has ever been standardized by ANSI. In the Western world,
"ANSI" in this context is normally windows-1252.

2) Which of those (if any) does Wordpad's "Unicode" correspond to?

UTF-16LE. This is generally what "Unicode" means in Windows software,
unless stated otherwise.

> 3) which of those will give the sort of file that, other factors not
> precluding it, the site validator.w3.org will agree is deserving of the
> following META tag (in the HEAD element)? -- :
>
> <meta http-equiv=Content-Type content="text/html; charset=UTF-8">
>
> I'd always assumed it was only Notepad's UTF-8 files, that both Notepad and
> Wordpad could open and resave nondestructively.

That is correct.

> Now I'm starting to wonder:
> will Wordpad's Unicode Text Documents work just as well?

No, it won't. It would require UTF-16 instead of UTF-8. Some browsers
support UTF-16, some don't. There are remarkably few UTF-16 encoded
documents on the web. Even though modern browsers generally cope with
them, there's no good reason to use UTF-16 for HTML documents on the
web, because UTF-8 is equally expressive and favored by IEEE, W3C, etc.
For some types of text data, UTF-16 saves space and thus bandwidth when
compared with UTF-8, but such concerns are hardly significant enough.

--
Yucca, http://www.cs.tut.fi/~jkorpela/

[Helmut Richter]

On Thu, 19 Apr 2012, Jukka K. Korpela wrote:

> [...]

Jukka, thank you very much for these explanations. I never saw it compiled
in such concise form.

> ObHTML: Contrary to widespread rumors around, even on W3C pages, the BOM (Byte
> Order Mark, U+FEFF, at the very start of a document) is useful in UTF-8
> encoded HTML documents rather than a problem (which it used to be in the
> previous millennium). Browsers have recognized it for a long time. It acts as
> a useful "signature" that tells that the data is UTF-8 encoded, even in the
> absence of HTTP headers, XML prologues, meta tags, and whatever.

This is news to me. This feature can be of great help, but there is, of
course, also a danger associated with it, to wit that now HTTP headers,
meta tags, and BOM might even specify three different encodings.

> No, it won't. It would require UTF-16 instead of UTF-8. Some browsers support
> UTF-16, some don't. There are remarkably few UTF-16 encoded documents on the
> web. Even though modern browsers generally cope with them, there's no good
> reason to use UTF-16 for HTML documents on the web, because UTF-8 is equally
> expressive and favored by IEEE, W3C, etc.

To sum up, there are applications, among them HTML, where UTF-8 is more
widespread than other Unicode encodings, but hardly any where usage of
UTF-8 is discouraged.

> For some types of text data, UTF-16 saves space and thus bandwidth when
> compared with UTF-8,

Texts in most European and Middle-East non-Latin scripts (Cyrillic, Greek,
Hebrew, Arabic, Armenian, phonetic (IPA)) have approximately the same
length in all three encodings, to wit 2 bytes/char. Interspersed ASCII
characters, e.g. spaces or HTML tags, makes UTF-8 encoding a bit shorter,
interspersed non-ASCII punctuation, e.g. dashes and quotes, makes UTF-8 a
bit longer -- the first effect will nearly always outweigh the latter.

Texts in Latin script are always shorter in UTF-8 because ASCII characters
account for most of the text even in languages making heavy use of
diacritics.

Text in far-East scripts are longer in UTF-8 than in UTF-16 by at most
50%. The more interspersed ASCII they contain, the less is the difference.

So there are not so many texts where there is a real difference.

> but such concerns are hardly significant enough.

--

Helmut Richter

[Andreas Prilop]

On Thu, 19 Apr 2012, Jukka K. Korpela wrote:

> In Notepad, UTF-8 is UTF-8 with BOM. An easy way to check
> this out is to launch Notepad, enter "foo", save as "UTF-8",
> and open the file in

… Notepad, with encoding “ANSI” selected. You could then even
get rid of the BOM by deleting the first three bytes.

--
In memoriam Alan J. Flavell
http://www.alanflavell.org.uk/charset/

[Andreas Prilop]

On Wed, 18 Apr 2012, tlvp wrote:

> Subject: Silly but important Unicode question

Your questions are not silly at all.

> <meta http-equiv=Content-Type content="text/html; charset=UTF-8">

This META tag is in US-ASCII. Therefore you can specify only an encoding
that contains US-ASCII as a subset, i.e. UTF-8. With UTF-16, your
META tag would look like

< m e t a h t t p - e q u i v = ……

(Spaces here mean actually null bytes.)
You would need to know the encoding in advance to be able to read this.

You cannot specify UTF-16 or UTF-32 with such a META tag.
You can specify UTF-16 or UTF-32 only with a BOM or a real HTTP header.

See also
http://www.unicode.org/faq/utf_bom.html

[David Stone]

In article <timstreater-799F...@news.individual.net>,
Tim Streater <timst...@greenbee.net> wrote:

> In article <Pine.LNX.4.64.12...@zen.rrzn.uni-hannover.de>,

> Andreas Prilop <prilo...@trashmail.net> wrote:
>
> > On Thu, 19 Apr 2012, Jukka K. Korpela wrote:
> >
> > > In Notepad, UTF-8 is UTF-8 with BOM. An easy way to check
> > > this out is to launch Notepad, enter "foo", save as "UTF-8",
> > > and open the file in
> >

> > â•œ Notepad, with encoding ╲ANSI╡ selected. You could then even

> > get rid of the BOM by deleting the first three bytes.
>

> TextWrangler (or BBedit) will read the file properly BOM or not. It'll
> then allow you to save it with or without the BOM.

Talking of encodings and BOM, I recently ran across this page,
which clarified quite a few things for me, anyway:

<http://www.joelonsoftware.com/articles/Unicode.html>

[dorayme]

In article <timstreater-799F...@news.individual.net>,
Tim Streater <timst...@greenbee.net> wrote:

> In article <Pine.LNX.4.64.12...@zen.rrzn.uni-hannover.de>,
> Andreas Prilop <prilo...@trashmail.net> wrote:
>

> > On Thu, 19 Apr 2012, Jukka K. Korpela wrote:
> >
> > > In Notepad, UTF-8 is UTF-8 with BOM. An easy way to check
> > > this out is to launch Notepad, enter "foo", save as "UTF-8",
> > > and open the file in
> >
> > … Notepad, with encoding “ANSI” selected. You could then even
> > get rid of the BOM by deleting the first three bytes.
>

> TextWrangler (or BBedit) will read the file properly BOM or not. It'll
> then allow you to save it with or without the BOM.

Yes, that's right. In BBedit, I have been saving without BOM for
years, mainly on advice - from the brilliant Toby Inkster - from years
back. Korpela's remarks have made me start to rethink this. Modern
browsers and the advantages of 'with' maybe should now make me change,
I just don't know enough about the issue and whether working on Macs
is anything to be taken special account of in the decision.

--
dorayme

[tlvp]

Jukka, thank you! You are a treasure-trove of valuable information! Cheers,

[Swifty]

On Fri, 20 Apr 2012 08:51:09 +1000, dorayme <dor...@optusnet.com.au>
wrote:

>I have been saving without BOM for >years

If you switch, beware of running into problems with language compilers
and interpreters.

Some of these will tell you that you have "illegal characters" in your
source. You open your source in your editor (which is now configured
to accept, if not impose BoMs) and you see no illegal characters.

Some editors hide the BoMs so effectively that you won't know that
they're there unless you view the content in hex. There may even be
editors that never show it. You'd be able to recognise these by their
users having clumps of hair torn out. :-)

--
Steve Swift
http://www.swiftys.org.uk/swifty.html
http://www.ringers.org.uk

[dorayme]

In article <1a32p7588rt185egu...@4ax.com>,

Swifty <steve....@gmail.com> wrote:

> On Fri, 20 Apr 2012 08:51:09 +1000, dorayme <dor...@optusnet.com.au>
> wrote:
>
> >I have been saving without BOM for >years
>
> If you switch, beware of running into problems with language compilers
> and interpreters.
>
> Some of these will tell you that you have "illegal characters" in your
> source. You open your source in your editor (which is now configured
> to accept, if not impose BoMs) and you see no illegal characters.
>
> Some editors hide the BoMs so effectively that you won't know that
> they're there unless you view the content in hex. There may even be
> editors that never show it. You'd be able to recognise these by their
> users having clumps of hair torn out. :-)

In Curb Your Enthusiasm, Larry David finds himself caught in the
middle of two rival street protests. On the one side are his own
people gathered to object to a new Palestinian eatery close to a long
established Jewish deli. On the other side are supporters of the deli.
The former have the weight of his cultural being crushing in on him,
his best friends are even there. The Palestinian magnet is (a) his
knowledge that they serve the best fried chicken in the world (b) a
sexy palestinian whom he has been out with not only reminds him from
the crowd that there is more to be enjoyed with her but that she has
beautiful and willing sisters and friends too. The episode ends with
him in the middle undecided, his neck twisting this way and that.

--
dorayme

[tlvp]

On Thu, 19 Apr 2012 08:03:45 +0300, Jukka K. Korpela answered:

> In Notepad, UTF-8 is UTF-8 with BOM. An easy way to check this out is to

> launch Notepad, enter "foo", save as "UTF-8", and ...
> ... right-click on the file icon in

> Windows and select Properties, you'll see that the size is 6 bytes.
> That's 3 bytes for the 3 Ascii letters plus 3 bytes for the UTF-8
> representation of the BOM.

This "BOM" -- is it the three bytes that poor, unhappy, mal-adjusted
newsreaders sometimes encounter, rendering them as " " ?

If so, is there an easy or obvious MS Windows-y way to strip those bytes
out? Or does a UTF-8/w/BOM file quit being UTF-8 at all without them?

[Manuel Collado]

El 21/04/2012 0:23, tlvp escribió:
> On Thu, 19 Apr 2012 08:03:45 +0300, Jukka K. Korpela answered:
>
>> In Notepad, UTF-8 is UTF-8 with BOM. An easy way to check this out is to
>> launch Notepad, enter "foo", save as "UTF-8", and ...
>> ... right-click on the file icon in
>> Windows and select Properties, you'll see that the size is 6 bytes.
>> That's 3 bytes for the 3 Ascii letters plus 3 bytes for the UTF-8
>> representation of the BOM.
>
> This "BOM" -- is it the three bytes that poor, unhappy, mal-adjusted
> newsreaders sometimes encounter, rendering them as " " ?
>
> If so, is there an easy or obvious MS Windows-y way to strip those bytes
> out?

Open the file in Notepad++, for instance, and
"Encoding" -> "Convert to UTF-8 without BOM"

> Or does a UTF-8/w/BOM file quit being UTF-8 at all without them?

UTF-8 without BOM is, of course, UTF-8.

The BOM is just an optional hint, than can be useful in absence of HTTP
headers or other ways of declaring the file encoding.

--
Manuel Collado - http://lml.ls.fi.upm.es/~mcollado

[tlvp]

Thank you, Manuel. Time to "invest" in Notepad++, I guess, after decades of
hesitating :-) . Cheers, -- tlvp

[Roedy Green]

On Wed, 18 Apr 2012 23:26:51 -0400, tlvp <mPiOsUcB...@att.net>
wrote, quoted or indirectly quoted someone who said :

>1) How exactly do Notepad's UTF-8, Unicode, and Unicode Big-endian differ?

see http://mindprod.com/jgloss/utf.html
see http://mindprod.com/jgloss/unicode.html

If you have a hex viewer/editor, once you know the expected possible
patterns, you can answer that question for yourself by looking at
sample files.
See http://mindprod.com/jgloss/hex.html

If this is flummoxing, email me some sample files with attachments and
I will tell you what they are. see
http://mindprod.com/contact/contact.html
for my email address.


--
Roedy Green Canadian Mind Products
http://mindprod.com
Programmers love to create simplified replacements for HTML.
They forget that the simplest language is the one you
already know. They also forget that their simple little
markup language will bit by bit become even more convoluted
and complicated than HTML because of the unplanned way it grows.
.

[tlvp]

On Sun, 29 Apr 2012 07:57:59 -0700, Roedy Green wrote:

> If you have a hex viewer/editor, once you know the expected possible
> patterns, you can answer that question for yourself by looking at
> sample files.

Thanks, Roedy. I think if I had a hex editor I'd not have had to ask :-) .

Actually, I *used* to have a hex editor -- the DOS Edit one -- but MS in
their ... umm ... "infinite wisdom" (?) ... chose to excise that from Win
Vista, so I have it, for what little it was worth, no longer.

[Roedy Green]

On Wed, 18 Apr 2012 23:26:51 -0400, tlvp <mPiOsUcB...@att.net>
wrote, quoted or indirectly quoted someone who said :

>1) How exactly do Notepad's UTF-8, Unicode, and Unicode Big-endian differ?

see http://mindprod.com/jgloss/notepad.html

[Jukka K. Korpela]

2012-04-29 22:13, Roedy Green wrote:

>> 1) How exactly do Notepad's UTF-8, Unicode, and Unicode Big-endian differ?
>
> see http://mindprod.com/jgloss/notepad.html

You keep advertizing your oddly named "Java glossary", but in this issue
at least, it is best ignored. To begin with, UTF-8 is definitely not
"8-bit chars".

--
Yucca, http://www.cs.tut.fi/~jkorpela/

[tlvp]

On Sun, 29 Apr 2012 12:13:53 -0700, Roedy Green wrote:

> On Wed, 18 Apr 2012 23:26:51 -0400, tlvp <mPiOsUcB...@att.net>
> wrote, quoted or indirectly quoted someone who said :
>
>>1) How exactly do Notepad's UTF-8, Unicode, and Unicode Big-endian differ?
>
> see http://mindprod.com/jgloss/notepad.html

Ah, thanks, that provides a starting point for me. Cheers, -- tlvp

[Warren Oates]

In article <timstreater-009A...@news.individual.net>,
Tim Streater <timst...@greenbee.net> wrote:

> You can't spell "advertising" that way, sorry. It's one of a number of
> words ("enterprise" is another) that are *always* spelt with an 's'.
>
> Here in Europe we spell all such words with an 's' anyway. It's only the
> Yanks, who study uglification at school, who insist on the 'z' whenever
> they can.

Well, Tim, I'll make a serious note of that. It seems, though, that Mr.
Korpela posts from Finland and it's very possible that English is not
his first language. He seems to have mastered it pretty well, though, so
don't kick his tyres (the ones with the aluminium rims on the motor
parked at the kerb).
--

... do not cover a warm kettle or your stock may sour. -- Julia Child

[John W Kennedy]

On 2012-04-30 11:39:46 +0000, Tim Streater said:

> In article <jnl1ar$d6g$1...@dont-email.me>,

> "Jukka K. Korpela" <jkor...@cs.tut.fi> wrote:
>
>> 2012-04-29 22:13, Roedy Green wrote:
>>
>>>> 1) How exactly do Notepad's UTF-8, Unicode, and Unicode Big-endian differ?
>>>
>>> see http://mindprod.com/jgloss/notepad.html
>>
>> You keep advertizing your oddly named "Java glossary", but in this
>> issue at least, it is best ignored. To begin with, UTF-8 is definitely
>> not "8-bit chars".
>

> You can't spell "advertising" that way, sorry. It's one of a number of
> words ("enterprise" is another) that are *always* spelt with an 's'.
>
> Here in Europe

A rumor has reached the shores here, in the Utter West, to the effect
that there are still those who question whether the land of roast beef,
haggis, toasted cheese, and Ulster Fry ought to be regarded as "Europe"
at all. Or is it, perhaps, Euromania that explains the otherwise
inexplicable passion of the orthographers of Albion for the
Frenchifications of "-our", "-re", etc., as though yet all had been
forgot, yea, even unto Agincourt?

> we spell all such words with an 's' anyway. It's only the Yanks, who
> study uglification at school, who insist on the 'z' whenever they can.

Don't talk rubbish. The "z", in addition to being a closer phonetic
transcription, more accurately represents the Greek root. Even the
bloody Oxford English Dictionary regards "-ize" as standard, and "-ise"
as less than secondary. The reason that "advertize" is an error (which
it is) is that the etymology does not involve Greek "-izein", although
it is often mistakenly thought to.

--
John W Kennedy
Read the remains of Shakespeare's lost play, now annotated!
http://www.SKenSoftware.com/Double%20Falshood

[tlvp]

John's point is well taken. There are different suffixes, "-ize" and "-ise"
-- they sound alike but have different origins and different functions.
Advertise and glamorize are two verbs incorporating those suffixes. Ponder,
if you will, what would happen were the suffixes there interchanged :-) .
"Advertize"? "Glamorise"? No more possible than "sise" or "prise" for
"size" and "prize", or "rize" or "wize" for "rise" and "wise" :-) .

I, for one, find Jukka's "advertize" far more acceptable than "sise" or
"wize" :-) . Indeed, that's usage that I've found in the "adverts" sections
of several British periodical publications, whence I could even imagine
it's officially sanctioned :-) .

[Dr J R Stockton]

In comp.infosystems.www.authoring.html message <py9kx84uju87.12qpxlhhmz4
5$.d...@40tude.net>, Sun, 29 Apr 2012 14:29:20, tlvp
<mPiOsUcB...@att.net> posted:

>On Sun, 29 Apr 2012 07:57:59 -0700, Roedy Green wrote:
>
>> If you have a hex viewer/editor, once you know the expected possible
>> patterns, you can answer that question for yourself by looking at
>> sample files.
>
>Thanks, Roedy. I think if I had a hex editor I'd not have had to ask :-) .
>
>Actually, I *used* to have a hex editor -- the DOS Edit one -- but MS in
>their ... umm ... "infinite wisdom" (?) ... chose to excise that from Win
>Vista, so I have it, for what little it was worth, no longer.

Then get MiniTrue, via sig line 4. It is a viewer, searcher, and
replacer, with ASCII and Hex/ASCII view; search/replace uses RegExps.

It's like (I think) a combination of SED, AWK, GREP : I don't say it
will do everything that those can do, but I believe it can do things
that they cannot. I've used it only in Win98 and Windup. But it is
essentially for 7-bit characters. A description is at <http://www.idiot
sdelight.net/minitrue/>, but the originator is inactive; see <http://ado
xa.3eeweb.com/minitrue/index.html>.

--
(c) John Stockton, nr London, UK. ?@merlyn.demon.co.uk Turnpike 6.05 Windup.
Web <http://www.merlyn.demon.co.uk/> - FAQ-type topics, acronyms, and links.
Command-prompt MiniTrue is useful for viewing/searching/altering files. Free,
DOS/Win/UNIX now 2.0.6; see <URL:http://www.merlyn.demon.co.uk/pc-links.htm>.

[John W Kennedy]

On 2012-04-30 21:58:45 +0000, Tim Streater said:

> In article <1xv3yqtel9iab.1b03732d9lr2u$.d...@40tude.net>,

> As I said, you're all steeped in uglification. The root of the word is
> a matter of indifference to me.

Most people regard "My ignorance is better than your knowledge" as a
peculiarly American form of idiocy. Are you quite certain that you’re
Briti -- excuse me, "European"? Do you like your steaks with catsup? Do
you drink only lager, straight from the fridge? Do you decorate your
home with the arms (which you call a "crest") of some random Briton of
whom you know nothing except that he happens to share your surname?
Remember, the first step is to admit that you have a problem.

>> I, for one, find Jukka's "advertize" far more acceptable than "sise" or
>> "wize" :-) . Indeed, that's usage that I've found in the "adverts" sections
>> of several British periodical publications, whence I could even imagine
>> it's officially sanctioned :-) .
>

> Ignorance is no excuse.

Unless it's yours, eh?

[tlvp]

On Mon, 30 Apr 2012 22:58:45 +0100, Tim Streater wrote:

> The root of the word is a
> matter of indifference to me.

As is your taste (or lack of it) to me :-) . Cheers, -- tlvp

[tlvp]

On Mon, 30 Apr 2012 18:45:43 +0100, Dr J R Stockton wrote:

> In comp.infosystems.www.authoring.html message <py9kx84uju87.12qpxlhhmz4
> 5$.d...@40tude.net>, Sun, 29 Apr 2012 14:29:20, tlvp
> <mPiOsUcB...@att.net> posted:
>

>>Actually, I *used* to have a hex editor -- the DOS Edit one -- but MS in
>>their ... umm ... "infinite wisdom" (?) ... chose to excise that from Win
>>Vista, so I have it, for what little it was worth, no longer.
>
> Then get MiniTrue, via sig line 4. It is a viewer, searcher, and
> replacer, with ASCII and Hex/ASCII view; search/replace uses RegExps.

Thanks for the suggestion: I'll certainly have a look. And, actually, they
didn't excise it, they just rendered it less helpful than its earlier
incarnations had been (displaying BOMs, CRs, LFs, etc, as DOS graphic
glyphs only now, without also reporting their ASCII code points).

> It's like (I think) a combination of SED, AWK, GREP : I don't say it
> will do everything that those can do, but I believe it can do things
> that they cannot. I've used it only in Win98 and Windup. But it is
> essentially for 7-bit characters. A description is at <http://www.idiot
> sdelight.net/minitrue/>, but the originator is inactive; see <http://ado
> xa.3eeweb.com/minitrue/index.html>.

BTW, was it aioe.org or your Turnpike that broke those URLs in mid-word (as
you see) for the sake of honoring a line-wrap parameter? Or something else?

(Easy enough to paste back together again, as

<http://www.idiotsdelight.net/minitrue/>
and
<http://adoxa.3eeweb.com/minitrue/index.html> ,

so I'm not so much upset as just curious.)

[tlvp]

On Mon, 30 Apr 2012 18:45:43 +0100, Dr J R Stockton wrote:

> Command-prompt MiniTrue is useful for viewing/searching/altering files. Free,
> DOS/Win/UNIX now 2.0.6; see <URL:http://www.merlyn.demon.co.uk/pc-links.htm>.

As tiny bit of compensation for your offering that link, allow me, FWIW, to
point out a tiny typo -- the "uitilies", in the line

> LISTPLUS Vernon Buerg uitilies including LIST v.9.6x ,

who'd surely rather become "utilities", eh? No harm done, of course :-) .
(But for how astonishingly long can that have gone unnoticed?)

Cheers, and thanks again, -- tlvp

[dorayme]

In article <timstreater-009A...@news.individual.net>,
Tim Streater <timst...@greenbee.net> wrote:

> In article <jnl1ar$d6g$1...@dont-email.me>,
> "Jukka K. Korpela" <jkor...@cs.tut.fi> wrote:
>

...

> >
> > You keep advertizing ...

>
> You can't spell "advertising" that way, sorry. It's one of a number of
> words ("enterprise" is another) that are *always* spelt with an 's'.
>

> Here in Europe we spell all such words with an 's' anyway. It's only the

> Yanks, who study uglification at school, who insist on the 'z' whenever
> they can.

Would not have thought it was even a bit optional with "enterprise"?
Anyway, I'm no yank, but I sometimes rather like the option to use a Z
instead of an S where it is optional.

But I am biased by a privileged access to the history of letters as a
whole, and the psychology of particular letters in particular. To use
a yank idea, letters have attitude and sometimes these determine how
they are to be used on any particular occasion. On some days, Ss are
sleepy and need to be respected and left alone, on these days Zs are
often keen to be picked for a role on stage.

<http://dorayme.netweaver.com.au/letters/proudZ.html>

--
dorayme

[tlvp]

On Tue, 01 May 2012 13:03:26 +1000, dorayme wrote:

> ... these days Zs are

> often keen to be picked for a role on stage.
>
> <http://dorayme.netweaver.com.au/letters/proudZ.html>

What a delightful design, dorayme, for a loudspeaker pedestal :-) .

Cheers, -- tlvp

[dorayme]

In article <1489ts4tm6drs.1c59kjaigztj7$.d...@40tude.net>,

tlvp <mPiOsUcB...@att.net> wrote:

> On Tue, 01 May 2012 13:03:26 +1000, dorayme wrote:
>
> > ... these days Zs are
> > often keen to be picked for a role on stage.
> >
> > <http://dorayme.netweaver.com.au/letters/proudZ.html>
>
> What a delightful design, dorayme, for a loudspeaker pedestal :-)

Z is like that, thinks nothing of using others for whatever. He is not
really a cockerel among hens, as some of the other jealous and less
confident letters have insinuated. His nature is proud, he does not do
low-class swaggering or strutting. He really is strong. I respect him.
And it is also why Americans like him, they respect force, up-front,
in your face, are-you-looking-at-me-mister? things.

I run a clinic for damaged letter *instances*, it is very very rare to
see a Z come through the doors. The stories I could tell you!

Anyway, here is another:

<http://dorayme.netweaver.com.au/letters/beforeTheWar.html>

--
dorayme

[tlvp]

On Tue, 01 May 2012 15:04:31 +1000, dorayme wrote:

> ...

> I run a clinic for damaged letter *instances*, it is very very rare to
> see a Z come through the doors. The stories I could tell you!

You should run such a clinic in Central Europe. Germany, Poland, Hungary --
how those countries' citizens mistreat their Zs. Shoot arrows right through
their proud spines, same as they do with their 7s, as were they fs or ts.
They both just grit their teeth, keep a stiff upper lip, and soldier on,
unbowed, despite the pain, with that arrow protruding fore and aft. Your
clinics could do such good in those benighted lands ... :-) .

>
> Anyway, here is another:
>
> <http://dorayme.netweaver.com.au/letters/beforeTheWar.html>

Now there I see Z as Mother Courage, protecting her upper-case flock of
fellow-majuscules from the depradations of a hostile swarm of lower-case
minuscules attacking in cloud formation. Gives new visual insight into the
meaning of the phrase, "To keep s'b'dy under one's wing" :-) .

[dorayme]

In article <1c6ju62d8bpew.1...@40tude.net>,

tlvp <mPiOsUcB...@att.net> wrote:

> > <http://dorayme.netweaver.com.au/letters/beforeTheWar.html>
>
> Now there I see Z as Mother Courage, protecting her upper-case flock of
> fellow-majuscules from the depradations of a hostile swarm of lower-case
> minuscules attacking in cloud formation. Gives new visual insight into the
> meaning of the phrase, "To keep s'b'dy under one's wing" :-) .

mm... yes, I see what you mean.

However, in this particular case, there is an objective truth. In
fact, this is no mere cartoon. It is based on a photograph taken just
before the war. Yes, the photographer was there and actually had been
accepted to be embedded in the home capital army. This was taken when
Z had lined up his troops, it was a military parade of sorts, a morale
boosting demonstration to calm the nerves of the great masses of
letters and signs.

The war did not go well. But that is another story, perhaps too O(ff)T
for this thread. At least so far we have been completely O(n)T.

--
dorayme

[Warren Oates]

In article <dorayme-491632...@news.albasani.net>,
dorayme <dor...@optusnet.com.au> wrote:

> <http://dorayme.netweaver.com.au/letters/beforeTheWar.html>

Very nice. But is he Commander Zee or Commander Zed? Some of my
favou?rite music years ago was by Zed Zed Top.

(One or zero times, cobras).

[Roedy Green]

On Sun, 29 Apr 2012 14:29:20 -0400, tlvp <mPiOsUcB...@att.net>

wrote, quoted or indirectly quoted someone who said :

>Thanks, Roedy. I think if I had a hex editor I'd not have had to ask :-) .

the link I gave http://mindprod.com/jgloss/hex.html
points to at least one free hex editor.

[Roedy Green]

On Mon, 30 Apr 2012 06:35:55 +0300, "Jukka K. Korpela"
<jkor...@cs.tut.fi> wrote, quoted or indirectly quoted someone who
said :

>You keep advertizing your oddly named "Java glossary", but in this issue
>at least, it is best ignored. To begin with, UTF-8 is definitely not
>"8-bit chars".

There is nothing for sale, so the word "advertising" is inappropriate.
All the information, all the utilities and all the source is available
free.

There answer to his question is formatted nicely there. It makes sense
to provide a link rather than inline in a newsgroup post. I can
update it later and people will see the updated version. If I answer
inline, they will see the out of date version. You seem to be
implying I have committed some sort of crime.

You might want to quibble about terminology, but UTF-8 does not have
an endian issue because the atoms it is composed of are 8-bit. That
difference does not justify ignoring the work I have put into the
glossary over the last 16 years. You have read only one entry, and
decided the whole thing is worthless. Your leaping to conclusions is
hardly something to be proud of.

I have been doing this since 1963, so my terminology may sometimes be
old fashioned. You might want to check out my entry on encodings to
see if you have quibbles with the terminology there.
http://mindprod.com/jgloss/encoding.html

The hex I show for the various encodings is checked by experiment. I
did not simply deduce them. The problem with deducing is it presumes
others are logical, which is not always a safe assumption.

[Roedy Green]

On Mon, 30 Apr 2012 06:35:55 +0300, "Jukka K. Korpela"

<jkor...@cs.tut.fi> wrote, quoted or indirectly quoted someone who
said :

>To begin with, UTF-8 is definitely not
>"8-bit chars".

You can think of UTF-8 as sort of extended ASCII. It handles the high
characters with sequences of 8-bit chars. Granted, there are other
ways of looking at it, but that way is not wrong. It reflects the
historical development. If you write a decoder, that is how you will
look at it.

[Jukka K. Korpela]

2012-05-01 15:53, Roedy Green wrote:

> You can think of UTF-8 as sort of extended ASCII.

Character codes are confusing enough; there is no need to add to the
confusion that way. It is all too common that people mistake UTF-8 for a
character code and not a transfer encoding.

> It handles the high
> characters with sequences of 8-bit chars.

“High characters” and “chars” make no sense. If you try to explain
things, you should use words that everyone knows, or words that you
define, not your own absurd terms. “Char” is not just a sloppy word
here; it is grossly misleading, since the obvious interpretation is that
it is just short for “character.”

--
Yucca, http://www.cs.tut.fi/~jkorpela/

[tlvp]

On Tue, 01 May 2012 05:29:30 -0700, Roedy Green wrote:

> On Sun, 29 Apr 2012 14:29:20 -0400, tlvp <mPiOsUcB...@att.net>
> wrote, quoted or indirectly quoted someone who said :
>
>>Thanks, Roedy. I think if I had a hex editor I'd not have had to ask :-) .
>
> the link I gave http://mindprod.com/jgloss/hex.html
> points to at least one free hex editor.

Yes, thanks Roedy, I only noticed that fact rather later.

Also, DOS Edit is still there, in Vista, and will display the
DOS CodePage glyphs for the various ASCII codes, but
doesn't seem to wish to show me what the codes are.

So the one you point me to should come in handy, thanks.

[tlvp]

On Tue, 1 May 2012 19:09:41 -0400, tlvp mis-wrote:

> So the one you point me to ...

Sorry, meant "the ones ...". Thanks again. Cheers, -- tlvp

[Dr J R Stockton]

In comp.infosystems.www.authoring.html message <oppu7dnd96q0$.105gpjbypo
0o3...@40tude.net>, Mon, 30 Apr 2012 19:24:14, tlvp
<mPiOsUcB...@att.net> posted:

>On Mon, 30 Apr 2012 18:45:43 +0100, Dr J R Stockton wrote:
>> In comp.infosystems.www.authoring.html message <py9kx84uju87.12qpxlhhmz4
>> 5$.d...@40tude.net>, Sun, 29 Apr 2012 14:29:20, tlvp
>> <mPiOsUcB...@att.net> posted:
>>
>>>Actually, I *used* to have a hex editor -- the DOS Edit one -- but MS in
>>>their ... umm ... "infinite wisdom" (?) ... chose to excise that from Win
>>>Vista, so I have it, for what little it was worth, no longer.
>>
>> Then get MiniTrue, via sig line 4. It is a viewer, searcher, and
>> replacer, with ASCII and Hex/ASCII view; search/replace uses RegExps.
>
>Thanks for the suggestion: I'll certainly have a look. And, actually, they
>didn't excise it, they just rendered it less helpful than its earlier
>incarnations had been (displaying BOMs, CRs, LFs, etc, as DOS graphic
>glyphs only now, without also reporting their ASCII code points).

By the way, just what is it that you mean by DOS Edit hex editor? if I
type edit at the command line I get what I expected, but the only hexy
thing I can see is got by, for example, edit /55 filename . See
footnote.

>> It's like (I think) a combination of SED, AWK, GREP : I don't say it
>> will do everything that those can do, but I believe it can do things
>> that they cannot. I've used it only in Win98 and Windup. But it is
>> essentially for 7-bit characters. A description is at <http://www.idiot
>> sdelight.net/minitrue/>, but the originator is inactive; see <http://ado
>> xa.3eeweb.com/minitrue/index.html>.
>
>BTW, was it aioe.org or your Turnpike that broke those URLs in mid-word (as
>you see) for the sake of honoring a line-wrap parameter? Or something else?

Turnpike did it. Any respectable news/mail reader will recognise line-
broken URLs as units if they are enclosed in < > brackets. When I
remember, I also put a hard line feed in front of them; but I cannot
recall the exact state of that part at the moment of pressing the Post
button.


Footnote : I don't seem to need to edit Hex nowadays, and LIST and
MiniTrue serve my viewing needs. But, once upon a time, I has a task
containing a simple inner loop which needed to be as fast as possible.
The IT guru wrote the assembly language, with correct syntax, and
assembled it to linkable code. But the calculation needed a floating-
point 2 (or 0.5?); and he just could not manage to write a floating 2 in
the assembler. Everything that he tried gave some other value. But he
knew perfectly well what the binary for 2.0 was. So he took the
80-column punched card in question, adjusted a few of the rectangular
holes between absent and present, copied the card, and, lo and behold,
it worked!

--
(c) John Stockton, nr London, UK. ?@merlyn.demon.co.uk Turnpike v6.05.
Website <http://www.merlyn.demon.co.uk/> - w. FAQish topics, links, acronyms
PAS EXE etc. : <http://www.merlyn.demon.co.uk/programs/> - see in 00index.htm
Dates - miscdate.htm estrdate.htm js-dates.htm pas-time.htm critdate.htm etc.

[Dr J R Stockton]

In comp.infosystems.www.authoring.html message <1ghfd8zvgik04$.lbvhsxnqw
tsm$.d...@40tude.net>, Mon, 30 Apr 2012 19:35:54, tlvp
<mPiOsUcB...@att.net> posted:

>On Mon, 30 Apr 2012 18:45:43 +0100, Dr J R Stockton wrote:
>
>> Command-prompt MiniTrue is useful for viewing/searching/altering files. Free,
>> DOS/Win/UNIX now 2.0.6; see <URL:http://www.merlyn.demon.co.uk/pc-links.htm>.
>
>As tiny bit of compensation for your offering that link, allow me, FWIW, to
>point out a tiny typo -- the "uitilies", in the line
>
>> LISTPLUS Vernon Buerg uitilies including LIST v.9.6x ,
>
>who'd surely rather become "utilities", eh? No harm done, of course :-) .
>(But for how astonishingly long can that have gone unnoticed?)

I only copied that pair of lines - dated Jul 29 2004, but could well
have been originated much earlier. Almost certainly, the writer was
Finnish.

Google (reluctantly) finds 21,200 instances, and 518,000,000 for
utilities. The _original_ error has already been corrected.

I have corrected my copy, using MiniTrue. Thanks.


QUERY : Does anyone here both
(a) Run a browser (maybe other than Chrome) in a UNIX-type system,
(b) Have, or be able to have, a small web site or site subtree on
that system ?

If so, it would be nice to have my www.authoring tool
<http://www.merlyn.demon.co.uk/linxchek.htm> tested on that system,
understanding the instructions being part of the test. Uses JavaScript.
Operation is, as explained, browser-dependent in my Win XP sp3.

--
(c) John Stockton, nr London, UK. ?@merlyn.demon.co.uk Turnpike v6.05 MIME.
Web <http://www.merlyn.demon.co.uk/> - FAQqish topics, acronyms and links;
Astro stuff via astron-1.htm, gravity0.htm ; quotings.htm, pascal.htm, etc.

[tlvp]

On Wed, 2 May 2012 18:51:24 +0100, Dr J R Stockton wrote:

> In comp.infosystems.www.authoring.html message <oppu7dnd96q0$.105gpjbypo
> 0o3...@40tude.net>, Mon, 30 Apr 2012 19:24:14, tlvp
> <mPiOsUcB...@att.net> posted:
>>On Mon, 30 Apr 2012 18:45:43 +0100, Dr J R Stockton wrote:
>>> In comp.infosystems.www.authoring.html message <py9kx84uju87.12qpxlhhmz4
>>> 5$.d...@40tude.net>, Sun, 29 Apr 2012 14:29:20, tlvp
>>> <mPiOsUcB...@att.net> posted:
>>>
>>>>Actually, I *used* to have a hex editor -- the DOS Edit one -- but MS in
>>>>their ... umm ... "infinite wisdom" (?) ... chose to excise that from Win
>>>>Vista, so I have it, for what little it was worth, no longer.
>>>
>>> Then get MiniTrue, via sig line 4. It is a viewer, searcher, and
>>> replacer, with ASCII and Hex/ASCII view; search/replace uses RegExps.
>>
>>Thanks for the suggestion: I'll certainly have a look. And, actually, they
>>didn't excise it, they just rendered it less helpful than its earlier
>>incarnations had been (displaying BOMs, CRs, LFs, etc, as DOS graphic
>>glyphs only now, without also reporting their ASCII code points).
>
> By the way, just what is it that you mean by DOS Edit hex editor? if I
> type edit at the command line I get what I expected, but the only hexy
> thing I can see is got by, for example, edit /55 filename . See
> footnote.

Back in the previous century, typing edit at the command prompt along with
a suitable switch, parameter, modifier, or whatnot, got you a display with
lines each displaying 16 two-digit, space-separated hex numbers, followed
by 16 glyphs (where available -- spaces otherwise) having those hex codes
as their ASCII codes. Available from DOS 5 or earlier on up through Win95
or later. Gone today. Safest used in read-only mode :-) .

>>> It's like (I think) a combination of SED, AWK, GREP : I don't say it
>>> will do everything that those can do, but I believe it can do things
>>> that they cannot. I've used it only in Win98 and Windup. But it is
>>> essentially for 7-bit characters. A description is at <http://www.idiot
>>> sdelight.net/minitrue/>, but the originator is inactive; see <http://ado
>>> xa.3eeweb.com/minitrue/index.html>.
>>
>>BTW, was it aioe.org or your Turnpike that broke those URLs in mid-word (as
>>you see) for the sake of honoring a line-wrap parameter? Or something else?
>
> Turnpike did it. Any respectable news/mail reader will recognise line-
> broken URLs as units if they are enclosed in < > brackets. When I
> remember, I also put a hard line feed in front of them; but I cannot
> recall the exact state of that part at the moment of pressing the Post
> button.

No matter -- just curious (and praying it wasn't me or my Dialog :-) .

> Footnote : I don't seem to need to edit Hex nowadays, and LIST and
> MiniTrue serve my viewing needs. But, once upon a time, I has a task
> containing a simple inner loop which needed to be as fast as possible.
> The IT guru wrote the assembly language, with correct syntax, and
> assembled it to linkable code. But the calculation needed a floating-
> point 2 (or 0.5?); and he just could not manage to write a floating 2 in
> the assembler. Everything that he tried gave some other value. But he
> knew perfectly well what the binary for 2.0 was. So he took the
> 80-column punched card in question, adjusted a few of the rectangular
> holes between absent and present, copied the card, and, lo and behold,
> it worked!

Lovely :-) ! Thanks; and cheers, -- tlvp

[tlvp]

On Wed, 2 May 2012 19:08:29 +0100, Dr J R Stockton wrote:

> In comp.infosystems.www.authoring.html message <1ghfd8zvgik04$.lbvhsxnqw
> tsm$.d...@40tude.net>, Mon, 30 Apr 2012 19:35:54, tlvp
> <mPiOsUcB...@att.net> posted:
>
>>On Mon, 30 Apr 2012 18:45:43 +0100, Dr J R Stockton wrote:
>>
>>> Command-prompt MiniTrue is useful for viewing/searching/altering files. Free,
>>> DOS/Win/UNIX now 2.0.6; see <URL:http://www.merlyn.demon.co.uk/pc-links.htm>.
>>
>>As tiny bit of compensation for your offering that link, allow me, FWIW, to
>>point out a tiny typo -- the "uitilies", in the line
>>
>>> LISTPLUS Vernon Buerg uitilies including LIST v.9.6x ,
>>
>>who'd surely rather become "utilities", eh? No harm done, of course :-) .
>>(But for how astonishingly long can that have gone unnoticed?)
>
> I only copied that pair of lines - dated Jul 29 2004, but could well
> have been originated much earlier. Almost certainly, the writer was
> Finnish.

Foof! 2004! That's long! ... Hmm ... Finnish ... And known to us both :-) ?

> Google (reluctantly) finds 21,200 instances, and 518,000,000 for
> utilities. The _original_ error has already been corrected.

My, how errors propagate :-) !

> I have corrected my copy, using MiniTrue. Thanks.

YVW. Glad to have been able to be of some small service.

> QUERY : Does anyone here both
> (a) Run a browser (maybe other than Chrome) in a UNIX-type system,
> (b) Have, or be able to have, a small web site or site subtree on
> that system ?

a) "browser ... other than Chrome": yes; "ina UNIX-type system": no.
b) "have small website" *mirror* "on that system": yes.

But that's probably of no use to you.

> If so, it would be nice to have my www.authoring tool
> <http://www.merlyn.demon.co.uk/linxchek.htm> tested on that system,
> understanding the instructions being part of the test. Uses JavaScript.
> Operation is, as explained, browser-dependent in my Win XP sp3.

Only XP (very limited HD free space) and Vista here, not UNIX (or U-type).
Sorry.

Cheers, -- tlvp

[John W Kennedy]

I abandoned the operating system of the damned some years ago, but I
seem to remember that either Textpad or EmEditor could turn the trick.

[Dr J R Stockton]

In comp.infosystems.www.authoring.html message <17od9lttbtsgr.1pxkc27s8y
pz2$.d...@40tude.net>, Wed, 2 May 2012 21:58:22, tlvp
<mPiOsUcB...@att.net> posted:

>On Wed, 2 May 2012 19:08:29 +0100, Dr J R Stockton wrote:

>> I only copied that pair of lines - dated Jul 29 2004, but could well
>> have been originated much earlier. Almost certainly, the writer was
>> Finnish.
>
>Foof! 2004! That's long! ... Hmm ... Finnish ... And known to us both :-) ?

Known in person? I doubt it - AFAIR, I've only really met two Finns
(except in passing), and its not likely to be either of those. But
without knowing who you are, one cannot be sure. I've not knowingly met
any of those who might have written it, though I've corresponded with
the leader of the group.

>> QUERY : Does anyone here both
>> (a) Run a browser (maybe other than Chrome) in a UNIX-type system,
>> (b) Have, or be able to have, a small web site or site subtree on
>> that system ?
>
>a) "browser ... other than Chrome": yes; "ina UNIX-type system": no.
>b) "have small website" *mirror* "on that system": yes.
>
>But that's probably of no use to you.

A mirror will be OK, though less useful to the tester.

>> If so, it would be nice to have my www.authoring tool
>> <http://www.merlyn.demon.co.uk/linxchek.htm> tested on that system,
>> understanding the instructions being part of the test. Uses JavaScript.
>> Operation is, as explained, browser-dependent in my Win XP sp3.
>
>Only XP (very limited HD free space) and Vista here, not UNIX (or U-type).
>Sorry.

Well, with XP/Vista, at least understandability of the instructions
could be checked. And it could be useful.

--
(c) John Stockton, nr London UK. ?@merlyn.demon.co.uk Turnpike v6.05 MIME.
<http://www.merlyn.demon.co.uk/> TP/BP/Delphi/&c., FAQqy topics & links;
<http://www.merlyn.demon.co.uk/clpb-faq.txt> RAH Prins : c.l.p.b mFAQ;
<ftp://garbo.uwasa.fi/pc/link/tsfaqp.zip> Timo Salmi's Turbo Pascal FAQ.

[tlvp]

I'm not sure how this reply evaded mine eyes until now, but, having found
it at last, let me ask in what way my mirroring something can help you?

[Dr J R Stockton]

In comp.infosystems.www.authoring.html message <1ftnby7q90efq.1wp3mpdgq0
3ns$.d...@40tude.net>, Fri, 25 May 2012 22:38:29, tlvp
<mPiOsUcB...@att.net> posted:

The cited page, <http://www.merlyn.demon.co.uk/linxchek.htm>, works for
me on my WinXP sp3 system - the browser table at #Status needs updating,
but remains a good guide - on the master of my own site, and on copies
of those of my Godson and his sister.

It would be nice to know whether it works on other systems, including
what I may soon buy, and on other web site copies. Part of that is
knowing whether the instructions are comprehensible.

But a real test needs a directory which is the root of a local copy of a
web site, with a starting page and links direct or indirect to other
local pages. It does not matter if other stuff is also in that
directory. And that is something that those here are likely to have
already.

I use it whenever I am not confident that the links and anchors within
my site are all connected, and sometimes for other reasons. You might
find it useful too.


It could, I think, easily be modified to run when in, but not
necessarily linking to or linked from, an actual Web-accessible site. I
don't support that, since I don't want random passers-by to erode my
download allowance by idly pressing a button and reading the whole site.
But if you privily put a copy on a site of yours, suitably modified,
you'll be using your own bandwidth, which does not concern me.

--

(c) John Stockton, nr London, UK. ?@merlyn.demon.co.uk Turnpike v6.05 MIME.
Web <http://www.merlyn.demon.co.uk/> - FAQqish topics, acronyms and links;
Astro stuff via astron-1.htm, gravity0.htm ; quotings.htm, pascal.htm, etc.

No Encoding. Quotes before replies. Snip well. Write clearly. Don't Mail News.

[tlvp]

Perhaps I'm being foolish by missing out on this, but I'll pass, thanks.