GUI TSE font bug with code page 65001 (UTF-8)

[Carlo Hogeveen]

If we are not using the Terminal font, then GUI TSE should internally consistently use code page 1252 whatever Windows' global code page is.
But GUI TSE does not do that consistently when Windows' global code page is 65001 (UTF-8).

To set Windows 11 Pro's code page to/from 65001, go to Setings -> Time & language -> Language & region -> the most-right down-symbol besides "Display Language", and turn "Beta: Use Unicode UTF-8 for worldwide language support" On or Off.
In my experience afterwards you need to restart Windows 11 Pro too for consistent results.
In Windows 10 the option is under Settings -> Time & Language -> Language -> (In top right:) Administrative language settings -> Change system locale.

To reproduce GUI TSE's inconsistency:
The Windows UTF-8 option is set to On and Windows was restarted afterwards.
In a new TSE installation where the font's properties were never changed:
- The font is NOT Terminal.
- TSE's ASCII Chart menu does not show real characters from character 128 upwards.

1.
If you open TSE's font menu and select OK, then TSE's ASCCII Chart menu gains proper characters for the upper 128 ones.

2.
If you Macro Execute (including the "execute"!)
execute string fn[255]='' integer fs=0 integer ff=0 GetFont(fn, fs, ff) SetFont(fn, fs, ff)
then the ASCII Chart menu loses proper characters for the upper 128 ones. (I see open block characters.)

Both these changes can be made persistent across TSE sessions by saving TSE's settings.

Here is the problem / bug:
As the second change suggests, it is not possible to use the SetFont() function in GUI TSE with a not-Terminal font if the external code page is 65001, because that makes GUI TSE lose its upper 128 characters.
This can also not be repaired or worked around by a macro.
Normally GUI TSE (without the Terminal font) internally uses code page 1252 whatever the external code page is, but for external code 65001 that fails.

This is an old bug that I (in less detail) reported years earlier.
One reason I had to stop using code page 65001 was because of this GUI TSE bug.
But now we have at least one other user that uses external code page 65001 .

Carlo

[eli...@partner.net.il]

Thanks, Carlo.
I've also gone back to using CODE PAGE 437.
Now the GuideLines v1.2.1 macro works fine.

Eliyahu Trigoub

--

---
You received this message because you are subscribed to the Google Groups "SemWare TSE Pro text editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email to semware+u...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/semware/000101dc6398%24fb4b7f00%24f1e27d00%24%40ecarlo.nl.

[S.E. Mitchell]

I was able to reproduce this, and this fixes it here.

Let me know if this fixes it for you:

SetFont("Courier New", 0, _FONT_CHARSET_, 0)

The first 0 is the point size.  0 means use default.

The last 0 is the font charset.  0 is the ANSI_CHARSET.

To view this discussion visit https://groups.google.com/d/msgid/semware/000201dc63b5%247b2e6220%24718b2660%24%40partner.net.il.

[Carlo Hogeveen]

Well done! Excellent!
What a great small solution to wake up to. :-)

I just finished extensively testing this, including for "all" fonts.
It showed, that the _FONT_CHARSET_ flag is the only relevant parameter for fixing the problem:
Leaving out SetFont()'s 4th parameter changes nothing.
Setting the 2nd parameter (size) works!
Setting _FONT_CHARSET_|_FONT_BOLD_ works.

That begged the question if always setting the _FONT_CHARSET_ flag has a downside.
It has exactly 1.
With the flag, Windows reverts to the "Courier New" font when we try to set (anything for) the Terminal font.

Re-summarizing:
The problem is that if Windows' UTF-8 setting is ON, then " SetFont() WITH parameters" turns TSE's upper 128 characters into the same open block character for all fonts but the Terminal font.
SetFont()'s "new" _FONT_CHARSET_ flag fixes this for all fonts except the Terminal font, which becomes unsattable.

For me as a macro programmer that means I have a functional work-around.

Obvious suggestion:
For regular users who will run into the problem, it would be helpful if TSE's "SetFont() WITH parameters" internally sets this flag ON for not-Terminal fonts, and OFF for the Terminal font.

Anyways, thank you very much for the work-around.
My Windows UTF-8 setting is ON again!

Carlo