TSE lost ANSI filename support

[Carlo Hogeveen]

Sammy,

Windows TSE v4.50.27 and upwards gained the ability to open and edit files with "Unicode filenames" by replacing them with short filenames.
This is a good thing!

Unfortunately this made TSE lose the ability to open and edit files that have an "ANSI filename".

Here "Unicode filename" means a filename that contains non-ANSI characters, and "ANSI filename" means a filename that contains non-ASCII characters.
This a generalization of Joachim's report of the degree character ° no longer being supported in filenames.
Any ANSI character that is not an ASCII character is no longer supported in a TSE filename, causing its filename to be shown as a short name instead. ☹

I am happy to see, that under the hood TSE still supports ANSI filenames.
For example, I can open the file "C:\Work\RSUM~1.txt", and then Macro Execute:
Execute ChangeCurrFilename('Résumé.txt', _dont_prompt_|_dont_expand_|_overwrite_)
This will show "Résumé.txt" above the opened file and in the open files list, and TSE has no problem editing and saving it.

I am looking into the possibility of having a macro apply this fix automatically,
but I would prefer if TSE fixed it at the root, namely when opening a file or changing its name via the menu.

TSE does have a significant amount of non-English users that might like to or need to see their filenames with accented characters again.

Carlo

[zhong zhao]

Do not call  FindFirstFileW, FindNextFileW;

call FindFirstFileA,FindNextFileA.

I guess.

[S.E. Mitchell]

As Zhong Zhao suggested, I can revert to FindFirstFileA.
It doesn't really matter to me.
And I agree, there might be more non-English users than English users
at this point. But at least there are quite a few.
Let me know if you think this is the best idea.

> --
>
> ---
> You received this message because you are subscribed to the Google Groups "SemWare TSE Pro text editor" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to semware+u...@googlegroups.com.
> To view this discussion visit https://groups.google.com/d/msgid/semware/1a08614a-0f06-45c5-9344-9ca19940a58bn%40googlegroups.com.

[Carlo Hogeveen]

Sammy wrote:
> And I agree, there might be more non-English users than English users at this point.

I did and would not go that far.

> As Zhong Zhao suggested, I can revert to FindFirstFileA.

> Let me know if you think this is the best idea.

I think that will not work in all cases, because that will not let you differentiate between fitted and not-fitted ANSI names.
For example, when using the Windows ANSI functions to open the files
"The letter ă.txt"
"The letter ā.txt"
"The letter ą.txt"
all these filenames will be returned as
"The letter a.txt"
How would you know this is not the file's real ANSI name?

Maybe this will inspire you.
I am still plodding away at opening a file from Windows File Explorer into TSE.
https://ecarlo.nl/tse/InDevelopment.html#UniExplorer
Simplified, I get the Unicode filename from the command line,
and convert it to ANSI and back to determine if the ANSI name was the correct one.
This works so far.
It does not work if the code page is a multi-byte one, like UTF-8 or Chinese:
That needs a different solution.

Carlo

To view this discussion visit https://groups.google.com/d/msgid/semware/CA%2BgEvkboov2pCW535-dXZXv23iJKUQ92Uoe34j%2BFHB9m4wL77A%40mail.gmail.com.

[Joe Souza]

I'm wondering if, as a stopgap measure until TSE fully supports wide-character Unicode names, it could perhaps use FindFirstFileW but then convert the returned name to UTF-8 for internal use.  It would do this by calling WideCharToMultiByte, specifying CP_UTF8 as the code page.  UTF-8 is designed to hold Unicode wide character strings in single-byte character arrays.  It could also be that I'm not fully understanding the problem here.

[Carlo Hogeveen]

Storing a filename UTF-8-encoded into a 1-byte character-array does not make UTF-8 a 1-byte character encoding.
For example, in such an UTF-8-encoded 1-byte character-array the character "é" would be stored as the "characters" (its UTF-8 bytes) "Ã" and "© ".
Because of its macro language, that TSE itself uses too, TSE is very open.
Therefore filenames can trickle into lots of parts of TSE, that would then all have to "know" that a character could also be 2-, 3- or 4-bytes.
I therefore think a simple stop-gap is not possible once you store a filename's multi-byte characters in TSE.
TSE v4.50.27's move to internally replace "Unicode filenames" with short filenames, which are ASCII-encoded, was therefore great, but went a step too far in also replacing "ANSI filenames" by short filenames.

Carlo

From: sem...@googlegroups.com <sem...@googlegroups.com> On Behalf Of Joe Souza
Sent: Tuesday, September 29, 2026 7:25 PM
To: sem...@googlegroups.com
Subject: Re: [TSE] Re: TSE lost ANSI filename support

I'm wondering if, as a stopgap measure until TSE fully supports wide-character Unicode names, it could perhaps use FindFirstFileW but then convert the returned name to UTF-8 for internal use. It would do this by calling WideCharToMultiByte, specifying CP_UTF8 as the code page. UTF-8 is designed to hold Unicode wide character strings in single-byte character arrays. It could also be that I'm not fully understanding the problem here.

...

[knud van eeden]

1. I assume that it worked OK for Zhong Zhao in some earlier version of TSE.

2. Which TSE version?

3. If yes, then Zhong Zhao could let us know since which TSE version exactly it still worked.

4. He could just load that Chinese character example file in different TSE versions going backwards in time and see where it starts working again.

4. Then e.g. Semware could make decisions based on that (version) information.

5. So it would probably be advised to roll back to that earlier TSE version as a workaround

because it is important that Zhong Zhao running TSE on a Chinese based Microsoft Windows operating system

can do his actions with TSE still fully and without hindrance.

To view this discussion visit https://groups.google.com/d/msgid/semware/CA%2BgEvkboov2pCW535-dXZXv23iJKUQ92Uoe34j%2BFHB9m4wL77A%40mail.gmail.com.

[H P]

It's not only a problem for filenames, in the editor using diacrits is no longer possible.

If for example I'll type á (and so on) I get a non-visible character.

After saving the file with non-visible characters and opening that file with notepad, well there they are and now visible. 

Met vriendelijke groet,
With kind regards,

Atenciosamente,
Mit Freundliche Gruß,

Tuyo sinceramente,


H. Pikaar
Henri...@gmail.com

Op wo 30 sep 2026 om 18:27 schreef 'knud van eeden' via SemWare TSE Pro text editor <sem...@googlegroups.com>:

To view this discussion visit https://groups.google.com/d/msgid/semware/759855411.2156222.1790785626912%40mail.yahoo.com.

[knud van eeden]

This should be a quite serious issue.

When I copy this 'á' and paste it using <Ctrl V> in any file in latest TSE version 4.50.30 it is not visible.

Not even visible in 'Options' > 'Hex view' as you see

in the screenshot.

So seemingly completely 'eaten' by the editor.

Pasting the same character using <Ctrl V> in my working version TSE version 4.50.rc24 shows that character á, no problem.

'

To view this discussion visit https://groups.google.com/d/msgid/semware/CAAR3jqFHdGOidW0F3bXYXh3vnfXdp2H6zqOdHP4-KV9c6hcv2w%40mail.gmail.com.

[knud van eeden]

I am checking further with <Shift Ins> because that

is default defined. This <Ctrl V> is not defined in general for a paste from Microsoft Windows clipboard.

To view this discussion visit https://groups.google.com/d/msgid/semware/2143830971.2462197.1790869446455%40mail.yahoo.com.

[knud van eeden]

At least copy 'á' from the email and doing a <Shift Ins> it successfully inserts this character 'á' 

in the text.

So it must be something in the way you type this character 'á'.

I guess you have a non-US English language set in your Microsoft Windows, then you type two keys in sequence, e.g. first an 'a' and then a forward tic or something like that. The operating system will then make 1 character of it.

[H P]

This was under "Courier New" always possible so I choose this font again but still nothing.

However when I look on the position I typed the á I found that the character is present with value 160 which according to extended ascii is a á it's just not visible. But this explains why after saving a file with notepad it becomes visible.

And for Knud, you type first the ' and the a.

Met vriendelijke groet,
With kind regards,

Atenciosamente,
Mit Freundliche Gruß,

Tuyo sinceramente,


H. Pikaar
Henri...@gmail.com

Op do 1 okt 2026 om 18:04 schreef 'knud van eeden' via SemWare TSE Pro text editor <sem...@googlegroups.com>:

To view this discussion visit https://groups.google.com/d/msgid/semware/2055617226.2459248.1790870638257%40mail.yahoo.com.

[Knud van Eeden]

> And for Knud, you type first the ' and the a.

Yes, what I already thought, but that would mean changing the (keyboard, language, ...) settings on my Microsoft Windows 11 Professional, so I will leave that further testing for now and or skip.

To view this discussion visit https://groups.google.com/d/msgid/semware/CAAR3jqHMiKPNuc%3DbgoQtUq9G15unjEEBT_2O4Ri%3DPDg%2BZNr4DQ%40mail.gmail.com.

[S.E. Mitchell]

I am very sorry about this - it definitely was not intentional, and at
this point, I don't even know what is causing it.
Do you have any idea which was the last version you used where this worked?
I've gone back to 4.5026q (Aug 20 2026) and it is broken there.
I'll keep going back until I find a working version and then run diffs
to see what has changed.

FYI: With Terminal font, it works fine. However with Consolas and
Courier New I can duplicate your behavior.

> To view this discussion visit https://groups.google.com/d/msgid/semware/CAAR3jqFHdGOidW0F3bXYXh3vnfXdp2H6zqOdHP4-KV9c6hcv2w%40mail.gmail.com.

[S.E. Mitchell]

I've gone back to the 2026 Jul 01 version: 4.50.26 - still broken there :(

[S.E. Mitchell]

Yikes! I've gone back to 2025 Dec 25, 4.50.19 - still broken there :(
Ok, no more updates until I actually find the last working version.

[S.E. Mitchell]

H. Pikaar - how do you enter the á?
If I enter it by pressing and holding alt, 0, 1, 6, 0, it is
invisible, reverting to 4.40a - I get the same result.
However, if I paste the character, it shows up. With Courier New or
Consolas, it has a value of 225.
--

[Carlo Hogeveen]

HP wrote:
> > > It's not only a problem for filenames, in the editor using diacrits is no longer possible.
> > > If for example I'll type á (and so on) I get a non-visible character.

For what it is worth, I cannot reproduce this additional bug in any version of the editor.
I enter é by pressing ' followed by e.
This works if Windows keyboard is set to an "... International" one, like "United States-International QUERTY".
Maybe also test what happens when selecting é or á from TSE's so-called "ASCII Chart".

(In Windows 11 this keyboard selection is in Settings -> Time & language -> Language & region -> click the three dots besides your language.)

Carlo

[H P]

Sammy, 

I went back to 4.50.26 and there it works.

I just put in a ' and then the a which gives me á, also by entering alt 160 not 0160.

I use the USA international keybord with qwerty, windows 11 with all available updates.

Met vriendelijke groet,
With kind regards,

Atenciosamente,
Mit Freundliche Gruß,

Tuyo sinceramente,


H. Pikaar
Henri...@gmail.com

Op vr 2 okt 2026 om 13:02 schreef Carlo Hogeveen <t...@ecarlo.nl>:

--

---
You received this message because you are subscribed to the Google Groups "SemWare TSE Pro text editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email to semware+u...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/semware/000101dd525d%2473a2ace0%245ae806a0%24%40ecarlo.nl.

[S.E. Mitchell]

Hmm. If I enter Alt+160, it works find for me in 4.50.30.
I tried it with Terminal, Consolas, and Courier New fonts.
I wonder what differences in our versions might be causing this?

I don't know how the `+a thing works, as that does not do anything
here in any version.

> To view this discussion visit https://groups.google.com/d/msgid/semware/CAAR3jqFk4wCx3EFisKYM2r2cuynjM_rwXg3jfHVCnP2HNg8YjA%40mail.gmail.com.

[Carlo Hogeveen]

HP,

What happens when you insert an "á" into your text by selecting it in TSE's so-called ASCII Chart?

The following is a far-fetched theory, but we are getting where we need to consider those.
Windows keyboard configuration is coupled to your selected Windows language.
You sign your messages in a lot of languages.
Do you perhaps (recently?) have a different Windows language selected, that is not coupled to an "... international" Windows keyboard?

Carlo

[H P]

I'm using dutch language. First jpg is done with 4.50.26 and Courier New 10;

1 char used ' and the a

2 alt 160

3 alt 0160 

4 ascii chart 160.

Second jpg is the same file but now with 4.50.26 and consolas 11.

The codepage used is 850 Multilingual (Latin I).

Met vriendelijke groet,
With kind regards,

Atenciosamente,
Mit Freundliche Gruß,

Tuyo sinceramente,


H. Pikaar
Henri...@gmail.com

Op vr 2 okt 2026 om 14:47 schreef Carlo Hogeveen <t...@ecarlo.nl>:

--

---
You received this message because you are subscribed to the Google Groups "SemWare TSE Pro text editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email to semware+u...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/semware/000001dd526c%242a4e67f0%247eeb37d0%24%40ecarlo.nl.

[Carlo Hogeveen]

HP,

I cannot reproduce the bug with those settings, so let us assume they are not the cause.
I notice that in your TSE v4.50.26 the colors are different. Maybe other things are too.
Can you reproduce your problem in a new, unchanged install of TSE v4.50.30?

Carlo

From: sem...@googlegroups.com <sem...@googlegroups.com> On Behalf Of H P
Sent: Friday, October 2, 2026 3:21 PM
To: sem...@googlegroups.com
Subject: Re: [TSE] Re: TSE lost ANSI filename support

[S.E. Mitchell]

Just to make sure I understand:

For you, inserting Alt+160 using Courier New in 4.26 displays the character, but in 4.30, with the same setup, the character appears blank.

Here, 4.26 and 4.30 work the same, as far as I can tell.

Since I cannot reproduce the problem, I'm at a loss for how to fix it.  Any suggestions?

--

To view this discussion visit https://groups.google.com/d/msgid/semware/CAAR3jqF%2BDJfMq3V9_2KRtsr1C_Tw89uR-J9EdfRV-DDH%3DWrb4Q%40mail.gmail.com.

[H P]

Something changed and I can't explain what.

With old version up to 4.50.24 it worked but no longer.

Strangely enough see tse30g32.jpg they diacritics are gone or changed to other characters. This jpg was made with g32.exe.

But tse30e32 shows the right ones but was made with e32.exe.

As you can see it shows the cmd-box with the right ones.

Met vriendelijke groet,
With kind regards,

Atenciosamente,
Mit Freundliche Gruß,

Tuyo sinceramente,


H. Pikaar
Henri...@gmail.com

Op vr 2 okt 2026 om 16:00 schreef S.E. Mitchell <sammy.m...@gmail.com>:

To view this discussion visit https://groups.google.com/d/msgid/semware/CA%2BgEvkZ3mY5E%2BwqgMi9hypg4tvn3jgVPU_B4WYUWg5HO58D%3D7w%40mail.gmail.com.

[H P]

Forgot the jpg's

Met vriendelijke groet,
With kind regards,

Atenciosamente,
Mit Freundliche Gruß,

Tuyo sinceramente,


H. Pikaar
Henri...@gmail.com

Op vr 2 okt 2026 om 17:24 schreef H P <henri...@gmail.com>:

[S.E. Mitchell]

Back to ANSI filename support.  Here is a directory sorted by filesize:

The blank after win .txt is actually an ascii 255.

Here is the same information in v4.40a:

You can't go into the Greek* directory.  Loading Peru*, ???.txt, what ???*.txt gives a new file, e.g., doesn't load the file from disk.

Here is the same information in v4.30:

You can load all the files, but you get the short filenames.

Here is the same information in v4.31:

Close to 4.30, except the three win*.txt files retain their "dir" names.  All of these load using the names shown.

To view this discussion visit https://groups.google.com/d/msgid/semware/CAAR3jqFuPwE%2B5%3DO%3D2Q5j195N111MXpGfnhSW%2B3D2yhXg%3Dh%3DNiA%40mail.gmail.com.

[H P]

Sammy, 

I found the reason why.

In the earlier versions up to 4.50.26 I used "Courier New 10" with fonttype "_OEM_" this showed the diacritics but now in 4.50.30 although changing the font "Consolas" to fonttype "_OEM_" and even changing back to "Courier New 10" with fonttype "_OEM_" the diacritics stay invisible so the change to this behaviour must lay somewhere between 4.50.26 and 4.50.30.

Met vriendelijke groet,
With kind regards,

Atenciosamente,
Mit Freundliche Gruß,

Tuyo sinceramente,


H. Pikaar
Henri...@gmail.com

Op vr 2 okt 2026 om 18:52 schreef S.E. Mitchell <sammy.m...@gmail.com>:

To view this discussion visit https://groups.google.com/d/msgid/semware/CA%2BgEvkYoiyKSGDzYQw2ONFDLXs4oM_3eBB_1gkOjhyu0Ft_bHA%40mail.gmail.com.

[S.E. Mitchell]

I moved this to its own thread, and these issues do not appear to be related.

I've spent much time comparing the .26 and .30 source, and I can't find any changes that I can see that would impact this.

However, there were some changes (I'm not sure exactly when) prior to .26 that can definitely impact this.

I wonder if the change that made the default font Consolas is the problem?

Try this:

Create a tse.cfg file if you don't have one.

Where it says:

FontName                = "Consolas"

Change that to:

FontName                = "Courier New"

Burn that into the editor, or create a stand-alone tsecfg.dat file.

Does that fix the problem?

To view this discussion visit https://groups.google.com/d/msgid/semware/CAAR3jqFKmmPw%2BXL7jweQKgy9%3DdkqXdqeApC7AGhYb%2Be1DgzNOw%40mail.gmail.com.

[Henriques18]

Found it and also with consolas.

The parameter älways Display Line Drawing Characters was ON after setting this to OFF I had my diacritics back and also after choosing Consolas as font and setting _OEM_ on.

Op zaterdag 3 oktober 2026 om 17:33:13 UTC+2 schreef S.E. Mitchell:

[S.E. Mitchell]

Yep, that makes sense.

With that setting on, some non-ASCII characters (0..31, 176..255) display as OEM characters.

Actually, that gives me a thought.

Why not open up the character translate tables so that users can display whatever characters they like?

Something like:

integer SetDisplayChar(integer index, integer new_value)

index: must be between 0..255 - is the character that will be set

new_value: must be between 0..255 - is the new value to use for whatever character is at chars_tab[index].

Returns the current value at index.

--

To view this discussion visit https://groups.google.com/d/msgid/semware/eac6b023-30d8-4003-af63-87e2e3501c2dn%40googlegroups.com.