TSE Windows 4.50.23 cmpfiles access violation

[Guy Rouillier]

I'm getting the 4.50.23 version installed and configured for both Windows and Linux. I'm using Windows 11 now mostly in a Virtualbox VM under Linux MATE 24.04.4 LTS. After composing this email, I'll try booting native Windows 7 and see the problem occurs there as well.

[EDIT] Getting the exact same access violation error under native Windows 7, with the same address.

In Windows 11, straight out of the zip file, without any customizations, if I try to run the console version e32.exe, Util - Potpourri - CmpFiles to compare the stock version of tse\ui\tse.ui to a version for an earlier TSE release, the compare completes, but upon exiting the compare, Windows pops up the Access violation Error box with the following info:

Title: Error

Message: I'm sorry, Dave. I'm afraid I can't do that.

Access violation error (c00000005) at 42f62e.

The Windows console session continues to show TSE, hung, with a message in the upper left corner saying "Restoring bookmarks".

The TSE About dialog has:

The SemWare Editor (Console) 4.50.23 - 2026 April 15

Internal version: 12435

Started from command line: yes

Memory used; 453 kB left: 3.93 GB

The GUI version g32.exe doesn't seem to have this issue.

-- 

Guy Rouillier

Virus-free.www.avast.com

[S.E. Mitchell]

Sorry for the problem.

This is a very frustrating one!

I was able to get it a few times, but I can't make it happen all the time.

And I can't get it to happen in the debugging version.

If you can find a way to make it crash each time, please let me know.

In the meantime, I'll keep trying to figure out what is causing it.

--

---
You received this message because you are subscribed to the Google Groups "SemWare TSE Pro text editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email to semware+u...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/semware/em39e943b4-9303-4ef8-9bc5-5493b02a4a88%40gmail.com.

[Guy Rouillier]

Here's a way I get it to fail consistently, in a Windows Command Prompt
window:

(1) Unzip  tse-setup-4.50.23.zip and run the installer.  I unzipped in
d:\util\temp, and installed in d:\util\temp\tse.  All the following steps are
from inside this install directory.

(2) Copy the unaltered tse.ui file from the newly installed ui\tse.ui file.  I
copied to ui\tse.ui to tse_45023.org.

(3) Copy the unaltered tse.ui file from a version 4.50.20 installation. I
copied from d:\util\tse-45020\ui\tse.ui to tse_45020.org.

(4) Now start up e32 with both of these files: 
e32 tse_45020.org tse_45023.org.

(5) Once inside the editor, through the menus, execute:

Util -> Potpourri -> CmpFiles

(6) Answer Yes to Compare currently loaded files

(7) Now just tap the Enter key until the macro gets to the bottom of both
files. At that point, tap the Esc key.

(8) When I do this, I get a popup Warning from TSE stating "No Block", with an
OK button. Then I press the Enter key for OK, I then get a "Search for:"
popup. When I Esc out of that, I then see a message "Restoring bookmarks"
under the menu bar, and the editor is hung.  Hiding behind the window is a
Windows popup with the Access Violation I detailed previously.

(9) I can close the Command Prompt window and start over.

> > Virus-free.www.avast.com
> >
> >  

--
Guy Rouillier

[S.E. Mitchell]

Yep, I get the same. Thanks!
It works in 4.50.22.
When cmpfiles returns, it goes into la-la land.
Trying to isolate why that happens.
Thanks for the report!

> --
>
> ---
> You received this message because you are subscribed to the Google Groups "SemWare TSE Pro text editor" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to semware+u...@googlegroups.com.

> To view this discussion visit https://groups.google.com/d/msgid/semware/36eb8f27cc49fd90c9832f57abde55d343bea3c2.camel%40gmail.com.

[Carlo Hogeveen]

> It works in 4.50.22.

That is not what I am seeing.
In the old TSE versions I have readily available, I can reproduce the abort back to TSE 4.42 (3 Jan 2022).

Carlo

[S.E. Mitchell]

How do you reproduce it?
In 4.50.23, I do:
e32 -e cmpfiles
And it goes off into the weeds.
cmpfiles() is not returning to the caller, but no idea why.
In 4.50.22, I do the same, and all seems well.

> --
>
> ---
> You received this message because you are subscribed to the Google Groups "SemWare TSE Pro text editor" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to semware+u...@googlegroups.com.

> To view this discussion visit https://groups.google.com/d/msgid/semware/000101dceaae%2429980530%247cc80f90%24%40ecarlo.nl.

[S.E. Mitchell]

I should have added:
I press <esc> at the cmpfiles prompt window, and then it goes off into
the weeds.

[Carlo Hogeveen]

I do the exact same thing, and press <Escape> on all the prompts that follow.
e32 -e cmpfiles
I have sampled lots of TSE versions between 4.42 and 4.50.23, and in all of them Dave shows up.

Carlo

[S.E. Mitchell]

Ok, I can now reproduce this in version 4.42.
I had to turn debugging off :(

> --
>
> ---
> You received this message because you are subscribed to the Google Groups "SemWare TSE Pro text editor" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to semware+u...@googlegroups.com.

> To view this discussion visit https://groups.google.com/d/msgid/semware/000401dceab1%24717b01b0%2454710510%24%40ecarlo.nl.

[Guy Rouillier]

I don't know how the TSE debugger works, but back before I retired and worked
in software development as an employee, we learned that C debuggers were known
to mask data initialization errors. A "feature" of the debugger (for example,
in Microsoft C IDE or Borland Turbo IDE) would nicely initialize all variables
to default values if a value wasn't explicitly assigned, e.g., zero to
integers and floating point, and either null or the empty string to strings.
So, problems related to data initialization would be masked or "act normally"
in the debugger, but still crash or throw and exception when run outside the
debugger. This made finding issues related to data initialization very
challenging to find.

--
Guy Rouillier

[S.E. Mitchell]

I've narrowed the bug somewhat.
The bug was introduced in the Nov 05 2007 version.
The bug is not reproducible in the Nov 03 2007 version.
Unfortunately, there are lots of differences between those versions.
I have narrowed down the reproduction steps. I would be glad if
someone could narrow it down even further:

integer hookI, hookBUD, hookAUD

proc SaveHookState()
hookI = SetHookState(OFF, _IDLE_)
hookBUD = SetHookState(OFF, _BEFORE_UPDATE_DISPLAY_)
hookAUD = SetHookState(OFF, _AFTER_UPDATE_DISPLAY_)
end

proc RestoreHookState()
SetHookState(hookI, _IDLE_)
SetHookState(hookBUD, _BEFORE_UPDATE_DISPLAY_)
SetHookState(hookAUD, _AFTER_UPDATE_DISPLAY_)
end

keydef FilenamePromptKeys
<Enter> EndProcess(TRUE)
<GreyEnter> EndProcess(TRUE)
end

proc FilenamePromptStartup()
Enable(FilenamePromptKeys)
end

proc FilenamePromptCleanup()
Disable(FilenamePromptKeys)
end

string temp_fn[_MAX_PATH_]

proc Main()
string st[_MAX_PATH_] = "*.zip"
string temp_dir[_MAX_PATH_] = GetEnvStr("TEMP")

temp_fn = QuotePath(MakeTempName(temp_dir))
ExecMacro("state -l -s -f" + temp_fn)

SaveHookState()

UpdateDisplay()
Hook(_PROMPT_STARTUP_, FilenamePromptStartup)
Hook(_PROMPT_CLEANUP_, FilenamePromptCleanup)
AskFilename('Compare "'
+ Upper(SplitPath(CurrFilename(), _NAME_ | _EXT_)) +
'" to: ',
st, _MUST_EXIST_)
UnHook(FilenamePromptStartup)
UnHook(FilenamePromptCleanup)

PurgeMacro(CurrMacroFilename())
RestoreHookState()

warn("calling state restore:"; temp_fn)
ExecMacro("state -l -r -f" + temp_fn)
warn("calling EraseDiskFile()")
EraseDiskFile(temp_fn)
warn("macro returning")
end Main

> To view this discussion visit https://groups.google.com/d/msgid/semware/8e82fc334127ee4430e600421264730c23433d24.camel%40gmail.com.

[zhong zhao]

I just use windbg to load e32.exe -e cmpfiles,and get call-stack list and save to file zz.txt when error.

Attached call-stack.zip which include e32.c,e32.lst and zz.txt,added every line call-chain commented by "//~call-stack {0d,0e,...,00}",

is helpfully to track call-chain.

[zhong zhao]

I just use g32.exe and e32.exe  run above macro:

integer hookI, hookBUD, hookAUD

proc SaveHookState()

...

end Main

result is g32.exe is OK. e32 pop dialog "Dave..."

[Guy Rouillier]

I put your code into a file called cmpbug.s, and after compiling and running,
verified it produced the editor hang and access violation. Through trial and
error, I narrowed the issue down to line 53:

warn("calling state restore:"; temp_fn)

// ExecMacro("state -l -r -f" + temp_fn)

When I commented out this ExecMacro, the error did not occur.

--
Guy Rouillier

[Carlo Hogeveen]

Sammy wrote:
> In the meantime, I've narrowed the bug somewhat.

> The bug was introduced in the Nov 05 2007 version.
> The bug is not reproducible in the Nov 03 2007 version.

I cannot reproduce that.
Based on those dates and TSE's read.me, the relevant versions I tested were v4.40.32 (Jun 26 2007) and v4.40.34 (Nov 14 2007).
Both did not abort using "e32 -ecmpfiles" and your test macro.

Instead, using "e32 -ecmpfiles" and your test macro, I find that the bug does not occur in v4.41.45 (13 Mar 2021) and does occur in v4.41.46 (17 Dec 2021).
v4.41.46 was your Christmas present: The first TSE version with the generous license.
v4.41.46 has a lot of documented changes in the read.me.
I checked: cmpfiles and all its called macros were not changed between v4.41.45 and v4.41.46.

As a test because of their read.me comment, I ported the old syncfg.si and syncfg2.si from v4.41.45 to v4.41.46.
The bug still occurred, so the syncfg and syncfg2 changes are not my suspects.

Carlo

[S.E. Mitchell]

Still trying to whittle it down, based on your note. I tried:

integer hookI, hookBUD, hookAUD

proc SaveHookState()
hookI = SetHookState(OFF, _IDLE_)
hookBUD = SetHookState(OFF, _BEFORE_UPDATE_DISPLAY_)
hookAUD = SetHookState(OFF, _AFTER_UPDATE_DISPLAY_)
end

proc RestoreHookState()
SetHookState(hookI, _IDLE_)
SetHookState(hookBUD, _BEFORE_UPDATE_DISPLAY_)
SetHookState(hookAUD, _AFTER_UPDATE_DISPLAY_)
end

string temp_fn[_MAX_PATH_]

proc Main()

string temp_dir[_MAX_PATH_] = GetEnvStr("TEMP")

temp_fn = QuotePath(MakeTempName(temp_dir))
ExecMacro("state -l -s -f" + temp_fn)

SaveHookState()

MsgBox("bam", "does this crash?")

RestoreHookState()

warn("calling state restore:"; temp_fn)
ExecMacro("state -l -r -f" + temp_fn)
warn("calling EraseDiskFile()")
EraseDiskFile(temp_fn)
warn("macro returning")
end Main

But it seems to work okay; I did not get a crash.

I'm still trying to find the smallest macro that causes a crash.

> To view this discussion visit https://groups.google.com/d/msgid/semware/89a2428370f5e2b21fdd07df6cce21225330310e.camel%40gmail.com.

[S.E. Mitchell]

Very interesting! I must have kept reintroducing that bug!

I did not release this one:

v4.40.34 Nov 5, 2007 is on the signon screen.

It crashes here.

> --
>
> ---
> You received this message because you are subscribed to the Google Groups "SemWare TSE Pro text editor" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to semware+u...@googlegroups.com.

> To view this discussion visit https://groups.google.com/d/msgid/semware/001901dcec2b%245888ee30%24099aca90%24%40ecarlo.nl.

[S.E. Mitchell]

Some more information:

The crash is actually a little bit of a red-herring.
Sometime towards the end of the macro in question, TSE encounters an
unknown opcode.
Usually this occurs just before cmpfiles exits.
A routine is called to print the "Unknown opcode" message, but that
routine crashes.

So the error appears to be some unknown interaction between the
ExecMacro("state ....
and something else the cmpfiles macro does. It keeps cmpfiles from
returning to the main loop, but rather keeps interpreter executing
opcodes, which will be garbage at this point.

Still trying to whittle it down to the smallest macro that can
reproduce the problem.

Thanks for all the help!

[S.E. Mitchell]

Thanks to some help from Stein, another culprit appears to be an early
PurgeMacro()
This now crashes:

integer hookI, hookBUD, hookAUD

proc SaveHookState()
hookI = SetHookState(OFF, _IDLE_)
hookBUD = SetHookState(OFF, _BEFORE_UPDATE_DISPLAY_)
hookAUD = SetHookState(OFF, _AFTER_UPDATE_DISPLAY_)
end

proc RestoreHookState()
SetHookState(hookI, _IDLE_)
SetHookState(hookBUD, _BEFORE_UPDATE_DISPLAY_)
SetHookState(hookAUD, _AFTER_UPDATE_DISPLAY_)
end

string temp_fn[_MAX_PATH_]

proc Main()

string temp_dir[_MAX_PATH_] = GetEnvStr("TEMP")

temp_fn = QuotePath(MakeTempName(temp_dir))
ExecMacro("state -l -s -f" + temp_fn)

SaveHookState()

MsgBox("bam", "does this crash?")

PurgeMacro(CurrMacroFilename()) // purge the macro

RestoreHookState()

warn("calling state restore:"; temp_fn)
ExecMacro("state -l -r -f" + temp_fn)
warn("calling EraseDiskFile()")
EraseDiskFile(temp_fn)
warn("macro returning")
end Main

Need to keep whittling this down.

[Carlo Hogeveen]

I went at the problem from another angle.
Why would "e32 -ecmplies" plus <Escape>s abort, but not "e32 -ecmplies" plus <Escape>s?
One difference is that GUI TSE has a font and Console and Linux TSE do not.
Quick and dirty (!):
I removed saving and restoring the font from state.s.
Now "e32 -ecmplies" plus <Escape>s no longer aborts.
The same improved happens when testing this with Sammy's test macro.
https://ecarlo.nl/tse/DemosAndTests.html#state

Carlo

[zhong zhao]

t1.s:

proc main()
    SetColorTableValue(_BACKGROUND_, 0,0x000000 )
end

At cmd:

sc32 t1.s

e32 -e t1

e32 -e t1

e32 -e t1

... ...

about 1 time Error on 10 times OK.

Attached is snapshot when Error and OK,and zz.txt(in windbg call-stack info when Error),e32.lst,e32.c(Comment //~call-stack 0d,0c,...,01,00,-1  at every line end) that is helpfully for track call-chain.

I guess the Error because multi-threads read/write one memory or other shared-resource conflicts

[zhong zhao]

I just try t1.s:

proc main()
    SetColorTableValue(_BACKGROUND_, 0,0x000000 )
end

on WinXP in VMWare,Error per time when 'e32 -e t1' in cmd.

Attached snapshot.

[S.E. Mitchell]

I got it down to this:

xxx15.s:

proc Main()
string temp_fn[_MAX_PATH_] = QuotePath(MakeTempName(GetEnvStr("TEMP")))

ExecMacro("state -l -s -f" + temp_fn)

PurgeMacro(CurrMacroFilename()) // purge the macro

ExecMacro("state -l -r -f" + temp_fn)

EraseDiskFile(temp_fn)
end

And found where the problem is. It is a small function in the macro interpreter:

long APPCALL interp(MACRO_FNTAB *m, XUINT macro_offset) {
MACRO_FNTAB *save_user_id;

if (m < &macro_fntab[0] || m > &macro_fntab[MAX_MACRO_FILES] ||
((UCHAR *)m - (UCHAR *)macro_fntab) % sizeof(MACRO_FNTAB))
return error(MACRO_LOST);

save_user_id = user_macro_id;
if (m != BIND_ID)
user_macro_id = m;
if (cmLockMacroId(m)) {
lo_interp(m, macro_offset);
cmUnlockMacroId(m);
}
PurgeMacros();
user_macro_id = save_user_id;

return accum.integer;
}

If I compile this function without optimizations, it works fine.
If I compile this function (well, the entire source file) with
optimizations, then after the call to lo_interp(), the value of "m"
gets changed to a different entry in the MACRO_FNTAB (table of
pointers to loaded macros).
And so the wrong macro gets unlocked (m->fn points to "state" coming
in, but gets revectored to "xxx15"), so "xxx15" gets purged right
before it is finished, which causes the crash.

If I add code to save\restore the value of "m" before the call to
lo_interp(), things work fine. But I should not have to do that.
Perhaps lo_interp() (or one of its subordinates) is (somehow) in
changing a value on the stack that does not belong to them?

I have been able to reproduce this one under the Turbo Debugger, but
have not been able to find where "m" gets changed. Will continue to
investigate around work and grandkids :)

> --
>
> ---
> You received this message because you are subscribed to the Google Groups "SemWare TSE Pro text editor" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to semware+u...@googlegroups.com.

> To view this discussion visit https://groups.google.com/d/msgid/semware/001c01dcec42%24bfe36620%243faa3260%24%40ecarlo.nl.

[Carlo Hogeveen]

Sorry for the illegibility of my previous message.
It had more typos than a group of universal blood donors.

Sammy, I think our investigative paths converge!

state.s saves and restores e32's non-existing font.
My tests say that in e32 SetFont()'s parameters are irrelevant for the following.
Calling SetFont() in e32 messes with what TSE thinks its loaded macros are.

After executing the demo macro below, the Macro Purge menu says that the macro is still loaded!

When applying the Macro Compile menu twice and both times selecting Execute, I get the following error message:
"Cannot overlay running macro: <macro name>"

Commenting the SetFont() statement "fixes" both these errors.

I think that the mispointed entry into your "table of pointers to loaded macros" that you reported, matches with what I am seeing SetFont() doing in my test results!

Demo macro:
proc Main()
SetFont('0', 0, 0)
PurgeMacro(CurrMacroFileName())
End Main

A simple way to invoke Dave with probably the same cause:
When e32 is started with "New File", apply the Macro Execute menu on the following line:
execute SetFont('Terminal', 12, 0)
Here too SetFont()'s parameters are irrelevant.

Carlo

[S.E. Mitchell]

You're a genius Carlo! :)
Yep, it was the SetFont() call, and only in the Console version.
It is fixed now!
Thanks!

> --
>
> ---
> You received this message because you are subscribed to the Google Groups "SemWare TSE Pro text editor" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to semware+u...@googlegroups.com.

> To view this discussion visit https://groups.google.com/d/msgid/semware/001d01dced06%24703dfe90%2450b9fbb0%24%40ecarlo.nl.