My 64-bit starpack silently exits when I call any tcl api function.

[ronald benn]

Bottom line: I am stuck creating a 64-bit starpack.

I create a 64-bit starpack executable. When I execute it, App_Init() is invoked which in turn calls Tcl_InitStubs(). Tcl_InitStubs() returns "8.6.13", the correct version. After that, any call to the Tcl api, say Tcl_GetVar(), exits the process silently. This is true for any tcl api function call.

Note that Tk_InitStubs also works and returns the correct version.

I have built 32-bit starpacks from the same applcation for years.

The main difference between 32-bit and 64-bit applications is the replacement of 32-bit binary extensions but as we never get to the point where any of these extensions are loaded, they are not the problem.

It's as if the linkage between the stubs code and the tcl library is broken.

My environment:

I installed BAWT and used it to build 64-bit Tcl Tk and extensions from scratch.

Then I used the extensions and tcl/tk libraries provided by the BAWT build. For example:

tclkit-win64-tcl.exe -> used to run the SPX kit
spx.kit -> the spx kit
wrap application.exe -> wraps application.vfs conaining the app dll
-runtime tclkit-win64-tk.exe -> app connected to this execuatble base
tclstubs86.lib -> linked to the stubs libraries (also the headers)
tkstubs86.lib

BAWT experts say this should work.

To build the 64-bit BAWT I started with a clean machine and installed:
- BAWT 2.3.0
- tcl 8.6.13
- mingw 8.1
- visual studio 17.4.4

Then I successfully ran the following to create my BAWT environment:

c:>bawt/bawt-2.3.0/Build-Windows.bat x64 vs2013+gcc Setup\TclDistribution.bawt update

I have tried a variety of tcl base executables. They all fail the same way so I guess it's not tcl/tk base executable's fault.

I compile my application as a dll with USE_TCL_STUBS and USE_TK_STUBS.

I am linking to tclstubs86.lib and tkstub86.lib, also taken from the BAWT build and produced by BAWT using visual C++.

Anybody encountered this problem before? It is my belief that somewhere I have missed a trick that only an experienced starpack creator would identify.

Ronald Benn

[Christian Gollwitzer]

Am 18.01.23 um 23:14 schrieb ronald benn:

> Bottom line: I am stuck creating a 64-bit starpack.
>
> I create a 64-bit starpack executable. When I execute it, App_Init() is invoked which in turn calls Tcl_InitStubs(). Tcl_InitStubs() returns "8.6.13", the correct version. After that, any call to the Tcl api, say Tcl_GetVar(), exits the process silently. This is true for any tcl api function call.

To me, it is unclear what part you wrote and what part not. If I
understand correctly, the tclkit-win64-tcl.exe is just the thing you got
from BAWT. Does it work to run this purely, without wrapping anything,
and you get a a Tcl prompt where you can do "package require Tk"? If
not, the Tclkit is broken.

Then, I understand you have a C extension which you wish to load. Does
your extension call Tcl_AppInit()? Because it shouldn't.

Your description of the stack trace sounds strange. If normal Tcl calls
fail, then it looks like the stubs table isn't initialized, or an
application was built embeds the Tcl interpreter rather than extends it,
and it links to the stubs library, when it should link to the real library.

You can also try these tclkits which are built by me on Github Actions:
https://github.com/auriocus/kbskit/releases

There is an application built using this: https://github.com/BessyHDFViewer

Christian

[ronald benn]

==========================================================================================
More details on the problem:

1 int Qw_Init(Tcl_Interp* Interpreter) {
2 const char* TclInitStubsResult=Tcl_InitStubs(Interpreter,TCL_VERSION,0);
3 cout<<"Qw_Init,1000.0,Tcl_InitStubs,result=="<<TclInitStubsResult<<endl;
4 if (TclInitStubsResult==0) {
5 return TCL_ERROR;
6 }
7 cout<<"Qw_Init,1000.1,"<<endl;
8 char* Buffer=(char*)malloc(100);
9 cout<<"rwb1_debug,Qw_Init,1000.0.3.3"<<endl;
10 ::strcpy(Buffer,"Hello World");
11 cout<<"Qw_Init,1000.2"<<endl;
12 cout<<"Qw_Init,buffer length=="<<::strlen(Buffer)<<endl;
13 cout<<"Qw_Init,1000.3"<<endl;
14 cout<<"QW_init,::argv=="<<Tcl_GetVar(Interpreter,"::argv",TCL_GLOBAL_ONLY)<<endl;
15 ...

The code above is the init function for qw.dll. The package name is qw. The function executes up to line 13 and then silently exists the process when calling Tcl_GetVar in line 13. The correct version is returned from Tcl_InitStubs abnd sent to standard out.

I added lines 8 t0 12 to prove we are successfully calling the c runtime library api as malloc(), strcpy(), strlen() and std::cout all execute and send to stdout as expected. Other output is send to stdout from main.tcl using the puts command, proving that the interpreter is running.

The last thing that comes out in stdout is "Qw_Init,1000.3".

Any tcl (or tk) api function exist silently. So does anyone know why we are not linking properly to the Tcl api stub functions? That is the question. Note that the same coed linked properly in 23-bit.

Ronald Benn

[Michael Soyka]

I can't speak to the linking possibility but you might check the Windows
event logs using "eventvwr". Application errors are sometimes recorded
there.

-mike

[ronald benn]

==============================================================
The event log only shows that a dll (my dll) cannot be loaded.

BTW: I tested the tclkit exe's,, both the shell and the gui versions, running them from the command line on test scripts, and they both passed the test. Seem to be on solid ground there.

Ron

[Christian Gollwitzer]

Am 19.01.23 um 20:15 schrieb ronald benn:

> ==========================================================================================
> More details on the problem:
>
> 1 int Qw_Init(Tcl_Interp* Interpreter) {
> 2 const char* TclInitStubsResult=Tcl_InitStubs(Interpreter,TCL_VERSION,0);
> 3 cout<<"Qw_Init,1000.0,Tcl_InitStubs,result=="<<TclInitStubsResult<<endl;
> 4 if (TclInitStubsResult==0) {
> 5 return TCL_ERROR;
> 6 }
> 7 cout<<"Qw_Init,1000.1,"<<endl;
> 8 char* Buffer=(char*)malloc(100);
> 9 cout<<"rwb1_debug,Qw_Init,1000.0.3.3"<<endl;
> 10 ::strcpy(Buffer,"Hello World");
> 11 cout<<"Qw_Init,1000.2"<<endl;
> 12 cout<<"Qw_Init,buffer length=="<<::strlen(Buffer)<<endl;
> 13 cout<<"Qw_Init,1000.3"<<endl;
> 14 cout<<"QW_init,::argv=="<<Tcl_GetVar(Interpreter,"::argv",TCL_GLOBAL_ONLY)<<endl;
> 15 ...
>
> The code above is the init function for qw.dll. The package name is qw. The function executes up to line 13 and then silently exists the process when calling Tcl_GetVar in line 13. The correct version is returned from Tcl_InitStubs abnd sent to standard out.

The code looks good to me. When you compile your code, is USE_TCL_STUBS
defined? i.e. on the command line for the MS compiler it would be
/DUSE_TCL_STUBS, in the IDE I don't know. Maybe you have different
settings for 32 bit vs 64 bit? You could also #define it in the file
before you include <tcl.h>

Christian

[ronald benn]

======================================================================
TCL_USE_STUBS and TK_USE_STUBS are defined using -D as I am using GNU g++ under MINGW.

I have verified that they are defined using an #error preprocessor statement to check them.

[Christian Gollwitzer]

Am 19.01.23 um 21:38 schrieb ronald benn:

Just a nitpick, it is USE_TCL_STUBS, not TCL_USE_STUBS (you probably
just mixed it up in this post, haven't you?)

>
> I have verified that they are defined using an #error preprocessor statement to check them.
>

You mentioned the MS compilers earlier on. When you use gcc anyway, is
there a reason that you link to the .lib files for the MS compiler?
While this should work AFAIK, wouldn't it be easier to link to the
stublib created by gcc? And, are you sure it is not picking up some
other version of that library somewhere from your system?



Christian

[Alan Grunwald]

On 19/01/2023 20:11, ronald benn wrote:
> On Thursday, January 19, 2023 at 2:49:32 PM UTC-5, mss...@gmail.com wrote:
>> On 01/19/2023 2:15 PM, ronald benn wrote:
>>> On Thursday, January 19, 2023 at 3:37:55 AM UTC-5, Christian Gollwitzer wrote:

Lots of stuff snipped

This is likely irrelevant and is certainly at the very limit of my
understanding of what's going on, but surely the code above is in your
dll, so if it is being executed, then your dll must have been loaded OK.

Alan

[ronald benn]

==================================================================
Code is in the dll and the dll definitely loaded ok.
Then first call to tcl api though stubs fails.
Also have determined that the failure is a gp. Silent exit is because I trap the GP and my error handling code has not been set up to process it properly (because we really haven't booted the app at all).

Also note that I did spell USE_TCL_STUBS properly in the code.
I have reinstalled BAWT to upgrade to 2.3.1 and to use gcc 7.2.0. Same problem comes up.
Now I am rebuilding BAWT in 32-bit mode to try that. BTW BAWT is a great piece of work.

[ronald benn]

==================================================================
Note I use the library produced by Visual C++ because the BAWT documentation says only the Visual C++ version is compaitble with stubs.

Ron

[Christian Gollwitzer]

Am 20.01.23 um 22:42 schrieb ronald benn:

> Code is in the dll and the dll definitely loaded ok.
> Then first call to tcl api though stubs fails.
> Also have determined that the failure is a gp. Silent exit is because I trap the GP and my error handling code has not been set up to process it properly (because we really haven't booted the app at all).

Yes, I have expected this. The stubs mechanism basicall replaces a call
like Tcl_GetVar() using a macro by (tclStubsPtr->tcl_GetVar)()

The InitStubs() call should initialize the stubsptr with a pointer to
the correct table. If it doesn't work properly, e.g. if ou forget the
call to InitStubs, then the pointer is zero or some garbage and the call
fails. Hence, I was suspecting that you link the wrong library,
inconsistent include files or macro.

BTW, can't you use a debugger to step into the code and see what goes
wrong? E.g. if your call to Tcl_GetVar returns NULL, it will also crash.

You can also try to compile a sampleextension against your Tcl using the
same Makefile, and see if it can be loaded. If yes, the problem is more
likely on your code.

Christian

[ronald benn]

=========================================================================
Still stuck on problem at same point.

I created a different but equivalent 64-bit tcl-base generated by Kleene's on-line kitcreator and ended up with exactly the same problem.

Therefore I have to conclude that the problem is mine and not Kleene or BAWT.

In App_Init, after calling Tcl_InitStubs, I retrieved the address of a typical tcl api function, i.e. tclStubsPtr->tcl_GetVar2Ex, and the address was 29878472570568704.

This address is extremely suspicious unless there's something about windows memory management for dll's that I am unaware of.

The address of tclStubPtr itself is 12042432, which is more reasonable as were several non-tcl variable and function addresses I retrieved as a sanity check.