Tcl_CreateInterp() hangs

[Jens Benner]

Hi all,

I tried to enhance an existing Application with an Tcl interpreter. This
Application starts several thread to parse some different files. The
threading is done by win32 mechanism.

In every thread id do the following things:

// On thread Startup
TclInterp = Tcl_CreateInterp();
Tcl_FindExecutable(NULL);
Tcl_CreateCommand(TclInterp, "myCallback", TclCallbackCmd,
(ClientData)this, (Tcl_CmdDeleteProc*) NULL);
Tcl_EvalFile(TclInterp, "MySript.tcl");

...

//Pass some data and do some work
while ( ... ) {

Tcl_Eval(TclInterp, "myCommad param1 param2");

}

...

// On thread shutdown
Tcl_DeleteCommand(TclInterp, "myCallback");
Tcl_DeleteInterp(TclInterp);

This runs fine so far for a lot of cycles but it randomly hangs up after
several hours or days. A closer look with the debugger shows, that it
hangs at the line TclInterp = Tcl_CreateInterp();

Is there may be something I forgot, or something I'm doing wrong?
Do I have to call Tcl_Finalize() after Tcl_DeleteInterp()?

Thanks in advance.
Jens

[Christian Nassau]

Jens Benner wrote:
> In every thread id do the following things:
>
> // On thread Startup
> TclInterp = Tcl_CreateInterp();
> Tcl_FindExecutable(NULL);
> Tcl_CreateCommand(TclInterp, "myCallback", TclCallbackCmd,
> (ClientData)this, (Tcl_CmdDeleteProc*) NULL);
> Tcl_EvalFile(TclInterp, "MySript.tcl");
> ...

The purpose of "Tcl_FindExecutable(NULL);" is to initialize all of the
subsystems that the other Tcl calls rely on. You should therefore call
it *only once* when your application starts, and *before* any worker
thread is spawned.

Apart from that your setup looks fine to me.

HTH,

--
=> Christian Nassau, http://www.nullhomotopie.de

[Don Porter]

Jens Benner wrote:
> This runs fine so far for a lot of cycles but it randomly hangs up after
> several hours or days. A closer look with the debugger shows, that it
> hangs at the line TclInterp = Tcl_CreateInterp();

It might make more sense to debug this via the Tracker at Tcl's
SourceForge project. Meanwhile, precisely what release of Tcl
shows the trouble you report?

--
| Don Porter Mathematical and Computational Sciences Division |
| donald...@nist.gov Information Technology Laboratory |
| http://math.nist.gov/~DPorter/ NIST |
|______________________________________________________________________|

[Jens Benner]

Don Porter schrieb:

> Jens Benner wrote:
>> This runs fine so far for a lot of cycles but it randomly hangs up after
>> several hours or days. A closer look with the debugger shows, that it
>> hangs at the line TclInterp = Tcl_CreateInterp();
>
> It might make more sense to debug this via the Tracker at Tcl's
> SourceForge project. Meanwhile, precisely what release of Tcl
> shows the trouble you report?
>

Thanks a lot for your help Don,

I'm using Tcl 8.4.2 and I linked against tcl84t.lib/tcl84t.dll

Regards
Jens

[Jens Benner]

Christian Nassau schrieb:

Hello Christian,

the application loads a lot of modules (dlls) via dynamically via
loadlibrary(). In one of these modules I integrated the Tcl interpreter.
So only this module is linked against tcl84t.lib/tcl84t.dll. The Module
is unloaded after work and loaded again if there is work to do.
So do I have to call Tcl_FindExecutable(NULL) every time the module is
loaded? Or do I have to link the complete Application to Tcl and call
Tcl_FindExecutable(NULL) at application Startup?
Is there a way to detect if Tcl_FindExecutable(NULL) was already called
or has to be called again?

Thanks
Jens

[Alexandre Ferrieux]

If the whole DLL linked against Tcl is unloaded, then obviously any
static memory area associated with it vanishes. Hence whatever is
stored by Tcl_FindExecutable is wiped.

Moreover, have a look at the Tcl_Finalize manpage. Not sure about the
"automatic" behavior mentioned there, but the manual call is guranteed
to be safe even if redundant, so I wouldn't hesitate...

-Alex

[Don Porter]

Jens Benner wrote:
> I'm using Tcl 8.4.2 and I linked against tcl84t.lib/tcl84t.dll

Hmmm. Let me confess that my interest in your problem is whether it
demonstrates some bug in Tcl. I'm far less interested in finding a bug
in the 2003 release of Tcl than I am in finding bugs in any of the 2008
releases of Tcl. Is there no way you can use Tcl 8.4.19 or 8.5.4 ?

In another followup you gave more details about your program. It is
multi-threaded and repeatedly loads and unloads tcl.dll. I think it's
fair to say this is a under-tested mode of using the Tcl library. It's
supposed to work, but I don't know of any example I can point to to show
it working. So, it would be really helpful if you could migrate the
stress-testing you are doing to an active release of Tcl where bug fixes
might be possible (or, if we're lucky, already fixed!).

[Don Porter]

Jens Benner wrote:
>>> // On thread Startup
>>> TclInterp = Tcl_CreateInterp();
>>> Tcl_FindExecutable(NULL);
>>> Tcl_CreateCommand(TclInterp, "myCallback", TclCallbackCmd,
>>> (ClientData)this, (Tcl_CmdDeleteProc*) NULL);
>>> Tcl_EvalFile(TclInterp, "MySript.tcl");

> the application loads a lot of modules (dlls) via dynamically via

> loadlibrary(). In one of these modules I integrated the Tcl interpreter.
> So only this module is linked against tcl84t.lib/tcl84t.dll. The Module
> is unloaded after work and loaded again if there is work to do.
> So do I have to call Tcl_FindExecutable(NULL) every time the module is
> loaded? Or do I have to link the complete Application to Tcl and call
> Tcl_FindExecutable(NULL) at application Startup?

The value of Tcl_FindExecutable(NULL) is that it internally calls
TclInitSubsystems() which is a vital part of getting the Tcl library
initialized for all of its tasks.

The routine Tcl_CreateInterp() also internally calls
TclInitSubsystems(). So, in your example code, I believe your
Tcl_FindExecutable(NULL) calls are unnecessary, though I believe
harmless.

[Don Porter]

Jens Benner wrote:
> This runs fine so far for a lot of cycles but it randomly hangs up after
> several hours or days. A closer look with the debugger shows, that it
> hangs at the line TclInterp = Tcl_CreateInterp();

There is no mutex in the Tcl_CreateInterp() call itself. Does your
debugger place the hang in that routine explicitly, or could it
be nested deeper in calls that Tcl_CreateInterp() makes? If so,
what deeper routine is the one hanging?

[Jens Benner]

Hello all,

I just tried the same thing with Tcl 8.5.5. I linked against
tcl85tg.lib/tcl85tg.dll and I added Tcl_Finalize(); after
Tcl_DeleteInterp(TclInterp);

Now I get crash after several cycles:

msvcr80d.dll!_NMSG_WRITE(int rterrnum=10) Zeile 198 C
msvcr80d.dll!abort() Zeile 59 + 0x7 Bytes C
tcl85tg.dll!Tcl_PanicVA(const char * format=0x05d61fd4, char *
argList=0x0426ebe0) Zeile 103 + 0x8 Bytes C
tcl85tg.dll!Tcl_Panic(const char * format=0x05d61fd4, ...) Zeile 132
+ 0xd Bytes C
tcl85tg.dll!BogusFind(Tcl_HashTable * tablePtr=0x05d71ba0, const char *
key=0x00000440) Zeile 941 + 0xf Bytes C
tcl85tg.dll!ThreadStorageGetHashTable(Tcl_ThreadId_ * id=0x00000440)
Zeile 212 + 0x11 Bytes C
tcl85tg.dll!TclpThreadDataKeyGet(Tcl_ThreadDataKey_ * *
keyPtr=0x05d71658) Zeile 336 + 0xb Bytes C
tcl85tg.dll!Tcl_GetThreadData(Tcl_ThreadDataKey_ * * keyPtr=0x05d71658,
int size=8) Zeile 87 + 0x9 Bytes C
tcl85tg.dll!TclFreeObj(Tcl_Obj * objPtr=0x06b57bb8) Zeile 912 + 0xc Bytes C
tcl85tg.dll!FreeListInternalRep(Tcl_Obj * listPtr=0x05e2c800) Zeile
1597 + 0x1e Bytes C
tcl85tg.dll!TclCleanupLiteralTable(Tcl_Interp * interp=0x05e59520,
LiteralTable * tablePtr=0x05e595ec) Zeile 137 + 0xe Bytes C
tcl85tg.dll!DeleteInterpProc(Tcl_Interp * interp=0x05e59520) Zeile 1252
+ 0x12 Bytes C
tcl85tg.dll!Tcl_EventuallyFree(void * clientData=0x05e59520, void (char
*)* freeProc=0x05c411f0) Zeile 300 + 0x9 Bytes C
tcl85tg.dll!Tcl_DeleteInterp(Tcl_Interp * interp=0x05e59520) Zeile 1175
+ 0xe Bytes C
MyApp.dll!MyApp::~MyApp() Zeile 79 + 0x10 Bytes C++
MyApp.dll!MyApp::`scalar deleting destructor'() + 0x14 Bytes C++
...

At tcl85tg.dll!BogusFind(Tcl_HashTable * tablePtr=0x05d71ba0, const char
* key=0x00000440) Zeile 941 + 0xf Bytes C
I found this:

static Tcl_HashEntry *
BogusFind(
Tcl_HashTable *tablePtr, /* Table in which to lookup entry. */
const char *key) /* Key to use to find matching entry. */
{
Tcl_Panic("called %s on deleted table", "Tcl_FindHashEntry");
return NULL;
}

Any idea what's going wrong? Is this a Tcl bug or is it my mistake?

Thanks in advance
Jens

Jens Benner schrieb: