info
Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
TCL 8.6.8 does not work with Tcl_SetCommandInfo on some commands
83 views
Skip to first unread message
Sebastian Biały
Nov 9, 2018, 3:34:14 AM11/9/18
to
Hello.
I'm trying to do something non-standard with TCL 8.6.8 that was working
just fine with 8.5.12.
Actually i want to rewrite proc/objProc for *every* command in every
namespace. Since tracing seems to be slower, this solution was used in
production code without issues. Basically I'm using Tcl_SetCommandInfo
with proc/objProc pointing to my functions that do proxy call to
original functions.
Bu it was broken in 8.6.8. Some functions, that are (I'm guessing now)
native byte code implemented, does not work at all as expected. Instead
of calling my proc/objProc I'm getting original behavior of command, my
functions are not called at all. Also, touching objClientData field
generates assertion (broken byte code).
Since this seems fatal for me, and I cannot find any sane solution,
there is minimal example that generates assertion in byte code interpreter:
#include "tcl.h"
#include <string>
#include <iostream>
#include <assert.h>
int
objProxy( ClientData clientData, Tcl_Interp *interp, int objc, struct
Tcl_Obj *const *objv )
{
assert( !"not executed at all" );
return TCL_OK;
}
int
proxy( ClientData clientData, Tcl_Interp *interp, int argc, CONST84 char
*argv[] )
{
assert( !"not executed at all" );
return TCL_OK;
}
void
doProxy( std::string const& _name, Tcl_Interp* _interp )
{
Tcl_CmdInfo commandInfo;
int a;
commandInfo.isNativeObjectProc = 0;
commandInfo.objClientData = &a; /// <---- important to trigger assertion
commandInfo.objProc = objProxy;
commandInfo.clientData = nullptr;
commandInfo.proc = proxy;
commandInfo.deleteData = nullptr;
commandInfo.deleteProc = nullptr;
Tcl_SetCommandInfo( _interp, _name.c_str(), &commandInfo );
}
int main()
{
std::string command( "::tcl::clock::seconds" );
auto interp = Tcl_CreateInterp();
doProxy( command, interp );
Tcl_RecordAndEval( interp, command.c_str(), TCL_EVAL_GLOBAL );
std::cout << Tcl_GetStringResult( interp ) << std::endl;
Tcl_DeleteInterp( interp );
return 0;
}
Example exits with assertion:
"clockRead instruction with unknown clock#", debugging tcl internals
swhows garbage instead of proper byte code.
In case commandInfo.objClientData is nullptr original command works just
fine, my functions are not called at all.
It works with ::tcl::clock::seconds in mysterious way. Other "normal"
commands works fine, like "puts" and many others. I'm not quite sure how
many other commands behave this way, but seems that majority of commands
just working fine with above code (except it is missing calling original
proc).
Now it is time for questions:
a) is it any other way to rewrite such commands?
b) Am I doing something illegal here?
PS. No, changing between TCL_EVAL_GLOBAL/TCL_EVAL_DIRECT does not work.
Also Tcl_RecordAndEval is critical, using standard eval works just fine
(but I expect this is just luck with memory garbage). I'm using VS 2015
with TCL compiled from sources.
I'm trying to do something non-standard with TCL 8.6.8 that was working
just fine with 8.5.12.
Actually i want to rewrite proc/objProc for *every* command in every
namespace. Since tracing seems to be slower, this solution was used in
production code without issues. Basically I'm using Tcl_SetCommandInfo
with proc/objProc pointing to my functions that do proxy call to
original functions.
Bu it was broken in 8.6.8. Some functions, that are (I'm guessing now)
native byte code implemented, does not work at all as expected. Instead
of calling my proc/objProc I'm getting original behavior of command, my
functions are not called at all. Also, touching objClientData field
generates assertion (broken byte code).
Since this seems fatal for me, and I cannot find any sane solution,
there is minimal example that generates assertion in byte code interpreter:
#include "tcl.h"
#include <string>
#include <iostream>
#include <assert.h>
int
objProxy( ClientData clientData, Tcl_Interp *interp, int objc, struct
Tcl_Obj *const *objv )
{
assert( !"not executed at all" );
return TCL_OK;
}
int
proxy( ClientData clientData, Tcl_Interp *interp, int argc, CONST84 char
*argv[] )
{
assert( !"not executed at all" );
return TCL_OK;
}
void
doProxy( std::string const& _name, Tcl_Interp* _interp )
{
Tcl_CmdInfo commandInfo;
int a;
commandInfo.isNativeObjectProc = 0;
commandInfo.objClientData = &a; /// <---- important to trigger assertion
commandInfo.objProc = objProxy;
commandInfo.clientData = nullptr;
commandInfo.proc = proxy;
commandInfo.deleteData = nullptr;
commandInfo.deleteProc = nullptr;
Tcl_SetCommandInfo( _interp, _name.c_str(), &commandInfo );
}
int main()
{
std::string command( "::tcl::clock::seconds" );
auto interp = Tcl_CreateInterp();
doProxy( command, interp );
Tcl_RecordAndEval( interp, command.c_str(), TCL_EVAL_GLOBAL );
std::cout << Tcl_GetStringResult( interp ) << std::endl;
Tcl_DeleteInterp( interp );
return 0;
}
Example exits with assertion:
"clockRead instruction with unknown clock#", debugging tcl internals
swhows garbage instead of proper byte code.
In case commandInfo.objClientData is nullptr original command works just
fine, my functions are not called at all.
It works with ::tcl::clock::seconds in mysterious way. Other "normal"
commands works fine, like "puts" and many others. I'm not quite sure how
many other commands behave this way, but seems that majority of commands
just working fine with above code (except it is missing calling original
proc).
Now it is time for questions:
a) is it any other way to rewrite such commands?
b) Am I doing something illegal here?
PS. No, changing between TCL_EVAL_GLOBAL/TCL_EVAL_DIRECT does not work.
Also Tcl_RecordAndEval is critical, using standard eval works just fine
(but I expect this is just luck with memory garbage). I'm using VS 2015
with TCL compiled from sources.
Christian Gollwitzer
Nov 9, 2018, 3:54:51 AM11/9/18
to
Am 09.11.18 um 09:34 schrieb Sebastian Biały:
coded commands, so maybe you could replicate this behaviour from C?
i.e Tcl_RenameObjCmd, which also handles all te nasty stuff like
modifying the compilation epoch counter etc. Unfortunately that requires
tclInt.h, but IMHO much better than messing with the internals of the
Tcl_Command structure.
Another option to consider, how about a safe interpreter? Maybe it can
redirect the execution to your liking? Are you writing a debugger,
profiler or trying to restrict access?
Christian
> Hello.
>
> I'm trying to do something non-standard with TCL 8.6.8 that was working
> just fine with 8.5.12.
>
> Actually i want to rewrite proc/objProc for *every* command in every
> namespace. Since tracing seems to be slower, this solution was used in
> production code without issues. Basically I'm using Tcl_SetCommandInfo
> with proc/objProc pointing to my functions that do proxy call to
> original functions.
Overriding a command from within Tcl using rename works also with byte
>
> I'm trying to do something non-standard with TCL 8.6.8 that was working
> just fine with 8.5.12.
>
> Actually i want to rewrite proc/objProc for *every* command in every
> namespace. Since tracing seems to be slower, this solution was used in
> production code without issues. Basically I'm using Tcl_SetCommandInfo
> with proc/objProc pointing to my functions that do proxy call to
> original functions.
coded commands, so maybe you could replicate this behaviour from C?
i.e Tcl_RenameObjCmd, which also handles all te nasty stuff like
modifying the compilation epoch counter etc. Unfortunately that requires
tclInt.h, but IMHO much better than messing with the internals of the
Tcl_Command structure.
Another option to consider, how about a safe interpreter? Maybe it can
redirect the execution to your liking? Are you writing a debugger,
profiler or trying to restrict access?
Christian
Sebastian Biały
Nov 9, 2018, 4:05:29 AM11/9/18
to
On 2018-11-09 09:54, Christian Gollwitzer wrote:
> Overriding a command from within Tcl using rename works also with byte
> coded commands, so maybe you could replicate this behaviour from C?
Will try. I'm not sure however about performance of this solution.
> Overriding a command from within Tcl using rename works also with byte
> coded commands, so maybe you could replicate this behaviour from C?
> Another option to consider, how about a safe interpreter?
talking about using interpreter instead of compiled one. Maybe, however
speed is critical, since tracing was slower than proxying I'm guessing
interpreting will be even slower.
> Are you writing a debugger,
> profiler or trying to restrict access?
this control also needs to be optimized we wish to know what is going on
in TCL to be able to optimize some of our behavior while script is
executed. Basically I need to know that certain command will be executed
before it is. It is quite critical for us.
We need all speed we can get from TCL. That is why I'm sceptic about
interpreter solution.
s.effe...@googlemail.com
Nov 9, 2018, 4:15:13 AM11/9/18
to
In doProxy you're pointing objClientData to a variable "a" on the stack, this address is no longer valid when you call Tcl_RecordAndEval. I assume that Tcl_SetCommandInfo doesn't work with the value, it simply remembers it. Within the call of Tcl_RecordAndEval the invalid address is accessed and the stack is corrupted (if it was a write access).
Sebastian Biały
Nov 9, 2018, 4:34:35 AM11/9/18
to
On 2018-11-09 10:15, s.effe...@googlemail.com wrote:
> In doProxy you're pointing objClientData to a variable "a" on the stack, this address is no longer valid when you call Tcl_RecordAndEval.
Does not really matter. If I change it to new int (and initialize it)
> In doProxy you're pointing objClientData to a variable "a" on the stack, this address is no longer valid when you call Tcl_RecordAndEval.
and put proper pointer to heap allocated object it works same way as
before. This is just minimal example, actually this code in production
is thousands of lines long and there is no missing pointers or not
initialized memory, as checked with valgrind and other tools.
I expect that clientData holds some internal data for this command and
does not want me to change it in any way, however setting it to null
works (!). Could not find such information in documentation, that is why
I'm here asking.
Sebastian Biały
Nov 9, 2018, 4:54:36 AM11/9/18
to
On 2018-11-09 09:34, Sebastian Biały wrote:
> Bu it was broken in 8.6.8. Some functions, that are (I'm guessing now)
> native byte code implemented, does not work at all as expected.
One more thing: It could be solution to detect such functions and
> Bu it was broken in 8.6.8. Some functions, that are (I'm guessing now)
> native byte code implemented, does not work at all as expected.
exclude them from proxying. As far as I could check it they only exists
in ensembles. But I have no way to prove it so I need some kind of test
that some command is not possible to be used with such proxying idea due
to such unexpected behavior.
Christian Gollwitzer
Nov 9, 2018, 5:16:54 AM11/9/18
to
Am 09.11.18 um 10:05 schrieb Sebastian Biały:
This was built into Tcl at the time when there was a Browserplugin in
order to have control, so that the plugin can be restricted e.g. for
access to the file system etc.
> I'm guessing that You
> talking about using interpreter instead of compiled one.
No, that has nothing to do with it. I'm not sure that it will solve your
problem, but it sounds similar and maybe you can use this mechanism.
Christian
> On 2018-11-09 09:54, Christian Gollwitzer wrote:
>> Overriding a command from within Tcl using rename works also with byte
>> coded commands, so maybe you could replicate this behaviour from C?
>
> Will try. I'm not sure however about performance of this solution.
>
>> Another option to consider, how about a safe interpreter?
>
> I'm not quite sure what is "safe intepretter",
See here: https://www.tcl.tk/man/tcl8.6/TclCmd/safe.htm
>> Overriding a command from within Tcl using rename works also with byte
>> coded commands, so maybe you could replicate this behaviour from C?
>
> Will try. I'm not sure however about performance of this solution.
>
>> Another option to consider, how about a safe interpreter?
>
> I'm not quite sure what is "safe intepretter",
This was built into Tcl at the time when there was a Browserplugin in
order to have control, so that the plugin can be restricted e.g. for
access to the file system etc.
> I'm guessing that You
> talking about using interpreter instead of compiled one.
problem, but it sounds similar and maybe you can use this mechanism.
Christian
Ralf Fassel
Nov 9, 2018, 5:24:51 AM11/9/18
to
* Sebastian Biały <he...@poczta.onet.pl>
typedef struct Tcl_CmdInfo {
int isNativeObjectProc;
Tcl_ObjCmdProc *objProc;
ClientData objClientData;
Tcl_CmdProc *proc;
ClientData clientData;
Tcl_CmdDeleteProc *deleteProc;
ClientData deleteData;
>>> Tcl_Namespace *namespacePtr;
} Tcl_CmdInfo;
which I would expect needs to get initialized, but even calling
assert (Tcl_GetCommandInfo( _interp, _name.c_str(), &commandInfo ));
at the start of doProxy() to initialize everything before resetting some
fields still asserts "clockRead instruction with unknown clock#"...
R'
| void
| doProxy( std::string const& _name, Tcl_Interp* _interp )
| {
| Tcl_CmdInfo commandInfo;
| int a;
| commandInfo.isNativeObjectProc = 0;
| commandInfo.objClientData = &a; /// <---- important to trigger assertion
| commandInfo.objProc = objProxy;
| commandInfo.clientData = nullptr;
| commandInfo.proc = proxy;
| commandInfo.deleteData = nullptr;
| commandInfo.deleteProc = nullptr;
>
| Tcl_SetCommandInfo( _interp, _name.c_str(), &commandInfo );
| }
The Tcl_CmdInfo has an additional field 'namespacePtr' in my documentation
| doProxy( std::string const& _name, Tcl_Interp* _interp )
| {
| Tcl_CmdInfo commandInfo;
| int a;
| commandInfo.isNativeObjectProc = 0;
| commandInfo.objClientData = &a; /// <---- important to trigger assertion
| commandInfo.objProc = objProxy;
| commandInfo.clientData = nullptr;
| commandInfo.proc = proxy;
| commandInfo.deleteData = nullptr;
| commandInfo.deleteProc = nullptr;
>
| Tcl_SetCommandInfo( _interp, _name.c_str(), &commandInfo );
| }
typedef struct Tcl_CmdInfo {
int isNativeObjectProc;
Tcl_ObjCmdProc *objProc;
ClientData objClientData;
Tcl_CmdProc *proc;
ClientData clientData;
Tcl_CmdDeleteProc *deleteProc;
ClientData deleteData;
>>> Tcl_Namespace *namespacePtr;
} Tcl_CmdInfo;
which I would expect needs to get initialized, but even calling
assert (Tcl_GetCommandInfo( _interp, _name.c_str(), &commandInfo ));
at the start of doProxy() to initialize everything before resetting some
fields still asserts "clockRead instruction with unknown clock#"...
R'
Sebastian Biały
Nov 9, 2018, 5:35:10 AM11/9/18
to
On 2018-11-09 11:24, Ralf Fassel wrote:
> The Tcl_CmdInfo has an additional field 'namespacePtr' in my documentation
Already tried this, just forgot to put it in this tiny example.
> The Tcl_CmdInfo has an additional field 'namespacePtr' in my documentation
> which I would expect needs to get initialized, but even calling
> assert (Tcl_GetCommandInfo( _interp, _name.c_str(),&commandInfo ));
Any way If I do perfect copy of Tcl_GetCommandInfo it still hits assert
if I touch objClientData field. This looks like either this structure is
fake for some commands or it is cast-ed into something different before
use and I'm just mangling with unknown fields.
Sebastian Biały
Nov 9, 2018, 5:36:16 AM11/9/18
to
On 2018-11-09 11:16, Christian Gollwitzer wrote:
> See here: https://www.tcl.tk/man/tcl8.6/TclCmd/safe.htm
Thanks. Will look into it.
> See here: https://www.tcl.tk/man/tcl8.6/TclCmd/safe.htm
Thanks. Will look into it.
Don Porter
Nov 13, 2018, 9:36:22 AM11/13/18
to
On 11/09/2018 03:34 AM, Sebastian Biały wrote:
> I'm trying to do something non-standard with TCL 8.6.8 that was working
> just fine with 8.5.12.
Please open a ticket.
> I'm trying to do something non-standard with TCL 8.6.8 that was working
> just fine with 8.5.12.
https://core.tcl-lang.org/tcl/tktnew
--
| Don Porter Applied and Computational Mathematics Division |
| donald...@nist.gov Information Technology Laboratory |
| http://math.nist.gov/~DPorter/ NIST |
|______________________________________________________________________|
Sebastian Biały
Nov 19, 2018, 7:07:35 AM11/19/18
to
On 2018-11-13 15:36, Don Porter wrote:
> Please open a ticket.
> https://core.tcl-lang.org/tcl/tktnew
Done:
> Please open a ticket.
> https://core.tcl-lang.org/tcl/tktnew
https://core.tcl-lang.org/tcl/tktview/9bdd5788e370acc24ddc8a57e5291832aada8e6b
0 new messages