[Bug Tcl 8.1] genStubs.tcl isn't generating the xxxStub.c files
davy...@bigfoot.com
Tcl 8.1 Bug: Generated by Scriptics' bug entry form at
http://www.scriptics.com/support/bugForm.html
Responses to this post are encouraged.
------
Submitted by: David Gravereaux
CVS: RCS: @(#) $Id: genStubs.tcl,v 1.5 1999/04/30 22:45:03 stanton Exp $
OperatingSystem: Windows NT
OperatingSystemVersion: 4.0 sp5
Synopsis: genStubs.tcl isn't generating the xxxStub.c files
ReproducibleScript:
nmake -f makefile.vc genstubs
ObservedBehavior:
The emitStubs procedure in genStubs.tcl isn't being called in the
creation of the Stubs files.
DesiredBehavior:
tclStubs.c and tclIntStubs.c are modified to reflect changes from
tcl.decl and tclInt.decl
Patch:
cvs -z9 diff genStubs.tcl (in directory E:\TclCvs\tcl8.1\tools\)
Index: genStubs.tcl
===================================================================
RCS file: /cvsroot/tcl/tools/genStubs.tcl,v
retrieving revision 1.5
diff -r1.5 genStubs.tcl
860a861
> emitStubs $name
PatchFiles:
genStubs.tcl
Comments:
I'm not sure if I'm correct about this.
David Gravereaux
> ....
>Comments:
> I'm not sure if I'm correct about this.
I'm not correct! Something is amiss here. What's the purpose of
tclStub.c and tclIntStub.c? They're in the CVS, but not referenced in
makefile.vc
I'm confused with the whole logic of Stubs. Can someone enlighten me,
please.
If an extension were to modify the Stub table for an internal Tcl
function (like Tcl_AppendResult), does this *actually* have any effect
on an application that got that function from the outside as an export
with LoadLibrary and GetProcAddress? Is this the purpose of
tclStub.c?
* David Gravereaux *
Tomahawk Software Group
Paul Duffin
>
> davy...@bigfoot.com wrote:
>
> > ....
> >Comments:
> > I'm not sure if I'm correct about this.
>
> I'm not correct! Something is amiss here. What's the purpose of
> tclStub.c and tclIntStub.c? They're in the CVS, but not referenced in
> makefile.vc
>
You mean tclStubs.c and tclIntStubs.c, there was a problem with the
brain dead backlinking mechanism on Solaris which was causing all sorts
of problems when building Tk using the Stub functions (as opposed to the
macros) because of the duplicate symbols. Therefore they where removed
from the Stubs library before 8.1.0 was released.
> I'm confused with the whole logic of Stubs. Can someone enlighten me,
> please.
>
Rather than rely on the operating system dependent dynamic linking
mechanism Stubs replaces this with calls through a function table
which has many benefits.
> If an extension were to modify the Stub table for an internal Tcl
> function (like Tcl_AppendResult), does this *actually* have any effect
> on an application that got that function from the outside as an export
> with LoadLibrary and GetProcAddress? Is this the purpose of
> tclStub.c?
>
No. No. It only affects those extensions which call through the Stub
table.
--
Paul Duffin
DT/6000 Development Email: pdu...@hursley.ibm.com
IBM UK Laboratories Ltd., Hursley Park nr. Winchester
Internal: 7-246880 International: +44 1962-816880
Darren New
and what they're for? And for that matter, how one would add a new data
type (like a "tree" or something)? The Welsh book doesn't have anything
in the index or the Tcl&C chapter, and searching Scriptics and the man
pages for "stubs" doesn't come up with anything. Is the only reference
in the C code for the interpreter itself?
--
Darren New / Senior Software Architect / MessageMedia, Inc.
San Diego, CA, USA (PST). Cryptokeys on demand.
Help outlaw Hydrogen Dioxide, a major component of acid rain!
Scott Redman
has something about the original implementation.
Stubs are new to the Tcl/Tk core and were not around when Brent Welch wrote
his book (except in the Plus Patch?). The decision to add them was made
earlier this year.
What are they? They are a better method of providing dynamically loadable
libraries than the methods provided by most versions of Unix and even Windows.
They allow an extension DLL to use the Tcl commands without linking to the Tcl
DLL/shared lib directly. When the extension DLL is loaded by the Tcl interp,
the stub table (or function table) is passed from the interpreter into the
DLL/shared lib, resolving the Tcl symbols at runtime (rather than load-time).
This was the main reason for adding stubs.
Tcl and Tk both provide stub tables, and it is possible for an extension
to do the same (for C-level APIs).
Adding a new data type is not any easier with stubs. Providing better
support is another issue altogether. It can be done, does anyone else
have a good example/documentation of how to do this?
-- Scott
Scott Redman
(that use the static-lib versions of Tcl and Tk, not the dynamic ones)
can now dynamically load stub-enabled extensions.
-- Scott
David Gravereaux
>> If an extension were to modify the Stub table for an internal Tcl
>> function (like Tcl_AppendResult), does this *actually* have any effect
>> on an application that got that function from the outside as an export
>> with LoadLibrary and GetProcAddress? Is this the purpose of
>> tclStub.c?
>>
>
>No. No. It only affects those extensions which call through the Stub
>table.
Yes, in the current mechanism. I understand how it isn't now.
But what if you had used back-linking with tclStubs.c? Would this
have given extensions the ability to manipulate the internal workings
of Tcl and also effect other extension that had implicitly linked to
Tcl as well as one that used GetProcAddress?
David Gravereaux
>Can anyone provide a pointer to a description of "stubs", how they work
>and what they're for? And for that matter, how one would add a new data
>type (like a "tree" or something)? The Welsh book doesn't have anything
>in the index or the Tcl&C chapter, and searching Scriptics and the man
>pages for "stubs" doesn't come up with anything. Is the only reference
>in the C code for the interpreter itself?
http://www.scriptics.com/support/howto/stubs.html
That page and also the TEA document, as well. The TEA document refers
to Tcl_Require() (or something like it) which was changed later to
Tcl_InitStubs(). Also notice the new Tcl_PkgRequireEx() and
Tcl_PkgProvideEx(). These are used to transfer Stub tables between
extensions.
At least, I think so ;) I'm still confused using it.
Paul Duffin
That would probably be platform dependent as it depends on the
backlinking mechanism. On Solaris it probably would allow that
behaviour.
As Stubs is designed to remove the platform dependencies this is
not a good thing to try and do. However with some minor changes to the
core this behaviour would be possible. By minor I mean.
Renaming all exported functions
Storing the addresses of the renamed functions in the Stub table
Linking with the Stub functions
Alexandre Ferrieux
>
> I forgot to mention that stubs also allows statically built shells
> (that use the static-lib versions of Tcl and Tk, not the dynamic ones)
> can now dynamically load stub-enabled extensions.
Yes, and this is an extremely important point:
It allows you to ProWrap in static mode (i.e. the generated EXE will not
need any external tcl80/tk80.DLL) an app that also loads some
third-party DLLs. This is invaluable if you don't have the time,
resource, or even option to statically link the third-party ext against
prowish for instance.
Another note: in this scheme, if you further want the third-party DLL
(or an EXE) itself to be also wrapped, you will need one additional step
because it is impossible at OS level to dlopen()/exec() (unix
terminology - I don't know the win equivalent...) something that is not
a standalone file. But it is easy to make it so: in your Tclscript, open
the wrapped DLL or EXE file, and use [puts -nonewline] to copy them as
standalone files somewhere on your disk. Then tinker with the relevant
Tcl globals to alter the various search paths, and off you go.
I guess Scriptics will provide a standard way of doing this as soon as a
Stub-enabled version of TclPro shows up. (of course the DLL too must be
stub-enabled). But in the meantime, you can still use this trick with a
stubless TclPro and a .EXE.
-Alex
Alexandre Ferrieux
>
> Paul Duffin <pdu...@mailserver.hursley.ibm.com> wrote:
>
> >> If an extension were to modify the Stub table for an internal Tcl
> >> function (like Tcl_AppendResult), does this *actually* have any effect
> >> on an application that got that function from the outside as an export
> >> with LoadLibrary and GetProcAddress? Is this the purpose of
> >> tclStub.c?
> >>
> >
> >No. No. It only affects those extensions which call through the Stub
> >table.
>
> Yes, in the current mechanism. I understand how it isn't now.
>
> But what if you had used back-linking with tclStubs.c? Would this
> have given extensions the ability to manipulate the internal workings
> of Tcl and also effect other extension that had implicitly linked to
> Tcl as well as one that used GetProcAddress?
The data structure maintained by the native loaders (i.e. the one
queried by GetProcAddress()/dlsym()) is AFAIK not easily writable by the
programmer, unless you want to reverse-engineer every single OS version.
Hence, I can see no feasible way to affect the result returned to
another extension by GetProcAddress.
The whole point of Stubs is to bring back this data structure in open
light, so that we can manipulate it at will, and regardless of the OS.
Of course, this means that to take advantage of it the other extension
must go through it explicitly (i.e. be stubs-aware).
Another point that is implicit in your questions is this: should we
allow/forbid extension A to change the 'generic Tcl' viewed by extension
B ? I believe yes. Of course, it means that Tcl itself must go through
the stubs table to call itself at some points (not everywhere for perf
reasons). But this yields total flexibility among well-behaved citizens.
This has happened for years in the MacOS with SetTrapAddress, which is
roughly a public API to a writable system Stub. Thanks to this the
Macintosh OS has evolved by progressively glueing extensions that were
initially 'stacked' (i.e. patched on top of one another). You may say,
yes but the MacOS has become such an horrible mess. Okay but it is
unrelated I believe; likewise, even without ext A being allowed to draw
the mat under B's feet, ext A alone can already crash the whole process
today !
-Alex
Chang LI
>
This scheme maybe useful as an install program.
> Another note: in this scheme, if you further want the third-party DLL
> (or an EXE) itself to be also wrapped, you will need one additional step
> because it is impossible at OS level to dlopen()/exec() (unix
> terminology - I don't know the win equivalent...) something that is not
> a standalone file. But it is easy to make it so: in your Tclscript, open
> the wrapped DLL or EXE file, and use [puts -nonewline] to copy them as
> standalone files somewhere on your disk. Then tinker with the relevant
> Tcl globals to alter the various search paths, and off you go.
>
> I guess Scriptics will provide a standard way of doing this as soon as a
> Stub-enabled version of TclPro shows up. (of course the DLL too must be
> stub-enabled). But in the meantime, you can still use this trick with a
> stubless TclPro and a .EXE.
>
> -Alex
--
--------------------------------------------------------------
Chang LI, Neatware
email: cha...@neatware.com
web: http://www.neatware.com
--------------------------------------------------------------
Scott Redman
wrapped EXE. Then, at runtime, opening the wrapped
"zip" area, pulling out the DLL, and copying it to
some tmp directory. Then using load to open it.
Putting an EXE (even another wrapped EXE) should
also be possible.
This could be done beginning with TclPro 1.3b1. There
was a bug in the wrapper code that truncated files of
exact multiples of 4k, which I fixed just before b1.
-- Scott
Alexandre Ferrieux wrote:
>
> Scott Redman wrote:
> >
> > I forgot to mention that stubs also allows statically built shells
> > (that use the static-lib versions of Tcl and Tk, not the dynamic ones)
> > can now dynamically load stub-enabled extensions.
>
> Yes, and this is an extremely important point:
> It allows you to ProWrap in static mode (i.e. the generated EXE will not
> need any external tcl80/tk80.DLL) an app that also loads some
> third-party DLLs. This is invaluable if you don't have the time,
> resource, or even option to statically link the third-party ext against
> prowish for instance.
>
Scott Redman
You could use the TclPro wrapper to package an installer
and all of the files to be installed into a single, self-
extracting, platform-dependent binary executable.
The TclPro installer is wrapped with the TclPro wrapper
(but the compressed files to be installed are currently
separate...for now).
-- Scott
Chang LI wrote:
>
> Alexandre Ferrieux wrote:
> >
>
> This scheme maybe useful as an install program.
>
> > Another note: in this scheme, if you further want the third-party DLL
> > (or an EXE) itself to be also wrapped, you will need one additional step
> > because it is impossible at OS level to dlopen()/exec() (unix
> > terminology - I don't know the win equivalent...) something that is not
> > a standalone file. But it is easy to make it so: in your Tclscript, open
> > the wrapped DLL or EXE file, and use [puts -nonewline] to copy them as
> > standalone files somewhere on your disk. Then tinker with the relevant
> > Tcl globals to alter the various search paths, and off you go.
> >
> > I guess Scriptics will provide a standard way of doing this as soon as a
> > Stub-enabled version of TclPro shows up. (of course the DLL too must be
> > stub-enabled). But in the meantime, you can still use this trick with a
> > stubless TclPro and a .EXE.
> >
> > -Alex
>
Scott Redman
TclPro Wrapper with 1.3. But, we use it internally
to build the other pieces of TclPro.
-- Scott
David Gravereaux
>David Gravereaux wrote:
>> Paul Duffin <pdu...@mailserver.hursley.ibm.com> wrote:
>>
>> >> If an extension were to modify the Stub table for an internal Tcl
>> >> function (like Tcl_AppendResult), does this *actually* have any effect
>> >> on an application that got that function from the outside as an export
>> >> with LoadLibrary and GetProcAddress? Is this the purpose of
>> >> tclStub.c?
>> >>
>> >
>> >No. No. It only affects those extensions which call through the Stub
>> >table.
>>
>> Yes, in the current mechanism. I understand how it isn't now.
>>
>> But what if you had used back-linking with tclStubs.c? Would this
>> have given extensions the ability to manipulate the internal workings
>> of Tcl and also effect other extension that had implicitly linked to
>> Tcl as well as one that used GetProcAddress?
>
>The data structure maintained by the native loaders (i.e. the one
>queried by GetProcAddress()/dlsym()) is AFAIK not easily writable by the
>programmer, unless you want to reverse-engineer every single OS version.
>
>Hence, I can see no feasible way to affect the result returned to
>another extension by GetProcAddress.
Here's what I see in tclStubs.c:
/* Slot 70 */
void
Tcl_AppendResult TCL_VARARGS_DEF(Tcl_Interp *,interp)
{
Tcl_Interp * var;
va_list argList;
var = (Tcl_Interp *) TCL_VARARGS_START(Tcl_Interp *,interp,argList);
(tclStubsPtr->tcl_AppendResultVA)(var, argList);
va_end(argList);
}
If that definition of Tcl_AppendResult WAS the export, then the
innards of Tcl are modifiable by changing the Stubs table. It's
this that got me confused and why I posted that bug report (which
isn't a bug). And compiling the core with -DUSE_TCL_STUBS will
reroute the innards through the Stub table.
As Paul Duffin says:
> However with some minor changes to the
> core this behaviour would be possible. By minor I mean.
>
> Renaming all exported functions
the real core functions get a new name like: real_Tcl_AppendResult
> Storing the addresses of the renamed functions in the Stub table
> Linking with the Stub functions
the functions in tclStubs.c ---^
About allowing the ability for extensions to change the core, I can't
really comment on that. It's a dangerous topic. I see your point
about the MacOS. I would like the flexibility, though.
Jean-Claude Wippler
> You could use the TclPro wrapper to package an installer
> and all of the files to be installed into a single, self-
> extracting, platform-dependent binary executable.
[Alexandre Ferrieux]
> > > Another note: in this scheme, if you further want the third-party DLL
> > > (or an EXE) itself to be also wrapped, you will need one additional step
> > > because it is impossible at OS level to dlopen()/exec() (unix
> > > terminology - I don't know the win equivalent...) something that is not
> > > a standalone file. But it is easy to make it so: in your Tclscript, open
> > > the wrapped DLL or EXE file, and use [puts -nonewline] to copy them as
> > > standalone files somewhere on your disk. Then tinker with the relevant
> > > Tcl globals to alter the various search paths, and off you go.
> > >
> > > I guess Scriptics will provide a standard way of doing this as soon as a
> > > Stub-enabled version of TclPro shows up. (of course the DLL too must be
> > > stub-enabled). But in the meantime, you can still use this trick with a
> > > stubless TclPro and a .EXE.
Ok, let me jump the gun by about a week and mention that I'm working on
another packaging structure, which splits everything into a generic
platform-specific Tcl "runtime", which I'm calling Tclkit, and a
platform-independent "scripted document", containing everything that
constitutes an application - and more :)
It's all being set up at the following URL, and will be officially
announced by May 31st:
http://www.equi4.com/tclkit/
One of the values of stub-enabled standalone executables, is that it
will allow precisely the approach described above. TclKit is already
stub-enabled, but since this was done before the "official" TEA summit,
it will be redone to match the definitive standard.
Right now, TclKit is work-in-progress, in a week or so, I hope to have
more information ready. Please feel free to comment, suggest, and
criticise. I'll be out for the next three days, but am very interested
in what people think of this new approach.
-- Jean-Claude
Cameron Laird
Jean-Claude Wippler <j...@equi4.com> wrote:
.
.
.
>Ok, let me jump the gun by about a week and mention that I'm working on
>another packaging structure, which splits everything into a generic
>platform-specific Tcl "runtime", which I'm calling Tclkit, and a
>platform-independent "scripted document", containing everything that
>constitutes an application - and more :)
>
>It's all being set up at the following URL, and will be officially
>announced by May 31st:
> http://www.equi4.com/tclkit/
.
.
>criticise. I'll be out for the next three days, but am very interested
>in what people think of this new approach.
>
>-- Jean-Claude
I like it, as <URL:http://
www.sunworld.com/swol-05-1999/swol-05-regex.html#2>
hints. In fact, I'm a TclKit customer and user.
Think about Oscar Wilde (apparently he several times
emitted some variant of "only superficial things are
deep"). Jean-Claude's description here of TclKit is
limited to its "packaging" aspect. All he's talking
about, in some sense, is slashing the file count of
an installation. That's not a profound problem, in
the sense that cached method dispatching or assertion
syntaxes or compression algorithms or real-time gar-
bage collection or SQL optimization are.
However, it's ENORMOUSLY rewarding for us working
stiffs trying to satisfy customers. I haven't yet
come up with a way to describe the benefits that
satisfies me. Start with this, though: 'you know
how bad installation is, how you're always wasting
time on it in every project? Suppose that TclKit
halves your time just on installation (and it
will)--would that interest you?
--
Cameron Laird http://starbase.neosoft.com/~claird/home.html
cla...@NeoSoft.com +1 281 996 8546 FAX