Extension Installation and Tcl Bug
Chang LI
I think it is an old topic. After checking several extension
installation I found there is no common standard. TEA is
supposed to have a standard but I do not know how it is doing.
There is no hint they are doing right or not.
Following is my suggestion and a patch for this purpose.
Acturally the solution is quite simple. All the extensions are
suppose to install in the lib directory that Tcl located. I use
BLT and Img as samples, both of them have the excellent installation
with different configurations. I suggested following configuration:
Tcl81
bin
lib
blt
dd_protocols
...
demo
...
help
...
2.4
blt24.dll
pkgIndex.tcl
...
2.5
blt25.dll
pkgIndex.tcl
...
img
demo
...
help
...
1.3
img13.dll
pkgIndex.tcl
...
1.4
img14.dll
pkgIndex.tcl
...
The directory structure is lib\extension\version. In this
structure uninstall an extension is much ease. It also allows
multi-version extensions to coexist. Any comments?
One problem is a Tcl bug in the init.tcl. It can not automatically
search more than one level of subdirectory in the lib. So I attached
the following patch to the init.tcl:
The init.tcl is located in the lib\tcl8.1, add the proc in the header,
#
# this is a recursive program to make automatically search
# subdirectory possible.
#
proc AppendFolder {start} {
set cur [pwd]
if {[catch {cd $start} err]} {
puts stderr $err
return
}
foreach f [glob -nocomplain *] {
if [file isdirectory $f] {
set d [file join $start $f]
lappend ::auto_path $d
AppendFolder $d
}
}
cd $cur
}
...
# add following command here
AppendFolder [file dirname [file join [pwd] [info library]]]
# Setup the unknown package handler
package unknown tclPkgUnknown
Best Regards,
--------------------------------------------------------------
Chang LI, Neatware
email: cha...@neatware.com
web: http://www.neatware.com
--------------------------------------------------------------
David Gravereaux
Wouldn't it be possible to merge the separate pkgIndex.tcl
files into one? Like this:
# Tcl package index file, version 1.0
package ifneeded Itcl 3.0 \
"[list load [file join $dir itcl30.dll] Itcl] \; namespace import ::itcl::*"
package ifneeded Itcl 3.1 [subst {
load "[file join $dir itcl31.dll]" Itcl
namespace import ::itcl::*
auto_mkindex_parser::slavehook { _%@namespace import -force ::itcl::* }
}]
That way different version DLLs (with a different filename of course)
could reside in the same Tcl/lib/<pkg> directory. Any consensus on an
accepted practice this would surely be an improvement.
Speaking about improvements. I came across this beauty of a catch22 a
number of months ago. You know how custom shells (itclsh31 for example)
are linked to Tcl's import library as a load time dependency? This
makes it necessary to have the shell in the Tcl/bin directory. This in
turn affects how the <pkg>StubLib.c is linked. itk31.dll needs
Tcl_PkgRequire() as a Tcl Stub link, but the itkwish31 shell needs
Tcl_PkgRequire() as an import link.
Nice circular dependency.
I hope that gets resolved one day, and tclsh.exe uses Stubs instead of
an implicit run-time link through an import library. To do this we need
a cross-platform shared library loader so we can grab Tcl_CreateInterp
from the outside to then call Tcl_InitStubs.
I do this in my projects, but I just do Win32. I think Jan Nijtmans
requested his dlopen routines be added to the core. I haven't followed
up on this to see if they did.
TTYL
* David Gravereaux *
Tomahawk Software Group
Chang LI
>
> Nice idea.
>
> Wouldn't it be possible to merge the separate pkgIndex.tcl
> files into one? Like this:
>
> # Tcl package index file, version 1.0
>
> package ifneeded Itcl 3.0 \
> "[list load [file join $dir itcl30.dll] Itcl] \; namespace import ::itcl::*"
>
> package ifneeded Itcl 3.1 [subst {
> load "[file join $dir itcl31.dll]" Itcl
> namespace import ::itcl::*
> auto_mkindex_parser::slavehook { _%@namespace import -force ::itcl::* }
> }]
>
> That way different version DLLs (with a different filename of course)
> could reside in the same Tcl/lib/<pkg> directory. Any consensus on an
> accepted practice this would surely be an improvement.
>
The reasons to separate pkgIndex file are upgrade and installation.
There are many bugs to generate the right pkgIndex file. So you may need
to edit the pkgIndex manually. When you try to uninstall an old version
of extension, independent pkgIndex file is easy to be deleted. When the
pkgIndex file is mergered you have to identify which part you need to
delete.
> Speaking about improvements. I came across this beauty of a catch22 a
> number of months ago. You know how custom shells (itclsh31 for example)
> are linked to Tcl's import library as a load time dependency? This
> makes it necessary to have the shell in the Tcl/bin directory. This in
> turn affects how the <pkg>StubLib.c is linked. itk31.dll needs
> Tcl_PkgRequire() as a Tcl Stub link, but the itkwish31 shell needs
> Tcl_PkgRequire() as an import link.
>
> Nice circular dependency.
>
I believe any new shell program like itclsh31 should be in the bin
directory.
> I hope that gets resolved one day, and tclsh.exe uses Stubs instead of
> an implicit run-time link through an import library. To do this we need
> a cross-platform shared library loader so we can grab Tcl_CreateInterp
> from the outside to then call Tcl_InitStubs.
>
tclsh.exe is a special starting program.
> I do this in my projects, but I just do Win32. I think Jan Nijtmans
> requested his dlopen routines be added to the core. I haven't followed
> up on this to see if they did.
>
> TTYL
> * David Gravereaux *
> Tomahawk Software Group
--
David Gravereaux
>The reasons to separate pkgIndex file are upgrade and installation.
>There are many bugs to generate the right pkgIndex file. So you may need
>to edit the pkgIndex manually. When you try to uninstall an old version
>of extension, independent pkgIndex file is easy to be deleted. When the
>pkgIndex file is mergered you have to identify which part you need to
>delete.
How about a single pkgIndex.tcl at the root that has a list of others
to load:
source [file join [file dirname [info script]] 2.4/pkgIndex.tcl]
source [file join [file dirname [info script]] 2.5/pkgIndex.tcl]
But that's messy, too. The uninstall script has to find and remove
it. How about this as the pkgIndex.tcl:
foreach subdir [glob [file dirname [info script]]/*] {
if {[file isdirectory $subdir] && \
[file exist [file join $subdir pkgIndex.tcl]]} {
source [file join $subdir pkgIndex.tcl]
}
}
if you remove 2.4 under blt it won't get sourced by the top level
pkgIndex.tcl
>I believe any new shell program like itclsh31 should be in the bin
>directory.
Why? This'll make a mess of the bin directory with a large amount of
shells. Just like the bin directory in TclPro. I think the
subdirectory under Lib is where everything related to an extension
should go. But that's just my belief.
What about Tcl applications that aren't shells? Ones that
LoadLibrary("tcl82.dll"). They need special care for using Stubs.
Exactly that method I described for resolving Tcl_CreateInterp first.
Not using Stubs and going implicit run-time linking, you lose all the
wonderful benefits like DLL name independence, notifier replacement at
run-time, and future features to be added later.
>tclsh.exe is a special starting program.
What's so special about it? Ok, so its level 0 of a multi-teered
relationship of extension dependencies. But tclAppInit.c is the
backbone for extending a Tcl shell for your project (whatever that may
be).
As soon as you get to level 2, you have a circular loop with level 1's
<pkg>StubLib.c. Where the extension that was built to the
<pkg>StubLib.c requires a -DUSE_TCL_STUBS compile, yet the level 2
shell that uses the lib needs a import link to Tcl instead. Causing
a circular dependency around Tcl_PkgRequire() in level 1's
<pkg>StubLib.c.
example:
1) itcl makes a itclstubs31.lib from itclStubLib.c w/ -DUSE_TCL_STUBS.
2) itk uses this for itk31.dll.
3) itkwish31.exe is implicitly linked for an import of tcl8x.dll.
Just like the tclAppInit.c template shows us.
4) itkwish31.exe now cannot link because tclStubPtr (referenced in
itclstubs31.lib) is missing and can't resolve what Tcl_PkgRequire
is replaced with in the Stubs macro for it.
Ok, just screw a Stubs table for itcl and go import lib in the
itkwish31.exe shell. NO GOOD. The dll is in lib/itcl31 and the shell
is in bin/. Crash on startup (itcl31.dll not found in path...)
Ok, move the dll into the bin then. NO GOOD. That's messy. Just
like TclPro.
Ok, let's add tclstubs82.lib to the link line and also build
itkwish31.exe with an implicit import link to tcl82.dll, too!
ACK!!! This stinks.
See the problem?
There's a way to hack this by compiling <pkg>StubsLib.c for your needs
regarding what resolving Tcl_PkgRequire should have, but the true
beauty in Stubs isn't being realized as the core template is wrong for
expanding. It's still a problem that should be looked into DEEPLY
before we get too entrenched with Stubs as it is and we all built
screwy shells.
Luckily all my own projects don't go beyond level 1 with regards to
Stubs dependencies.
Harald Kirsch
[how to organize extensions]
This is a windows solution.
o On Unix, if I am the admin, I decide where software is installed and
it might end up somewhere else than *within* the distribution
directory of some other software.
o There may be extensions which work over a range of
Tcl-versions. With your proposal, removing the Tcl-dir to install a
new version will blow off the installed extensions. ... And I am not
yet talking about a multi-platform nfs-server, where it should suffice
to install Tcl-only extensions just once, and not once for every
served platform.
o On Unix, if I am a mere user, I want to be able to install
extensions for my own purposes, even if the admin is busy or has other
complaints.
You see, there are quite a few reasons why there must be some freedom
of choice about the directory tree where to install extensions.
Harald Kirsch
--
P.S.: Never ever mail me copies of your posts.
---------------------+---------------------------------------------
Harald Kirsch (@home)| Don't like `make' ?
k...@iitb.fhg.de | Look at http://wsd.iitb.fhg.de/~kir/brashome/
Hemang Lavana
> it. How about this as the pkgIndex.tcl:
>
> foreach subdir [glob [file dirname [info script]]/*] {
> if {[file isdirectory $subdir] && \
> [file exist [file join $subdir pkgIndex.tcl]]} {
> source [file join $subdir pkgIndex.tcl]
> }
> }
I have been using something similar, to search beyond 1-level deep
in a sub-directory, for quite sometime now.
# pkgIndex.tcl
# Note that $dir is expected to be set when this file is sourced
set olddir $dir
foreach idxfile [glob -nocomplain [file join $dir * pkgIndex.tcl]] {
set dir [file dirname $idxfile]
source $idxfile
}
set dir $olddir ;# Reset the $dir value
Hemang.
Chang LI
>
> This is a windows solution.
>
> o On Unix, if I am the admin, I decide where software is installed and
> it might end up somewhere else than *within* the distribution
> directory of some other software.
>
I think it is a solution for individual user. For multiple users I agree
that we need another solution. There are also no standard there and more
complicated.
> o There may be extensions which work over a range of
> Tcl-versions. With your proposal, removing the Tcl-dir to install a
> new version will blow off the installed extensions. ... And I am not
> yet talking about a multi-platform nfs-server, where it should suffice
> to install Tcl-only extensions just once, and not once for every
> served platform.
>
It is necessary to distinguish the multiple Tcl versions and multiple
extension versions. For example, there are many Tcl 8.0 extensions,
they can not be run on the Tcl8.1. So it is better to install 8.0's
extension under the 8.0's directory, and 8.1's under the 8.1's
directory.
> o On Unix, if I am a mere user, I want to be able to install
> extensions for my own purposes, even if the admin is busy or has other
> complaints.
>
> You see, there are quite a few reasons why there must be some freedom
> of choice about the directory tree where to install extensions.
>
The choice maybe good for Tcl masters but it maybe not good for newbies.
I do not think extension scattering is good for management.
> Harald Kirsch
> --
> P.S.: Never ever mail me copies of your posts.
> ---------------------+---------------------------------------------
> Harald Kirsch (@home)| Don't like `make' ?
> k...@iitb.fhg.de | Look at http://wsd.iitb.fhg.de/~kir/brashome/
--
Chang LI
>
> But that's messy, too. The uninstall script has to find and remove
> it. How about this as the pkgIndex.tcl:
>
> foreach subdir [glob [file dirname [info script]]/*] {
> if {[file isdirectory $subdir] && \
> [file exist [file join $subdir pkgIndex.tcl]]} {
> source [file join $subdir pkgIndex.tcl]
> }
> }
>
That is a good solution. pkg_mkIndex needs to rewrite.
> if you remove 2.4 under blt it won't get sourced by the top level
> pkgIndex.tcl
>
> >I believe any new shell program like itclsh31 should be in the bin
> >directory.
>
> Why? This'll make a mess of the bin directory with a large amount of
> shells. Just like the bin directory in TclPro. I think the
> subdirectory under Lib is where everything related to an extension
> should go. But that's just my belief.
>
The best solution is that we need only one shell and others as the
loadable extension.
> What about Tcl applications that aren't shells? Ones that
> LoadLibrary("tcl82.dll"). They need special care for using Stubs.
> Exactly that method I described for resolving Tcl_CreateInterp first.
>
The application without shells can added to the bin. So everytime
when you launch an application you select an executable file in the
bin.
> Not using Stubs and going implicit run-time linking, you lose all the
> wonderful benefits like DLL name independence, notifier replacement at
> run-time, and future features to be added later.
>
> >tclsh.exe is a special starting program.
>
> What's so special about it? Ok, so its level 0 of a multi-teered
> relationship of extension dependencies. But tclAppInit.c is the
> backbone for extending a Tcl shell for your project (whatever that may
> be).
>
I do not expect it to be special, in fact it is. By using stub it makes
relationship complicate. Stub dependency is an advanced topic. The
relationship between a stub shell and a stub extension is not quite
clear.
I believe it should be no problem that two stub extensions could call
each
other.
Do you means we can not put itkwish31.exe in the bin directory,
and the itcl31.dll and itk31.dll in the lib\itcl31 directory?
Is itcl a special Tcl extension like Tk?
--
David Gravereaux
>The best solution is that we need only one shell and others as the
>loadable extension.
Yes, that would work, but it limits how someone could use Tcl.
>> What about Tcl applications that aren't shells? Ones that
>> LoadLibrary("tcl82.dll"). They need special care for using Stubs.
>> Exactly that method I described for resolving Tcl_CreateInterp first.
>>
>
>The application without shells can added to the bin. So everytime
>when you launch an application you select an executable file in the
>bin.
I couldn't do that. My IRC client sits in it's own directory under
c:\program files\xircon with about 12 files there and about 150 in
various subdirectories. It's a whole windows application that only
uses the services of Tcl with a LoadLibrary() at startup.
There's so many ways to use Tcl. It's wonderful :)
>> >tclsh.exe is a special starting program.
>>
>> What's so special about it? Ok, so its level 0 of a multi-teered
>> relationship of extension dependencies. But tclAppInit.c is the
>> backbone for extending a Tcl shell for your project (whatever that may
>> be).
>>
>
>I do not expect it to be special, in fact it is. By using stub it makes
>relationship complicate. Stub dependency is an advanced topic. The
>relationship between a stub shell and a stub extension is not quite
>clear.
Yes, it's very confusing. And even more so when you realize
<pkg>stub.lib is not a general purpose way to provide the module's
Stub table to other modules that will use it's services.
First you have this level 2 resolving problem with Tcl_PkgRequire.
Extensions using that <pkg>stub.lib want it as a Stubs resolve to
tclStubsPtr->tclPkgRequire, but if you make a shell there is no
tclStubsPtr, as the linking to Tcl is with an import library.
Second, what about when you make a debug build and static vs. dynamic
C run-time. That's 4 combos right there.
There just doesn't seem to be a way to get around needing the actual
source of the module you're building against. You have to compile
it's <pkg>StubLib.c for your own combo of needs.
Look at tclstubs82.lib that comes with the windows distribution. It's
worthless for all but one combo: dynamic C run-time and release build.
Add on to that, the problem of the "level 2 Tcl_PkgRequire circularity
thing" for custom Stubs tables and I've come to the conclusion that
distributing a <pkg>stubs.lib is a total waste. You will just always
need the source of the module you are building against to make that
custom <pkg>stubs.lib fit your needs... for that specific compile...
you're doing right then.
The benefits of Stubs do out-way the drawbacks, but Stubs isn't pure
yet. Not until tclshxx.exe gets a dynamic loading mechanism for
tclxx.dll and use it's Stubs table will this problem go away. That's
when it reaches purity.
>I believe it should be no problem that two stub extensions could call
>each
>other.
>
>Do you means we can not put itkwish31.exe in the bin directory,
>and the itcl31.dll and itk31.dll in the lib\itcl31 directory?
Correct. It won't work. As it stands in scriptics' NetCVS, the itk
shell needs much more attention to the code to fix this problem and
get the use of Stubs to be pure.
Dynamic loading of itk31.dll into wish82.exe works fine.
>Is itcl a special Tcl extension like Tk?
[incr Tcl] (itcl31.dll) adds an object-oriented paradigm to using Tcl
scripts. [incr Tk] (itk31.dll) extends [incr Tcl] to allow the
construction of "mega widgets". I love it. I use it all over my
projects. http://www.tcltk.com/itcl
lvi...@cas.org
:>I believe any new shell program like itclsh31 should be in the bin
:>directory.
:
:Why? This'll make a mess of the bin directory with a large amount of
:shells. Just like the bin directory in TclPro. I think the
On the other hand, _users_ of tcl that I deal with prefer adding _one_
directory to get at Tcl code rather than dozens (one for each extension).
--
<URL: mailto:lvi...@cas.org> Quote: Saving the world before bedtime.
<*> O- <URL: http://www.purl.org/NET/lvirden/>
Unless explicitly stated to the contrary, nothing in this posting
should be construed as representing my employer's opinions.
lvi...@cas.org
According to Chang LI <cha...@neatware.com>:
:Harald Kirsch wrote:
:I think it is a solution for individual user. For multiple users I agree
:complicated.
Let's try to find one solution that can suit everyone - building extensions
that have to install one place for a stand alone user and some totally
different structure for 'multiple' users is more nuisance than should
have to be faced.
David Gravereaux
>:>I believe any new shell program like itclsh31 should be in the bin
>:>directory.
>:
>:Why? This'll make a mess of the bin directory with a large amount of
>:shells. Just like the bin directory in TclPro. I think the
>
>On the other hand, _users_ of tcl that I deal with prefer adding _one_
>directory to get at Tcl code rather than dozens (one for each extension).
I'm in the middle of a similar discussion with someone regarding a
product I made that upgrades a program to use the real scriptics
distributed Tcl 8.x rather than the custom compile of 7.6p2 that had
no features regarding packages and extensions.
I'm being told by him (and a few other) that the new features of 8.x
over 7.6 are outstanding but the windows install for Tcl is 2.3 megs
and that's way too big. He wants to trim it down to only the files it
needs so he can redistribute it in a smaller package.
My reply to him was very simple:
"You will solely take all responsibility regarding the bugs you create
by removing parts of the Tcl core to suit your purpose."
I see the Tcl distribution as a development system. Not as a user
run-time. For his needs, he wants it a small run-time that's not used
for development by the end user.
A trimmed down minimal Tcl run-time system would be worth a thought.
A "script" could be distributed as a single compressed file with the
needed encoding files and extensions and pieces in a virtual directory
that's extracted to memory on use. Similar to ProWrap, but without
adding tcl8x.dll and a few other things regarding what is already
pre-installed.
I'm sure that idea has already has come up and been discussed before.
Actually, I haven't looked at ProWrap since last year. It might
already do this.
My full blown windows app that's using Tcl takes up over 33 megs of
space with all accessories and source development. This isn't
counting Tcl itself. It wouldn't be proper to place stuff in Tcl's
bin directory in my situation.
Paul Duffin
>
> Chang LI <cha...@neatware.com> wrote:
>
> >I believe any new shell program like itclsh31 should be in the bin
> >directory.
>
> Why? This'll make a mess of the bin directory with a large amount of
> shells. Just like the bin directory in TclPro. I think the
> should go. But that's just my belief.
>
> LoadLibrary("tcl82.dll"). They need special care for using Stubs.
> Exactly that method I described for resolving Tcl_CreateInterp first.
>
> wonderful benefits like DLL name independence, notifier replacement at
> run-time, and future features to be added later.
>
>
> What's so special about it? Ok, so its level 0 of a multi-teered
> relationship of extension dependencies. But tclAppInit.c is the
> backbone for extending a Tcl shell for your project (whatever that may
> be).
>
> <pkg>StubLib.c. Where the extension that was built to the
> <pkg>StubLib.c requires a -DUSE_TCL_STUBS compile, yet the level 2
> shell that uses the lib needs a import link to Tcl instead. Causing
> a circular dependency around Tcl_PkgRequire() in level 1's
> <pkg>StubLib.c.
>
> example:
> 1) itcl makes a itclstubs31.lib from itclStubLib.c w/ -DUSE_TCL_STUBS.
> 2) itk uses this for itk31.dll.
> 3) itkwish31.exe is implicitly linked for an import of tcl8x.dll.
> Just like the tclAppInit.c template shows us.
> 4) itkwish31.exe now cannot link because tclStubPtr (referenced in
> itclstubs31.lib) is missing and can't resolve what Tcl_PkgRequire
> is replaced with in the Stubs macro for it.
>
> Ok, just screw a Stubs table for itcl and go import lib in the
> itkwish31.exe shell. NO GOOD. The dll is in lib/itcl31 and the shell
> is in bin/. Crash on startup (itcl31.dll not found in path...)
itkwish31.exe should not be using the itclstubs library, it should be
linking to itcl31.dll and itk31.dll, just like wish links directly to
both tcl.dll and tk.dll.
The fact that this doesn't work because the DLLs cannot be found has
more to do with the Itcl installation and Windows than it has to do
with stubs.
As Itcl is an extension you do not really need to have a itkwish31.exe,
you could just as easily have an itkwish31.tcl.
>
> Ok, move the dll into the bin then. NO GOOD. That's messy. Just
> like TclPro.
>
It is not really a TclPro problem, more like a Windows feature.
Most Windows installations allow the user to change the directory
in which an application is installed so it is not possible to
place an absolute path to the library inside the executable at compile
time. However it would be nice to be able to add a relative path.
> Ok, let's add tclstubs82.lib to the link line and also build
> itkwish31.exe with an implicit import link to tcl82.dll, too!
>
> ACK!!! This stinks.
>
Why ? It works doesn't it.
> See the problem?
>
> There's a way to hack this by compiling <pkg>StubsLib.c for your needs
> regarding what resolving Tcl_PkgRequire should have, but the true
> beauty in Stubs isn't being realized as the core template is wrong for
> expanding. It's still a problem that should be looked into DEEPLY
> before we get too entrenched with Stubs as it is and we all built
> screwy shells.
>
Stubs does not solve the problem with creating shells, as you have said
if you want to create a "proper" shell you need to make it behave the
same on all platforms which requires that you dynamically load it
yourself which is really another kettle of fish.
> Luckily all my own projects don't go beyond level 1 with regards to
> Stubs dependencies.
>
--
Paul Duffin
DT/6000 Development Email: pdu...@hursley.ibm.com
IBM UK Laboratories Ltd., Hursley Park nr. Winchester
Internal: 7-246880 International: +44 1962-816880
David Gravereaux
>itkwish31.exe should not be using the itclstubs library, it should be
>linking to itcl31.dll and itk31.dll, just like wish links directly to
>both tcl.dll and tk.dll.
But then the extension DLLs all have to be in the bin/ directory, not
a sub directory under lib/ which is where they should reside. Not
only that, but look at the features you get with Stubs. At run-time,
an "upper" module could change the itcl Stub table to have a global
effect for that process. By limiting the shell to an import library
the enhanced features are thrown away.
Stubs was supposed to stop this dependency problem and allow modules
to reside outside the bin directory. This ideal is not being met in
the current architecture.
Actually, it's very easy to use a Stubs table in a shell.
Tcl_PkgRequireEx does it for you.
>The fact that this doesn't work because the DLLs cannot be found has
>more to do with the Itcl installation and Windows than it has to do
>with stubs.
No. I'm not about to put everything in the bin directory to fix a
linking problem. A dynamic loading mechanism for the tclxx.dll will
fix this and make Stubs pure. So that Tcl apps are using Stubs
tables. Allowing one part of the puzzle to be stuck as an implicit
load-time dependency is the one bad apple that ruins it for everyone.
>As Itcl is an extension you do not really need to have a itkwish31.exe,
>you could just as easily have an itkwish31.tcl.
True. But who's to say you can't? If so, shells will be another old
technique discarded for something new when the real problem can be
fixed.
GIVE TCLSH82.EXE A DYNAMIC LOADING MECHANISM FOR TCL82.DLL AND STOP
THIS DEPENDACY PROBLEM NOW.
Create a Stubs table for your own extension. Then make another
extension based from the first using it's Stub table. Then make a
shell for that one and look closely at the architecture. Stubs falls
to pieces at level 2. Removing the implicit link to tclxx.dll for the
shell will solve this completely.
Chang LI
>
> Paul Duffin <pdu...@mailserver.hursley.ibm.com> wrote:
>
> >itkwish31.exe should not be using the itclstubs library, it should be
> >linking to itcl31.dll and itk31.dll, just like wish links directly to
> >both tcl.dll and tk.dll.
>
> But then the extension DLLs all have to be in the bin/ directory, not
> a sub directory under lib/ which is where they should reside. Not
> only that, but look at the features you get with Stubs. At run-time,
> an "upper" module could change the itcl Stub table to have a global
> effect for that process. By limiting the shell to an import library
> the enhanced features are thrown away.
>
> Stubs was supposed to stop this dependency problem and allow modules
> to reside outside the bin directory. This ideal is not being met in
> the current architecture.
>
I agree your argument.
> Actually, it's very easy to use a Stubs table in a shell.
> Tcl_PkgRequireEx does it for you.
>
> >The fact that this doesn't work because the DLLs cannot be found has
> >more to do with the Itcl installation and Windows than it has to do
> >with stubs.
>
> No. I'm not about to put everything in the bin directory to fix a
> linking problem. A dynamic loading mechanism for the tclxx.dll will
> fix this and make Stubs pure. So that Tcl apps are using Stubs
> tables. Allowing one part of the puzzle to be stuck as an implicit
> load-time dependency is the one bad apple that ruins it for everyone.
>
So I am wondering why the blt24.dll for Tcl8.1 can not be put in the
lib/blt24 directory. There are two own shell programs: bltwish and
bltwish24. David, that maybe in your case. In my own extension without
my own shells like the Img extension there is no problem.
> >As Itcl is an extension you do not really need to have a itkwish31.exe,
> >you could just as easily have an itkwish31.tcl.
>
> True. But who's to say you can't? If so, shells will be another old
> technique discarded for something new when the real problem can be
> fixed.
>
> GIVE TCLSH82.EXE A DYNAMIC LOADING MECHANISM FOR TCL82.DLL AND STOP
> THIS DEPENDACY PROBLEM NOW.
>
Yes. I do not understand why the tclsh should be special.
> Create a Stubs table for your own extension. Then make another
> extension based from the first using it's Stub table. Then make a
> shell for that one and look closely at the architecture. Stubs falls
> to pieces at level 2. Removing the implicit link to tclxx.dll for the
> shell will solve this completely.
>
There are more to dig out.
> * David Gravereaux *
> Tomahawk Software Group
--
lvi...@cas.org
According to David Gravereaux <davy...@bigfoot.com>:
:A trimmed down minimal Tcl run-time system would be worth a thought.
Have you taken a look at Jean-Claude Wippler's Tclkit?
<URL: http://www.purl.org/NET/Tcl-FAQ/part4.html>:
What: TclKit (Wippler)
Where: <URL: http://www.equi4.com/tclkit/>
Description: Standalone version of Tcl/Tk v8.0.4 with plus patches and
mk4tcl linked together. Binary versions for Linux/Intel, Solaris,
FreeBSD, and Windows are available above.
Updated: 06/1999
Contact: <URL: mailto:j...@equi4.com> (Jean-Claude Wippler)
Paul Duffin
that shells can take advantage of all of the advantages that stubs provides to extensions.
A shell in this case is an application which contains a variable amount of code ranging
from minimal tclsh like code, through to millions of lines of code.
Stubs is a linking mechanism, not a loading mechanism and as such was never intended to
solve this problem.
It is certainly possible to do something like this but it would involve quite a lot of work.
1. Provide a cross platform loading mechanism, possibly by ripping the loading code out
of Tcl and packaging it up as a library, both static and dynamic.
2. Extend the tcl stub library so it provides a Tcl_LoadTcl which searches for the Tcl
library in a platform independent way, loads it and initialises the stub table.
Is there any interest in this from the Unix world, otherwise if it is a Windows only
problem then it would be easier to do.
Chang LI
>
> So just to clarify, you want a dynamic loading mechanism which is available to shells so
> that shells can take advantage of all of the advantages that stubs provides to extensions.
>
> A shell in this case is an application which contains a variable amount of code ranging
> from minimal tclsh like code, through to millions of lines of code.
>
> Stubs is a linking mechanism, not a loading mechanism and as such was never intended to
> solve this problem.
I believe both the linking and loading can be integrated.
>
> It is certainly possible to do something like this but it would involve quite a lot of work.
> 1. Provide a cross platform loading mechanism, possibly by ripping the loading code out
> of Tcl and packaging it up as a library, both static and dynamic.
>
I like this idea.
> 2. Extend the tcl stub library so it provides a Tcl_LoadTcl which searches for the Tcl
> library in a platform independent way, loads it and initialises the stub table.
>
That is sound great.
> Is there any interest in this from the Unix world, otherwise if it is a Windows only
> problem then it would be easier to do.
>
That is not Windows only problem. It is architecture problem.
> --
> Paul Duffin
> DT/6000 Development Email: pdu...@hursley.ibm.com
> IBM UK Laboratories Ltd., Hursley Park nr. Winchester
> Internal: 7-246880 International: +44 1962-816880
--
Paul Duffin
>
> Paul Duffin wrote:
> >
> > So just to clarify, you want a dynamic loading mechanism which is available to shells so
> > that shells can take advantage of all of the advantages that stubs provides to extensions.
> >
> > A shell in this case is an application which contains a variable amount of code ranging
> > from minimal tclsh like code, through to millions of lines of code.
> >
> > Stubs is a linking mechanism, not a loading mechanism and as such was never intended to
> > solve this problem.
>
> I believe both the linking and loading can be integrated.
>
They could be....
> >
> > It is certainly possible to do something like this but it would involve quite a lot of work.
> > 1. Provide a cross platform loading mechanism, possibly by ripping the loading code out
> > of Tcl and packaging it up as a library, both static and dynamic.
> >
>
> I like this idea.
>
> > 2. Extend the tcl stub library so it provides a Tcl_LoadTcl which searches for the Tcl
> > library in a platform independent way, loads it and initialises the stub table.
> >
>
> That is sound great.
>
> > Is there any interest in this from the Unix world, otherwise if it is a Windows only
> > problem then it would be easier to do.
> >
>
> That is not Windows only problem. It is architecture problem.
>
I disagree, at least for the moment.
Unix (at least the ones that I know of) provide the ability for an
executable to decide where to find its libraries, Windows doesn't, you
either have to modify global environment variables, or place the
library in a few specific locations.
David does not want to do any of the above for perfectly reasonable
reasons and so because Tcl does not provide the ability, he has had
to do it himself. If he worked on Unix he would not have that problem
because he could specify in the executable where the libraries where,
but he may want to allow the user to install in a different location
to the default which is something which Windows does well (because it
has to due to the limitations of its file systems) but Unix doesn't do
that well if at all (because Unix has more flexible file systems and so
does not have to worry as much about the hard disks and partitions). If
he did then he would probably come back to the same solution.
The reason I asked the question is I want to know whether Unix people
have actually come across this problem and would like a solution or
whether this is a purely Windows problem which can be solved with a
Windows only solution. Make no mistake, cross platform is VERY important
to me but only where it really makes sense and has benefits.
Assume for the moment that a general cross platform solution is
required and that we have a cross platform loading mechanism
which we can use.
When I run tclsh where should it look for the Tcl library, assuming that
it does not have a hard coded path (which I presume it shouldn't).
Using an environment variable ?
TCL_LIBRARY_PATH=....
David Gravereaux
>When I run tclsh where should it look for the Tcl library, assuming that
>it does not have a hard coded path (which I presume it shouldn't).
>
>Using an environment variable ?
> TCL_LIBRARY_PATH=....
>
Sure, that should be first.
if not set, second check (on windows) might be the registry for:
HKEY_LOCAL_MACHINE\SOFTWARE\Scriptics\Tcl
and choosing the highest version, unless requested for a specific one.
Third and last fallback might be the path:
if (SearchPath(NULL, "TCL82", ".DLL", 0, NULL, NULL) != 0) ....
Paul Duffin
>
> Paul Duffin <pdu...@mailserver.hursley.ibm.com> wrote:
>
> >When I run tclsh where should it look for the Tcl library, assuming that
> >it does not have a hard coded path (which I presume it shouldn't).
> >
> >Using an environment variable ?
> > TCL_LIBRARY_PATH=....
> >
>
> Sure, that should be first.
>
> if not set, second check (on windows) might be the registry for:
> HKEY_LOCAL_MACHINE\SOFTWARE\Scriptics\Tcl
>
This would break the cross platform nature of this solution.
> and choosing the highest version, unless requested for a specific one.
>
How do you decide on 'highest' ?
> Third and last fallback might be the path:
> if (SearchPath(NULL, "TCL82", ".DLL", 0, NULL, NULL) != 0) ....
>
--
David Gravereaux
>David Gravereaux wrote:
>> if not set, second check (on windows) might be the registry for:
>> HKEY_LOCAL_MACHINE\SOFTWARE\Scriptics\Tcl
>>
>
>This would break the cross platform nature of this solution.
But taking advantage of options available on a specific platform is
good. Ok, so this part won't be used on an AIX. That's fine. It's
an OS specific solution to OS specific data.
>> and choosing the highest version, unless requested for a specific one.
>>
>
>How do you decide on 'highest' ?
tcl83.dll > tcl82.dll > tcl81.dll
or
libtcl8.2.so > libtcl8.1.so
or with directory names, if that's part of the comparison on other
systems. Remember, I only do windows :) So that's my point of
reference.
David Gravereaux
Here's the logic as far as I can see through my "rosey" WIN32 goggles
:)
How's this look so far?
tclWinDlLoad.c gets dropped in tcl8.2b1/win/. That's an edit on Jan's
dlopen() routines he posted on c.l.t back in May and could be extended
to cross-over into TclpLoadFile.
Chang LI
>
> Paul,
>
David,
I do not see your code related to the stub. It is a tclxx.dll load
program.
Are you trying to replace the Tcl_Init?
> ---------------------------------------------------------------
>
> Name: main.c
> Part 1.2 Type: Plain Text (text/plain)
> Encoding: 7bit
>
> Name: tclWinDlLoad.c
> Part 1.3 Type: Plain Text (text/plain)
> Encoding: 7bit
David Gravereaux
>I do not see your code related to the stub. It is a tclxx.dll load
>program.
>Are you trying to replace the Tcl_Init?
Stub_TclLoad will get called at the beginning of the Tcl_AppInit
somewhere. I'm getting there. It's sloppy. I'm only a day into it
:)
Yes, all it is is a loader. When the logic for the loading is done
(and cleaned up) I'll grab Tcl_CreateInterp with dlsym, create an
interp, call Tcl_StubInit on it, and dump the interp.
Jan Nijtmans
> Yes, all it is is a loader. When the logic for the loading is done
> (and cleaned up) I'll grab Tcl_CreateInterp with dlsym, create an
> interp, call Tcl_StubInit on it, and dump the interp.
That's not the right way to do it. It should be something like:
int myAppInitProc(Tcl_Interp *interp)
{
if (Tcl_InitStubs (interp, "8.1", 0) != TCL_OK) {
return TCL_ERROR;
}
.......
}
main()
{
....
handle = dlopen("tcl81.dll", ....);
tclMainPtr = dlsym(handle, "Tcl_Main");
tclMainPtr (argc, argv, myAppInitProc);
....
}
This way you don't have to create and interpreter and then
delete it. Just perform the stub initialization in the beginning
of your tclAppInit() function. Further on, compile this application
with USE_TCL_STUBS and link with the plus-patch stub-library
(just in order to get access to the cross-platform versions
of dlopen and dlsym), then it should work for all platforms
(except that the name of the dll is different)!!!!!!
Hope this helps,
--
Jan Nijtmans, CMG Arnhem B.V.
email: Jan.Ni...@wxs.nl (private)
Jan.Ni...@cmg.nl (work)
url: http://home.wxs.nl/~nijtmans/
Chang LI
>
This is clearer in architecture. It is possible to break tcl81.dll
into several .dlls and load what you needed. So we can reduce the Tcl
kernel to a min size. Before we have tried to make Tk as a dynamic
loadable extension, now Tcl itself should be a loadable extension
based on stub. It means that the Tcl kernel is loadable. So what
can we do? We may have a scalable kernel!
> That's not the right way to do it. It should be something like:
>
> int myAppInitProc(Tcl_Interp *interp)
> {
> if (Tcl_InitStubs (interp, "8.1", 0) != TCL_OK) {
> return TCL_ERROR;
> }
I think no Tcl_Init is needed here. It can be integrated
into the Tcl_InitStubs(). So this init procedure will be
very simple.
> .......
> }
>
> main()
> {
> ....
> handle = dlopen("tcl81.dll", ....);
It is possible to use
handle = dlopen("smalltcl81.dll", ...);
> tclMainPtr = dlsym(handle, "Tcl_Main");
> tclMainPtr (argc, argv, myAppInitProc);
> ....
> }
>
> This way you don't have to create and interpreter and then
> delete it. Just perform the stub initialization in the beginning
> of your tclAppInit() function. Further on, compile this application
> with USE_TCL_STUBS and link with the plus-patch stub-library
> (just in order to get access to the cross-platform versions
> of dlopen and dlsym), then it should work for all platforms
> (except that the name of the dll is different)!!!!!!
>
It is better to integrate the stub/dlopen/dlsym into a module.
> Hope this helps,
> --
> Jan Nijtmans, CMG Arnhem B.V.
> email: Jan.Ni...@wxs.nl (private)
> Jan.Ni...@cmg.nl (work)
> url: http://home.wxs.nl/~nijtmans/
--
David Gravereaux
So now what? You guys want the diffs?
Index: tclAppInit.c
===================================================================
RCS file: /cvsroot/tcl/win/tclAppInit.c,v
retrieving revision 1.5
diff -c -r1.5 tclAppInit.c
*** tclAppInit.c 1999/04/16 00:48:07 1.5
--- tclAppInit.c 1999/08/09 02:29:40
***************
*** 53,58 ****
--- 53,66 ----
int argc; /* Number of command-line arguments. */
char **argv; /* Values of command-line arguments. */
{
+ #ifdef USE_TCL_STUBS
+ void *(*tclMainProc) _ANSI_ARGS_((int argc, char **argv,
+ Tcl_AppInitProc *appInitProc));
+
+ Stub_LoadTcl(TCL_VERSION, /*exact*/1, /*debug*/1);
+ tclMainProc = Stub_GetMain();
+ #endif
+
/*
* Set up the default locale to be standard "C" locale so parsing
* is performed correctly.
***************
*** 61,67 ****
--- 69,80 ----
setlocale(LC_ALL, "C");
setargv(&argc, &argv);
+
+ #ifdef USE_TCL_STUBS
+ tclMainProc(argc, argv, Tcl_AppInit);
+ #else
Tcl_Main(argc, argv, Tcl_AppInit);
+ #endif
return 0; /* Needed only to prevent compiler warning. */
}
***************
*** 89,98 ****
Tcl_AppInit(interp)
Tcl_Interp *interp; /* Interpreter for application. */
{
if (Tcl_Init(interp) == TCL_ERROR) {
return TCL_ERROR;
}
-
#ifdef TCL_TEST
if (Tcltest_Init(interp) == TCL_ERROR) {
return TCL_ERROR;
--- 102,113 ----
Tcl_AppInit(interp)
Tcl_Interp *interp; /* Interpreter for application. */
{
+ if (Tcl_InitStubs(interp, TCL_VERSION, 1) == NULL) {
+ return TCL_ERROR;
+ }
if (Tcl_Init(interp) == TCL_ERROR) {
return TCL_ERROR;
}
#ifdef TCL_TEST
if (Tcltest_Init(interp) == TCL_ERROR) {
return TCL_ERROR;
***************
*** 196,202 ****
}
}
}
! argSpace = (char *) Tcl_Alloc(
(unsigned) (size * sizeof(char *) + strlen(cmdLine) + 1));
argv = (char **) argSpace;
argSpace += size * sizeof(char *);
--- 211,217 ----
}
}
}
! argSpace = (char *) malloc(
(unsigned) (size * sizeof(char *) + strlen(cmdLine) + 1));
argv = (char **) argSpace;
argSpace += size * sizeof(char *);
I have to revert back to the stock makefile.vc so I can diff it.
Mine is too edited to gen a good diff.
Paul Duffin
>
> Jan Nijtmans wrote:
> >
>
> This is clearer in architecture. It is possible to break tcl81.dll
> into several .dlls and load what you needed. So we can reduce the Tcl
> kernel to a min size. Before we have tried to make Tk as a dynamic
> loadable extension, now Tcl itself should be a loadable extension
> based on stub. It means that the Tcl kernel is loadable. So what
> can we do? We may have a scalable kernel!
>
> > That's not the right way to do it. It should be something like:
> >
> > int myAppInitProc(Tcl_Interp *interp)
> > {
> > if (Tcl_InitStubs (interp, "8.1", 0) != TCL_OK) {
> > return TCL_ERROR;
> > }
>
> I think no Tcl_Init is needed here. It can be integrated
> into the Tcl_InitStubs(). So this init procedure will be
> very simple.
>
An extra Tcl_Init is needed because it Tcl_Init is only needed if you
want to make the interpreter behave like a standard tclsh by
evaluating init.tcl.
> > .......
> > }
> >
> > main()
> > {
> > ....
> > handle = dlopen("tcl81.dll", ....);
>
> It is possible to use
>
> handle = dlopen("smalltcl81.dll", ...);
>
> > tclMainPtr = dlsym(handle, "Tcl_Main");
> > tclMainPtr (argc, argv, myAppInitProc);
> > ....
> > }
> >
> > This way you don't have to create and interpreter and then
> > delete it. Just perform the stub initialization in the beginning
> > of your tclAppInit() function. Further on, compile this application
> > with USE_TCL_STUBS and link with the plus-patch stub-library
> > (just in order to get access to the cross-platform versions
> > of dlopen and dlsym), then it should work for all platforms
> > (except that the name of the dll is different)!!!!!!
> >
>
> It is better to integrate the stub/dlopen/dlsym into a module.
>
The dlopen / dlsym definitely need to be in their own library (both
static and dynamic) but the stub stuff is Tcl specific and should not
be in with the dlopen stuff.
David Gravereaux
This is sweet. I just dropped it into my main Tcl app for testing.
It's working great. The exe size went up only 5k, but I can dump that
ini file :)
Robin Becker
>The dlopen / dlsym definitely need to be in their own library (both
>static and dynamic) but the stub stuff is Tcl specific and should not
>be in with the dlopen stuff.
>
off and handled via unknown?
--
Robin Becker
Volker Hetzer
>
> ....
finding the commands according to some (preferably definable) rules.
Then, core development could concentrate on caching and such without having to release
a whole tcl each time something is finished.
Volker
--
Hi! I'm a signature virus! Copy me into your signature file to help me spread!
Chang LI
>
> >
> > It is better to integrate the stub/dlopen/dlsym into a module.
> >
>
> static and dynamic) but the stub stuff is Tcl specific and should not
> be in with the dlopen stuff.
>
In pure sense Tcl kernel itself may be designed to be a stub extension.
The shell is a startup program. The stub/load will be a protocl.
This protocl should be platform independent. The glue is the stub others
are objects. It is good to separate this glue into a standalong unit.
> --
> Paul Duffin
> DT/6000 Development Email: pdu...@hursley.ibm.com
> IBM UK Laboratories Ltd., Hursley Park nr. Winchester
> Internal: 7-246880 International: +44 1962-816880
--
Paul Duffin
>
> Paul Duffin wrote:
> >
> > >
> > > It is better to integrate the stub/dlopen/dlsym into a module.
> > >
> >
> > The dlopen / dlsym definitely need to be in their own library (both
> > static and dynamic) but the stub stuff is Tcl specific and should not
> > be in with the dlopen stuff.
> >
>
> In pure sense Tcl kernel itself may be designed to be a stub extension.
> The shell is a startup program. The stub/load will be a protocl.
> This protocl should be platform independent. The glue is the stub others
> are objects. It is good to separate this glue into a standalong unit.
>
Agreed, although this standalone unit would use the dynamic load library,
not be part of it.