Questions about "package require"
Mark A. Wege
command, and I'm not sure if it's a bug, or a mis-understanding of
what the command does.
I just compiled Tcl 8.3, as well as TclX 8.2 and Oratcl 2.5. I entered
an interactive wish session and issued following commands:
% package names
Tk Tcl
% package require Tclx
8.2
wish8.3>package names
http Oratcl tcltest msgcat opt Tkx Tclx Tk Tcl
wish8.3>package versions Oratcl
2.5
wish8.3>oralogon /
invalid command name "oralogon"
My questions are:
1) Why, when I require TclX, do I also appear to get Oratcl,
et. al.?
2) Does the "package names" command return only what is actually
loaded, or anything that _could_ be loaded? (I read the man
page, but it wasn't clear, at least to me)
3) If it does return anything that could be loaded, is there
another command that tells you what is actually loaded?
Mark Wege
Don Porter
> % package names
> Tk Tcl
> % package require Tclx
> 8.2
> wish8.3>package names
> http Oratcl tcltest msgcat opt Tkx Tclx Tk Tcl
> 1) Why, when I require TclX, do I also appear to get Oratcl,
> et. al.?
>
> 2) Does the "package names" command return only what is actually
> loaded, or anything that _could_ be loaded? (I read the man
> page, but it wasn't clear, at least to me)
[package names] returns a list of all the packages which the
interpreter knows about. This includes all packages which are
loaded, and all packages it knows how to load.
Like many of Tcl's builtin commands, [package] is lazy. It
doesn't do anything until it has to. In particular it doesn't
try to learn what packages are available for loading until
you actually try to load one.
Your [package require TclX] above actually loads TclX (and whatever
other packages TclX loads). It also learns how to load the other
packages which are added to the [package names] list, but does not
load them.
> 3) If it does return anything that could be loaded, is there
> another command that tells you what is actually loaded?
[package present Oratcl] or [package provide Oratcl]
Also note that [package versions] returns a list of all the
versions of the package which could be loaded; it says nothing
about what is or isn't loaded.
--
| Don Porter, D.Sc. Mathematical and Computational Sciences Division |
| donald...@nist.gov Information Technology Laboratory |
| http://math.nist.gov/mcsd/Staff/DPorter/ NIST |
|______________________________________________________________________|
Mark A. Wege
Thanks for the reply.
If I had seen the "package present" command, I would have probably figured
this
out for myself. Unfortunately, when I was looking at the man page, it was
the
8.0 version, where the "present" subcommand either didn't exist or wasn't
listed,
not the 8.3 version. I have an alias which just looks at the Tcl man pages,
and I
forgot to update the alias to point to the new version.
Mark
lvi...@cas.org
According to Don Porter <d...@cam.nist.gov>:
:[package names] returns a list of all the packages which the
:interpreter knows about. This includes all packages which are
:loaded, and all packages it knows how to load.
What needs to be done so that package names knows about all the packages
installed though?
$ tclsh8.3
% package names
Tcl
% package require math
can't find package math
% package names
http Efftcl tcltest msgcat opt Tcl
% ls /usr/tcl83/lib
Img1.2 libBLTlite24.a libpng.so.2.1.0 libtk8.3.so
blt2.4 libBLTlite24.so libswig.a libtkstub8.3.a
bwidget1.2.1 libexpect5.31.a libswigpl.a libz.so
efftcl1.0 libitcl3.1.so libswigpl.so libz.so.1
expect5.31 libitclstub3.1.a libswigtcl.a libz.so.1.1
ftplib1.2 libitk3.1.so libswigtcl.so libz.so.1.1.3
itcl3.1 libitkstub3.1.a libtcl8.3.so oratcl2.6
itk3.1 libjpeg.so libtclXtSend.a swig_lib
iwidgets libjpeg.so.62 libtclXtSend.so tcl8.3
iwidgets2.2.0 libjpeg.so.62.0 libtclXtSend.so.1.2 tclConfig.sh
iwidgets3.0.0 libjpeg.so.62.0.0 libtclstub8.3.a tcllib0.1
libBLT.a liboratcl2.6.so libtiff.so tix4.1
libBLT24.a libpng.so libtiff.so.3 tk8.3
libBLT24.so libpng.so.2 libtiff.so.3.5 tkConfig.sh
libBLTlite.a libpng.so.2.1 libtiff.so.3.5.4
%
In the above example, I would have expected to see expect, incr tcl, oratcl
and the packages in tcllib - or at least tcllib itself. Right now I am not
running X11, so the Tk based extensions not showing up I guess makes sense.
--
<URL: http://dev.scriptics.com/>
<URL: mailto:lvi...@cas.org> <URL: http://www.purl.org/NET/lvirden/>
Unless explicitly stated to the contrary, nothing in this posting
should be construed as representing my employer's opinions.
Don Porter
> What needs to be done so that package names knows about all the packages
> installed though?
When I want to fill the [package names] list without loading any
packages, I use:
eval [package unknown] Tcl [package provide Tcl]
> % package names
> http Efftcl tcltest msgcat opt Tcl
> % ls /usr/tcl83/lib
> Img1.2 libBLTlite24.a libpng.so.2.1.0 libtk8.3.so
> blt2.4 libBLTlite24.so libswig.a libtkstub8.3.a
> bwidget1.2.1 libexpect5.31.a libswigpl.a libz.so
> efftcl1.0 libitcl3.1.so libswigpl.so libz.so.1
> expect5.31 libitclstub3.1.a libswigtcl.a libz.so.1.1
> ftplib1.2 libitk3.1.so libswigtcl.so libz.so.1.1.3
> itcl3.1 libitkstub3.1.a libtcl8.3.so oratcl2.6
> itk3.1 libjpeg.so libtclXtSend.a swig_lib
> iwidgets libjpeg.so.62 libtclXtSend.so tcl8.3
> iwidgets2.2.0 libjpeg.so.62.0 libtclXtSend.so.1.2 tclConfig.sh
> iwidgets3.0.0 libjpeg.so.62.0.0 libtclstub8.3.a tcllib0.1
> libBLT.a liboratcl2.6.so libtiff.so tix4.1
> libBLT24.a libpng.so libtiff.so.3 tk8.3
> libBLT24.so libpng.so.2 libtiff.so.3.5 tkConfig.sh
> libBLTlite.a libpng.so.2.1 libtiff.so.3.5.4
>
> In the above example, I would have expected to see expect, incr tcl, oratcl
> and the packages in tcllib - or at least tcllib itself.
What is the value of $::auto_path ? My guess is /usr/tcl83/lib is not on
it. If that guess is wrong, then check the existence/readability of
the pkgIndex.tcl files.
You don't have a custom [package unknown] registered, do you?
> Right now I am not
> running X11, so the Tk based extensions not showing up I guess makes sense.
It might, if those extensions had carefully crafted pkgIndex.tcl files
which only called [package ifneeded] in interpreters which already
have Tk loaded. I believe that is the proper way to construct pkgIndex.tcl,
but in my experience, few extensions do this. Also, few extensions which
depend on a particular version of Tcl/Tk, verify that version before
calling [package ifneeded].
$ tclsh
% info patchlevel
8.3.0
% eval [package unknown] Tcl [package provide Tcl]
% package names
http BLT opt tcltest msgcat Iwidgets Itk Tcl Tclpp Itcl Mk4tcl
% package require Iwidgets
can't find package Tk 8.0
% package require BLT
couldn't load file "/usr/local/lib/libBLT24.so": libtk8.2.so: cannot open shared object file: No such file or directory
Paul Duffin
>
> <lvi...@cas.org> wrote:
> > Right now I am not
> > running X11, so the Tk based extensions not showing up I guess makes sense.
>
> It might, if those extensions had carefully crafted pkgIndex.tcl files
> which only called [package ifneeded] in interpreters which already
> have Tk loaded. I believe that is the proper way to construct pkgIndex.tcl,
> but in my experience, few extensions do this. Also, few extensions which
> depend on a particular version of Tcl/Tk, verify that version before
> calling [package ifneeded].
>
So do you think something like.
if { [package present Tk 8.3] } {
package ifneeded Foo 1.0 ...
}
is better than
package ifneeded Foo 1.0 ...
where the Foo package is a binary extension and itself does a
Tk_InitStubs ("Tk", "8.3") which does a Tcl_PkgRequire internally.
Personally, I prefer the latter as the pkgIndex.tcl is simpler which considering
it has to be automatically generated is always useful, the dependencies are
listed in one place, the user does not have to know what I depend on in order
to use me (although there are other issues involved) and in the future when Tk is
fully dynamically loadable into Tcl [package require Foo] will work just the same
in tclsh as it does in wish.
lvi...@cas.org
:packages, I use:
:
: eval [package unknown] Tcl [package provide Tcl]
This is one of several things that made a difference. Something else
was getting rid of an older version of tclsh8.3 that was in my path <blush>.
$ tclsh8.3
% info patchlevel
8.3.0
% package names
Tcl
% eval [package unknown] Tcl [package provide Tcl]
% package names
http pop3 math BLT tcllib tcltest opt cmdline Efftcl struct Img BWidget msgcat tkpiechart mime Itk Iwidgets ncgi FTP profiler switched Tcl smtp stooop Itcl base64 Oratcl Tix fileutil Tixsam
% puts $::auto_path
/usr/tcl83/lib/tcl8.3 /usr/tcl83/lib /usr/tcl83/lib/tcllib0.1
% ls /usr/tcl83/lib
Img1.2 libBLTlite24.so libswigpl.a libtkstub8.3.a
blt2.4 libexpect5.31.a libswigpl.so libz.so
bwidget1.2.1 libitcl3.1.so libswigtcl.a libz.so.1
efftcl1.0 libitclstub3.1.a libswigtcl.so libz.so.1.1
expect5.31 libitk3.1.so libtcl8.3.so libz.so.1.1.3
ftplib1.2 libitkstub3.1.a libtclXtSend.a oratcl2.6
itcl3.1 libjpeg.so libtclXtSend.so stooop-4.1
itk3.1 libjpeg.so.62 libtclXtSend.so.1.2 swig_lib
iwidgets libjpeg.so.62.0 libtclstub8.3.a tcl8.3
iwidgets2.2.0 libjpeg.so.62.0.0 libtiff.so tclConfig.sh
iwidgets3.0.0 liboratcl2.6.so libtiff.so.3 tcllib0.1
libBLT.a libpng.so libtiff.so.3.5 tix4.1
libBLT24.a libpng.so.2 libtiff.so.3.5.4 tk8.3
libBLT24.so libpng.so.2.1 libtix4.1.8.0.so tkConfig.sh
libBLTlite.a libpng.so.2.1.0 libtixsam4.1.8.0.so tkpiechart-6.2
libBLTlite24.a libswig.a libtk8.3.so
:>
:> In the above example, I would have expected to see expect, incr tcl, oratcl
:> and the packages in tcllib - or at least tcllib itself.
:
:What is the value of $::auto_path ? My guess is /usr/tcl83/lib is not on
:it. If that guess is wrong, then check the existence/readability of
:the pkgIndex.tcl files.
:
:You don't have a custom [package unknown] registered, do you?
I don't intentionally have a custom package unknown registered...
I still don't see an expect and Tk extension, and I am uncertain what
else I should expect to see that I don't.
Another frustrating thing if you check out those package names is the fact
that some are all lower case, some have the first letter capitalized, at
least one has the first two letters capitalized, and at least one has
all letters capitalized. Another useful thing to have would be a
script that would match package names from the directories from which
that package came. For instance, there is in the above example an ftplib
extension installed. But did the FTP package come from there, or as a
part of some other package? I can certainly, in this case, go poking
around to see. It would however be really better to have tcl indicate
for certain where it found a package. In the case of attempting to
debug a user environment, there might be a TCL_LIBRARY or TK_LIBRARY
defined which causes a conflict...
laurent....@cgi.ca
> I still don't see an expect and Tk extension, and I am uncertain what
> else I should expect to see that I don't.
>
Well, your expect is compiled statically so maybe that's why. As for Tk, I'm
not sure it's a real extension so that's probably why it don't show up.
L
--
Laurent Duperval "Montreal winters are an intelligence test,
mailto:laurent....@cgi.ca and we who are here have failed it."
-Don Camilli
Penguin Power! ***Nothing I say reflects the views of my employer***
Don Porter
> I still don't see an expect and Tk extension, ...
Judging from what you posted, they're not installed as loadable
extensions, so that is normal.
> Another frustrating thing if you check out those package names is the fact
> that some are all lower case, some have the first letter capitalized, at
> least one has the first two letters capitalized, and at least one has
> all letters capitalized.
I would hope that is one thing addressed by TEA. (Perhaps with some
grandfather clauses).
> Another useful thing to have would be a
> script that would match package names from the directories from which
> that package came. For instance, there is in the above example an ftplib
> extension installed.
The man pages library(n) and tclvars(n) advise that each "application"
foo should store its "library" directory in a global variable,
foo_library. I believe this advice dates to a time before namespaces
and even before packages, so in the present day, I would re-interpret
that as advice that each package foo should store its home directory in
the variable ::foo::library.
That leaves a few problems.
1) Packages which contain both C and Tcl code are typically
installed in two directories, the binary parts under
TCL_EXEC_PREFIX and the platform-independent parts under
TCL_PREFIX. Judging by Tk, the practice has been to store
the directory containing the platform-independent parts in
foo_library, with no mechanism for finding the binary parts,
perhaps because it is difficult (impossible?) to equip a
cross-platform binary load mechanism with the means to
discover and store the directory from which its binary
library was retrieved.
2) Of all the many overloaded meanings of "library" in Tcl
discourse, I find this one is the least appropriate.
Unfortunately it is also the one with the longest heritage.
If it could be done without objection, the move of the
variable from a global to a namespace variable would also be
a good time to rename it to something less generic.
Victor Wagner
: % package names
: Tk Tcl
: % package require Tclx
: 8.2
: wish8.3>package names
: http Oratcl tcltest msgcat opt Tkx Tclx Tk Tcl
: wish8.3>package versions Oratcl
: 2.5
: wish8.3>oralogon /
: invalid command name "oralogon"
: My questions are:
: 1) Why, when I require TclX, do I also appear to get Oratcl,
: et. al.?
Becouse upon your first request Tcl have scanned all the pkgIndex.tcl
files and now shows you all the packages you have on your machine.
: 2) Does the "package names" command return only what is actually
: loaded, or anything that _could_ be loaded? (I read the man
: page, but it wasn't clear, at least to me)
Second.
: 3) If it does return anything that could be loaded, is there
: another command that tells you what is actually loaded?
info loaded does this, but only for compilied packages
lvi...@cas.org
According to <laurent....@cgi.ca>:
:On 27 Mar, lvi...@cas.org wrote:
:> I still don't see an expect and Tk extension, and I am uncertain what
:> else I should expect to see that I don't.
:>
:
:Well, your expect is compiled statically so maybe that's why. As for Tk, I'm
Sigh... I didn't realize that until you mentioned it. The other
extensions seem to default to building as dynamically loadable extensions.
Thanks!
Don Porter
> So do you think something like.
>
> if { [package present Tk 8.3] } {
> package ifneeded Foo 1.0 ...
> }
>
> is better than
>
> package ifneeded Foo 1.0 ...
>
> where the Foo package is a binary extension and itself does a
> Tk_InitStubs ("Tk", "8.3") which does a Tcl_PkgRequire internally.
I do, but I imagine that will spark a bit of controversy (stay tuned).
Let me make a non-controversial (I hope) claim first.
If a package knows that it cannot successfully be loaded into a Tcl
interpreter, it should not advertise to that Tcl interpreter (via
[package ifneeded]) loading instructions which are doomed to fail.
For example, if binary extension Foo 1.0 was compiled against Tcl 8.0
(before stubs) and can only successfully be loaded into an 8.0.x
interpreter, its pkgIndex.tcl file should look like:
if {[catch {package require -exact Tcl 8.0}]} {return}
package ifneeded Foo 1.0 ...
Likewise, when Foo 1.1 is released, using stubs (and requiring at least
Tcl 8.1), its pkgIndex.tcl file should look like:
if {![package vsatisfies [package provide Tcl] 8.1]} {return}
package ifneeded Foo 1.1 ...
And when Foo 1.2 is released, which makes internal use of [string is],
its pkgIndex.tcl file should look like:
if {![package vsatisfies [package provide Tcl] 8.2]} {return}
package ifneeded Foo 1.2 ...
When indexed that way, I can have Tcl 8.0, 8.1, 8.2 and 8.3 all
installed on my machine and all sharing the same installation of
packages, yet when each evaluates [package require Foo 1], it will
get a working version of Foo for that interpreter. Without that
style of indexing, all interpreters would try to load Foo 1.2, and
the apps using 8.0 and 8.1 interpreters would fail, even though there's
a perfectly usable version of Foo 1 installed for each of them.
Don Porter
> So do you think something like.
>
> if { [package present Tk 8.3] } {
> package ifneeded Foo 1.0 ...
> }
>
> is better than
>
> package ifneeded Foo 1.0 ...
>
> where the Foo package is a binary extension and itself does a
> Tk_InitStubs ("Tk", "8.3") which does a Tcl_PkgRequire internally.
Yes I do.
The operative principle is that the global namespace is the property
of the application, and no package should place anything there unless
the application asks it to.
Tk is not a good example, since it isn't yet loadable and doesn't yet
confine itself to its own namespace, so lets assume package Foo depends
on package Bar. When the app evaluates [package require Foo], it's
asking package Foo to load itself in the current interpreter. Implied
in that request is an invitation to create the namespace ::Foo and
*only* that in the global namespace.
However, if [package require Foo] "recursively" calls [package require
Bar], then package Foo also creates the namespace ::Bar . That's more
than it was invited to do by the application. When packages place
more than what the application asks in the global namespace, the
application can't prevent potential conflicts among those "extras".
This leads to the problem of surprise conflicts when two packages which
are explicitly [require]d by the application, each [require] different
major versions of the same package. See
http://www.deja.com/=dnc/getdoc.xp?AN=439550081
When I wrote that a year ago, I could see that "recursive" [package
require] leads to the problem of hidden package conflicts. However,
I hadn't yet grokked namespaces, so I was proposing wacky ideas like
having packages live in separate interpreters. I think my mind is
clearer now.
This problem is no different in principle from the problem of two
packages trying to create a command in the global namespace with the
same name. In both cases it's the responsibility of the packages to
stay in their own namespaces and the responsibility of the application
to resolve the conflict. With conflicting commands, the application
decides which (if either) command gets into the global namespace via
[namespace import]. It should also be left entirely to the application
to resolve conflicting package requirements. The way to do that is
to require the application to load all packages it needs (and all
packages they need, and so on), and have no packages "recursively" load
their own dependencies.
> Personally, I prefer the latter as the pkgIndex.tcl is simpler which
> considering it has to be automatically generated is always useful, the
> dependencies are listed in one place, the user does not have to know
> what I depend on in order to use me (although there are other issues
> involved)
I agree with you about every one of these problems with what I'm
proposing. However, I don't think that indicates a problem with the
principle I am advocating. It just illustrates that the [package]
command doesn't provide a powerful enough set of tools for managing
packages.
I am working (off and on, mostly off recently) on an extended set of
subcommands for [package] that will address these problems. In a c.l.t.
article I posted on Feb 4, I hoped aloud that I would post a prototype
"next week". Obviously, I've been "off" much more than "on". :)
I held off replying to your question because I'm still not ready to
release an alpha version of the prototype of the extended [package].
Then I realized I'm corresponding with the author of Feather, so I
think that gives me some wiggle room. :) RSN, really.
> ... and in the future when Tk is fully dynamically loadable into
> Tcl [package require Foo] will work just the same in tclsh as it does
> in wish.
Yeah, but [package require Tk; package require Foo] will also work
in wish now, and work the same in both when Tk becomes loadable.
Paul Duffin
>
> Paul Duffin <pdu...@hursley.ibm.com> wrote:
> > So do you think something like.
> >
> > if { [package present Tk 8.3] } {
> > package ifneeded Foo 1.0 ...
> > }
> >
> > is better than
> >
> > package ifneeded Foo 1.0 ...
> >
> > where the Foo package is a binary extension and itself does a
> > Tk_InitStubs ("Tk", "8.3") which does a Tcl_PkgRequire internally.
>
> Let me make a non-controversial (I hope) claim first.
>
> If a package knows that it cannot successfully be loaded into a Tcl
> interpreter, it should not advertise to that Tcl interpreter (via
> [package ifneeded]) loading instructions which are doomed to fail.
>
Agreed.
> For example, if binary extension Foo 1.0 was compiled against Tcl 8.0
> (before stubs) and can only successfully be loaded into an 8.0.x
> interpreter, its pkgIndex.tcl file should look like:
>
> if {[catch {package require -exact Tcl 8.0}]} {return}
> package ifneeded Foo 1.0 ...
>
Agreed.
> Likewise, when Foo 1.1 is released, using stubs (and requiring at least
> Tcl 8.1), its pkgIndex.tcl file should look like:
>
> if {![package vsatisfies [package provide Tcl] 8.1]} {return}
> package ifneeded Foo 1.1 ...
>
Agreed.
> And when Foo 1.2 is released, which makes internal use of [string is],
> its pkgIndex.tcl file should look like:
>
> if {![package vsatisfies [package provide Tcl] 8.2]} {return}
> package ifneeded Foo 1.2 ...
>
Agreed.
> When indexed that way, I can have Tcl 8.0, 8.1, 8.2 and 8.3 all
> installed on my machine and all sharing the same installation of
> packages, yet when each evaluates [package require Foo 1], it will
> get a working version of Foo for that interpreter. Without that
> style of indexing, all interpreters would try to load Foo 1.2, and
> the apps using 8.0 and 8.1 interpreters would fail, even though there's
> a perfectly usable version of Foo 1 installed for each of them.
>
Agreed. Gosh that was relatively painless.
Paul Duffin
>
> Paul Duffin <pdu...@hursley.ibm.com> wrote:
> > So do you think something like.
> >
> > if { [package present Tk 8.3] } {
> > package ifneeded Foo 1.0 ...
> > }
> >
> > is better than
> >
> > package ifneeded Foo 1.0 ...
> >
> > where the Foo package is a binary extension and itself does a
> > Tk_InitStubs ("Tk", "8.3") which does a Tcl_PkgRequire internally.
>
>
> The operative principle is that the global namespace is the property
> of the application, and no package should place anything there unless
> the application asks it to.
>
Agreed most strongly.
> Tk is not a good example, since it isn't yet loadable and doesn't yet
> confine itself to its own namespace, so lets assume package Foo depends
> on package Bar. When the app evaluates [package require Foo], it's
> asking package Foo to load itself in the current interpreter. Implied
> in that request is an invitation to create the namespace ::Foo and
> *only* that in the global namespace.
>
> However, if [package require Foo] "recursively" calls [package require
> Bar], then package Foo also creates the namespace ::Bar . That's more
> than it was invited to do by the application. When packages place
> more than what the application asks in the global namespace, the
> application can't prevent potential conflicts among those "extras".
>
That is what I was referring to in parentheses when I said
...the user does not have to know what I depend on in order
to use me (although there are other issues involved)...
I'll reserve judgement on this until I see how you intend to solve this.
Another solution to the problem would be to allow multiple different
versions of a package to be loaded at once and handle all the possible
conflicts between them...
I often think that Tcl needs to know a lot more about what a package's
effects are. At the moment there is no way of determining the effect of
a package once it has been loaded, e.g. what commands it creates, what
namespaces it creates, what packages it loads because there is no
'marker' which the package leaves behind, no trail to follow. This also
means that it is not possible to 'nicely' cleanup the package.
e.g.
Package Foo loads extension foo.so.
Extension foo.so creates a command "bar",
a namespace,
a Tcl_ObjType.
The command "bar" creates lots of other 'object' commands,
bar1, bar2, bar3, bar4, bar5, ...
It would be nice to be able to determine that bar1 is 'owned',
'implemented', whatever you want to call it by package Foo.
I have not yet decided what I would do with this but I feel that it could
provide a solution to this packaging clash by providing different views of
the Tcl interpreter.
e.g.
If package Foo requires package Bar 2.0 then Bar 2.0 lives
'inside' package Foo somehow.
If package Grok requires package Bar 3.0 then Bar 3.0 lives
'inside' package Grok somehow.
If package Clunk requires package Bar 3.0 then Bar 3.0 lives
'inside' package Clunk somehow.
Application A requires Foo, Grok, Clunk so Bar 2.0 lives 'inside'
package Foo and Bar 3.0 is shared between 'Grok' and 'Clunk'.
'inside' above does not really indicate containment but rather perception.
So when Foo looks at the Bar package it sees Bar 2.0 but when Grok looks
at the Bar package it sees Bar 3.0.
Obviously there will still be packages which are mutually exclusive but
the number will be drastically reduced.
The 'package' trail is relatively easy to implement (with minimal impact
on most extensions), the interpreter simply has a stack of 'package'
markers which are manipulated as follows.
Before [package require Foo 1.0] calls the script registered
by [package ifneeded Foo 1.0] it pushes a new 'Foo 1.0' marker on
the stack and pops it off afterwards.
When [proc] is called to create a new command it associates the
current (top) package marker on the stack with it. So if the
[package ifneeded] script calls [proc] then the new command is
associated with 'Foo 1.0'.
Similarly for Tcl_CreateObjCommand....
When a command/proc is called it pushes its marker onto the
stack before it executes the body and pops it off afterwards.
Similarly for other callbacks into Tcl or C code.
What is not so easy to follow are the consequences and uses which can be
made of this information and also the pitfalls.
> I am working (off and on, mostly off recently) on an extended set of
> subcommands for [package] that will address these problems. In a c.l.t.
> article I posted on Feb 4, I hoped aloud that I would post a prototype
> "next week". Obviously, I've been "off" much more than "on". :)
>
> I held off replying to your question because I'm still not ready to
> release an alpha version of the prototype of the extended [package].
> Then I realized I'm corresponding with the author of Feather, so I
> think that gives me some wiggle room. :) RSN, really.
>
That was nasty and uncalled for, I am going to cry now.... :-(
,
,
,
,
> > ... and in the future when Tk is fully dynamically loadable into
> > Tcl [package require Foo] will work just the same in tclsh as it does
> > in wish.
>
> Yeah, but [package require Tk; package require Foo] will also work
> in wish now, and work the same in both when Tk becomes loadable.
>
I seem to agree with you on most things in principle and look forward
to seeing your practical solutions.
Don Porter
Paul Duffin <pdu...@hursley.ibm.com> wrote:
> Another solution to the problem would be to allow multiple different
> versions of a package to be loaded at once and handle all the possible
> conflicts between them...
Is that possible with binary packages?
Can both libfoo1.0.so and libfoo1.1.so be loaded into the same
process when both of them want to define the routine Foo_Init() ?
I don't think even stubs can avoid a conflict on Foo_Init() can they?
I don't really know what I'm talking about here, so I'm hoping the
Stubs Man can shed some light on the subject.
> I often think that Tcl needs to know a lot more about what a package's
> effects are. ... This also
> means that it is not possible to 'nicely' cleanup the package.
I agree that both having greater introspection about the contents of
a package, and having the ability to cleanup or unload a package
would be nice. Do constraints on what is possible (cross-platform)
with binary packages limit us, though? There's some reason there
is no [unload] to go with [load].
> The 'package' trail is relatively easy to implement (with minimal impact
> on most extensions), the interpreter simply has a stack of 'package'
> markers which are manipulated as follows.
>
> Before [package require Foo 1.0] calls the script registered
> by [package ifneeded Foo 1.0] it pushes a new 'Foo 1.0' marker on
> the stack and pops it off afterwards.
>
> When [proc] is called to create a new command it associates the
> current (top) package marker on the stack with it. So if the
> [package ifneeded] script calls [proc] then the new command is
> associated with 'Foo 1.0'.
>
> Similarly for Tcl_CreateObjCommand....
>
> When a command/proc is called it pushes its marker onto the
> stack before it executes the body and pops it off afterwards.
>
> Similarly for other callbacks into Tcl or C code.
Hmmm. The entangling of [package] with [proc] and Tcl_Create*Command
bothers me. Something in my notion of programming elegance wants the
[package] subsystem to be more modular than that. However, even if the
Tcl core evolves to a tiny kernel which loads other modules, I suppose
[package] would have to be part of the innermost kernel, so there's no
reason not to entangle it with other things in the kernel.
Juan Carlos Gil Montoro
>
> [SNIP]
>
> The operative principle is that the global namespace is the property
> of the application, and no package should place anything there unless
> the application asks it to.
>
>
> This problem is no different in principle from the problem of two
> packages trying to create a command in the global namespace with the
> same name. In both cases it's the responsibility of the packages to
> stay in their own namespaces and the responsibility of the application
> to resolve the conflict. With conflicting commands, the application
> decides which (if either) command gets into the global namespace via
> [namespace import]. It should also be left entirely to the application
> to resolve conflicting package requirements. The way to do that is
> to require the application to load all packages it needs (and all
> packages they need, and so on), and have no packages "recursively" load
> their own dependencies.
>
One of the modules of my Simple Library (which I expect to release next
week) is SimplePackage. This package improves the Tcl facilities for package
loading and version control. It works by each package declaring itself.
This declaration includes such information as the namespaces used by the
package, its exported procedures or the packages it requires, so that this
information can not only be queried later, but also used for other purposes.
As an example, it is possible to obtain the list of exported procedures from
a package plus those exported from all packages it may require, recursively,
before actually modifying other namespaces not owned by it (i.e., before
actually installing the package)
The following is an example from the documentation:
------------------------------------------------------------------------
# Install the package
package require SimplePackage
# Define an ifneeded script for a package which does not declare itself
package ifneeded UndeclaredPackage 1.0 {
package provide UndeclaredPackage 1.0
}
# Define an ifneeded script for a package declaring itself
package ifneeded DeclaredPackage 1.0 {
::Simple::Package::declare DeclaredPackage 1.0 {
{UndeclaredPackage 1.0}
} {
::Declared
} {
::Declared::export
} {
puts {Installing DeclaredPackage 1.0}
set ::Declared::PackageVariable 1
} {
puts { for the first time!}
proc ::Declared::export {} {puts export}
proc ::Declared::public {} {puts public}
} {
puts {Uninstalling DeclaredPackage 1.0}
}
}
# No packages initially installed
# This displays the following:
# installed:
puts "installed: [::Simple::Package::information installed *eclared*]"
# Now require the declared package
::Simple::Package::require DeclaredPackage
# The undeclared package has been installed
# This displays the following:
# installed: {UndeclaredPackage 1.0}
puts "installed: [::Simple::Package::information installed *eclared*]"
# Which procedures are exported by the declared package ?
# This displays the following:
# export: export
puts "export: [::Simple::Package::information export DeclaredPackage]"
# Would those collide with other procedures in the global namespace ?
# This displays the following:
# importcollisions:
puts "importcollisions:\
[::Simple::Package::information importcollisions DeclaredPackage]"
# No collisions, we may safely install the declared package
# This displays the following:
# Installing DeclaredPackage 1.0
# for the first time!
::Simple::Package::install DeclaredPackage
# Increase and display the package variable
# This displays the following:
# 2
puts [incr ::Declared::PackageVariable]
# Into which namespaces has the declared package been imported ?
# This displays the following:
# importedinto: ::
puts "importedinto:\
[::Simple::Package::information importedinto DeclaredPackage]"
# Which procedures are provided by the declared package ?
# This displays the following:
# publicandexport: ::Declared::export ::Declared::public
puts "publicandexport:\
[::Simple::Package::information publicandexport DeclaredPackage]"
# Ok, use them
# This displays the following:
# public
::Declared::public
# This displays the following:
# export
export
# We are done, let's get rid of the declared package
# This displays the following:
# Uninstalling DeclaredPackage 1.0
::Simple::Package::unload DeclaredPackage
# The package procedures are no longer available
# This displays the following:
# invalid command name "export"
catch {export} result
puts $result
# The undeclared package remains installed
# This displays the following:
# installed: {UndeclaredPackage 1.0}
puts "installed: [::Simple::Package::information installed *eclared*]"
# We need the declared package again!
# This displays the following:
# Installing DeclaredPackage 1.0
::Simple::Package::load DeclaredPackage
# Verify that the package variable has its default value
# This displays the following:
# 1
puts $::Declared::PackageVariable
------------------------------------------------------------------------
Juan Carlos---
lvi...@cas.org
:(before stubs) and can only successfully be loaded into an 8.0.x
:interpreter, its pkgIndex.tcl file should look like:
This will work in two of three cases:
1. the author knows that the extension will not work with an older extension
2. the author knows that, currently the extension will not work with a newer
extension.
However, if the author wrote the extension before stubs was introduced, then
they can't be faulted for not knowing that an incompatibility was on its
way.
Or is your proposal that an extension always match the exact range of
versions that an author knows, at the time of writing, the software will
work, and then explicitly add new versions to the extension when it is
proven that it works in newer ones? That doesn't sound too bad - except
for all the various extensions that people have abandoned... someone would
need to go in, each release, and test and update the extensions for the
release.
But frankly, it sure would be nice if there were a team of people willing to
test the various extensions and who reported via some database regarding
the compatibility of the various extensions in relationship to the various
releases.
Bob Techentin
>
> One of the modules of my Simple Library (which I expect to release next
> week) is SimplePackage. This package improves the Tcl facilities for package
> loading and version control. It works by each package declaring itself.
> This declaration includes such information as the namespaces used by the
> package, its exported procedures or the packages it requires, so that this
> information can not only be queried later, but also used for other purposes.
> As an example, it is possible to obtain the list of exported procedures from
> a package plus those exported from all packages it may require, recursively,
> before actually modifying other namespaces not owned by it (i.e., before
> actually installing the package)
>
What a great idea! Give the package mechanism the same introspection
capability that exists for most of the rest of Tcl! Then you could
write procs to let packages determine if they are compatible with each
other, instead of relying on an ever-more-complex and ever-more-subtle
hardcoded packaging mechanism!!
Bob
--
Bob Techentin techenti...@mayo.edu
Mayo Foundation (507) 284-2702
Rochester MN, 55905 USA http://www.mayo.edu/sppdg/sppdg_home_page.html
Paul Duffin
> > versions of a package to be loaded at once and handle all the possible
> > conflicts between them...
>
>
> Can both libfoo1.0.so and libfoo1.1.so be loaded into the same
> process when both of them want to define the routine Foo_Init() ?
>
Yes, they can (or at least on any sensible operating system they should)
work because Tcl looks for the Foo_Init function within the particular
object that it has loaded. e.g.
handle = dlopen (library);
initFn = dlsym (handle, "Foo_Init");
There are certainly a lot of problems to solve when thinking about
multiple versions of the same package being loaded but this is not one of
them. At least I don't think so. Other problems are....
Clashes between names of things.
Namespaces.
Tcl object types.
Keys used in AssocData.
:
:
> > I often think that Tcl needs to know a lot more about what a package's
> > effects are. ... This also
> > means that it is not possible to 'nicely' cleanup the package.
>
> I agree that both having greater introspection about the contents of
> a package, and having the ability to cleanup or unload a package
> would be nice. Do constraints on what is possible (cross-platform)
> with binary packages limit us, though? There's some reason there
> is no [unload] to go with [load].
>
Scriptics did recently add code to unload libraries when Tcl was closed
down, I presume to allow an application to clean up an embedded Tcl
properly without exiting the application. Unfortunately this hit a
major problem in that many extensions, especially C++ ones would cause
Tcl to crash on exit because the extensions where registering atexit
handlers to cleanup the code which obviously did not work if the object
which contained the handler was unloaded prior to exit.
Note this was not necessarily the fault of the extension writers but of the
C++ compilers and linkers which use atexit to cleanup file scope objects
and writers of other libraries which the extensions interface to.
> > The 'package' trail is relatively easy to implement (with minimal impact
> > on most extensions), the interpreter simply has a stack of 'package'
> > markers which are manipulated as follows.
> >
> > Before [package require Foo 1.0] calls the script registered
> > by [package ifneeded Foo 1.0] it pushes a new 'Foo 1.0' marker on
> > the stack and pops it off afterwards.
> >
> > When [proc] is called to create a new command it associates the
> > current (top) package marker on the stack with it. So if the
> > [package ifneeded] script calls [proc] then the new command is
> > associated with 'Foo 1.0'.
> >
> > Similarly for Tcl_CreateObjCommand....
> >
> > When a command/proc is called it pushes its marker onto the
> > stack before it executes the body and pops it off afterwards.
> >
> > Similarly for other callbacks into Tcl or C code.
>
> Hmmm. The entangling of [package] with [proc] and Tcl_Create*Command
> bothers me. Something in my notion of programming elegance wants the
> [package] subsystem to be more modular than that. However, even if the
> Tcl core evolves to a tiny kernel which loads other modules, I suppose
> [package] would have to be part of the innermost kernel, so there's no
> reason not to entangle it with other things in the kernel.
>
The description I gave above was an example of how to determine the
'ownership' of 'objects' within a Tcl application and therefore is
not just restricted to [proc] s and commands but to other things as
well such as namespaces, Tk canvas items, interpreters, Tcl object
types, ... The list goes on and on and is not just restricted to 'objects'
in the Tcl code but in extensions as well.
This is very similar to the ownership that a process has over things
created in its namespace.
Donal K. Fellows
Don Porter <d...@cam.nist.gov> wrote:
> Can both libfoo1.0.so and libfoo1.1.so be loaded into the same
> process when both of them want to define the routine Foo_Init() ?
On Unix (well, on Solaris and SunOS for sure) the answer is yes. This
is because each dynamically loaded object file has its *own* namespace
for C functions. I've even, with the aid of a hard link IIRC (or
maybe it was use of cp that caused this) had the same library loaded
twice into the same executable, running completely independently. It
made things much more difficult to debug, I can tell you...
Donal.
--
Donal K. Fellows http://www.cs.man.ac.uk/~fellowsd/ fell...@cs.man.ac.uk
-- The small advantage of not having California being part of my country would
be overweighed by having California as a heavily-armed rabid weasel on our
borders. -- David Parsons <o r c @ p e l l . p o r t l a n d . o r . u s>
Jeffrey Hobbs
> Don Porter wrote:
> > a package, and having the ability to cleanup or unload a package
> > would be nice. Do constraints on what is possible (cross-platform)
> > with binary packages limit us, though? There's some reason there
> > is no [unload] to go with [load].
>
> Scriptics did recently add code to unload libraries when Tcl was closed
> down, I presume to allow an application to clean up an embedded Tcl
> properly without exiting the application. Unfortunately this hit a
> major problem in that many extensions, especially C++ ones would cause
> Tcl to crash on exit because the extensions where registering atexit
> handlers to cleanup the code which obviously did not work if the object
> which contained the handler was unloaded prior to exit.
>
> Note this was not necessarily the fault of the extension writers but of the
> C++ compilers and linkers which use atexit to cleanup file scope objects
> and writers of other libraries which the extensions interface to.
Since 8.2.3 the libraries have actually not been unloaded explicitly,
due to problems as noted above (where 'atexit' may be used). For 8.4,
we are looking to overhaul the load command to allow the user to hint
for us (in this case the extension writer), that a library shouldn't
be unloaded. It'd also be nice to have more generic dll access (a la
ffidl or dllcaller).
--
Jeffrey Hobbs The Tcl Guy
jeffrey.hobbs at scriptics.com Scriptics Corp.
Paul Duffin
>
> Paul Duffin wrote:
> > Don Porter wrote:
> > > a package, and having the ability to cleanup or unload a package
> > > would be nice. Do constraints on what is possible (cross-platform)
> > > with binary packages limit us, though? There's some reason there
> > > is no [unload] to go with [load].
> >
> > Scriptics did recently add code to unload libraries when Tcl was closed
> > down, I presume to allow an application to clean up an embedded Tcl
> > properly without exiting the application. Unfortunately this hit a
> > major problem in that many extensions, especially C++ ones would cause
> > Tcl to crash on exit because the extensions where registering atexit
> > handlers to cleanup the code which obviously did not work if the object
> > which contained the handler was unloaded prior to exit.
> >
> > Note this was not necessarily the fault of the extension writers but of the
> > C++ compilers and linkers which use atexit to cleanup file scope objects
> > and writers of other libraries which the extensions interface to.
>
> Since 8.2.3 the libraries have actually not been unloaded explicitly,
> due to problems as noted above (where 'atexit' may be used). For 8.4,
> we are looking to overhaul the load command to allow the user to hint
> for us (in this case the extension writer), that a library shouldn't
Shouldn't it be the other way around, unload only if the extension writer
provides the hint.
> be unloaded. It'd also be nice to have more generic dll access (a la
> ffidl or dllcaller).
>
It most certainly would.