Proposed improvement to package loading
John Ousterhout
One of the problems with the current Tcl package loader use that it's
hard to make a new package available to a collection of users: either
everyone has to modify their TCLLIBPATH environment variable to include
the new package's directory, or the new package has to share an
existing directory that's already in auto_path or TCLLIBPATH, and this
means merging pkgIndex.tcl files (and unmerging them if you want to
uninstall the package).
I propose to change the package loader for Tcl 7.6 so that it searches
not only the directories in auto_path but also their immediate subdirectories.
The idea is to have a single directory for installing packages, with each
package installed in its own subdirectory. That way, once everyone has the
toplevel directory in their auto_path, you can install a new package simply
by creating a new subdirectory for the package. You can uninstall a package
by removing its subdirectory.
I've implemented this change in our sources and I'm including a patch below
that you can apply to the Tcl 7.6 version of init.tcl to make the change.
I'd appreciate it if some of you could try this out and let me know if
either (a) you find that it doesn't work as advertised or (b) you think
this approach is a bad idea. In the latter case, please post your
comments on comp.lang.tcl for discussion. If I don't hear about any
significant problems, this will be in the Tcl 7.6 final release (coming
up in a few weeks).
------- init.tcl -------
*** /tmp/da006vt Wed Dec 31 16:00:00 1969
--- init.tcl Tue Sep 17 14:45:06 1996
***************
*** 523,530 ****
# tclPkgUnknown --
# This procedure provides the default for the "package unknown" function.
# It is invoked when a package that's needed can't be found. It scans
! # the auto_path directories looking for pkgIndex.tcl files and sources any
! # such files that are found to setup the package database.
#
# Arguments:
# name - Name of desired package. Not used.
--- 523,531 ----
# tclPkgUnknown --
# This procedure provides the default for the "package unknown" function.
# It is invoked when a package that's needed can't be found. It scans
! # the auto_path directories and their immediate children looking for
! # pkgIndex.tcl files and sources any such files that are found to setup
! # the package database.
#
# Arguments:
# name - Name of desired package. Not used.
***************
*** 541,547 ****
--- 542,556 ----
set dir [lindex $auto_path $i]
set file [file join $dir pkgIndex.tcl]
if [file readable $file] {
+ puts "reading $file"
source $file
}
+ foreach file [glob -nocomplain [file join $dir * pkgIndex.tcl]] {
+ if [file readable $file] {
+ set dir [file dirname $file]
+ puts "reading $file"
+ source $file
+ }
+ }
}
}
Jeffrey Hobbs
In article <51n7of$k...@engnews2.Eng.Sun.COM>,
John Ousterhout <ous...@tcl.eng.sun.com> wrote:
>I propose to change the package loader for Tcl 7.6 so that it searches
>not only the directories in auto_path but also their immediate subdirectories.
This is an excellent idea, but I think that it should go one step further
and go fully recursive into the subdirectories. Back before packages, I
used a recursive auto_mkindex procedure to allow me to have to add only
one path to auto_path (as you suggest), but from that directory I could
subdivide as much and as often as I wanted.
For example, take an upcoming package like VerTcl. That should take up a
directory in itself in this top-level directory. However, it will be
comprised of the packages TkCon, TkEdit, TkWWW and more. In turn, TkCon
will have the packages TkConInspect, TkConUtil, etc...
At some point, I can have sibling packages in one directory, but for large
applications, I can see the desire to have a deeper nesting of directories.
I suppose it's possible for a package to figure out any related directories
by capturing [info script] and doing the leg-work itself, but I believe that
using a recursive pkgIndex.tcl directory search method is better.
--
Jeffrey Hobbs office: 541.683.7891
Nomad of the 'Net email: jho...@cs.uoregon.edu
URL: http://www.cs.uoregon.edu/~jhobbs/
John Ousterhout
The patch I sent in my original posting had some debugging statements left
in it; here's a cleaner version that doesn't have them:
------- init.tcl -------
*** /tmp/da006zE Wed Dec 31 16:00:00 1969
--- init.tcl Tue Sep 17 15:23:28 1996
***************
*** 523,530 ****
# tclPkgUnknown --
# This procedure provides the default for the "package unknown" function.
# It is invoked when a package that's needed can't be found. It scans
! # the auto_path directories looking for pkgIndex.tcl files and sources any
! # such files that are found to setup the package database.
#
# Arguments:
# name - Name of desired package. Not used.
--- 523,531 ----
# tclPkgUnknown --
# This procedure provides the default for the "package unknown" function.
# It is invoked when a package that's needed can't be found. It scans
! # the auto_path directories and their immediate children looking for
! # pkgIndex.tcl files and sources any such files that are found to setup
! # the package database.
#
# Arguments:
# name - Name of desired package. Not used.
***************
*** 543,547 ****
--- 544,554 ----
if [file readable $file] {
source $file
}
+ foreach file [glob -nocomplain [file join $dir * pkgIndex.tcl]] {
+ if [file readable $file] {
+ set dir [file dirname $file]
+ source $file
+ }
+ }
}
}
Juergen Schoenwaelder
John Ousterhout (ous...@tcl.eng.sun.com) wrote:
: I propose to change the package loader for Tcl 7.6 so that it searches
: not only the directories in auto_path but also their immediate subdirectories.
: The idea is to have a single directory for installing packages, with each
: package installed in its own subdirectory. That way, once everyone has the
: toplevel directory in their auto_path, you can install a new package simply
: by creating a new subdirectory for the package. You can uninstall a package
: by removing its subdirectory.
The problem with this approach is that you have to install packages in
the [info library] directory (because that is the only place which is in
everyones auto_path). This means that packages are automatically bound to
the Tcl version. Lets assume you install a package for Tcl 7.6 in [info
library] of Tcl version 7.6. Now you upgrade to Tcl 7.7. You can't use
a package installed for Tcl version 7.6 without at least creating symbolic
links, but most likely to re-install the packages. I therefore propose a
scheme where tclPkgUnknown does not look for pkgIndex.tcl files in the
subdirectories of auto_path, but in directories located next to the
directories in auto_path. So the general setup would look like
.../lib/tcl7.5
.../lib/tcl7.6
.../lib/tk4.1
.../lib/tk4.2
.../lib/foo1.3
.../lib/foo1.4
.../lib/bar2.8
which allows to use a package with different versions of Tcl and to use
different versions of a package with a single version of Tcl. Below
is a patch which implements this scheme and which I have sent to this
list a few weeks ago. (It is possible to make some minor improvements
but I am using this version on UNIX and Windows platforms for some
time now.)
Note, I don't think we need recursion to handle large packages which
consist of multiple smaller packages because you can adjust auto_path
as required once the top-level package is found by Tcl. Such a
recursion would create some significant overhead if you have a couple
of packages that install their of sub-directories but do not make
use of such a feature.
Juergen
*** library/init.tcl.orig Mon Aug 26 22:14:32 1996
--- library/init.tcl Thu Sep 5 08:50:37 1996
***************
*** 542,547 ****
--- 546,575 ----
set file [file join $dir pkgIndex.tcl]
if [file readable $file] {
source $file
+ }
+ }
+
+ # Look for pkgIndex.tcl files that are installed in a directory
+ # which is next to the Tcl library directory and called like the
+ # package we are looking for. This allows extension writers to
+ # install packages whithout modifying the Tcl init.tcl script (to
+ # adapt the auto_path) or requiring users to set the TCLLIBPATH
+ # variable.
+
+ set dir [file dirname [info library]]
+ set pkg [string tolower $name$version]
+ set file [file join $dir $pkg pkgIndex.tcl]
+ if [file readable $file] {
+ source $file
+ }
+ if {[string compare $exact "-exact"] != 0} {
+ if {! [catch {glob [file join $dir $pkg*]} dlist]} {
+ foreach dir $dlist {
+ set file [file join $dir pkgIndex.tcl]
+ if [file readable $file] {
+ source $file
+ }
+ }
}
}
}
--
Juergen Schoenwaelder sch...@cs.utwente.nl http://www.cs.utwente.nl/~schoenw
Computer Science Department, University of Twente, (Fax: +31-53-489-3247)
P.O. Box 217, NL-7500 AE Enschede, The Netherlands. (Tel. +31-53-489-3678)
Eric Vought
> >I propose to change the package loader for Tcl 7.6 so that it searches
> >not only the directories in auto_path but also their immediate subdirectories.
>
> and go fully recursive into the subdirectories. Back before packages, I
> used a recursive auto_mkindex procedure to allow me to have to add only
> one path to auto_path (as you suggest), but from that directory I could
> subdivide as much and as often as I wanted.
I actually don't like this idea from a modularization standpoint. Going
into the first set of subdirectories is fine, but going fully recursive
exposes internal packages to the outside world. What might be better is
for each package in a main subdirectory to add its own directory to the
autopath so that its immediate subpackages become available to it (as
the unknown procedure looks one directory down), but internal packages
aren't exposed to other, unrelated packages. One of my reasons for
concern is in naming- if you recurse only one directory down and only
add other trees as needed, then you don't have to have universally
unique package names, only mostly unique package names. I have software
projects with literally dozens of subpackages and a handful of
collections of subpackages. The chances of being able to come up with
universally unique package names that fit in a single line of text are
slim to none- I can come up with mostly unique names however.
The way this works in my project, is by creating a package to setup the
autopath for the packages it contains. So, I'll have a "CVS" package,
which, when loaded, (the pkgindex directly sources the file), adds its
subdirectory to the autopath. The "CVS" package contains sub-packages
for manipulating CVS from an application. Subpackages include
"CVS_Commands", etc. These only become available when the "CVS" package
is loaded. With judicious use of multiple interpreters, they only become
available to the packages that require the cvs packages. The proposal to
automatically recurse through the entire tree would seriously torpedo
this technique.
What might solve my criticism is some way to make packages which are
entirely internal and whose names are known only to the owning package.
That would seem to entail a major
change.
Bruce Stephens
>>>>> "Jeffrey" == Jeffrey Hobbs <jho...@cs.uoregon.edu> writes:
> For example, take an upcoming package like VerTcl. That should take
> up a directory in itself in this top-level directory. However, it
> will be comprised of the packages TkCon, TkEdit, TkWWW and more. In
> turn, TkCon will have the packages TkConInspect, TkConUtil, etc...
Shouldn't at least some of these be at the top level? TkWWW for
example. Otherwise there's a danger that I'll have several packages,
each of which includes its own copy of TkWWW.
How about just making it easier for packages (when they know they need
to) to include subdirectories?
Extending glob, so it allows an extra couple of patterns, would surely
do this: add the zsh pattern "**" to match any number of directories,
and perhaps the terminating "/" which matches only directories. So I
could write "glob /fred/**/*.tcl" to get all tcl files anywhere under
/fred/, and I could write "glob /fred/**/" to get all directories
under /fred/. Zsh has many, many, other patterns, but the only other
one that's potentially relevant is "***" which also traverses symbolic
links to directories: this is more tricky to implement, though,
because you have to consider the possibility of loops.
--
Bruce Stephens | email: B.Ste...@math.ruu.nl
Utrecht University | telephone: +31 30 2534630
Department of Mathematics | telefax: +31 30 2518394
P.O. Box 80010, 3508 TA Utrecht, The Netherlands
Jeffrey Hobbs
>> Jeffrey Hobbs <jho...@cs.uoregon.edu> writes:
>> This is an excellent idea, but I think that it should go one step further
>> and go fully recursive into the subdirectories. Back before packages, I
>> For example, take an upcoming package like VerTcl. That should take
>> up a directory in itself in this top-level directory. However, it
>> will be comprised of the packages TkCon, TkEdit, TkWWW and more. In
Eric Vought <ad...@ids2.idsonline.com> wrote:
>I actually don't like this idea from a modularization standpoint. Going
Bruce Stephens <step...@math.ruu.nl> wrote:
>Shouldn't at least some of these be at the top level? TkWWW for
>example. Otherwise there's a danger that I'll have several packages,
>each of which includes its own copy of TkWWW.
Giving it another 2 secs thought, I agree on both points. However, the
'modularization' point made by Eric isn't entirely valid because avoiding
naming conflicts will always be a major concern until we have namespaces
(although they won't go away entirely then either) which is what you hinted
at in your last paragraph. However, I don't think there will be much worry
about a plethora of package names.
Bruce's point just shows that I haven't really reached the point of deciding
how to package up and release everything. I suppose there will be more
top-level packages and I can figure out any sub-packages on my own.
John Ousterhout
In article <51o3gi$h...@pandora.cs.utwente.nl>, sch...@cs.utwente.nl (Juergen Schoenwaelder) writes:
|> John Ousterhout (ous...@tcl.eng.sun.com) wrote:
|>
|> : I propose to change the package loader for Tcl 7.6 so that it searches
|> : not only the directories in auto_path but also their immediate subdirectories.
|> : package installed in its own subdirectory. That way, once everyone has the
|> : toplevel directory in their auto_path, you can install a new package simply
|> : by creating a new subdirectory for the package. You can uninstall a package
|> : by removing its subdirectory.
|>
|> The problem with this approach is that you have to install packages in
|> the [info library] directory (because that is the only place which is in
|> everyones auto_path). This means that packages are automatically bound to
|> the Tcl version.
I think that the real issue here is that auto_path doesn't have enough
directories in it right now; in addition to [info library], which is just
for Tcl scripts, it should also have the toplevel package directory. I
didn't make this change because I haven't figured out what directory should
be used for this, and how to give sites flexibility in choosing it. For
now I assume that people will modify init.tcl to add a site-specific
package directory, or else people will use the TCLLIBPATH environment
variable. Neither of these solutions is ideal for the long term, of course,
but they both have the advantage that you do one-time setup and then don't
have to do any additional reconfiguration when you add or delete packages.
Once we figure out the right way to specify this directory in auto_path
we'll add support for that to init.tcl.
|> I therefore propose a
|> scheme where tclPkgUnknown does not look for pkgIndex.tcl files in the
|> subdirectories of auto_path, but in directories located next to the
|> directories in auto_path.
I'm not wild about this proposal, for a couple of reasons:
1. I think it's trickier to manage, because you may accidentally pickup
packages you don't want just because they happen to be in siblings
of the auto_path directories. Said another way, with my proposal
you only have to worry about what's *in* the auto_path directories
to understand all the package loading issues; with Juergen's proposal
you also have to worry about what's *near* the auto_path directories.
2. The current directory that's in auto_path contains scripts, which
are platform-independent. Packages are more likely to be binary and
hence platform-dependent, which means that in multi-platform sites
they are likely to be in a very different place in the file system.
The right place for this is unlikely to be a sibling of [info library].
Better to name this place explicitly and independently in auto_path.
Christopher C. Wood
In article <323FF45F...@ids2.idsonline.com>, Eric Vought <ad...@ids2.idsonline.com> writes:
|> Jeffrey Hobbs wrote:
|> > In article <51n7of$k...@engnews2.Eng.Sun.COM>,
|> > John Ousterhout <ous...@tcl.eng.sun.com> wrote:
|> > >I propose to change the package loader for Tcl 7.6 so that it
|> > >searches not only the directories in auto_path but also their
|> > >immediate subdirectories.
|> > This is an excellent idea, but I think that it should go one step
|> > further and go fully recursive into the subdirectories. Back
|> > before packages, I used a recursive auto_mkindex procedure to
|> > allow me to have to add only one path to auto_path (as you
|> > suggest), but from that directory I could subdivide as much and
|> > as often as I wanted.
I am wary of a fully-recursive search. It may cycle. What about a
more Macintosh-like approach: have "the system" "know" where packages
are. Most systems don't have necessary file-system hooks to do that
automatically, so you'd have to maintain the database manually. It
could be as simple as a file in the same directory as tclsh/wish (or
identified by TCL_LIBPATH, or auto_path) that has path names to
packages.
I'm not clear on how any of these proposals would make packages available
to some users, but not others. This could be my own
(mis)interpretation of John's original proposal.
|> I actually don't like this idea from a modularization standpoint. Going
|> into the first set of subdirectories is fine, but going fully recursive
|> exposes internal packages to the outside world. What might be better is
|> for each package in a main subdirectory to add its own directory to the
|> autopath so that its immediate subpackages become available to it (as
|> the unknown procedure looks one directory down), but internal packages
|> aren't exposed to other, unrelated packages.
I like this idea as well.
|> One of my reasons for concern is in naming- if you recurse only one
|> directory down and only add other trees as needed, then you don't
|> have to have universally unique package names, only mostly unique
|> package names. I have software projects with literally dozens of
|> subpackages and a handful of collections of subpackages. The
|> chances of being able to come up with universally unique package
|> names that fit in a single line of text are slim to none- I can
|> come up with mostly unique names however.
This does become a scalability/management issue. On the other hand,
hierarchical naming schemes have tackled the naming issue before.
|> The way this works in my project, is by creating a package to setup the
|> autopath for the packages it contains. So, I'll have a "CVS" package,
|> which, when loaded, (the pkgindex directly sources the file), adds its
|> subdirectory to the autopath. The "CVS" package contains sub-packages
|> for manipulating CVS from an application. Subpackages include
|> "CVS_Commands", etc. These only become available when the "CVS" package
|> is loaded. With judicious use of multiple interpreters, they only become
|> available to the packages that require the cvs packages. The proposal to
|> automatically recurse through the entire tree would seriously torpedo
|> this technique.
|> What might solve my criticism is some way to make packages which
|> are entirely internal and whose names are known only to the owning
|> package. That would seem to entail a major change.
Chris
--
Speaking only for myself, of course.
Chris Wood chr...@lexis-nexis.com ca...@CFAnet.com
Juergen Schoenwaelder
John Ousterhout (ous...@tcl.eng.sun.com) wrote:
: I'm not wild about this proposal, for a couple of reasons:
: 1. I think it's trickier to manage, because you may accidentally pickup
: packages you don't want just because they happen to be in siblings
: of the auto_path directories. Said another way, with my proposal
: you only have to worry about what's *in* the auto_path directories
: to understand all the package loading issues; with Juergen's proposal
: you also have to worry about what's *near* the auto_path directories.
Sorry, my description was not 100% correct. My patch only looks for
directories next to [info library] and nowhere else. It does not change
the way auto_path is used to find packages. So you only have to know
what is *near* [info library] and you do not need to worry what is
*in* all the directories listed in your auto_path. So it becomes even
simpler to manage the packages available on a system.
: 2. The current directory that's in auto_path contains scripts, which
: are platform-independent. Packages are more likely to be binary and
: hence platform-dependent, which means that in multi-platform sites
: they are likely to be in a very different place in the file system.
: The right place for this is unlikely to be a sibling of [info library].
: Better to name this place explicitly and independently in auto_path.
The directory only has to contain the pkgIndex.tcl file. It is possible
to keep this file platform-independent if the platform-dependent files
are installed in a 'reasonable' way and if the installation procedure is
smart enough. Note, your solution does not address this issue either.
It basically comes down to the question if you want to decouple the
version of a package from the Tcl version or not. I would like to
see a solution that has this property. A solution that does not have
this property removes much of the power of the versioning system
introduced by the package command. (I don't care much if you pick
up the scheme I proposed here or if you choose to define a toplevel
package directory. Everything that allows to easily install a package
in a common place so that it is found automatically by a Tcl
interpreter regardless of the version of the Tcl interpreter is fine
with me. And I would like to see such a solution in Tcl 7.6.)
Juergen
J. Adrian Zimmer
Essentially, we're talking about extending the idea of a search path to
include some subdirectories of directories in the path.
Including immediate subdirectories but not the entire subtree is, I think,
a bad compromise between
1) Wanting the ability to have some subdirectories included
2) Fearing the nasty surprises that will surely come when one loses
intellectual control of what's in the search space.
Instead, consider modifying the syntax for the search path so that it is
possible to designate some directories as recursively searchable.
Since the default would be the current search method, nobody would get
burned. Then too, Jeffrey Hobbs would get all the flexibility he wants.
...........................
J Adrian Zimmer jazi...@acm.org
(701) 746-8187 fax: 746-4971
http://www.skillshare.com/skillshare/softdoc.html
Michael Salmon
John Ousterhout wrote:
>
> In article <51o3gi$h...@pandora.cs.utwente.nl>, sch...@cs.utwente.nl (Juergen Schoenwaelder) writes:
> |>
> |> : I propose to change the package loader for Tcl 7.6 so that it searches
> |> : not only the directories in auto_path but also their immediate subdirectories.
> |> : package installed in its own subdirectory. That way, once everyone has the
> |> : toplevel directory in their auto_path, you can install a new package simply
> |> : by creating a new subdirectory for the package. You can uninstall a package
> |> : by removing its subdirectory.
> |>
> |> The problem with this approach is that you have to install packages in
> |> the [info library] directory (because that is the only place which is in
> |> everyones auto_path). This means that packages are automatically bound to
> |> the Tcl version.
>
> I think that the real issue here is that auto_path doesn't have enough
> directories in it right now; in addition to [info library], which is just
> for Tcl scripts, it should also have the toplevel package directory. I
> didn't make this change because I haven't figured out what directory should
> be used for this, and how to give sites flexibility in choosing it. For
> now I assume that people will modify init.tcl to add a site-specific
> package directory, or else people will use the TCLLIBPATH environment
> variable. Neither of these solutions is ideal for the long term, of course,
> but they both have the advantage that you do one-time setup and then don't
> have to do any additional reconfiguration when you add or delete packages.
> Once we figure out the right way to specify this directory in auto_path
> we'll add support for that to init.tcl.
Isn't the easiest way to handle the problem just to make TCLLIBPATH a
:/; separated list. There are already precedents for this in both Unix
and ms-dos. Other operating systems probably require other though
equivalent solutions i.e. a search list. In unix these are simple to
manipulate e.g.:
TCLLIBPATH=<mypath>${TCLLIBPATH+":$TCLLIBPATH"}
in a wrapper script. Lists are also easy for tcl to handle.
Another possibility is to use TCLPKGPATH-<package> and only have it
affect the require proc.
One thing that I believe is very important is that every user should
have control over his or her environment but not over others. It is
nearly as important to have control over usual and test sessions,
environment variables are about the only thing that fit these
requirements.
--
© 1995,1996 Michael Salmon
All opinions expressed in this article remain the property of
Michael Salmon. Permission is hereby granted for use in
followup articles, FAQ's and digests.
Bruce Stephens
>>>>> "Jeffrey" == Jeffrey Hobbs <jho...@cs.uoregon.edu> writes:
> Bruce's point just shows that I haven't really reached the point of
> deciding how to package up and release everything. I suppose there
> will be more top-level packages and I can figure out any
> sub-packages on my own.
I don't see any reason why you shouldn't expect some help in figuring
out the sub-packages, though. On the other hand, probably my
suggestion of extending glob patterns to search recursively is OTT:
typically you'll only have one or two levels of sub-packages, at
most.
Tim MacEachern
>> |> John Ousterhout (ous...@tcl.eng.sun.com) wrote:
>> |>
>> |> : I propose to change the package loader for Tcl 7.6 so that it searches
>> |> : not only the directories in auto_path but also their immediate subdirectories.
>> |> : The idea is to have a single directory for installing packages, with each
>> |> : package installed in its own subdirectory. That way, once everyone has the
>> |> : toplevel directory in their auto_path, you can install a new package simply
>> |> : by creating a new subdirectory for the package. You can uninstall a package
>> |> : by removing its subdirectory.
How about having an optional maximum recursion depth in the path list?
TCL_LIB_PATH=/path1/sub1(99):/path2/sub2(1) ? Giving effectively full
recursion (without an infinite loop possibility) in one and first-level
only in the second. What you lose are parentheses in path names (well,
not really as you could code /p1/p2(xx)/..(4) ).
-------------------------------------------------------------------------
Tim MacEachern |Tim_Mac...@atl.sofkin.ca, Software Specialist.
Software Kinetics Ltd.|Single father of four-year-old Laura, still I
Dartmouth, Nova Scotia|play golf, piano, bridge, read, run ...
Canada |Meet us at http://ccn.cs.dal.ca/~ae721/Profile.html
David S. Cargo
I am in favor of generalizing package loading. I do think it might
be worthwhile to avoid subdirectories that are named RCS. There may
be other directories that arise as a consequence of a development
environment that would make sense to exclude. (I use the emacs
interface to RCS; I don't know what directories might be used by SCCS or
CVS.)
David S. Cargo
J. Adrian Zimmer
John Ousterhout -- When discussing a need for a new way to search for package
directories, said that, as things are, new packages must:
share an existing directory that's already in auto_path or TCLLIBPATH,
and this means merging pkgIndex.tcl files (and unmerging them if you
want to uninstall the package).
John Ousterhout -- When discussing Schoenwaelder's patch:
I think that the real issue here is that auto_path doesn't have enough
directories in it right now ...
Juergen Schoenwaelder:
It basically comes down to the question if you want to decouple the
version of a package from the Tcl version or not. I would like to
see a solution that has this property.
J Adrian Zimmer -- current:
There are two kinds of initializations necessary for Tcl, those pertaining
to the current version and those pertaining to the system in which that
version is installed. Diddling with "init.tcl" to accomplish the second kind
of initialization
1) forces system administraters to fool with something that isn't broken
2) forces system administraters to do it again every time there is
a new tcl version
Seen this way, the obvious solution is to have a second initialization
procedure for local purposes. Another path name in the installation
scripts can locate this procedure. As shipped, that path name would
be empty and the second initialization procedure would not be invoked.
Otherwise, the path name can be set to a locally written procedure that
diddles with auto_path, or whatever.
Having the second initialization procedure enables Tcl versions to be
decoupled from package versions and any customization that the local system
administrator feels are necessary.
We don't need to be thinking about more rigid ways to solve this problem
because we have a powerful language for performing such customizations at
hand.
.....
One problem the second initialization procedure cannot not quite solve
is John's initial problem of installing new packages. This is because it
forces the guy who has just created or obtained a new version of a package
to make a change to the second initialization procedure and re-initialize.
Here is a way that the auto_path mechanism could be extended so that the
auto_path could dynamically change without requiring explicit invocation of a
procedure to make the alteration:
Where Tcl uses auto_path, alter the evaluation from a simple variable
substitution to something more complicated. (As stated so far, this seems to
have been done for DOS systems already.) The new evaluation would run "eval"
on evaluates path elements that are sublists which begin with "eval". The
result would of course be substituted the way command substitutions are.
Other sublists, for example "C:/Program Files/$ForU", would be not be
treated in this new way.
.....
Taken together these suggestions alter Tcl in a way that:
1) preserves it's Tcl-ness
2) does not adversely affect those who have no use for the alterations.
3) provides great flexibility. I think that both the Ousterhout and
the Schoenwaelder patches could be implemented without patching
If I wanted, I could arrange to have a particular directory tree
recursively searched that I could organize package versions in
a natural way.
Other things beyond our current imaginations would be possible because we
are introducing the full power of Tcl in two places it doesn't now exist.
Some will say they don't want the full power of Tcl in their auto_paths. I
don't want dangerous things in mine either, but I like the ability to be able
to make the decison about what is dangerous and what is useful for myself.
Larry W. Virden
John, while this change is taking place, could you also add in the code
to check not only in $prefix/lib but $exec_prefix/lib ? That way folk
who are trying to separate the text libraries from the binary libraries
can do so and not have to mess with environment variables or hard coded
paths?
And extension writers, be sure to install lib*.so or lib*.a type files
as well as a.out type executables, in $exec_prefix/lib and /bin rather
than in $prefix/lib and /bin.
--
:s Larry W. Virden INET: lvi...@cas.org
:s <URL:http://www.teraform.com/%7Elvirden/> <*> O- "We are all Kosh."
:s Unless explicitly stated to the contrary, nothing in this posting should
:s be construed as representing my employer's opinions.
Christophe Muller
Hi,
In article <51p6s8$3...@engnews2.Eng.Sun.COM> ous...@tcl.eng.sun.com (John Ousterhout) writes:
> I think that the real issue here is that auto_path doesn't have enough
> directories in it right now;
Agreed.
> in addition to [info library], which is just
> for Tcl scripts, it should also have the toplevel package directory. I
> didn't make this change because I haven't figured out what directory should
> be used for this,
I prefer your solution to the two other proposals that I have seen so far:
1) recursive search is probably not very useful and certainly time-consuming
2) searching sub-directories is better IMHO than siblings
You are right that Tcl packages (i.e., libraries of TCL code or .so/.sa files
to be loaded), are different from the TCL_LIBRARY. Why not choose at compile
time the TCL_LIBRARY and TCLLIBPATH with possible configure options or
variables?.
(BTW, I don't think packages will be only platform-dependent: TCL code will
probably be platform-independent and .so libraries will be platform-dependent).
What do you think of the following suggestions? Examples are in Unix syntax.
[info library]
- would be specified by --prefix ($(prefix)/lib/tcl$(VERSION))
- default: /usr/local/share/lib/tcl7.5/
- dynamic redefinition: TCL_LIBRARY
platform-independent packages top directory
- would be specified by --prefix ($(prefix)/lib/tcl/)
- default: /usr/local/share/lib/tcl/
- dynamic redefinition: TCLLIBPATH
platform-dependent packages top directory
- would be specified by --exec-prefix ($(exec_prefix)/lib/tcl)
- default: /usr/local/lib/tcl/
- dynamic redefinition: TCLARCHLIBPATH
> |> I therefore propose a
> |> scheme where tclPkgUnknown does not look for pkgIndex.tcl files in the
> |> subdirectories of auto_path, but in directories located next to the
> |> directories in auto_path.
If we completely disconnect the two concepts, as suggested by Dr.O, this is
not useful anymore..
> 2. The current directory that's in auto_path contains scripts, which
> are platform-independent. Packages are more likely to be binary and
> hence platform-dependent, which means that in multi-platform sites
> they are likely to be in a very different place in the file system.
> The right place for this is unlikely to be a sibling of [info library].
> Better to name this place explicitly and independently in auto_path.
I agree that the right place is unlikely to be a sibling of [info library],
but I think three "default" directories would probably be better than only two
in auto_path (see examples above).
My two cents..
Cheers,
Christophe.
= There are three roads to ruin; women, gambling and technicians. The most =
= pleasant is with women, the quickest is with gambling, but the surest is =
= with technicians. -- Georges Pompidou (1911-1974) =
Grant Reaber
In article <51rm5s$c...@news.nstn.ca>,
Tim MacEachern <Tim_Mac...@atl.sofkin.ca> wrote:
>
>How about having an optional maximum recursion depth in the path list?
>TCL_LIB_PATH=/path1/sub1(99):/path2/sub2(1) ? Giving effectively full
>recursion (without an infinite loop possibility) in one and first-level
>only in the second. What you lose are parentheses in path names (well,
>not really as you could code /p1/p2(xx)/..(4) ).
This would make it impossible to write packages that would work correctly on
all systems. I tend to like JO's proposal, but if you decide to allow a
choice, the choice should be the package writer's. So you could have a file
in the first subdirectory called "searchlist" that lists all the directories
to search. Or you could have a naming convention: search directories that
don't end in ".int" (apply recursively).
Grant
Eric Vought
> interface to RCS; I don't know what directories might be used by SCCS or
> CVS.)
Generally, "SCCS" and "CVS". This probably won't be a problem, because
these directories won't have a packageindex file in them (even if they
did, it would be named "pkgIndex.tcl,v" or something like that depending
on which version control software is being used).
Robin Becker
In article <51p2u1$g...@cronkite.polaristel.net>, "J. Adrian Zimmer"
<jazi...@acm.org> writes
>Essentially, we're talking about extending the idea of a search path to
>include some subdirectories of directories in the path.
>
>Including immediate subdirectories but not the entire subtree is, I think,
>a bad compromise between
>
> 1) Wanting the ability to have some subdirectories included
> 2) Fearing the nasty surprises that will surely come when one loses
> intellectual control of what's in the search space.
>
>Instead, consider modifying the syntax for the search path so that it is
>possible to designate some directories as recursively searchable.
>
>Since the default would be the current search method, nobody would get
>burned. Then too, Jeffrey Hobbs would get all the flexibility he wants.
>
>J Adrian Zimmer jazi...@acm.org
>(701) 746-8187 fax: 746-4971
> http://www.skillshare.com/skillshare/softdoc.html
>
if { [file exists [file dirname $tcl_library]/localtcl] } {
lappend auto_path [file dirname $tcl_library]/localtcl
}
which is just a hook to allow localisation; of course this doesn't
address all the problems. Making the search path more complex is a BAD
idea since it's already too complex I guess any tcl programmer gets
bitten at least once. Auto loading itself is the problem not the answer.
The intention of this discussion seems to be to make autoloading more
flexible and safer which IMHO is impossible. Tcl is inherently unsafe
from the language design point of view ie it's impossible to modularise
as any package/proc can rename even the most basic commands; this has
been seen as a strength. Name spaces, more complicated auto_load paths,
version control, packages and the like are an attempt at binding control
which will surely fail unless the core language is changed.
--
Robin Becker
Eric Vought
Reposting article removed by rogue canceller.
Robin Becker
Reposting article removed by rogue canceller.
In article <51p2u1$g...@cronkite.polaristel.net>, "J. Adrian Zimmer"
Jan Nijtmans
I have no objections to the proposal to search for siblings. Only,
it doesn't really help to find packages which are implemented as
a shared library in <exec_prefix>/lib, such as Tk and TclX. At
least this directory should be included in the search. The
disadvantage is indeed that multiple pkgIndex.tcl files
should be merged. But this is easy to do with a tcl-script
(the plus-patches already do that). And if shared libraries
are to be used both in executables (as tcl and wishx) and
as packages, it is not very handy to included all paths to
the LD_LIBRARY_PATH variables.
Here is my proposed tclPkgUnknown function. In addition to JO's
suggestion, it checks a few more directories which are very
usefull.
For example, assume that you are running tclsh7.6 (version b1)
from its build directory. If in your scripts you would execute
"package require Tk 4.2", and you have a pkgIndex.tcl file in
the Tk4.2b1 build directory, it will make an additional search
in the following directories ($tail could be "unix", "mac"
or "win"):
.
../lib (would work if tclsh is in <exec_prefic>/bin)
../../tk4.2b1/$tail (only if patchlevel is a or b)
../../tk4.2/$tail
../../tk/$tail
This makes all sibling packages available to tclsh from the
build directories without the need for adapting TCLLIBPATH
or anything else.
Jan Nijtmans
NICI (Nijmegen Institute of Cognition and Information)
email: nijt...@nici.kun.nl
url: http://www.cogsci.kun.nl/~nijtmans/
B.T.W. It is better to do "catch {source ...}" than checking
for the file to be readable first. This makes the
tclPkgUnknown function usable with the Nescape plugin
too, because there "file readable" is not available.
=============================================================
proc tclPkgUnknown {name version {exact {}}} {
global auto_path tcl_version
if [catch {set dirlist $auto_path}] {
set dirlist ""
}
for {set i [expr [llength $dirlist] - 1]} {$i >= 0} {incr i -1} {
set dir [lindex $dirlist $i]
set file [file join $dir pkgIndex.tcl]
catch {source $file}
foreach file [glob -nocomplain [file join $dir * pkgIndex.tcl]] {
set dir [file dirname $file]
catch {source $file}
}
}
set dir [file dirname [info nameofexecutable]]
set dirlist [list $dir]
lappend dirlist [file join [file dirname $dir] lib]
set tail [file tail $dir]
set dir [file join [file dirname [file dirname $dir]] "[string
tolower $name]"]
if [regexp {[ab][12345]} [info patchlevel] patch] {
lappend dirlist [file join $dir$version$patch $tail]
}
lappend dirlist [file join $dir$version $tail]
lappend dirlist [file join $dir $tail]
foreach dir $dirlist {
set file [file join $dir pkgIndex.tcl]
catch {source $file}
if {[lsearch $name [package names]] >= 0} {
break;
}
}
}
Heribert Dahms
In <51v726$t...@anubis.reed.edu> gre...@reed.edu (Grant) writes:
: Or you could have a naming convention: search directories that
: don't end in ".int" (apply recursively).
That won't work on VMS where "." is the directory separator (which are
always named *.dir;1).
Anybody knows about a platform where appending "_int" does *not* work?
Bye, Heribert (da...@ifk20.mach.uni-karlsruhe.de)
Maurice DIAMANTINI
In article (Dans l'article) <51n7of$k...@engnews2.Eng.Sun.COM>,
ous...@tcl.eng.sun.com (John Ousterhout) wrote (écrivait) :
>I propose to change the package loader for Tcl 7.6 so that it searches
>not only the directories in auto_path but also their immediate subdirectories.
>The idea is to have a single directory for installing packages, with each
>package installed in its own subdirectory. That way, once everyone has the
>toplevel directory in their auto_path, you can install a new package simply
>by creating a new subdirectory for the package. You can uninstall a package
>by removing its subdirectory.
I like this idee, but you should also provide a mean to know in which
order the subdirectories of the lib path are scanned.
- alphabetic order?
- metaTclIndex file where one could record all subdirectories in an
arbitrary order?
--
Maurice DIAMANTINI - di...@ensta.fr
John Ousterhout
Right now the order is fairly arbitrary (whatever glob returns). I think
I'd rather keep it this way since (a) it's not clear that any particular
order is better than any other and (b) I'd like to keep the mechanism
as simple as possible. I think that subdirectories should be used for
different packages or different versions of the same package; in this
case it doesn't matter what order they are read in since none of the
index information will overwrite any other information. If there are
alternative implementations of the same version of the same package, so
that index information can only be retained for one of them, then the
alternative implementations should be in very different places so that
a single entry in auto_path can't cause both to be indexed. In this
case you can use auto_path to select which one you want. I think this
is a pretty uncommon case anyway.
By the way, I'm also leaning against the various proposals for extending
the search to be either more inclusive (e.g. full recursive search) or less
inclusive (e.g. exclude certain subdirectories). All of these proposals
add complexity, and I think that the same effect can be achieved by other
techniques (such as computing your own auto_path at start-up or putting
additional scripts in pkgIndex.tcl files that source additional index
files). I'd like to keep the basic mechanism simple (handle the 90%
of the cases that are most common) and let people use extension
mechanisms to handle the less common cases.
The one change I do plan to make is to add one additional directory to
auto_path, namely the directory where libtcl.* gets installed (e.g.
/usr/local/lib). With this scheme, if you install packages in subdirectories
of that directory, they will automatically be picked up by Tcl.