Hi Charles,
Tcl has a library of "common" functions and utilities known as
"tcllib". While it's not part of the "core" of tcl, it's readily
available and easy to load and use. Among (many) other things, that
library contains a module named "fileutil", which in turn contains
"fileutil::find" - which is a Tcl variation of the Unix "find"
command.
That said, the Tcl core does contain *many* file (and filesystem)
related commands, most of which are collected under the "file"
ensemble command. So, check out "file" and its many sub-commands in
the standard Tcl man-pages for most of the built-in file support, and
take a look at the "fileutil" module of tcllib for other (mostly)
higher-level file utilities.
The tcllib library ships as a standard part of the ActiveState
ActiveTcl distribution, so that's likely the best place to get
everything all at once. Or, if you'd like, you can download the
library separately from http://tcllib.sourceforget.net .
Jeff
If you are doing sysadmin tasks, you will want to add the Tcllib and
some of the other Tcl extensions to your toolbox, especially Tclx,
Expect and maybe some others...
Michael
Jeff and Michael have made some good suggestions. I'd just like to
remind you that the "find" command is not normall part of your shell.
So just do in Tcl what you usually do in your shell script: execute the
find command!
# assigning result to a variable:
set x [exec find]
# parse result into a proper list and assign it to a variable:
set y [split [exec find] \n]
Generally, unless you're on a system where the regular unix commands
are completely built-in to the shell and not available as executables
(and I know some embedded systems where this is indeed the case), Tcl
can do anything shell scripts can do with a much cleaner syntax and a
much more powerful substitution engine. Not to mention that you also
get to use things which you normally can't do in a regular shell script
like opening a socket etc..
My suggestion is when in a hurry and you need to get the job done then
there is nothing wrong with using Tcl simply as a glue language to
execute regular Unix commands like the "find" example above. As you get
more comfortable with the Tcl core functions and extensions (you really
should install tcllib at the very least) you'll find the built in
functions much easier to work with and in some cases much more
powerful. But exec (and open) is your friend - use them to make the
learning curve as gentle as possible.
> Jeff and Michael have made some good suggestions. I'd just like to
> remind you that the "find" command is not normall part of your shell.
> So just do in Tcl what you usually do in your shell script: execute the
> find command!
>
> # assigning result to a variable:
> set x [exec find]
>
> # parse result into a proper list and assign it to a variable:
> set y [split [exec find] \n]
>
> Generally, unless you're on a system where the regular unix commands
> are completely built-in to the shell and not available as executables
> (and I know some embedded systems where this is indeed the case), Tcl
> can do anything shell scripts can do with a much cleaner syntax and a
> much more powerful substitution engine. Not to mention that you also
> get to use things which you normally can't do in a regular shell script
> like opening a socket etc..
One wart is that if you [exec] a command that writes to stderr you'll need
to [catch] it since it'll return an error when exec returns, though Tip
#267, if accepted, would remedy this. (See:
http://www.tcl.tk/cgi-bin/tct/tip/267.html)
That said, I agree wholeheartedly with using Tcl "as a glue language to
execute regular Unix commands."
> My suggestion is when in a hurry and you need to get the job done then
> there is nothing wrong with using Tcl simply as a glue language to
> execute regular Unix commands like the "find" example above. As you get
> more comfortable with the Tcl core functions and extensions (you really
> should install tcllib at the very least) you'll find the built in
> functions much easier to work with and in some cases much more
> powerful. But exec (and open) is your friend - use them to make the
> learning curve as gentle as possible.
Michael
You've gotten some good advice already, I'd just like to add a command that
hasn't been mentioned yet, [glob]. This command does pattern expansion
against the filesystem. In the GUI-based help systems, it is filed under
"System Related" commands, not under "Output". It's not a total find
solution, but it is probably an important part if you choose to roll your
own.
--
Rich Wurth / rwu...@att.net / Rumson, NJ USA
Unlike using the Tcl functionality to do this, this command fails on
directories that have files with embedded newlines in their names. Much
better to use the correct Tcl commands.
Of course, if you control what's in the directory, that's less of a problem.
--
Darren New / San Diego, CA, USA (PST)
This octopus isn't tasty. Too many
tentacles, not enough chops.
uwe
Speaking of which, that title ought to be "Input/Output" but the
embedded / causes problems with filesystems and the extractor is
stupid... :-)
Donal.
I'm using cygwin with bash console interface (X-windows not installed),
mostly for fortran console applications. The cygwin distribution of
tcl/tk (tk seems to work even without X-windows!) does not include
tcllib, so I got the sourceforge distribution and followed the
installation instructions. However, the tutorials I have found do not
explain how to load and invoke packages. What is the command syntax to
test my tcllib installation by invoking, for example, fileutil::cat
under tclsh?
My current incentive to look for alternatives to the unix shell is that
my old scripts fail when filenames contain spaces, and fixups like
-print0 above are not automatic, obvious or readable. I don't like or
use filenames with spaces, which destroy the elegant simplicity of the
unix command line, but it is no longer possible to avoid them. It would
be nice if the tcl counterparts were simpler and more transparent than
the unix commands.
see the man page for package, but it is simply
package require fileutil
fileutil::cat $file
Also, if you look at the docs for fileuitl (or any of the tcllib stuff)
is should show the packlage rquire needed for the commands in the synopsis.
An online locatin of the docs is hosted by activestate here:
http://aspn.activestate.com/ASPN/docs/ActiveTcl/8.4/tcllib/fileutil/fileutil.html
Bruce
Did you also download the Tcl documentation package from activestate?
Arnulf
Thanks. I evidently have a corrupted installation or configuration:
$ tclsh
% package require fileutil
can't find package fileutil
I need tcllib to run under cygwin, which is not one of the platforms
supported by activestate. tcl/tk is running fine, so the fix is
probably simple, but I am an absolute beginner, so I'll try to get help
from the cygwin mailing list.
> Also, if you look at the docs for fileuitl (or any of the tcllib stuff)
> is should show the packlage rquire needed for the commands in the synopsis.
>
> An online locatin of the docs is hosted by activestate here:
>
> http://aspn.activestate.com/ASPN/docs/ActiveTcl/8.4/tcllib/fileutil/fileutil.html
Manpages are not the easiest place for a beginner to start. The
terminology is unfamiliar and there are few if any examples. Can you
suggest a reference, preferably online, with simple examples of invoking
code from an external library?
Activestate does not appear to support cygwin, but I have a functioning
tcl/tk distribution from cygwin. I have tcl and tcllib documentation
from Sourceforge and several tcl tutorials from the web. Unfortunately,
none of the tutorials seem to cover use of external libraries, and it is
hard for a beginner to find and understand the relevant parts of the
manpages.
I don't think you have a corrupted installation, simply you don't have
tcllib installed.
> I need tcllib to run under cygwin, which is not one of the platforms
> supported by activestate. tcl/tk is running fine, so the fix is
> probably simple, but I am an absolute beginner, so I'll try to get help
> from the cygwin mailing list.
Normally on windows I'd strongly recommend running Tcl native by
installing the Activestate distro but I guess you have good reasons for
using cygwin. In which case you can download Tcllib from sourceforge:
http://tcllib.sourceforge.net/
To install tcllib simply untar it and do the usual ./configure; make
install (or run the installer.tcl script, in any case read the
INSTALL.txt file). You may need to change --prefix if you have tcl
installed in strange places.
Unix-style man pages suck horridly (you have to already know something
in order to look for it). Gnu info pages are not much better. But it
really does tcl/tk a terrible injustice to call its documentation
"manpages": they're written in html (http://www.tcl.tk ->
Documentation), so you can browse them effectively. They're pretty
well crosslinked, too.
But what surprises me is that the wiki hasn't been emphasized in this
thread. It's full of examples, explanations, neat tricks, workarounds,
and all other sorts of wonderful resources. And it's searchable on your
own keywords, both in the titles and in the text. Give it a try!
John Perry
Charles,
Here are the steps I took:
I unzipped the file into my cygwin directory, creating /tcllib-1.8
Opened a cygwin window.
Changed the current directory to /tcllib-1.8
Started tclsh
Entered source installer.tcl
Un-iconized the new window created by the installer.
Clicked on Install
(I have X in my cygwin, so the details of the last two steps may differ)
Watched a lot of stuff getting installed
Started a new tclsh
Typed package require fileutil
Got the response 1.8
HTH,
Gerry
Actually, the Tcl/Tk documentation is written in nroff/troff and there's
a Tcl script that converts them into HTML. Much other documentation
(e.g. for tcllib) is written using a tcllib package called doctools. In
fact, the only major documentation written directly in HTML is the
tutorial, and that's still something of a work-in-progress.
But if there's anything that you think is inadequately documented,
please let us know! We can help! We're just usually too close to the
trees to see the wood, if you know what I mean, so the perspectives (and
questions) of a newcomer are very valuable for making things better.
Donal.
>
> But if there's anything that you think is inadequately documented,
> please let us know! We can help! We're just usually too close to the
> trees to see the wood, if you know what I mean, so the perspectives (and
> questions) of a newcomer are very valuable for making things better.
I found the explanation of environment variables unhelpful on the topic
of how to tell tcl where to look for the tcllib files. There seems to be
only one relevant shell variable, TCL_LIBRARY. Setting and exporting it
in the bash shell from which I call tcl seemed to have no effect. Can
the search path include multiple directories? Can you set a separate
path for tcllib? Can you set environment variables directly from the
tclsh prompt? How can you tell what search path is being used?
That does it! Thanks.
The README mentions that as an alternative installation procedure, but
since it said that the makefile ends up calling installer.tcl anyway, it
did not seem likely to matter which procedure was used.
The TCL_LIBRARY environment variable controls where Tcl finds its own
library files, but in a correct installation you should never need to
set it yourself as the Tcl binary library should have the right default
setting baked into it. When it comes to loading packages, the relevant
variable is actually the Tcl global variable auto_path (seeded from the
TCL_LIBRARY env-var if appropriate) which contains a Tcl list of places
to look (so add to it using [lappend] of course). However, when tcllib
is correctly installed it should already be in a location where Tcl
looks by default.
You can set environment variables directly at the tclsh prompt by
setting things in the global env array, like this:
set ::env(FOO) "bar boo"
These environment variables are propagated to subprocesses, but not to
the calling process. Nor are they persistent. (And that's because that's
how environment variables have worked on all platforms for many years.)
When you say search path, I have to ask "searching for what?" :-) When
Tcl is trying to find a binary program to execute as a subprocess, it
uses the ::env(PATH) and when it is trying to find a package, it uses
::auto_path. There are a few other paths about, but you can ignore them
under normal circumstances.
I've no idea if this answers your questions properly. If not, ask some
more so that I can better grok your problems. :-)
Donal.
...and the corresponding relevant environment variable is TCLLIBPATH.
--
| Don Porter Mathematical and Computational Sciences Division |
| donald...@nist.gov Information Technology Laboratory |
| http://math.nist.gov/~DPorter/ NIST |
|______________________________________________________________________|
What does :: signify?
>
> When you say search path, I have to ask "searching for what?" :-) When
> Tcl is trying to find a binary program to execute as a subprocess, it
> uses the ::env(PATH) and when it is trying to find a package, it uses
> ::auto_path. There are a few other paths about, but you can ignore them
> under normal circumstances.
Are they all listed in one place somewhere in the documentation? I now
have a working installation, which shows:
tcl_library=
C:/cygwin/usr/share/tcl8.4
auto_path=
C:/cygwin/usr/share/tcl8.4 C:/cygwin/usr/share C:/cygwin/lib
How does tcl know to look in share/tcllib1.8 and not in all the other
subdirectories under share/ ? Is this set somewhere?
It doesn't know. It looks in all subdirectories.
I think they're all covered in the docs for Tcl's script library:
http://tmml.sourceforge.net/doc/tcl/library.html
Found this covered in http://www.tcl.tk/man/tcl8.5/tutorial/tcltutorial.html
man tclvars
Of course, man is a horrible mechanism for finding documentation since
you need to know where to look in the first place. Fortunately tcl's
manuals also exist in HTML form:
http://aspn.activestate.com/ASPN/docs/ActiveTcl/8.4/at.pkg_index.html
The tcl counterpart "glob" should be since it was written to run
natively on OSes where spaces in filenames are the norm (heck, on the
classic Mac even the system directory has a space in its name: "System
Folder"). Also read up on the "file" command since it contains lots of
subcommands to properly do filename mangling. You really shouldn't be
using string operations on filenames in Tcl.
I've been using TCL scripts to automate my Linux boxen for... wow back
to Redhat 4.2.
I use TCL because I personally find it easier, and it (at least in my
opinion) is simpler
to slap together library and procedures. I put together subroutines
that have to be
executed in a particular order and worm them into a procedure. I do a
lot of
TCLHTTPD programming, so my shell scripts leverage the libraries I
write
for the webserver and vice versa. My scripts have also had to support
multiple
Linux distros, Mac OS X, and Windows simultaneosly. It's hard to
imagine a Borne
script that can easily adapt to that many places.
My trick is to put all of my libraries and environment into one path,
in my case
/usr/odie (under Linux) or /sw/odie (with a simlink to /usr/odie) for
OS X. Inside
/usr/odie I have one script "init.tcl" that will set up all the paths,
load my packages,
establish connections to my database backend, etc.
For organization purposes /usr/odie has several subdirectories:
/usr/odie/bin (for daemons)
/usr/odie/scripts (with explanitory subdirectories for mail, ldap, tape
backup, etc.)
/usr/odie/lib (gets added to my auto path, contains a copy of all the
libraries I need
so I have to rely on the distro as little as
possible)
/usr/odie/lib/binaries/[operatingsystem] - any custom built binaries
I use rsync to keep all of my Linux and OSX boxes on the same page.
One definite advantage TCL has over BASH is the ability to interface
directly to
sqlite. I use that a lot because, being the lazy programmer that I am,
I use TCL
not just for scripting. I use it to build a lot of the conf files in my
/etc directory.
I have routines the sniff out dhcp leases, pair them up with new static
IP addresses,
and then update the DNS and DHCP server. Other routines maintain my
/etc/hosts
file, my NFS automount, LDAP configuration, etc.
And of course, you can still dump out to the command line with the EXEC
command,
which I do regularly and often.
For an incremental backup system, I highly recommend rsync. Sure, I
have written my
own routines in TCL. But at the end of the day, I keep coming back to
what works. RSYNC
coupled with shared RSA keys and sshd just can't be beat for
reliablity, flexibility,
scriptability.
Of course, I use TCL to build my RSYNC scripts. Take my routine to
backup my scripts:
#! /usr/bin/tclsh
###
# Sean's RSYNC Backup/Restore Script
###
if {[llength $argv] < 1 } {
puts "Usage: sync \[download|upload\]"
exit 1
}
set direction [lindex $argv 0]
set opts [lrange $argv 1 end]
set flags -rztl
if { [lsearch $opts "-preview"] >= 0 } {
append flags n
}
if { [lsearch $opts "-stomp"] < 0 } {
append flags u
}
set delete --stats
if { [lsearch $opts "-merge"] < 0 } {
set delete --delete
}
###
# all of the ::odie array is built here
# catch {source /usr/odie/init.tcl}
# for now we'll just use dummy variables
###
array set odie {
httpdroot /home/httpd
root /usr/odie
}
set server lachesis.internal.fi.edu
set base /opt/common
set paths {}
lappend paths root@${server}:[file join $base rsync/tclhttpd]
$::odie(httpdroot)
lappend paths root@${server}:[file join $base rsync/code] $::odie(root)
switch $direction {
upload {
foreach {remote local} $paths {
puts [list $local > $remote]
exec rsync $flags -e ssh --progress $delete \
--exclude "/sites" --exclude ".DS_Store" \
--exclude "/var" --exclude "/documents" --exclude "~*"
\
--exclude "classidx.sqlite" \
$local/ $remote/ >&@ stdout
}
}
download {
foreach {remote local} $paths {
puts [list $remote > $local]
exec rsync $flags -e ssh --progress $delete \
--exclude "/sites" --exclude ".DS_Store" \
--exclude "/var" --exclude "/documents" --exclude "~*"
\
--exclude "classidx.sqlite" \
$remote/ $local/ >&@ stdout
}
}
default {
puts "Bad direction, must be upload or download"
exit 1
}
}
###
# To use
# ./sync upload -> Upload files to server
# ./sync download -> Download file from server
#
# Default behaviors:
# * Preserve files that are newer at the destination
# * Delete any files at the destination that are not present at the
source
#
# Options:
# -stomp - Overwrite newer files
# -merge - Don't delete files
# -preview - Do a dry run
###
To keep the system from prompting you for a password each time,
set up a shared ssh key between the machines.
I have two copies at the office. One I lend, and the other is mine,
mine, mine.
I've had the book since '98 and I still go back to it when I'm not sure
of how a particular
widget works or some guidance and the Tcl API.
(Brent, if you are reading this, that Book turned me into a Tcler
overnight.
Just what I was hoping for.
I do as well. I am hoping that when 8.5 comes out that it will get an
updating. I have a few suggestions in that area. : )
:Robert
The answer appears to be yes. I like the fact that glob and
tcllib::find by default quote filenames with spaces. I've compiled a
table of equivalent commands, but have not found built-in equivalents to
cp -r and rm -r. Do they exist? Are there important omissions in the
following table?
ls: glob
find: tcllib::find
cat: tcllib::cat
pwd: pwd
cd: cd
cp: file copy
cp -r:
rm: file delete
rm -r:
mkdir: file mkdir
mv: file rename
grep: fileutil::grep
touch: fileutil::touch
> The answer appears to be yes. I like the fact that glob and
> tcllib::find by default quote filenames with spaces. I've compiled a
> table of equivalent commands, but have not found built-in equivalents to
> cp -r and rm -r. Do they exist? Are there important omissions in the
> following table?
A quick look in man file shows - just apply it on directories, and
maybe add -force:
file copy ?-force? ?- -? source target
file copy ?-force? ?- -? source ?source ...? targetDir
The first form makes a copy of the file or directory source under the
pathname target. If target is an existing directory, then the second
form is used. The second form makes a copy inside targetDir of each
source file listed. If a directory is specified as a source, then the
contents of the directory will be recursively copied into targetDir.
Existing files will not be overwritten unless the -force option is
specified.
file delete ?-force? ?- -? pathname ?pathname ... ?
Removes the file or directory specified by each pathname argument.
Non-empty directories will be removed only if the -force option is
specified. When operating on symbolic links, the links themselves will
be deleted, not the objects they point to. Trying to delete a
non-existent file is not considered an error.
>
> A quick look in man file shows - just apply it on directories, and
> maybe add -force:
>
> file copy ?-force? ?- -? source target
> file copy ?-force? ?- -? source ?source ...? targetDir
Thanks.
I infer that a pair of ?? enclose an argument that is optional, but what
does "?- -?" mean?
It's actually meant to be ?--? (no space) but some piece of software
seems to have have decided to use that character sequence as a chew-toy.
It refers to the (optional) "end of options" option.
Donal.
Someone may get a laugh out of the incident that drove me to look at
tcl. Filenames with spaces break my old Bourne shell scripts, and in
trying to fix them I came across the following:
A Guide to Unix Shell Quoting:
http://www.mpi-sb.mpg.de/~uwe/lehre/unixffb/quoting-guide.html
A look at that prompted me to try another language.