Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

Making sure a source file is used correctly

69 views
Skip to first unread message

Cecil Westerhof

unread,
Jan 4, 2018, 11:59:07 AM1/4/18
to
I am learning to work with tcl. At the moment I am working with a
source file to include generally used functionalities. It would be
better to use package, but that is the next step.

In my opinion the file should never be called directly and not be
sourced more as once. So I use the following:
if {[info exists ::dcblUtilities::utilScriptname]} {
error "${::dcblUtilities::utilScriptname} should only be sourced once"
}
namespace eval ::dcblUtilities {
variable utilScriptname [info script]
}
if { ${::dcblUtilities::utilScriptname} eq ${::argv0} } {
error "${::dcblUtilities::utilScriptname} should only be used to include with source"
}

Is that a good way to do this, or can it be improved?

--
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof

Gerald Lester

unread,
Jan 4, 2018, 12:14:49 PM1/4/18
to
On 01/04/2018 10:54 AM, Cecil Westerhof wrote:
> I am learning to work with tcl. At the moment I am working with a
> source file to include generally used functionalities. It would be
> better to use package, but that is the next step.
>
> In my opinion the file should never be called directly and not be
> sourced more as once. So I use the following:
> if {[info exists ::dcblUtilities::utilScriptname]} {
> error "${::dcblUtilities::utilScriptname} should only be sourced once"
> }
> namespace eval ::dcblUtilities {
> variable utilScriptname [info script]
> }
> if { ${::dcblUtilities::utilScriptname} eq ${::argv0} } {
> error "${::dcblUtilities::utilScriptname} should only be used to include with source"
> }
>
> Is that a good way to do this, or can it be improved?

My first question is, why do you care if it is sourced in more than once
or run directly?

Let's consider each case....

If it is run directly, it will do nothing and exit -- so what is the harm?

If it is sourced in more than once, the procedures just get redefined,
again no harm. If you are initializing variables or declaring classes
or creating widget, just wrap them in an "if does not exists, then
do..." construct.


--
+----------------------------------------------------------------------+
| Gerald W. Lester, President, KNG Consulting LLC |
| Email: Gerald...@kng-consulting.net |
+----------------------------------------------------------------------+

Brad Lanam

unread,
Jan 4, 2018, 12:43:02 PM1/4/18
to
On Thursday, January 4, 2018 at 8:59:07 AM UTC-8, Cecil Westerhof wrote:
> I am learning to work with tcl. At the moment I am working with a
> source file to include generally used functionalities. It would be
> better to use package, but that is the next step.
>
> In my opinion the file should never be called directly and not be
> sourced more as once. So I use the following:
> if {[info exists ::dcblUtilities::utilScriptname]} {
> error "${::dcblUtilities::utilScriptname} should only be sourced once"
> }
> namespace eval ::dcblUtilities {
> variable utilScriptname [info script]
> }
> if { ${::dcblUtilities::utilScriptname} eq ${::argv0} } {
> error "${::dcblUtilities::utilScriptname} should only be used to include with source"
> }
>
> Is that a good way to do this, or can it be improved?

I think you are re-creating the 'package' functionality.
But...
I don't like the fact that this code is not re-usable for a second
source'd script. How about:

variable vars

proc srconce { fn } {
variable vars

set ffn [file normalize $fn]
if { [info exists vars($ffn)] } [
return
}
try {
source $ffn
set vars($ffn) 1
} on error {err res} {
puts "error sourcing $ffn: $res"
}
}

Now put this proc in your utilScriptname file...
Oh, wait...

I have a slightly different use case that does a similar thing, where I source
the script, but then I need to call the main entry point. I don't want to
re-source the script, but I want to start up the process again.

# the caller

srcmanage::sourcefile thisone thisone.tcl

# some big package with a main script... (thisone.tcl)

namespace eval thisone {
proc init { } {
srcmanage::register thisone ::thisone::main
}

proc main { } {
init
}
}
::thisone::main

# srcmanage package

package provide srcmanage 1.0

namespace eval ::srcmanage {
variable vars

proc register { tag mainproc } {
variable vars

set vars($tag) $mainproc
}

proc sourcefile { tag path } {
variable vars

if { [info exists vars($tag)] && $vars($tag) ne {} } {
$vars($tag) ; # call the main procedure
return true
}

uplevel #0 [list source $path]
return false ; # not cached
}
}

Rich

unread,
Jan 4, 2018, 1:18:19 PM1/4/18
to
Cecil Westerhof <Ce...@decebal.nl> wrote:
> I am learning to work with tcl. At the moment I am working with a
> source file to include generally used functionalities. It would be
> better to use package, but that is the next step.
>
> In my opinion the file should never be called directly and not be
> sourced more as once.
> ... snip
> Is that a good way to do this, or can it be improved?

Can be much improved. Make it either a package or a module (for
modules see the "tm" man page).

Both packages and modules provide built in support for "not sourcing
more than once" so you don't ever have to worry about handling that
yourself.

Modules have an advantage that they are just a single file, but their
loading mechanism is slightly less flexible. But if your utility
file is just a set of Tcl procs then the loss of flexibility is not
likely going to be missed.

Packages require a directory, at least one source file, and a pkg_Index
script (there is a pkg_Index generator in the Tcl library as the
"pkg_mkIndex" command so you don't have to write your own pkg_Index
file for simple packages).

Both of course have to reside at a location where the Tcl interpreter
will search for them, which means (for packages) in one of the default
locations in the auto_path list (or you need to lappend the location to
auto_path at startup) or (for modules) at one of the locations returned
by the ::tcl::tm::path list (or you add a custom location via the
"::tcl::tm::path add" sub command).

Rich

unread,
Jan 4, 2018, 1:22:21 PM1/4/18
to
Cecil Westerhof <Ce...@decebal.nl> wrote:
> I am learning to work with tcl. At the moment I am working with a
> source file to include generally used functionalities. It would be
> better to use package, but that is the next step.

Also, seriously consider putting your custom utilities into their own
namespace. You can make them exportable and import them into other
code if you don't want to use namespace prefixes for them all the time.
But doing so gives you a lot of freedom to name your custom procs
however you like (and to have namespace specific 'global' data) without
direct name conflicts with other packages you might use.

Cecil Westerhof

unread,
Jan 4, 2018, 1:44:06 PM1/4/18
to
Gerald Lester <Gerald...@KnG-Consulting.net> writes:

> On 01/04/2018 10:54 AM, Cecil Westerhof wrote:
>> I am learning to work with tcl. At the moment I am working with a
>> source file to include generally used functionalities. It would be
>> better to use package, but that is the next step.
>>
>> In my opinion the file should never be called directly and not be
>> sourced more as once. So I use the following:
>> if {[info exists ::dcblUtilities::utilScriptname]} {
>> error "${::dcblUtilities::utilScriptname} should only be sourced once"
>> }
>> namespace eval ::dcblUtilities {
>> variable utilScriptname [info script]
>> }
>> if { ${::dcblUtilities::utilScriptname} eq ${::argv0} } {
>> error "${::dcblUtilities::utilScriptname} should only be used to include with source"
>> }
>>
>> Is that a good way to do this, or can it be improved?
>
> My first question is, why do you care if it is sourced in more than once
> or run directly?

Maybe I am more Catholic than the pope. ;-)

Long ago I remember problems that where created because multiple
includes. Even when immediately returning when something was already
included. This would have a bad influence on performance. But nowadays
that is probably not a real problem. Instead of generating an error I
could print a warning.

So I changed it to:
################ Only source if not already done
if {[info exists ::dcblUtilities::utilScriptName]} {
puts stderr "Already sourced ${::dcblUtilities::utilScriptName}"
return
}
namespace eval ::dcblUtilities {
variable utilScriptName [file normalize [info script]]
}


> Let's consider each case....
>
> If it is run directly, it will do nothing and exit -- so what is the harm?
>
> If it is sourced in more than once, the procedures just get redefined,
> again no harm. If you are initializing variables or declaring classes
> or creating widget, just wrap them in an "if does not exists, then
> do..." construct.

--

Cecil Westerhof

unread,
Jan 4, 2018, 2:14:06 PM1/4/18
to
Brad Lanam <brad....@gmail.com> writes:

> On Thursday, January 4, 2018 at 8:59:07 AM UTC-8, Cecil Westerhof wrote:
>> I am learning to work with tcl. At the moment I am working with a
>> source file to include generally used functionalities. It would be
>> better to use package, but that is the next step.
>>
>> In my opinion the file should never be called directly and not be
>> sourced more as once. So I use the following:
>> if {[info exists ::dcblUtilities::utilScriptname]} {
>> error "${::dcblUtilities::utilScriptname} should only be sourced once"
>> }
>> namespace eval ::dcblUtilities {
>> variable utilScriptname [info script]
>> }
>> if { ${::dcblUtilities::utilScriptname} eq ${::argv0} } {
>> error "${::dcblUtilities::utilScriptname} should only be used to include with source"
>> }
>>
>> Is that a good way to do this, or can it be improved?
>
> I think you are re-creating the 'package' functionality.

I first need to learn to crawl, before I learn to run. ;-)

But that will be a next step.


> But...
> I don't like the fact that this code is not re-usable for a second
> source'd script. How about:

I already changed that: see my other reply.


I will study your code.

Cecil Westerhof

unread,
Jan 4, 2018, 2:44:06 PM1/4/18
to
Rich <ri...@example.invalid> writes:

> Cecil Westerhof <Ce...@decebal.nl> wrote:
>> I am learning to work with tcl. At the moment I am working with a
>> source file to include generally used functionalities. It would be
>> better to use package, but that is the next step.
>>
>> In my opinion the file should never be called directly and not be
>> sourced more as once.
>> ... snip
>> Is that a good way to do this, or can it be improved?
>
> Can be much improved. Make it either a package or a module (for
> modules see the "tm" man page).

I had seen packaging and source, but not module. So that would be a
good one to get into.

I thought package to be a bit to much (at the moment), but it seems
that module could be an good alternative.

Cecil Westerhof

unread,
Jan 4, 2018, 2:44:06 PM1/4/18
to
I will do that also.

Brad Lanam

unread,
Jan 4, 2018, 2:49:16 PM1/4/18
to
On Thursday, January 4, 2018 at 11:14:06 AM UTC-8, Cecil Westerhof wrote:
> I first need to learn to crawl, before I learn to run. ;-)

Using packages is pretty easy.
At the top of every module that has a package require I have:

set ap [file join [file dirname [info script]] code]
if { $ap ni $::auto_path } {
lappend ::auto_path $ap
}
unset ap

In this case, I am adding the code/ sub-directory to the ::auto_path
variable. I use [file dirname [info script]] to get the path to the
current location of this script (probably needs a file normalize).

The 'ni' test prevents duplication (though since not normalized, I
may have some). The modules in the code/ directory have:

set ap [file join [file dirname [info script]]
if { $ap ni $::auto_path } {
lappend ::auto_path $ap
}
unset ap

I also remember all the include file problems, and I like to make
sure that a 'package require' for any module will
work and each module includes its own dependencies.

I wrote a script to check the dependencies, and every now and then I run
it to make sure I don't have missing or unused dependencies.

So that little snippet of code with the appropriate path
appears in every module that has 'package require' statements .

Then all of my 'package require' statements will pick up the
pkgIndex.tcl file in the code/ sub-directory.

The pkgIndex.tcl file has many lines like:
package ifneeded autosel 1.0 [list source [file join $dir autosel.tcl]]

Brad Lanam

unread,
Jan 4, 2018, 3:03:54 PM1/4/18
to
On Thursday, January 4, 2018 at 11:49:16 AM UTC-8, Brad Lanam wrote:
> set ap [file join [file dirname [info script]]

Copy/paste error, that should be:

set ap [file dirname [info script]]


Rich

unread,
Jan 4, 2018, 3:18:09 PM1/4/18
to
Cecil Westerhof <Ce...@decebal.nl> wrote:
> Gerald Lester <Gerald...@KnG-Consulting.net> writes:
>
>> On 01/04/2018 10:54 AM, Cecil Westerhof wrote:
>>> I am learning to work with tcl. At the moment I am working with a
>>> source file to include generally used functionalities. It would be
>>> better to use package, but that is the next step.
>>>
>>> In my opinion the file should never be called directly and not be
>>> sourced more as once. So I use the following:
>>> if {[info exists ::dcblUtilities::utilScriptname]} {
>>> error "${::dcblUtilities::utilScriptname} should only be sourced once"
>>> }
>>> namespace eval ::dcblUtilities {
>>> variable utilScriptname [info script]
>>> }
>>> if { ${::dcblUtilities::utilScriptname} eq ${::argv0} } {
>>> error "${::dcblUtilities::utilScriptname} should only be used to include with source"
>>> }
>>>
>>> Is that a good way to do this, or can it be improved?
>>
>> My first question is, why do you care if it is sourced in more than once
>> or run directly?
>
> Maybe I am more Catholic than the pope. ;-)
>
> Long ago I remember problems that where created because multiple
> includes. Even when immediately returning when something was already
> included. This would have a bad influence on performance. But nowadays
> that is probably not a real problem. Instead of generating an error I
> could print a warning.

C would complain about multiple definitions of the same function.
Which is why C include files protected themselves from multiple
inclusion.

But the reasons why for C don't apply to Tcl. If your "included" file
does nothing but define procs, multiple sourcings of the same file will
have (generally) no effect on net performance, nor on correctness. And
Tcl does not complain if you redefine the same proc multiple times.

Now, if you are sourcing the file inside a loop with other work, yes,
that will hurt performance, but that is more a bug than anything, as
sourcing library scripts really should only occur once, at the top of
each script. But if a script sources two library files, and those two
files each source the same third library (so the third is sourced
twice) the performance difference is negligible.

Rich

unread,
Jan 4, 2018, 3:19:47 PM1/4/18
to
Cecil Westerhof <Ce...@decebal.nl> wrote:
> Rich <ri...@example.invalid> writes:
>
>> Cecil Westerhof <Ce...@decebal.nl> wrote:
>>> I am learning to work with tcl. At the moment I am working with a
>>> source file to include generally used functionalities. It would be
>>> better to use package, but that is the next step.
>>>
>>> In my opinion the file should never be called directly and not be
>>> sourced more as once.
>>> ... snip
>>> Is that a good way to do this, or can it be improved?
>>
>> Can be much improved. Make it either a package or a module (for
>> modules see the "tm" man page).
>
> I had seen packaging and source, but not module. So that would be a
> good one to get into.
>
> I thought package to be a bit to much (at the moment), but it seems
> that module could be an good alternative.

Modules are simpler than packages. They are also newer than packages.

Packages are more flexible in what they can ultimately perform during
package require time.

Both modules and packages are loaded via "package require".

lmn...@gmail.com

unread,
Jan 4, 2018, 6:02:32 PM1/4/18
to
Have you tried using the "auto_path" and "tclIndex" capabilities of tcl?

I believe that solves the problem.
I use this method extensively and find it very useful to remove complications that you describe...

Plus: The compiler does a better optimization of the code this way versus 'source'ing in-line.

Example: Create/generate a ./tclIndex file:

./hello.tcl
proc hello {args} {
puts {hello world!}
}

% tclsh
% auto_mkindex [pwd] *.tcl

Output:
./tclIndex
# Tcl autoload index file, version 2.0
# This file is generated by the "auto_mkindex" command
# and sourced to set up indexing information for one or
# more commands. Typically each line is a command that
# sets an element in the auto_index array, where the
# element name is the name of a command and the value is
# a script that loads the command.

set auto_index(hello) [list source [file join $dir hello.tcl]]


Example: Add a path to TCL's "auto_path" search mechanism.

% tclsh
% lappend auto_path <path>
% hello
hello world!

In practice, to make this uniform in a development project and leverage it in an application -- I create a script in each tcl/src directory:

./update_tclIndex
#! /usr/bin/env tclsh
set dirlist [list "."]
foreach procdir $dirlist {
auto_mkindex $procdir *.tcl
}

And I execute this script anytime a new tcl procedure is added and this can be either a new tcl file or an existing tcl file that contains multiple procedures -- this works for both/any scenarios.

Using the 'update_tclIndex' script in combination with version control on the directory (svn for example) I can quickly see if content of tclIndex changed whenever ./update_tclIndex is ran. (svn status -u)

With respect to using this in a "main/program", I have implemented a convention which is to maintain a "init.tcl" file in the src directory of the tcl/program.

The first line in the "tcl/main/program" is to source the "init.tcl" file and then call the 'main' tcl procedure.

./main
#! /usr/bin/env tclsh
source <program_installation_path>/init.tcl
main <args>

The init.tcl file is responsible for knowing all the src directories the program needs for itself.

./init.tcl
lappend auto_path <program_installation_path>/tcl
lappend auto_path <program_installation_path>/tcl/subdir
etc...


This init.tcl file is usually more flexible, and uses $::env(PROGRAM_INSTALL) variables for the path and some additional checks to print nice messages if the variables aren't defined or we don't find basic/sanity checks that we expect -- so that users are given 'nice/pretty' messages and instructions on how the program is expecting to be ran, etc...
For example the hello proc can be called in the 'init.tcl' script to test that it found the procedure before proceeding to 'main'.

The 'main' program is a tcl procedure as well:

./main.tcl
proc main {args} {
<body>
}


Example User Calling the Program:
% main <args>

Hope that's helpful,
Jim

PS. If you follow all that and find it remotely useful -- the next question, typically (at least in my mind that's what happened), is how to pass arguments to a procedure (non-position-ally that is). If you're at that point and have that question -- feel free to post a new ticket and I'll share the code I have to do that...

i.e: main -arg <value> -arg <value> -arg <value>

Cecil Westerhof

unread,
Jan 5, 2018, 6:28:06 PM1/5/18
to
Cecil Westerhof <Ce...@decebal.nl> writes:

> I am learning to work with tcl. At the moment I am working with a
> source file to include generally used functionalities. It would be
> better to use package, but that is the next step.

It proved a lot easier to make a package as I thought, so I created a
package now.

Donal K. Fellows

unread,
Jan 6, 2018, 2:35:44 AM1/6/18
to
On 04/01/2018 16:54, Cecil Westerhof wrote:
> Is that a good way to do this, or can it be improved?

Since you're already planning to make it a package, there's really no
reason to bother. (If you do [package require] twice it will load the
package just once, unless you fiddle around with [package forget] and
then you know you are doing something unusual.) Yes, you can do the sort
of thing that you've implemented so far, but it is purely for you as a
developer of the package and doesn't give you any meaningful benefit
once you deploy it.

Or at least that's how I see it. I guess I'm lazy. :-)

Donal.
--
Donal Fellows — Tcl user, Tcl maintainer, TIP editor.

Cecil Westerhof

unread,
Jan 6, 2018, 9:28:06 AM1/6/18
to
"Donal K. Fellows" <donal.k...@manchester.ac.uk> writes:

> On 04/01/2018 16:54, Cecil Westerhof wrote:
>> Is that a good way to do this, or can it be improved?
>
> Since you're already planning to make it a package, there's really no
> reason to bother. (If you do [package require] twice it will load the
> package just once, unless you fiddle around with [package forget] and
> then you know you are doing something unusual.) Yes, you can do the sort
> of thing that you've implemented so far, but it is purely for you as a
> developer of the package and doesn't give you any meaningful benefit
> once you deploy it.

I expected it to be a lot of work to write a package, but I bite the
bullet and rewrote it to a package. This proved to be a lot less work
as I expected. So this thread can be retired to the dustbin. ;-)

I expand the package a little and will share it and some scripts I
wrote through GitHub next week.

Maybe I should make a page about making a package.

Schelte Bron

unread,
Jan 9, 2018, 8:30:12 AM1/9/18
to
Cecil Westerhof wrote:
> In my opinion the file should never be called directly and not be
> sourced more as once.

May I suggest you stop fighting Tcl features by imposing limitations
that you have become accustomed to from other languages?

The fact that you can source a file multiple times is actually a
very useful feature of Tcl. It greatly simplifies development by
allowing you to modify your code and then reload the modified source
file in a running process. That saves you from having to go through
all the setup steps again required to get to a failure point. You
can just directly retry the action that exhibited a problem.

For this reason, I normally split my applications into multiple
files and take care that they can be reloaded. For example, instead
of
set variable ""
I use:
append variable ""
That creates an empty variable the first time, but does not change
its value any subsequent times.

And I define oo classes in two steps:
catch {oo::class create myclass}
oo::define myclass {
# Class definition
}


Schelte.

Cecil Westerhof

unread,
Jan 9, 2018, 9:14:05 AM1/9/18
to
Schelte Bron <nos...@wanadoo.nl> writes:

> Cecil Westerhof wrote:
>> In my opinion the file should never be called directly and not be
>> sourced more as once.
>
> May I suggest you stop fighting Tcl features by imposing limitations
> that you have become accustomed to from other languages?

I already did. ;-)


> The fact that you can source a file multiple times is actually a
> very useful feature of Tcl. It greatly simplifies development by
> allowing you to modify your code and then reload the modified source
> file in a running process. That saves you from having to go through
> all the setup steps again required to get to a failure point. You
> can just directly retry the action that exhibited a problem.
>
> For this reason, I normally split my applications into multiple
> files and take care that they can be reloaded. For example, instead
> of
> set variable ""
> I use:
> append variable ""
> That creates an empty variable the first time, but does not change
> its value any subsequent times.

That is an interesting one.
0 new messages