Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

Larry Wall, on Tcl

47 views
Skip to first unread message

Bryan Oakley

unread,
Dec 7, 2007, 12:59:48 PM12/7/07
to
State of the Onion, http://www.perl.com/pub/a/2007/12/06/soto-11.html

His comments about Tcl:

--- begin quote ---

...After Perl came Tcl, which in a sense is a purer scripting language
than Perl. Perl just pretends that everything is a string when it's
convenient, but Tcl really believes that as a controlling metaphor. The
string metaphor tends to have bad performance ramifications, but that's
not why Tcl languished, I think. There were two reasons for that.

First, Tcl stayed in the Unix mindset that controlling tools was the
opposite of creating tools, so they didn't optimize much. The fast parts
can always be written in C, after all.

The second reason was the lack of a decent extension mechanism, so you
ended up with separate executables for expect, incr-tcl, etc.

I must say, though, that I've always admired Tcl's delegational model of
semantics. But it fell into the same trap as LISP by expecting everyone
to use the One True Syntax..."

--- end quote ---


I think he totally missed the boat with the "lack of a decent extension
mechansim" comment, but that statement was true at one point in time.
Perhaps his version of Tcl is stuck at 7.6.


--
Bryan Oakley
http://www.tclscripting.com

Jan Kandziora

unread,
Dec 7, 2007, 3:04:46 PM12/7/07
to
Bryan Oakley schrieb:

>
> I think he totally missed the boat with the "lack of a decent extension
> mechansim" comment, but that statement was true at one point in time.
> Perhaps his version of Tcl is stuck at 7.6.
>
Well, desinformation *must* leave out the newest facts. If that doesn't
help, it has to leave out even older facts, too.

Kind regards

Jan

Torsten Berg

unread,
Dec 7, 2007, 3:14:31 PM12/7/07
to
Oh no, not again ...

> The string metaphor tends to have bad performance ramifications

Does this never stop? Do Perl people always have to nag about speed? I
don't care most of the time, if a script runs in 0.12ms or 0.5ms. I
can't type so fast. For me, Tcl is fast. People, who perhaps only know
pre-8.0 Tcl, should be careful.

> The fast parts can always be written in C, after all.

Yeah, great, isn't it? Can Perl do that as easy as Tcl? With such a
fine API as Tcl's? And TEA? And stubs?

> The second reason was the lack of a decent extension mechanism, so you
> ended up with separate executables for expect, incr-tcl, etc.

Yawn. Recent information is the path to wisdom ...

> I must say, though, that I've always admired Tcl's delegational model of semantics.

Thanks!

> But it fell into the same trap as LISP by expecting everyone
> to use the One True Syntax..."

Help! I'm trapped in Tcl! And I even like it :-) I like not to be
confused with Three True Syntaxes.


Well, Larry gave his personal opinion. So he is not to blame. But
since he does this publicly, he should at least use recent knowledge
and be careful.

Just my $0.02

Torsten

Donal K. Fellows

unread,
Dec 7, 2007, 5:54:44 PM12/7/07
to
Jan Kandziora wrote:
> Well, disinformation *must* leave out the newest facts. If that doesn't

> help, it has to leave out even older facts, too.

QOTW!

Donal.

Rob Ratcliff

unread,
Dec 7, 2007, 11:42:35 PM12/7/07
to

>
> Well, Larry gave his personal opinion. So he is not to blame. But
> since he does this publicly, he should at least use recent knowledge
> and be careful.
>

We just won't let him into our secret society of evil geniuses now! :-)

(See
http://groups.google.com/group/comp.lang.tcl/tree/browse_frm/thread/8ba7b3e2e65dda9e/1b2cc741a3a2f844
for details)

Robert Hicks

unread,
Dec 8, 2007, 9:52:52 AM12/8/07
to
On Dec 7, 3:14 pm, Torsten Berg <b...@typoscriptics.de> wrote:
<snip>

> Well, Larry gave his personal opinion. So he is not to blame. But
> since he does this publicly, he should at least use recent knowledge
> and be careful.
>

Of course he is to blame. If he doesn't take the time to find
something out he shouldn't be giving his personal opinion on it.

Robert

Donal K. Fellows

unread,
Dec 8, 2007, 12:55:13 PM12/8/07
to
Robert Hicks wrote:
> Of course he is to blame. If he doesn't take the time to find
> something out he shouldn't be giving his personal opinion on it.

I disagree. I give my opinion on all sorts of things all the time
without finding out about them. The difference is, I'm usually right.

Donal.

tom.rmadilo

unread,
Dec 8, 2007, 8:53:04 PM12/8/07
to
On Dec 8, 9:55 am, "Donal K. Fellows"

I don't waste my time commenting on unimportant topics or obsessing
over why something I don't use (anymore) is languishing. I doesn't
make me feel any better about Tcl to fully investigate the many ways
Perl sucks. It does some things pretty well and I use it for those
things. But it sure doesn't appear either faster or slower than Tcl
for any comparable application. Since applications are the most useful
level to compare languages for speed, not a few isolated test
problems, I wonder how this misinformation persists, and why
programmers don't simply laugh when these comparisons are done.

Same goes with this syntax nonsense. You could look at it in a number
of ways. Tcl determines words and does substitutions on non-braced
words. The first word is a command and the rest are arguments. After
that each command is free to use whatever syntax it wants on the
arguments.

Robert Hicks

unread,
Dec 8, 2007, 9:37:09 PM12/8/07
to
On Dec 8, 12:55 pm, "Donal K. Fellows"

: )

tom.rmadilo

unread,
Dec 9, 2007, 3:28:55 AM12/9/07
to
On Dec 7, 9:59 am, Bryan Oakley <oak...@bardo.clearlight.com> wrote:
> State of the Onion,http://www.perl.com/pub/a/2007/12/06/soto-11.html

After reading the article, I think it is worth pointing out a few
facts.

First is that Perl is such a poor language that it needs to be
reinvented. A not so subtle admission of a system which hasn't stood
the test of time.

Second: what is this guy talking about? Apparently he bought in to a
number of idiotic theories over the years and slowly discovered they
were idiotic. Most of these were associated with new-age scripting
languages. Perl6 sounds like a therapy session for some dude with a
lot of bad experiences.

I found exactly one example of a concrete recommendation for Perl6:
being able to support for 1..Inf.

This is described as being able to work with infinite lists! He admits
that you need to find some way to actually exit before Inf is reached,
although I disagree. Given a healthy computer and electricity, there
should be no problem. The usual glitch is dumb humans which see no
reason to do a for 1..Inf.

In Tcl one way to achieve this holy grail is something almost any
Tcl'er could discover by pure accident:

for {set i 1} {$i > 0} {incr i} {
...
}

See, Tcl can work for infinite lists!

Of course Perl could handle this so much faster, you might actually
reach Inf, who knows.

Bryan Oakley

unread,
Dec 9, 2007, 9:43:33 AM12/9/07
to
tom.rmadilo wrote:
> On Dec 7, 9:59 am, Bryan Oakley <oak...@bardo.clearlight.com> wrote:
>
>>State of the Onion,http://www.perl.com/pub/a/2007/12/06/soto-11.html
>
>
> After reading the article, I think it is worth pointing out a few
> facts.
>
> First is that Perl is such a poor language that it needs to be
> reinvented. A not so subtle admission of a system which hasn't stood
> the test of time.

I think that's a bit too inflamatory. Lets keep language bashing to a
minimum, please.

I think most serious programmers would disagree completely with your
statement. Perl is a fine language which has quite well stood the test
of time from what I can see. It's just that ther Perl community, or
Larry Wall, wants to make it better. What's wrong with that?

Our own community has had a "Tcl 9" page on the wiki that includes all
sorts of potentially backwards-incompatible suggestions. Does that mean
Tcl hasn't stood the test of time either?

Tcl is certainly no more nor less perfect than Perl. Different
priorities, different languages.


Óscar Fuentes

unread,
Dec 9, 2007, 10:16:12 AM12/9/07
to
"tom.rmadilo" <tom.r...@gmail.com> writes:

[snip]

> I found exactly one example of a concrete recommendation for Perl6:
> being able to support for 1..Inf.
>
> This is described as being able to work with infinite lists! He admits
> that you need to find some way to actually exit before Inf is reached,
> although I disagree. Given a healthy computer and electricity, there
> should be no problem. The usual glitch is dumb humans which see no
> reason to do a for 1..Inf.
>
> In Tcl one way to achieve this holy grail is something almost any
> Tcl'er could discover by pure accident:
>
> for {set i 1} {$i > 0} {incr i} {
> ...
> }
>
> See, Tcl can work for infinite lists!

Your example is just a loop from 1 to 2^31 (in my platform). Loosely
explained, infinite lists on functional programming are generators,
not loops, and are regular objects. You ask for an element, then for
the next, etc, so those objects are referred as lists and the language
syntax allows using them as such. A typical example is a pseudo-random
number generator. A 1..Inf generator is useful too. Take a walk for
some Haskell site for details, there are plenty of nifty applications
for those constructs there.

[snip]

--
Oscar

tom.rmadilo

unread,
Dec 9, 2007, 12:49:29 PM12/9/07
to
On Dec 9, 6:43 am, Bryan Oakley <oak...@bardo.clearlight.com> wrote:

> tom.rmadilo wrote:
> > First is that Perl is such a poor language that it needs to be
> > reinvented. A not so subtle admission of a system which hasn't stood
> > the test of time.
>
> I think that's a bit too inflamatory. Lets keep language bashing to a
> minimum, please.
...

> I think most serious programmers would disagree completely with your
> statement. Perl is a fine language which has quite well stood the test
> of time from what I can see. It's just that ther Perl community, or
> Larry Wall, wants to make it better. What's wrong with that?

Ummm, that is my interpretation of Larry Wall's discussion, not my
assessment of Perl. I already stated that I use Perl applications, no
problem with that. I'm not bashing any language, just the use (by
Wall) of a few dozen bad trends in Perl and other scripting languages
as an excuse to make language changes. Maybe Per is in need of change,
but the users are the ones who demonstrate bad uses. Some languages
are designed with bad use in mind. Wall identifies these. And Tcl's
bad design decisions is 'considering everything a string', among
others. The Wall article is full of reactionary bashing, and it
appears he's turned the bashing right back on Perl, not me.

> Our own community has had a "Tcl 9" page on the wiki that includes all
> sorts of potentially backwards-incompatible suggestions. Does that mean
> Tcl hasn't stood the test of time either?

When I find something I can't do in Tcl, maybe I'll read up on Tcl 9.
Personally one of the most clear expressions of language bashing is
usually from within the community. Someone has heard of a neat concept-
of-the-year and wants to be able to do it in their language, so it'll
be cool. These are not language designers, these are feature whores.
Every language does not have to have every feature. In Tcl you add
features with procedures and packages, not by redesigning the
language. My advice for anyone considering language changes is to try
to write it in normal Tcl, show how much easier it would be in another
language which uses the feature, and provide a detailed example of how
the new feature would simplify a significant are of Tcl coding. And
simplify means more than just less code, which can usually be achieve
with replacing verbose code with a procedure call, simplicity also
means the ability to help developers think about a problem. [lassign]
is a good example. It does more than [foreach] with a [break],
although I recently discovered that I wish we had an [assign] which
does exactly [foreach $varList $dataList {break}], which is something
of an extension of [set]. A similar command for arrays would also be
useful [array assign arrayName] would create local variables based
upon the array names. These are not incompatible language changes, and
I use regular Tcl code to express the concept, but it makes code more
obscure than necessary, or maybe not. Maybe [array assign arrayName
$varList] would be better. Non-existing vars set to ""?


> Tcl is certainly no more nor less perfect than Perl. Different
> priorities, different languages.

Hopefully language developers will respect the priorities of their
language as they seek to improve it. Simply wanting to change a
language doesn't make the cause noble, or place the proposal above
question.

Darren New

unread,
Dec 9, 2007, 1:29:35 PM12/9/07
to
tom.rmadilo wrote:
> although I recently discovered that I wish we had an [assign] which
> does exactly [foreach $varList $dataList {break}], which is something
> of an extension of [set]. A similar command for arrays would also be
> useful [array assign arrayName] would create local variables based
> upon the array names.

So..... write them, and put them in your library or in TclLib? One nice
thing about Tcl is you *can* add these things at the language level.

--
Darren New / San Diego, CA, USA (PST)
It's not feature creep if you put it
at the end and adjust the release date.

Donal K. Fellows

unread,
Dec 9, 2007, 2:09:18 PM12/9/07
to
Darren New wrote:
> So..... write them, and put them in your library or in TclLib? One nice
> thing about Tcl is you *can* add these things at the language level.

I should note here that some of the ensemble-like commands in 8.5 really
are ensembles, so you can add your own commands to them. Not all of them
though; I ran out of time. :-)

Donal (and [array] is one I didn't do IIRC).

Cameron Laird

unread,
Dec 9, 2007, 4:18:19 PM12/9/07
to
In article <2facb385-f794-4802...@d21g2000prf.googlegroups.com>,
tom.rmadilo <tom.r...@gmail.com> wrote:
.
.
.

>are designed with bad use in mind. Wall identifies these. And Tcl's
>bad design decisions is 'considering everything a string', among
>others. The Wall article is full of reactionary bashing, and it
.
.
.
EIAS isn't a good thing?

Darren New

unread,
Dec 9, 2007, 10:57:03 PM12/9/07
to
Cameron Laird wrote:
> EIAS isn't a good thing?

It has its drawbacks. Like, no garbage collection of referenced objects.

tom.rmadilo

unread,
Dec 9, 2007, 11:01:17 PM12/9/07
to
On Dec 9, 1:18 pm, cla...@lairds.us (Cameron Laird) wrote:

> tom.rmadilo <tom.rmad...@gmail.com> wrote:
.
> >are designed with bad use in mind. Wall identifies these. And Tcl's
> >bad design decisions is 'considering everything a string', among
> >others. The Wall article is full of reactionary bashing, and it
.
> EIAS isn't a good thing?

My point exactly! Wall identifies things he considers bad in a number
of languages. Actual badness in most, but he just seems to miss the
point about Tcl. Maybe he was looking for easy to identify features
and bash those? However, since I know so little about these other
languages, maybe he misses the point in those as well, and his bashing
only sounds more plausible due to my complete ignorance? More likely.

My assumption is that the language designers knew enough to write a
language which attracted critical mass. This is obviously true for
Tcl, and Perl. What's more is that the attraction was to the language-
as-it-is, not some unknown improvement.

The EIAS concept is so fundamental to Tcl and an actual fact for the
environment in which it is expected to help out the most, that you
would think from reading these sorts of comments that other languages
have never had a need for I/O conversions or additional type
validation of any kind. EveryThing Is A String, as soon as it passes
the bounds of the programming language itself, like to the OS and into
memory and the CPU. The real il-lusion of de-lusion is thinking that
your little integer object traverses the different layers on you
computer or the internet as an integer. Maybe for some instant in
time? No, but maybe in the sweet dreams of semantic nonsense.

BTW, it is pretty hard for me to honestly take Wall seriously, it may
be just a typical rant (like mine). When you finish off a three page
bash session comparing the future many Perl6s coiled up like the
fictional extra dimensions in M theory, probably not to serious. Ya'
know, our language is just too big to fit in three+one dimensions of
spacetime! Now that is saying something.

tom.rmadilo

unread,
Dec 9, 2007, 11:39:41 PM12/9/07
to
On Dec 9, 11:09 am, "Donal K. Fellows"

<donal.k.fell...@manchester.ac.uk> wrote:
> Darren New wrote:
> > So..... write them, and put them in your library or in TclLib? One nice
> > thing about Tcl is you *can* add these things at the language level.
>

Please note that I am not complaining. If I notice something like
this, I start to pay attention to it. Maybe the hacks I use are best
but need to be standardized, at least in my own code. I only noticed
the [assign] possibility when working on my templating engine. Since
it rewrites commands as it compiles the template, it is easy to just
rewrite it into a foreach. Then I noticed how clean it looked without
a trailing {break}, but really that is the only benefit, I'm not sure
if that would have much of a performance gain, maybe it would be more
clear, dunno yet. An array command like this would probably be a
bigger gain, and would condense into one statement what the intent
was.

The problem with simply adding these convenience subcommands is that
if you really want to be clear what you are doing in a program, you
might want to wrap something like this into a nice specific, maybe
namespace specific proc:

namespace eval ::deckOfCards {
variable Cards ; # later add all 54 cards
}

proc ::deckOfCards::removeCards { args } {
variable Cards
foreach card $args {array unset Cards $card}
}

But in a different situation, you may need to remove cards that were
already grouped into a hand:

proc ::deckOfCards::dealToHand { handDealt } {
variable Cards
foreach card $handDealt {array unset Cards $card}
}

Using a specialized array command would not be as clear, and the
developer and maintainer must both learn two new subcommands, and then
provide full pathnames to the array. And when the error message shows
up, wouldn't it be nice to see an error message from [dealToHand]
instead of [array unset ...]?

> I should note here that some of the ensemble-like commands in 8.5 really
> are ensembles, so you can add your own commands to them. Not all of them
> though; I ran out of time. :-)
>
> Donal (and [array] is one I didn't do IIRC).

I noticed this, and think it is very interesting. Is this done at the
C level, and if so, did you keep any notes on how to do it (other than
looking as source code), or maybe pointers to good examples of
particular issues? Is this similar or completely different from the
new mathfunc extension? So easy to add your own functions, although I
wish they worked in every expression (like [if] and [while]).

suchenwi

unread,
Dec 10, 2007, 3:39:56 AM12/10/07
to
On 10 Dez., 05:01, "tom.rmadilo" <tom.rmad...@gmail.com> wrote:
> EveryThing Is A String, as soon as it passes
> the bounds of the programming language itself, like to the OS and into
> memory and the CPU. The real il-lusion of de-lusion is thinking that
> your little integer object traverses the different layers on you
> computer or the internet as an integer.

No - at least what regards memory and CPU. Since Tcl 8.0, values
(Tcl_Objects) used as integers will have a binary integer
representation in addition to their string rep in memory; and when the
CPU does arithmetics with it, this integer rep is used.

George Peter Staplin

unread,
Dec 10, 2007, 3:54:30 AM12/10/07
to
Bryan Oakley wrote:
> tom.rmadilo wrote:
>> On Dec 7, 9:59 am, Bryan Oakley <oak...@bardo.clearlight.com> wrote:
>>
>>>State of the Onion,http://www.perl.com/pub/a/2007/12/06/soto-11.html
>>
>>
>> After reading the article, I think it is worth pointing out a few
>> facts.
>>
>> First is that Perl is such a poor language that it needs to be
>> reinvented. A not so subtle admission of a system which hasn't stood
>> the test of time.
>
> I think that's a bit too inflamatory. Lets keep language bashing to a
> minimum, please.
>
> I think most serious programmers would disagree completely with your
> statement. Perl is a fine language which has quite well stood the test
> of time from what I can see. It's just that ther Perl community, or
> Larry Wall, wants to make it better. What's wrong with that?


So, Larry Wall can be inflamatory, and lie, but we can't? That makes
sense.

"Put anger away, abandon pride, overcome every attachment, cling not to
Mind and Body and thus be free from sorrow.

One, who controls his anger when aroused, is like a clever driver who
controls a fast going carriage; the others are like those who merely
hold the reins.

Conquer the angry man by love; conquer the ill-natured man by goodness;
conquer the miser with generosity; conquer the liar with truth."

So, I do believe I'll conquer Larry Wall with truth, whilst loving him
as a person. A difficult thing to do.


George

Arjen Markus

unread,
Dec 10, 2007, 5:01:28 AM12/10/07
to
On 9 dec, 16:16, Óscar Fuentes <o...@wanadoo.es> wrote:
> Oscar- Tekst uit oorspronkelijk bericht niet weergeven -
>
> - Tekst uit oorspronkelijk bericht weergeven -

If my memory serves me the Tclers' Wiki has some examples
of how to deal with infinite lists. Maybe not as elegant as
Haskell, but we can still manage. (One of these pages was written by
me, but it has been a few years, so I may have
been too inexperienced and too enthousiastic to fully
appreciate its quirks ;))

Regards,

Arjen

Arjen Markus

unread,
Dec 10, 2007, 5:03:13 AM12/10/07
to

And of course, Tcl is not the only victim of this
practice. My other favourite programming language, Fortran,
gets the same treatment ... Boy, I know how to pick
my languages!

Regards,

Arjen

Larry W. Virden

unread,
Dec 10, 2007, 8:42:06 AM12/10/07
to
On Dec 8, 12:55 pm, "Donal K. Fellows"
<donal.k.fell...@manchester.ac.uk> wrote:

>
> I disagree. I give my opinion on all sorts of things all the time
> without finding out about them. The difference is, I'm usually right.
>
> Donal.

I know this thread already generated a nominee for Quote of the Week -
but I have to submit this one as well... for it is true.

Alan Anderson

unread,
Dec 10, 2007, 6:44:28 PM12/10/07
to
Darren New <dn...@san.rr.com> wrote:

> Cameron Laird wrote:
> > EIAS isn't a good thing?
>
> It has its drawbacks. Like, no garbage collection of referenced objects.

What is a "referenced object" and why should I care?

(I do know the answer to the first question, making it rhetorical, but
the second question is legitimate.)

Darren New

unread,
Dec 10, 2007, 8:04:30 PM12/10/07
to
Alan Anderson wrote:
> Darren New <dn...@san.rr.com> wrote:
>
>> Cameron Laird wrote:
>>> EIAS isn't a good thing?
>> It has its drawbacks. Like, no garbage collection of referenced objects.
>
> What is a "referenced object" and why should I care?

Because garbage collection is good? Especially of external resources
like file handles? If everything is a string, you have no pointers, and
hence no garbage collection.

I write long-running, robust software in Tcl. It would be enhanced by
the ability to use OO features without mistakes leading to leaks, for
example.

tom.rmadilo

unread,
Dec 10, 2007, 9:34:08 PM12/10/07
to
On Dec 10, 5:04 pm, Darren New <d...@san.rr.com> wrote:
> Alan Anderson wrote:
> > Darren New <d...@san.rr.com> wrote:
>
> >> Cameron Laird wrote:
> >>> EIAS isn't a good thing?
> >> It has its drawbacks. Like, no garbage collection of referenced objects.
>
> > What is a "referenced object" and why should I care?
>
> Because garbage collection is good? Especially of external resources
> like file handles? If everything is a string, you have no pointers, and
> hence no garbage collection.
>
> I write long-running, robust software in Tcl. It would be enhanced by
> the ability to use OO features without mistakes leading to leaks, for
> example.

I never could figure this out. I always assumed that if I didn't take
care to take out the garbage and also provide a clean slate for new
work, I was introducing potential unexplained behavior. I just fixed a
bug in this category today. I've never heard of an ideal garbage
collection system, so I assume that it is a black art, subject to a
lot of hand waving.

In Tcl, any object (variable, namespace, proc) with a name is a
pointer, if it is ultimately stored in a hash it must be a pointer
with a nice name in a limited scope.

But the idea that something must have a reference count that would
inform the need to garbage collect seems impossible in Tcl. If you
open a file and don't close it, it is still open. A namespace exists
until it is deleted. How do you count references on that?

So the first thing is to define what a non-referenced object would
look like. For many types of referenced objects, the only reasonable
idea (in Tcl) is to have an ownership relationship with a parent
object. When the parent object dies, so must the child objects. And
when parent objects are initialized or reinitialized, things are
cleaned up. This implies a controller above the level of the objects,
and if the controller fails, the entire system must fail and allow the
OS to do cleanup. So GC could be done with a registration process
which triggers on object exit, or you could use a centralized
registration with the controller. On some interval, the controller
will look at the list of registered objects. If the object exists,
nothing is done, otherwise the controller would destroy the child
objects.

I think there is more of a gap between ideal behavior and reality in
this area than in most because you must rely on some parallel or
separate mechanism to enforce cleanup, and require everyone to use the
conventions. And this is the problem. GC requires work, but usually
the offending code is the code which didn't do the work.

Darren New

unread,
Dec 10, 2007, 11:07:01 PM12/10/07
to
tom.rmadilo wrote:
> I've never heard of an ideal garbage
> collection system, so I assume that it is a black art, subject to a
> lot of hand waving.

Depends what you mean by "ideal".

> In Tcl, any object (variable, namespace, proc) with a name is a
> pointer,

In a sense.

> if it is ultimately stored in a hash it must be a pointer
> with a nice name in a limited scope.

Yes. That doesn't mean it gets collected in Tcl. It just means it's easy
to find.

> But the idea that something must have a reference count that would
> inform the need to garbage collect seems impossible in Tcl. If you
> open a file and don't close it, it is still open. A namespace exists
> until it is deleted. How do you count references on that?

Yes, exactly. Note that there *are* languages where you *can* do that.
EIAS prevents it to a large extent, especially since I can store "file"
and "13" in separate strings, then concatenate them and pass them to
[read] and expect it to work.

Alan Anderson

unread,
Dec 10, 2007, 11:31:36 PM12/10/07
to
Darren New <dn...@san.rr.com> wrote:

> > What is a "referenced object" and why should I care?
>
> Because garbage collection is good? Especially of external resources
> like file handles?

Garbage collection is good, but it requires knowing that something is
garbage. If I'm writing a program, I figure that *I* know best when
something can be thrown away.

> If everything is a string, you have no pointers, and
> hence no garbage collection.

You have no *automatic* garbage collection service in Tcl, but you can
still unset variables and close files and generally delete unnecessary
stuff yourself if you want to.

> I write long-running, robust software in Tcl. It would be enhanced by
> the ability to use OO features without mistakes leading to leaks, for
> example.

I don't follow you. How does a reference-counting garbage collection
system help you avoid making such mistakes? It seems to me that leaving
garbage collection up to "the system" makes it *easier* to accidentally
leave something in an unused but still-present state, because you don't
get in the habit of cleaning up after yourself.

Darren New

unread,
Dec 10, 2007, 11:59:43 PM12/10/07
to
Alan Anderson wrote:
> Garbage collection is good, but it requires knowing that something is
> garbage. If I'm writing a program, I figure that *I* know best when
> something can be thrown away.

Unfortunately, many who think that are mistaken.

>> If everything is a string, you have no pointers, and
>> hence no garbage collection.
>
> You have no *automatic* garbage collection service in Tcl, but you can
> still unset variables and close files and generally delete unnecessary
> stuff yourself if you want to.

That's not "garbage collection". :-)

>> I write long-running, robust software in Tcl. It would be enhanced by
>> the ability to use OO features without mistakes leading to leaks, for
>> example.
>
> I don't follow you. How does a reference-counting garbage collection
> system help you avoid making such mistakes?

Where did "reference counting" come into it? Reference counting is an
exceptionally poor (i.e., "broken") form of garbage collection.

> It seems to me that leaving
> garbage collection up to "the system" makes it *easier* to accidentally
> leave something in an unused but still-present state, because you don't
> get in the habit of cleaning up after yourself.

You've not actually used a garbage-collected language, have you? :-)

Uwe Klein

unread,
Dec 11, 2007, 4:09:22 AM12/11/07
to
With introspection you can't loose anything. or can you?

info commands, info vars, ...
chanel names, array names ,...
what nots

with c doing

char *lotsofmem;
lotsofmem = malloc(*largenumber);
lotsofmem = malloc(42);

has just lost you a strapping amount of memory
and no way of reclaiming without killing the app.

uwe

Donal K. Fellows

unread,
Dec 11, 2007, 5:47:14 AM12/11/07
to
Alan Anderson wrote:
> What is a "referenced object" and why should I care?

You shouldn't care. It's easier (for many programs) to instead write Tcl
code so that you can tie resource lifetime to some scripted scope, for
example like this (which uses 8.5-isms):

proc processFile {filename channelVariable script} {
upvar 1 $channelVariable ch
set ch [set chan [open $filename]]
catch {uplevel 1 $script} msg opt
close $chan
return -options $opt $msg
}

That procedure makes it much easier to work with files, like this:

processFile /foo/bar.dat f {
while {![eof $f]} {
frobnicate [read $f 10]
}
}

Guess what? Even if there's an error, the file will get safely closed
and you don't have to worry. There are alternate ways of doing this
based on variable unset traces; they too work well in non-namespace
scopes. Tying to a namespace's lifespan also works, but requires manual
cleanup as we can't guess for you when the NS is surplus. (GC is about
automagically guessing when resources are no longer needed.)

Donal.

tom.rmadilo

unread,
Dec 11, 2007, 11:25:14 AM12/11/07
to
On Dec 10, 8:07 pm, Darren New <d...@san.rr.com> wrote:

> Note that there *are* languages where you *can* do that.
> EIAS prevents it to a large extent, especially since I can store "file"
> and "13" in separate strings, then concatenate them and pass them to
> [read] and expect it to work.

The vast majority of strings in Tcl *do not* point to actual file
handles, just like the vast majority of valid looking references in
any other language.

In Tcl, objects (anyone object to me calling things in Tcl objects?)
are disconnected from each other in a sense, but they are all still
connected to the hash arrays which allow these objects to be referred
to. Maybe in some language you could have an object which creates a
child object. When the object dies, but fails to clean up the child,
this object becomes disconnected from the object tree. Then it would
be obvious to a GC routine which has an additional reference to the
object, that no other object can reference this object without trying
to guess where in memory it is located. At this point the GC routine
can also remove the reference and allow the memory to be reused. This
is exactly how you would do garbage collection in Tcl. At some point
in time a routine can look for left over references or temporary
namespaces and remove them. But this is just the result of a decision
that someone made. Anyone who wants or needs garbage collection in Tcl
can have it. This is how you write robust programs...you plan for
failure and make a decision on what needs to be done upon failure.

An open but unused file handle is probably the best example of a
failed programming strategy. How could the file handle be lost without
the environment which created the handle knowing about it? Maybe
something more important than a little bit of memory was lost here.
Being satisfied with cleaning up that is really throwing out the baby
with the bath water.

Donal's example below is a good example of how easy it is to write an
intelligent garbage collection routine. I call it explicit
programming. You make a decision on what defines failure and what to
do in case of failure. The processFile proc is a controller object. It
is more elegant than a 'finally' block used in some languages because
it can be reused and because it provides a new specialized word which
captures a larger concept. A 'finally' block is just an anonymous
script.

Darren New

unread,
Dec 11, 2007, 11:37:34 AM12/11/07
to
tom.rmadilo wrote:
> The vast majority of strings in Tcl *do not* point to actual file
> handles, just like the vast majority of valid looking references in
> any other language.

I haven't used IncrTcl a lot, but I understood that most of the
"object-oriented" objects actually wound up as commands, just like
widgets do.

> In Tcl, objects (anyone object to me calling things in Tcl objects?)
> are disconnected from each other in a sense, but they are all still
> connected to the hash arrays which allow these objects to be referred
> to.

You're saying Tcl gives you introspection. So?

> Donal's example below is a good example of how easy it is to write an
> intelligent garbage collection routine.

It's not garbage collection if you have to implement it yourself for
each application, any more than assembler is "object-oriented" because
it's possible to code OO-style applications in assembler.

Darren New

unread,
Dec 11, 2007, 11:38:24 AM12/11/07
to
Uwe Klein wrote:
> With introspection you can't loose anything. or can you?

But you can't clean it up, either.

OK, so call [file channels], and explain how I know which one is still
being used somewhere in an event loop, and which one isn't.

Darren New

unread,
Dec 11, 2007, 11:40:06 AM12/11/07
to
Donal K. Fellows wrote:
> Guess what? Even if there's an error, the file will get safely closed
> and you don't have to worry.

Manual resource reclamation is always easier with trivial programs. Now
go back into the event loop with a dozen sockets open with [after]
setting timeouts and cleaning up, and show the same mechanism?

tom.rmadilo

unread,
Dec 11, 2007, 11:45:27 AM12/11/07
to
On Dec 10, 8:07 pm, Darren New <d...@san.rr.com> wrote:
> tom.rmadilo wrote:
> > I've never heard of an ideal garbage
> > collection system, so I assume that it is a black art, subject to a
> > lot of hand waving.
>
> Depends what you mean by "ideal".

I'll restate it: ideal means no surprises. That means I know what is
going to happen, how and when. Since GC systems are necessarily
complex and work behind the scenes, hard to imagine any approaching
ideal.

tom.rmadilo

unread,
Dec 11, 2007, 12:02:15 PM12/11/07
to
On Dec 11, 8:40 am, Darren New <d...@san.rr.com> wrote:
> Donal K. Fellows wrote:
> > Guess what? Even if there's an error, the file will get safely closed
> > and you don't have to worry.
>
> Manual resource reclamation is always easier with trivial programs. Now
> go back into the event loop with a dozen sockets open with [after]
> setting timeouts and cleaning up, and show the same mechanism?

I guess the real question is how does this get you to the robust
programming model that you are after. Simple C programs have no
trouble handling this, it is called error detection. Programming is
not shake-n-bake, if you can't provide order to the process why do you
think GC is going to help?

Programming is the technique of making what seemed complex into the
trivial. If you think your program is non-trivial, then it needs some
work.

Otherwise if using an event loop makes things more complex it must be
a defect to use it. Language features are intended to simplify stuff
in a predictable way, in some situations the simplifications are not
helpful.

Uwe Klein

unread,
Dec 11, 2007, 12:12:22 PM12/11/07
to
Darren New wrote:
> Uwe Klein wrote:
>
>> With introspection you can't loose anything. or can you?
>
>
> But you can't clean it up, either.
>
> OK, so call [file channels], and explain how I know which one is still
> being used somewhere in an event loop, and which one isn't.
>
What good is a glas dagger.

You would need an DWISHW wrapping interpreter.

What kind of mechanism do you have in mind that
would do more than just translate this problem
between levels of removal?

uwe

DWITIW ~= Do What I Should Have Wanted

Darren New

unread,
Dec 11, 2007, 12:28:03 PM12/11/07
to
tom.rmadilo wrote:
> I guess the real question is how does this get you to the robust
> programming model that you are after.

Sorry. I can't condense 30 years of professional programming experience
in dozens of different languages and environments into a newsgroup post. :-)

> Simple C programs have no trouble handling this, it is called error detection.

Right. That's why you never see a virus spread via a mistake in a C
program, nor a core dump for C programs. Because, you know, error
detection is so trivial in C that nobody has any trouble with it.

> Programming is
> not shake-n-bake, if you can't provide order to the process why do you
> think GC is going to help?

Because GC allows diverse parts of the program to cooperate without
explicit source-level linkages between them. If six pieces of your
program need to use a resource, but you need to clean it up when all of
them are done with it, how do you do that in a way you don't need to
change any of those six when you add a seventh program portion using
that same resource?

> Programming is the technique of making what seemed complex into the
> trivial. If you think your program is non-trivial, then it needs some
> work.

No, some things are essentially non-trivial. If you don't understand
this, then no, you probably don't need garbage collection in your programs.

Tell you what: why don't you write C programs on an operating system
that doesn't deallocate allocated memory or close files or sockets when
your process exits, and get back to me on how easy C is. BTDTGTTS.

tom.rmadilo

unread,
Dec 11, 2007, 12:28:31 PM12/11/07
to
On Dec 11, 9:12 am, Uwe Klein <uwe_klein_habertw...@t-online.de>
wrote:

> DWITIW ~= Do What I Should Have Wanted

I think the correct acro is IDK,AIDK.JDI!: I don't know, and I don't
care. Just do it!

RHS

unread,
Dec 11, 2007, 2:05:45 PM12/11/07
to
There seems to be two (only partially related) topics here...

On Dec 10, 11:59 pm, Darren New <d...@san.rr.com> wrote:
> > I don't follow you. How does a reference-counting garbage collection
> > system help you avoid making such mistakes?
>
> Where did "reference counting" come into it? Reference counting is an
> exceptionally poor (i.e., "broken") form of garbage collection.

Different garbage collection types have different characteristics.
Reference Counting has benefits and limitations, but it's certainly
not broken. For example, it doesn't require the system resources that
a "sweep and mark" (or whatever it's called) GC needs, which can be
invaluable in real time systems.


> > It seems to me that leaving
> > garbage collection up to "the system" makes it *easier* to accidentally
> > leave something in an unused but still-present state, because you don't
> > get in the habit of cleaning up after yourself.
>
> You've not actually used a garbage-collected language, have you? :-)

Ok, I'll bite. I have used many different languages, including various
garbage collected ones. Taking Java as an example, you still need to
clean up after yourself (closing files, etc) because there's no
promises as to when your objects will be cleaned up (or that they ever
will) and, hence, no finalizers. If you get into the habit of not
cleaning up after yourself, this type of thing can bite you. That
being said, I'd love to see Tcl be able to clean up more things on
it's own (objects, for instance). However, this leads to the other
part of the conversation...

Afaict, Tcl, as a language, cannot be extended to clean up referenced
"things" (objects, file descriptors, etc) because of the way the
language works. This is due to the possibility of doing things like:

set text [read fd$index]

There's just no way to know if the program is "done" with a given
reference. Is it possible to make changes that allow this to be done?
Sure, but it's a pretty major language change to do so... one that
removes some of the flexibility/power of the language.

Some people would be willing to give up that flexiblity/power...
Some people would not be willing to give it up.

Robert

bill...@alum.mit.edu

unread,
Dec 11, 2007, 2:41:38 PM12/11/07
to
This whole discussion assumes an understanding of memory management in
Tcl, that is, not only of the facilities that the language provides,
like unset, but of what the implementation does and does not do. I'm
not aware of any good description of this. Is there a good description
of Tcl memory management? It seems like their ought to be.

tom.rmadilo

unread,
Dec 11, 2007, 3:21:00 PM12/11/07
to

Unless you write some extension in C (regular C or Tcl's C), you can
always reach all named items/objects in Tcl. That is why you can't
employ a generic garbage collection for Tcl. At the script level, Tcl
manages memory so you don't have to, and deallocates it when you tell
it to. This is the point! In many languages you have references which
are not connected to a central namespace (hash) except through a
parent object. If the parent goes away and doesn't clean up these
children, they take up space or continue to hold onto some other
resource. Since only the parent had a direct link, the program now has
no way of accessing the disconnected object. This is probably true of
any system where objects can create private data.

Given that you can always find all object in regular Tcl, all you have
to do is to decide to clean stuff up at certain points during a
process and apply some rules to do the cleanup. These points can
differ depending upon the garbage. Hey, it might not be garbage. If an
object self-destructs and leaves behind an open file, it could also
forget to return the shared handle so the next object can use it.
Closing a file is easy, returning an opaque handle has to be
specifically scheduled. A timed wait on an open channel is another
form of garbage collection. Without it, a dead client could tie up
resources which should be released.

In Tcl, garbage represents a programming mistake if it isn't cleaned
up 'in time'. The length of time that garbage can remain around
without causing operational problems depends very much on the type of
garbage, just like in the real world.

bill...@alum.mit.edu

unread,
Dec 11, 2007, 3:56:50 PM12/11/07
to
I think that many people assume some form of automatic deallocation.
For example, if I do:

set foo [gets $fh]
set foo [string trim $foo]

offhand I don't recall any documentation saying what happens to the
original string, that is, whether it is automatically freed since
there is no longer any way of accessing it. In this case of course one
might write:

set foo [string trim [gets $fh]]

but for one reason or another one may not do that.

What one does not often see, in my experience, is something like:

set foo [gets $fh]
set bar [string trim $foo]
unset foo

which is what one would expect in the absence of garbage collection.

So, it seems to me that neither the reference manual nor the books
that I have read provide much information about Tcl's memory
management, and that it would be useful to have an explicit
description available.

tom.rmadilo

unread,
Dec 11, 2007, 4:57:54 PM12/11/07
to
On Dec 11, 12:56 pm, billpo...@alum.mit.edu wrote:

> What one does not often see, in my experience, is something like:
>
> set foo [gets $fh]
> set bar [string trim $foo]
> unset foo

> So, it seems to me that neither the reference manual nor the books


> that I have read provide much information about Tcl's memory
> management, and that it would be useful to have an explicit
> description available.

The extra bytes are returned to the process, but not back to the OS.
If foo was large and you followed the last sequence, the process would
end up with a larger memory footprint, but it could reuse the memory
later; so if you looped over that sequence, you would not grow any
larger than the first loop. If looping over the same code where you
unset the created variables, or reuse them, memory should not grow. If
it does, something is leaking.

As far as memory management, you could check out how the Tcl C API
allocates and cleans up. There is something called a DString for
dynamic string. Any code written at the C level must take care to
handle allocation and cleanup and I think that mostly means following
the conventions and API designed specifically for that purpose. Tcl
includes the C API, discussing Tcl simply as a scripting language, and
not as a method of quickly adding safe script level access to your
application or library miss the larger issue: application
development.

bill...@alum.mit.edu

unread,
Dec 11, 2007, 5:33:06 PM12/11/07
to
I think we're ships passing in the night here: I'm not asking for my
own sake what happens in the cases I mentioned, but rather pointing
out the lack of documentation for this. How is the programmer using
Tcl, even one familiar with such issues, to know what Tcl does, short
of reading the source? Somewhere there should be a description of what
Tcl does.

Donal K. Fellows

unread,
Dec 11, 2007, 6:32:44 PM12/11/07
to
tom.rmadilo wrote:
> Donal's example below is a good example of how easy it is to write an
> intelligent garbage collection routine.

No, it's Resource Allocation Is Initialization, a resource management
strategy developed originally in C++ that's trivial to adapt.

Donal.

Donal K. Fellows

unread,
Dec 11, 2007, 6:48:36 PM12/11/07
to
Darren New wrote:
> Manual resource reclamation is always easier with trivial programs. Now
> go back into the event loop with a dozen sockets open with [after]
> setting timeouts and cleaning up, and show the same mechanism?

So, you jump straight from the easiest case to close-on the most
complex one. Yeah, that's going to be *real* representative...

In any case, you think that GC would make that case easier? From the
floundering mess I've seen people get into with Java-implemented
service engines, the answer to that particular rhetorical question is
resoundingly in the negative. (Indeed, it's far easier to get Tcl
right in this area; you just need to remember to close the socket at
the right point in the main fileevent callback.)

Donal.

Donal K. Fellows

unread,
Dec 11, 2007, 6:49:55 PM12/11/07
to
tom.rmadilo wrote:
> The extra bytes are returned to the process, but not back to the OS.

Actually, that depends on all sorts of factors, many beyond our
control. But high-water-mark malloc()s are the most common kind.

Donal.

Darren New

unread,
Dec 11, 2007, 9:35:49 PM12/11/07
to
RHS wrote:
> Reference Counting has benefits and limitations, but it's certainly
> not broken.

"Broken" in the sense that it can't collect circular references. Got a
doubly-linked list? Better not try to reference count it.

Works great when you're reference-counting *values*, sure. Which is why
it works well for Tcl. EIAS.

> Ok, I'll bite. I have used many different languages, including various
> garbage collected ones. Taking Java as an example, you still need to
> clean up after yourself (closing files, etc)

Yes, because files aren't garbage collected. Java isn't a really great
example.

> Afaict, Tcl, as a language, cannot be extended to clean up referenced
> "things" (objects, file descriptors, etc) because of the way the
> language works. This is due to the possibility of doing things like:
>
> set text [read fd$index]
>
> There's just no way to know if the program is "done" with a given
> reference.

Exactly. Hence, my assertion that EIAS has some drawbacks. Which is
where I came into this discussion.

Darren New

unread,
Dec 11, 2007, 9:41:33 PM12/11/07
to
Donal K. Fellows wrote:
> Darren New wrote:
>> Manual resource reclamation is always easier with trivial programs. Now
>> go back into the event loop with a dozen sockets open with [after]
>> setting timeouts and cleaning up, and show the same mechanism?
>
> So, you jump straight from the easiest case to close-on the most
> complex one.

From the straightforward case to the realistic kind of things I work
with, I'd say.

> Yeah, that's going to be *real* representative...

'tis for me! Indeed, that's *exactly* the problem I logged a bug about.

RAII doesn't work unless you have LIFO-based allocations of everything
you're RAIIing, which is rarely the case at least in my programs.

> In any case, you think that GC would make that case easier?

Yes. Still not trivial, but easier.

I'm not saying Tcl is broken. I'm simply saying that EIAS prevents GC,
which means you have to manually clean up your resources, unlike some
other languages.

tom.rmadilo

unread,
Dec 11, 2007, 9:47:40 PM12/11/07
to
On Dec 11, 3:49 pm, "Donal K. Fellows" <donal.k.fell...@man.ac.uk>
wrote:

It is very helpful to have input from those who use the Tcl C API, I
hope we get more, I was only hoping to provide some general guidance
to someone who hasn't found an easily understood description of memory
management in Tcl.

In this respect, the first thing to know is that Tcl has two APIs, one
in C and the more famous scripting API. In addition, it is the
responsibility of those creating new scripting API to handle
allocation and provide for freeing of resources.

If you don't take into account the two distinct APIs and the implied
contract that the scripting API is safe from resource leaks, it will
be difficult to understand Tcl. The Tcl scripting API has no direct
control over memory management and must rely completely on the C code
which created the scripting API.

George Peter Staplin

unread,
Dec 12, 2007, 1:29:26 AM12/12/07
to
Darren New wrote:
> RHS wrote:
>> Reference Counting has benefits and limitations, but it's certainly
>> not broken.
>
> "Broken" in the sense that it can't collect circular references. Got a
> doubly-linked list? Better not try to reference count it.

Knuth's TAOCP has a reference to a paper that includes a solution for
handling circular references with reference-counted GC.


George

suchenwi

unread,
Dec 12, 2007, 3:35:09 AM12/12/07
to
On 11 Dez., 21:56, billpo...@alum.mit.edu wrote:
> What one does not often see, in my experience, is something like:
>
> set foo [gets $fh]
> set bar [string trim $foo]
> unset foo
>
> which is what one would expect in the absence of garbage collection.

The recommended way is to put code in procs, and use local variables
wherever possible. Those are cleaned up on return, so the user doesn't
have to bother with [unset] most any time.

davidn...@gmail.com

unread,
Dec 12, 2007, 3:35:44 AM12/12/07
to
From what I've seen, files are something that can be dealt with
explicitly, more or less, but where real GC would really help is in
dealing with objects. OO objects, to be clear. Having to clean them
up yourself puts you at a disadvantage compared to Python and Ruby.

Donal K. Fellows

unread,
Dec 12, 2007, 4:11:53 AM12/12/07
to
tom.rmadilo wrote:
> It is very helpful to have input from those who use the Tcl C API, I
> hope we get more, I was only hoping to provide some general guidance
> to someone who hasn't found an easily understood description of memory
> management in Tcl.

It's a lot of work, and is subject to frequent change. Start a page on
the Wiki... :-)

Donal.

Donal K. Fellows

unread,
Dec 12, 2007, 4:13:37 AM12/12/07
to
suchenwi wrote:
> The recommended way is to put code in procs, and use local variables
> wherever possible. Those are cleaned up on return, so the user doesn't
> have to bother with [unset] most any time.

It should be noted that there are other very good reasons for using
procedures too. In particular, code goes faster. :-)

Donal.

Alexandre Ferrieux

unread,
Dec 12, 2007, 6:04:08 AM12/12/07
to
On Dec 12, 9:35 am, "davidnwel...@gmail.com" <davidnwel...@gmail.com>
wrote:

Yes. And it may help to analyze what makes something GC-able or not in
EIAS context. To me, everything is linked to the transparency of the
object:

- if it's transparent, in that it is completely equivalent to its
string rep (no hidden state, no loss of functionality in O->S->O
across process or machine boundary), then it can be GCed just like
other Tcl values.

- otherwise (keys in some independent table, like handles or proc
names), it must be handled manually.

Then, looking at OO, it turns out that most of the time object
"values" are of the "handle" variety. I'm no OO expert, so I don't
know how many systems violate this "rule". But if I were to invent an
Tcl OO system with GC-ed objects, I'd start this way:

- implement objects as lists or dicts[*]
- use [unknown] to allow for the traditional object notation

$obj method args....

[*] yes, dicts have just become bidirectionally equivalent to their
string reps with Donal's addition of order-of-insertion remembrance.

Of course, one constraint is that the transparency be recursive: the
object must not contain opaque handles inside; or at least the
programmer mustn't believe that those handles will be intelligently
dealt with, since there is no Tcl-level [delete] method to intercept
GC-deallocation.

-Alex

Donal K. Fellows

unread,
Dec 12, 2007, 10:07:18 AM12/12/07
to
Alexandre Ferrieux wrote:
> Yes. And it may help to analyze what makes something GC-able or not in
> EIAS context. To me, everything is linked to the transparency of the
> object:

That's a reasonable analysis.

Donal.

tom.rmadilo

unread,
Dec 12, 2007, 11:04:02 AM12/12/07
to
On Dec 11, 3:32 pm, "Donal K. Fellows" <donal.k.fell...@man.ac.uk>
wrote:

Good to know the technical term. Resource management gets right at the
heart
of the matter with Tcl. Maybe I have a simple view, but if you manage
your
resources (chans, memory, handles) correctly, you might not have too
much
garbage to collect. In most of my code, each namespace deals with a
particular set of data and the namespace has an [init] command.
Correct
operation would call this at the beginning of a particular chunk of
work and
potentially register cleanup:

namespace eval ::twt::db::handle {

variable initialized 0
variable ids
}

# First time a db handle is needed for a conn:
proc ::twt::db::handle::init { } {

ns_atclose ::twt::db::handle::reset
variable initialized 1
variable ids

array unset ids
}

proc ::twt::db::handle::reset { } {

variable initialized
variable ids

foreach id [array names ids] {
set pool [lindex $ids($id) 3]
set dataSource [lindex $ids($id) 1]
::twt::db::datasource::addPool $dataSource $pool
}

array unset ids
set initialized 0
}

So this namespace manages an array 'ids'. Now the user can use the
commands in
this namespace and not worry about initialization and resource
management.

Of course there are several other layers below this where the actual
resources are managed. This code just gets an exclusive reference to a
resource and makes sure it gets returned at the end of the work cycle.
This is kind of like how a public library works. You check out a book,
the library records the fact and relies on you to return it on time.
But in case that doesn't work out, they have ways of dealing with the
situation because they have a record of what was supposed to happen.

Alan Anderson

unread,
Dec 12, 2007, 9:31:29 PM12/12/07
to
Darren New <dn...@san.rr.com> wrote:

> If six pieces of your
> program need to use a resource, but you need to clean it up when all of
> them are done with it, how do you do that in a way you don't need to
> change any of those six when you add a seventh program portion using
> that same resource?

Is this a trick question? The problem is weakly specified, but the
obvious answer is to use a semaphore to keep track of the resource
usage, and to clean it up when the semaphore shows that it has become
unused.

Darren New

unread,
Dec 12, 2007, 9:45:08 PM12/12/07
to
Alan Anderson wrote:
> Is this a trick question? The problem is weakly specified, but the
> obvious answer is to use a semaphore to keep track of the resource
> usage, and to clean it up when the semaphore shows that it has become
> unused.

Well, yeah. How do you think garbage collection works? Are you saying
"Duh, it's possible to implement garbage-collection techniques manually
every time you need it?" Sure. Welcome to turing completeness.

Alan Anderson

unread,
Dec 12, 2007, 10:02:11 PM12/12/07
to
Darren New <dn...@san.rr.com> wrote:

> Alan Anderson wrote:
> > Is this a trick question? The problem is weakly specified, but the
> > obvious answer is to use a semaphore to keep track of the resource
> > usage, and to clean it up when the semaphore shows that it has become
> > unused.
>
> Well, yeah. How do you think garbage collection works? Are you saying
> "Duh, it's possible to implement garbage-collection techniques manually
> every time you need it?" Sure. Welcome to turing completeness.

My philosophy is this: I'm the one writing the program, so I'm the one
who knows best when it's appropriate to clean things up. I know which
"garbage-collection technique" is most appropriate for the specific
issue I'm facing. I don't like the indeterminacy that goes along with
"automatic" garbage collection. Since my formative years were spent
programming what are now called "embedded systems", I have a strong urge
to be as spartan as possible and to clean up after myself. A garbage
collection system that tries to clean things up for me is almost always
less efficient than something I can implement easily and explicitly, and
is sometimes a source of subtly inconsistent behavior.

Darren New

unread,
Dec 13, 2007, 12:17:43 AM12/13/07
to
Alan Anderson wrote:
> My philosophy is this: I'm the one writing the program, so I'm the one
> who knows best when it's appropriate to clean things up.

That may be true, but given the number of core dumps and such out there,
it's clearly not true for everyone. GC isn't always for efficiency. It's
also for correctness.


> A garbage collection system that tries to clean things up for me is almost always
> less efficient than something I can implement easily and explicitly,

You probably dislike compilers ignoring your "register" declarations
too, I suspect. :-)

tom.rmadilo

unread,
Dec 13, 2007, 1:00:08 AM12/13/07
to
On Dec 12, 9:17 pm, Darren New <d...@san.rr.com> wrote:
> > A garbage collection system that tries to clean things up for me is almost always
> > less efficient than something I can implement easily and explicitly,
>
> You probably dislike compilers ignoring your "register" declarations
> too, I suspect. :-)

I just wish I could figure out why you are using Tcl. Tcl doesn't have
a garbage collection problem, doesn't have registers. Maybe just point
everyone to the GC system you are talking about instead of describing
how well it works. You already said it isn't Java. Can't be C, since
you have trouble with that language. Instead of knocking down every
reasonable solution to your ambiguous super-complex problem, get
specific. Philosophy doesn't do much in the programming world. What GC
system are you talking about?

When I point out that you can't lose things in Tcl, you say so what,
that is just introspection. No it isn't just introspection.
Introspection is possible because you can't lose things in Tcl. If you
can, it is a bug in the supporting C code. Wait! Tcl is written in C,
but you said it is too difficult to write robust programs in C. All
those Tcl C API viruses. Why are you using Tcl, virus prone Tcl?

It is like you are the first person to ever write a program which must
share resources. God help us if you ever figure it out. One thing is
for sure, you are not the first programmer that thinks you are going
to get something like GC for free, but why anyone would ever GC an
open socket or file and think that is robust, please just point me to
the textbook which lead you down that dead end.

So, maybe some references on a non-Tcl GC which can do what you want.

Maybe you can also send a reference to the OS you are using which
doesn't close open files etc. when the process exits. Is this the
system you are writing the robust application on? Or is that just
another useless example that nobody is talking about?

davidn...@gmail.com

unread,
Dec 13, 2007, 4:17:25 AM12/13/07
to
> I just wish I could figure out why you are using Tcl. Tcl doesn't have
> a garbage collection problem, doesn't have registers. Maybe just point
> everyone to the GC system you are talking about instead of describing
> how well it works. You already said it isn't Java. Can't be C, since
> you have trouble with that language. Instead of knocking down every
> reasonable solution to your ambiguous super-complex problem, get
> specific. Philosophy doesn't do much in the programming world. What GC
> system are you talking about?

Primarily, people are choosing Python and Ruby right now, both of
which have more "advanced" GC systems than Tcl does. Of course, GC is
not the reason they're ignoring Tcl, but those languages are worth
looking at as examples of what the market seems to want at the
moment. I think it's a bit silly to have a "high level" language that
makes you clean up after yourself just like in C. If I wanted that, I
would have just used C, which is faster in any case. Tcl actually
does a decent amount of cleaning up as things stand now, so why
distinguish between simple values like strings, lists and numbers, and
more complex objects?

EKB

unread,
Dec 13, 2007, 9:01:15 AM12/13/07
to
On Dec 13, 1:00 am, "tom.rmadilo" <tom.rmad...@gmail.com> wrote:
> On Dec 12, 9:17 pm, Darren New <d...@san.rr.com> wrote:
>
>
> When I point out that you can't lose things in Tcl, you say so what,
> that is just introspection. No it isn't just introspection.
> Introspection is possible because you can't lose things in Tcl. If you
> can, it is a bug in the supporting C code. Wait! Tcl is written in C,
> but you said it is too difficult to write robust programs in C. All
> those Tcl C API viruses. Why are you using Tcl, virus prone Tcl?

I've just got to point out that he's not saying it's impossible to
write robust C programs, just that it leaves a lot more work for the
programmer.

As an example, I use a memory management library for my C programs. It
helps a lot, but I have to explicitly use it and link it every time.
It's something I have to think about. A GC system would make it so
that I don't need to think about it. (But FWIW my favorite languages
are still non-GC languages: C & Tcl.)

>
> It is like you are the first person to ever write a program which must
> share resources. God help us if you ever figure it out. One thing is
> for sure, you are not the first programmer that thinks you are going
> to get something like GC for free, but why anyone would ever GC an
> open socket or file and think that is robust, please just point me to
> the textbook which lead you down that dead end.
>

I think all he's saying is that GC can help take care of this for you.
It's not *impossible* in Tcl or any other language, just harder. For
the applications that I create it is not too much of a burden, so it's
not enough of a draw to make me want to use a GC language.

Also, I think he's right that it's really hard in Tcl to make
automated GC because EIAS. As far as I can tell, that's really all
he's saying. The discussion I've seen here tells me that there are
many Tcl-specific facilities that can help you keep track of
resources and clean up after them, but the language is sufficiently
flexible (yay!!) that it's hard to make a generic automated GC.

My 2 cents.

Eric


Darren New

unread,
Dec 13, 2007, 10:32:15 AM12/13/07
to
tom.rmadilo wrote:
> [lots of blathering straw-man arguments trying to be insulting]

(plonk)


> Maybe you can also send a reference to the OS you are using which
> doesn't close open files etc. when the process exits.

AmigaDOS. Verix.

tom.rmadilo

unread,
Dec 13, 2007, 11:08:29 AM12/13/07
to
On Dec 13, 7:32 am, Darren New <d...@san.rr.com> wrote:

> tom.rmadilo wrote:
> > Maybe you can also send a reference to the OS you are using which
> > doesn't close open files etc. when the process exits.
>
> AmigaDOS. Verix.

Okay, I'll avoid those when I get my next server, oh, wait. I know how
to close files when I'm done with them, so nevermind. Tcl does too,
doesn't even wait for the OS to do it. I seem to remember discussing
this here WRT [exit]. So if Tcl runs on AmigaDOS or Verix, this isn't
an issue.

Do you have the GC reference? And a reference to using GC to close
open files and manage shared resources?

Alan Anderson

unread,
Dec 13, 2007, 8:17:41 PM12/13/07
to
EKB <er...@kb-creative.net> wrote:

> I've just got to point out that he's not saying it's impossible to
> write robust C programs, just that it leaves a lot more work for the
> programmer.

I guess I don't understand the mindset that makes behind-the-scenes
garbage collection a priority. So far as I'm concerned, it *is* the job
of the programmer to make the programs work. I probably don't create
the kind of programs that could benefit from having the system do such
things for me, because I don't really know what I would do differently
if there were a garbage collection service running. I just don't see it
making any less work for me.

EKB

unread,
Dec 13, 2007, 8:48:12 PM12/13/07
to
On Dec 13, 8:17 pm, Alan Anderson <arand...@insightbb.com> wrote:

Actually, for my own work, I feel the same way. I don't see a strong
need for it, unlike the strong need that I felt for dynamic arrays.
(I'm willing to type "malloc" hundreds of times in a C program, but it
doesn't mean I like it.) What I suspect, though, is that it isn't so
much a mindset as a class of applications that calls for GC, most
likely the type of app I'm not working on. I haven't got large numbers
of shared resources, and I know just where each of my resources is and
what it's doing, because that part of my program is pretty tame. When
I write complicated programs, the complexities lie elsewhere (usually
in the maths or the user interface). But I can imagine that for some
problems GC could give a huge productivity boost.

From my own humble experience working with Tcl I suspect that there
are a number of clever things you could do with Tcl to make it easier
to build apps where GC is a productivity booster. But I don't know
enough about those sorts of problems.

Hm... I just read what I wrote and it looked kind of lame. I'm
basically saying "I dunno!" But I felt like I had to say something,
because I'm very curious about the topic, but this thread was
producing more heat than light for me.

Darren New

unread,
Dec 13, 2007, 9:19:45 PM12/13/07
to
Alan Anderson wrote:
> I guess I don't understand the mindset that makes behind-the-scenes
> garbage collection a priority.

I didn't say it was a priority. I simply said EIAS made it much more
difficult to do anything automated in the way of garbage collection (for
objects other than strings, of course).

Tcl is pretty good at what it does. But then I code in Tcl differently
than I code in languages that have GC and pointer-style references and such.

tom.rmadilo

unread,
Dec 13, 2007, 11:32:11 PM12/13/07
to
On Dec 13, 5:48 pm, EKB <e...@kb-creative.net> wrote:
> Hm... I just read what I wrote and it looked kind of lame. I'm
> basically saying "I dunno!" But I felt like I had to say something,
> because I'm very curious about the topic, but this thread was
> producing more heat than light for me.

I admit being in the same category. I am very interested in anything
which would make my job easier and my code more reliable. When I say
"I dunno!" what I mean is that I have never run into this issue, and
it isn't because I don't write large applications, but because I have
an organic aversion to complexity. I simply do not understand the
difference between taking care of one shared resource and taking care
of hundreds or thousands of them. I don't understand the difference
between code which correctly handles a single resource's life cycle
and correctly handling the life cycle of many resources. Code either
works or it doesn't. Doing something twice doesn't change what must be
done. Doing two things in parallel doesn't change what must be done.
An algorithm does not change based upon how many times it runs or the
number of simultaneous invocations which exist. Management and data
may change, but not the algorithm. And that includes lots of things
which have nothing to do with garbage collection.

This thread is really about uninformed comments by otherwise
intelligent programmers. Talking about GC in the Tcl scripting
environment makes no sense. A bug free Tcl scripting environment
cannot create a non-referenced object. Programmers who somehow insist
that the Tcl scripting environment needs GC do not understand Tcl.
Some of us (including me) have tried to stretch the definition of GC
to include closing files, and other things which fall into the
category of cleanup, this is really a mistake. It is easy to deal with
unreferenced memory. GC does not handle anything more complex than
memory, and if it does, it is an obvious bug. Fortunately, in Tcl GC
is meaningless, so anything we try to put under that category is also
meaningless.

Think about this: the basic contract with Tcl is that you cannot
create disconnected objects (memory or otherwise). If you create a Tcl
command which violates this contract, it should be considered a bug.
Therefore GC in Tcl should be impossible. The only reason that memory
is consumed is due to programming code. The only reason memory is not
released is because the program has not released it. This property of
the Tcl language is a huge benefit. If you notice your Tcl process
increasing in memory usage when it shouldn't, you know you have a
potential memory leak. In Tcl you can separate these into two
categories: C level 'real' leaks. and Tcl level programming mistakes.

davidn...@gmail.com

unread,
Dec 14, 2007, 4:11:38 AM12/14/07
to
> This thread is really about uninformed comments by otherwise
> intelligent programmers. Talking about GC in the Tcl scripting
> environment makes no sense. A bug free Tcl scripting environment
> cannot create a non-referenced object. Programmers who somehow insist
> that the Tcl scripting environment needs GC do not understand Tcl.

Tom, I wouldn't be quite so quick to say this.

> Think about this: the basic contract with Tcl is that you cannot
> create disconnected objects (memory or otherwise). If you create a Tcl
> command which violates this contract, it should be considered a bug.
> Therefore GC in Tcl should be impossible. The only reason that memory
> is consumed is due to programming code. The only reason memory is not
> released is because the program has not released it. This property of
> the Tcl language is a huge benefit. If you notice your Tcl process
> increasing in memory usage when it shouldn't, you know you have a
> potential memory leak. In Tcl you can separate these into two
> categories: C level 'real' leaks. and Tcl level programming mistakes.

The problem is that Tcl has one mechanism for keeping track of EIAS
style objects: strings, numbers, lists, dicts, and so on.

proc foo {} {
set bar [list a b c d e]
}

bar will go away when foo passes out of scope, and gets nicely cleaned
up without the programmer having to worry about unsetting it
themselves.

Now, consider this:

package require Itcl

proc main {} {

itcl::class foo {
method bop {} {
puts bop
}
}

set bar [foo bar]
$bar bop
}

main

bar bop

bar hasn't gone away at all! It has been created as a named object.
That means that, rather than a variable holding a reference to it,
there's an internal hash table. *This* is where the problem lies.
Anything that can't truly be represented as a string (files, objects,
etc...) in Tcl works like this: you create an internal hash table,
where you put a made up string, like "foo", or "file8", or whatever,
and the value of the hash entry is a pointer to whatever fancy object
you have created. These hash tables are *not* automatically cleaned
up in any way, shape or form.

Even the humble http command creates things that it leaves lying
around, waiting for manual cleanup by the programmer:

package require http

proc main {} {
set tok [http::geturl http://www.tcl.tk]
puts $tok
}

main

puts [http::status ::http::1]

It's still accessible even though it's out of scope! A little
different than the Incr Tcl example in that Tcl itself still has a
reference to the data, and with 8.5 perhaps the whole thing could be
cleaned up to utilize a dict, but still...

In Ruby or Python, you don't have to worry about cleaning things like
that up. They are garbage collected for you when your *program* no
longer references them.

EKB

unread,
Dec 14, 2007, 7:37:07 AM12/14/07
to
On Dec 14, 4:11 am, "davidnwel...@gmail.com" <davidnwel...@gmail.com>
wrote:
> set tok [http::geturlhttp://www.tcl.tk]

> puts $tok
>
> }
>
> main
>
> puts [http::status ::http::1]
>
> It's still accessible even though it's out of scope! A little
> different than the Incr Tcl example in that Tcl itself still has a
> reference to the data, and with 8.5 perhaps the whole thing could be
> cleaned up to utilize a dict, but still...
>
> In Ruby or Python, you don't have to worry about cleaning things like
> that up. They are garbage collected for you when your *program* no
> longer references them.

Thanks! That helps clarify things for me.

> I simply do not understand the difference between taking care of one
> shared resource and taking care of hundreds or thousands of them.

To me, the difference is in the probability of a hard-to-track-down
bug entering the program. I'm careful, but not perfect, so there's a
small but nonzero probability each time I do something that I'll
introduce a bug, and the more of them there are in a program, the more
likely it is that I'll introduce a bug. That's why I use a memory
management library in C. I have *lots* of mallocs and free()'s, and
it's nice to have this lightweight system counting references for me
when I'm in debug mode. If there were only a handful, it wouldn't be
worth it. With a lot of them, it is.

Donal K. Fellows

unread,
Dec 14, 2007, 8:12:35 AM12/14/07
to
tom.rmadilo wrote:
> But the idea that something must have a reference count that would
> inform the need to garbage collect seems impossible in Tcl.

It's not impossible as it happens, since you can do tricks with the
reference counting in Tcl_Objs. It's not something that's strictly
within the language semantics (boo!) but it works pretty well anyway, as
users of Jacl, TclBlend and TCOM can attest. (It's not full-blown GC,
but that's actually not needed so often as there's not that much need to
do circular structure references in Tcl.)

There are two main problems with it: 1) references are not synthesizable
2) references are fragile. Both are flip-sides of the same coin, and
that is due to the fact that these magical values are not transparent.
But it seems to not be a particularly significant problem in practice.

Donal (non-manifest types are another issue).

tom.rmadilo

unread,
Dec 14, 2007, 9:25:13 AM12/14/07
to
On Dec 14, 5:12 am, "Donal K. Fellows"

<donal.k.fell...@manchester.ac.uk> wrote:
> tom.rmadilo wrote:
> > But the idea that something must have a reference count that would
> > inform the need to garbage collect seems impossible in Tcl.
>
> It's not impossible as it happens, since you can do tricks with the
> reference counting in Tcl_Objs. It's not something that's strictly
> within the language semantics (boo!) but it works pretty well anyway, as
> users of Jacl, TclBlend and TCOM can attest. (It's not full-blown GC,
> but that's actually not needed so often as there's not that much need to
> do circular structure references in Tcl.)

Right, but Jacl, TclBlend and TCOM are Tcl applications. If you get to
the C level, where pointers exist you can do this, if you expose
pointers to the Tcl scripting level, I guess you could do it too, but
AFAIK, regular Tcl you can't lose a reference and not consider that a
bug, since there would be no way to remove it.

I wish someone would explain why the need to GC could be considered a
good thing if you use a language that strictly speaking cannot create
garbage.

Donal K. Fellows

unread,
Dec 14, 2007, 10:42:56 AM12/14/07
to
tom.rmadilo wrote:
> Right, but Jacl, TclBlend and TCOM are Tcl applications.

They're not applications. They're extensions (or, in the case of Jacl, a
reimplementation of the language).

> If you get to
> the C level, where pointers exist you can do this, if you expose
> pointers to the Tcl scripting level, I guess you could do it too, but
> AFAIK, regular Tcl you can't lose a reference and not consider that a
> bug, since there would be no way to remove it.

As far as I know, regular Tcl has none of those sorts of bugs (which
would manifest as memory leaks). Regular Tk is not so great, though
there are steps there that can be taken to limit the impact.

Donal.

tom.rmadilo

unread,
Dec 14, 2007, 12:22:44 PM12/14/07
to
On Dec 14, 7:42 am, "Donal K. Fellows"

<donal.k.fell...@manchester.ac.uk> wrote:
> tom.rmadilo wrote:
> > Right, but Jacl, TclBlend and TCOM are Tcl applications.
>
> They're not applications. They're extensions (or, in the case of Jacl, a
> reimplementation of the language).

According to Ousterhout's _Tcl and the Tk Toolkit_ (Part III: Writing
Tcl Applications in C), extensions written in C are called
applications. That is where I get the usage from. But if you are
speaking about the way the software is used, TCOM probably isn't an
application, but a loadable library. TclBlend is a specialized tclsh,
and tclsh is a Tcl application, at least according to every manpage
I've ever seen: http://www.tcl.tk/man/tcl8.5/

Donal K. Fellows

unread,
Dec 14, 2007, 4:59:04 PM12/14/07
to
tom.rmadilo wrote:

> Donal K. Fellows wrote:
>> tom.rmadilo wrote:
>>> Right, but Jacl, TclBlend and TCOM are Tcl applications.
>> They're not applications. They're extensions (or, in the case of Jacl, a
>> reimplementation of the language).
>
> According to Ousterhout's _Tcl and the Tk Toolkit_ (Part III: Writing
> Tcl Applications in C), extensions written in C are called
> applications. That is where I get the usage from.

That's really out of date. Indeed, it was a usage that didn't last much
beyond when that book was written: people quite quickly distinguished
between writing reusable chunks of code (extensions/packages/modules)
and one-off code (applications). The introduction of the [load] command
made a great many people's lives easier, since it meant that most of us
could give up on the building of "Big Wish" applications; by reinforcing
the difference between extensions and applications, tremendous strength
was gained. Nowadays people (well, canny Tcl programmers) won't think
twice about mixing code from many different binary authors into a single
application, a usage that was a major feat of technical gymnastics in
Tcl 7.4 (I know; I remember still...)

It's a shame that TclBlend can't be a normal loadable extension; I
suspect that the problem is on the Java side. Probably relates to Java's
use of C++ for it's FFI... :-\

Donal.

Darren New

unread,
Dec 14, 2007, 6:21:39 PM12/14/07
to
tom.rmadilo wrote:
> I wish someone would explain why the need to GC could be considered a
> good thing if you use a language that strictly speaking cannot create
> garbage.

Your view of "garbage" is too limited.

tom.rmadilo

unread,
Dec 14, 2007, 9:38:03 PM12/14/07
to
On Dec 14, 3:21 pm, Darren New <d...@san.rr.com> wrote:
> tom.rmadilo wrote:
> > I wish someone would explain why the need to GC could be considered a
> > good thing if you use a language that strictly speaking cannot create
> > garbage.
>
> Your view of "garbage" is too limited.

All you need to do is to define what you think it is, and that means
identifying a strategy for determining what is garbage and what isn't.
I'm not trying to limit my view of garbage, but without a strategy to
identify it, you can't have any view of what it is, and nobody can
agree or disagree with something which isn't defined.


Darren New

unread,
Dec 14, 2007, 9:57:41 PM12/14/07
to
tom.rmadilo wrote:
> All you need to do is to define what you think it is, and that means
> identifying a strategy for determining what is garbage and what isn't.

There ya go. When EIAS, that's much harder.

So, is the file handle pointed to in a Tcl process by the string "file6"
garbage?

How about the file handle pointed to be a (say) "open_file_handle"
struct in a language with pointers, but which no pointer points to and
hence cannot be referenced by any construct in the language?

Does
{ char * x = malloc(1000);
x = NULL; }
generate any garbage in a C program running as a process under Linux?
By your definition, no, because the kernel can still free up the space
pointed to by the pointer. I.e., it's in a "hash table" somewhere.

How about a temp file you create, write to, and close, but never delete?
Is that garbage? Can Tcl "strictly" do this?

tom.rmadilo

unread,
Dec 14, 2007, 10:07:27 PM12/14/07
to
On Dec 14, 1:59 pm, "Donal K. Fellows"

<donal.k.fell...@manchester.ac.uk> wrote:
> tom.rmadilo wrote:
> > Donal K. Fellows wrote:
> >> tom.rmadilo wrote:
> >>> Right, but Jacl, TclBlend and TCOM are Tcl applications.
> >> They're not applications. They're extensions (or, in the case of Jacl, a
> >> reimplementation of the language).
>
> > According to Ousterhout's _Tcl and the Tk Toolkit_ (Part III: Writing
> > Tcl Applications in C), extensions written in C are called
> > applications. That is where I get the usage from.
>
> That's really out of date. Indeed, it was a usage that didn't last much
> beyond when that book was written: people quite quickly distinguished
> between writing reusable chunks of code (extensions/packages/modules)
> and one-off code (applications). The introduction of the [load] command
> made a great many people's lives easier, since it meant that most of us
> could give up on the building of "Big Wish" applications; by reinforcing
> the difference between extensions and applications, tremendous strength
> was gained. Nowadays people (well, canny Tcl programmers) won't think
> twice about mixing code from many different binary authors into a single
> application, a usage that was a major feat of technical gymnastics in
> Tcl 7.4 (I know; I remember still...)

This is real nonsense.
1. tclsh either is or isn't an application, your choice.
2. some loadable module is or isn't an application, your choice.
3. combine the two so you can use the loadable module or extension,
and only that extension. Is that an application or not?

I would say that 3 must be an application. But 3 could also be built
by adding the commands from 2 inside the AppInit.

Regardless of what you call a chunk of code in 2, it is still written
in the Tcl C API, and that is what allows it to create commands which
can become disconnected, and leak memory.

> It's a shame that TclBlend can't be a normal loadable extension; I
> suspect that the problem is on the Java side. Probably relates to Java's
> use of C++ for it's FFI... :-\

It probably needs much more control over startup and the runtime (like
GC) than you can get with a regular tclsh. Adding code to a tclsh can
be a simple one line addition.

Alan Anderson

unread,
Dec 14, 2007, 10:25:48 PM12/14/07
to
Darren New <dn...@san.rr.com> wrote:

> tom.rmadilo wrote:
> > I wish someone would explain why the need to GC could be considered a
> > good thing if you use a language that strictly speaking cannot create
> > garbage.
>
> Your view of "garbage" is too limited.

In the context of "garbage collection", isn't the view rather well
defined? I wonder what yours is. My view is that it's a resource which
can be used and later returned to availability, but has somehow been
misplaced before being returned. Nothing is actively using it, or *can*
actively use it, because the way to use it has been lost.

I suppose one can intentionally create something like garbage in Tcl by
discarding the result of a command like [open] or [image create] (though
it's probably still theoretically possible to recover such handles with
suitable introspection). But that strikes me as a contrived example.

Alan Anderson

unread,
Dec 14, 2007, 10:46:00 PM12/14/07
to
"tom.rmadilo" <tom.r...@gmail.com> wrote:

> This is real nonsense.
> 1. tclsh either is or isn't an application, your choice.
> 2. some loadable module is or isn't an application, your choice.
> 3. combine the two so you can use the loadable module or extension,
> and only that extension. Is that an application or not?

I think the terminology confusion here arises from your giving things
like Jacl and TCOM the label "Tcl application". I'd have expected that
to mean an application written in Tcl. I can see where it might be
appropriate to use it to refer to a program like wish, but that would be
using "Tcl" to name what the program *is* rather than what it is written
in.

Darren New

unread,
Dec 15, 2007, 12:31:41 AM12/15/07
to
Alan Anderson wrote:
> In the context of "garbage collection", isn't the view rather well
> defined?

Yes, but the definition doesn't cross levels. For example:

> I wonder what yours is. My view is that it's a resource which
> can be used and later returned to availability, but has somehow been
> misplaced before being returned.

Obviously, if it was *truly* misplaced, it couldn't be reclaimed. The GC
subsystem needs to be able to find objects even when there is no longer
any way to reach them from the application, for example. Memory "leaked"
in a C program can still be reached from the kernel when you exit the
process, etc.

To claim that discarding the result of [open] before a corresponding
[close] doesn't create garbage because you can get to it with [file
channels] misses the whole point. Sure, you can get to it with [file
channels], but you can't know whether other parts of the application are
still using that handle or not.

Alan Anderson

unread,
Dec 15, 2007, 11:44:05 AM12/15/07
to
Darren New <dn...@san.rr.com> wrote:

> To claim that discarding the result of [open] before a corresponding
> [close] doesn't create garbage because you can get to it with [file
> channels] misses the whole point.

Would you mind stating the point explicitly for me? My guesses all seem
to be missing the mark, and I'd rather have something concrete to
address.

> Sure, you can get to it with [file
> channels], but you can't know whether other parts of the application are
> still using that handle or not.

I'm starting to feel like I'm in a foreign city and I don't recognize
the street signs. I read your words, and they form understandable
sentences, but they don't seem to address anything I would consider a
real problem.

If you're the programmer and you don't know whether the application is
done using a file, ask the application. If the application doesn't know
whether it's done using a file, that's a bug. In my view, such an
application is broken and needs to be fixed. I don't understand how
fixing it properly can be done by invoking a garbage collection system.

Obviously I don't see the need for automatic garbage collection. I'd
honestly appreciate an example where it is necessary, or even where it
solves a problem that isn't easy to prevent in the first place.

Ramon Ribó

unread,
Dec 15, 2007, 12:13:56 PM12/15/07
to

Look at these examples:

package require tdom


proc a {} {
set aa [dict create a b c d]
}

proc b {} {
set doc [dom parse "<a/>"]
}

proc c {} {
dom parse "<a/>" doc
}

proc d {} {
set doc [dom parse "<a/>"]
$doc delete
}

in proc "a", variable "aa" is automatically unset when proc finishes
in proc "b" we leak memory, as the doc document is not deleted at the end
proc "c" is a simple form of garbage collection in tdom, doc is deleted
at the end
proc "d" is the way to finish correctly proc "b"

As you can see, TCL garbage collects very well its own variables when
they are not more needed, but gives not a simple way to the packages
or the extensions to do something similar with their objects and handles

Something very similar happens with object oriented frameworks as snit
Other packages make their own tricks, like tcom, but in a dangerous
way, not very reliable.

Of course, it is possible to work without these facilities, but it would
be nice to have them.


Alan Anderson escribió:

Donal K. Fellows

unread,
Dec 15, 2007, 12:42:05 PM12/15/07
to
tom.rmadilo wrote:
> This is real nonsense.

That's not a nice thing to say.

> 1. tclsh either is or isn't an application, your choice.
> 2. some loadable module is or isn't an application, your choice.
> 3. combine the two so you can use the loadable module or extension,
> and only that extension. Is that an application or not?
>
> I would say that 3 must be an application. But 3 could also be built
> by adding the commands from 2 inside the AppInit.

You're being really foolish, you know. An application is a piece of
code that can be executed using the operating system's main execve()
call (or equivalent on Windows). A loadable module is a file
(containing code) that can be mapped into a process using dlopen() (or
equivalent). If you wish to use other definitions that leave yourself
unable to be understood by anyone else, be my guest. But nobody here
will be able to grok what you're talking about. :-)

> Regardless of what you call a chunk of code in 2, it is still written
> in the Tcl C API, and that is what allows it to create commands which
> can become disconnected, and leak memory.

No. It's not difficult to enumerate *all* commands defined in a
namespace. It's not difficult to enumerate all namespaces. It's not
difficult to enumerate all interpreters descended from a master
interpreter. Commands clean themselves up when they are deleted; Tcl's
properly careful about that. The only thing of any difficulty at all
is deciding when a command is no longer needed.

Donal.

Óscar Fuentes

unread,
Dec 15, 2007, 12:50:45 PM12/15/07
to
Alan Anderson <aran...@insightbb.com> writes:

[snip]

> If you're the programmer and you don't know whether the application is
> done using a file, ask the application. If the application doesn't know
> whether it's done using a file, that's a bug. In my view, such an
> application is broken and needs to be fixed. I don't understand how
> fixing it properly can be done by invoking a garbage collection system.
>
> Obviously I don't see the need for automatic garbage collection. I'd
> honestly appreciate an example where it is necessary, or even where it
> solves a problem that isn't easy to prevent in the first place.

You are confusing GC with finalizers. A GC system just manages
memory. This relieves the programmer of one nasty task, namely,
releasing data objects that are no longer needed. I think nobody
rejects something that alleviates work, so GC is useful on lots of
scenarios. Applications that use GC tend to require more memory than
the same non-GC applications, use more CPU and, depending on the GC
technique used, may produce apparent application mini-freezes while
the GC does its stuff. In exchange you have one less task, which means
less work and less bugs.

Another major advantage of GC is that it makes natural certain
programming practices, such as Functional Programming, with its
closures, functors, etc. and attaching data context to code in
general. This is a very important "feature" of GC. Those interested on
the matter (highly recommended) can take a look at Scheme or Ocaml, to
name two easy languages. Learning some language where Functional
Programming techniques are natural is an eye-opener.

So, essentially, GC ensures that data will be around as long as it may
be used by the program, and eventually dealing with non-reachable
memory as available for reuse. Here, "eventually" means "at some point
of time", which, depending on the GC technique, can be "right now" or
"maybe never". This is essential to understand the next point.

Someone thought that executing a data-type-dependent procedure just
before the data is marked as free would be useful, and so finalizers
was invented. The problem with GC and finalizers is the above
mentioned indetermination on when they are executed. So, if you have a
finalizer that closes file handlers stored on some object, it may
never be executed, leaving the file open, which is not what you
want. Some GCs guarantee that the finalizers will be called within
certain predictable conditions, but other GCs doesn't. Most GCs I have
seen doesn't have finalizers at all, because finalizers is just an
extra of a GC.

There is another reason for not relating GC and general-purpose
resource handling. GC tracks memory use, and just that. The GC doesn't
know if you store the same file handle on two different places. So if
your finalizer closes the file when the GC detects that object A is
dead, object B may be still alive allowing the use of the closed file
handler.

How all this relates to Tcl? Well, I hope it's clear now that GC will
not magically solve problems with file handles and other resources,
except memory. It's true that EIAS makes a GC unnecessary (well, a
sophisticated GC, because reference counting is just a simple GC
method) but this fact shows, IMO, a limitation of Tcl, not a
virtue. GC is not required because the approach of Tcl to data
representation is so simple that you can't build certain data
structures, such as linked lists and everything else that relies on
data references. A not so glaring problem with Tcl that a GC system
(and, hence, the existence of data references) would solve is that
the upvar hack would be no longer necessary, and current applications
of upvar would be a subset of the programming techniques typical of
Functional Programming.

--
Oscar

Alan Anderson

unread,
Dec 15, 2007, 12:55:40 PM12/15/07
to
Ramon Ribó <ram...@compassis.com> wrote:

> Look at these examples:
>
> package require tdom

> #...


> proc b {} {
> set doc [dom parse "<a/>"]
> }

> #...


> proc d {} {
> set doc [dom parse "<a/>"]
> $doc delete
> }

> #...


> in proc "b" we leak memory, as the doc document is not deleted at the end

> ...

That looks exactly analogous to my earlier example of discarding the
result of [open], and seems just as contrived. One certainly wouldn't
write a program that way intentionally.

> proc "d" is the way to finish correctly proc "b"

Just as you would [close] a file when you're done with it.

My essential confusion is as strong as it always was. I'm still waiting
for an example where automatic garbage collection is a better answer
than simply not creating garbage in the first place. Making a resource
unaccessible sounds like a programming bug, plain and simple.

Maybe I'm just too deeply indoctrinated in keeping track of things
myself, and I'd find myself a lot happier if I let go and trusted "the
system" to take care of it? But I can't help feeling that doing it that
way would be the moral equivalent of mismatched braces. :)

Mark Janssen

unread,
Dec 15, 2007, 1:17:46 PM12/15/07
to

Why use procedures? I am perfectly capable of pushing the arguments to
the stack and jumping to the location of the next code to be executed.
The answer of course is, it relieves the programmer of the job to take
care of this. The same goes for garbage collection, you don't have to
trace the memory allocations of your program, the GC system takes care
of it for you.
Even though you are perfectly capable of doing this yourself, you must
be an exception, because insidious memory leaks in C applications are
certainly no exception. Of course creating the memory leak in the
first place is a programming bug, but that's not the point. The point
of GC is that you don't have to do the bookkeeping anymore allowing
you to focus on more pressing tasks like actually creating a
functional application.
Note that even with GC it is possible to create 'memory leaks' be
inadvertently keeping a reference to the allocated resource.

Mark

Alan Anderson

unread,
Dec 15, 2007, 1:23:58 PM12/15/07
to
Oscar Fuentes <o...@wanadoo.es> wrote:

> A GC system just manages
> memory. This relieves the programmer of one nasty task, namely,
> releasing data objects that are no longer needed.

If that's what GC means, then Tcl already has it. Every "data object"
in Tcl is a string. As soon as a variable goes out of scope, or if a
variable's value is modified to require a different amount of memory,
the Tcl interpreter releases the memory originally used. The garbage
collection is immediate, perfect, and unassailable.

But if that's all GC means, it seems rather a trivial thing to get all
worked up over. The label "garbage collection" to me implies something
a lot more involved than simply reclaiming the memory used by unneeded
data. I thought the whole point of GC was to determine *when* data
objects are no longer needed, so that they can be released without
explicit action by the programmer.

> Another major advantage of GC is that it makes natural certain
> programming practices, such as Functional Programming, with its
> closures, functors, etc. and attaching data context to code in
> general. This is a very important "feature" of GC.

I'll admit to being mostly ignorant on the fine points of FP. My
partially informed impression is that data is temporary, and doesn't
stick around when it's not needed. I'd imagine GC in such a case is
also immediate, and thus barely qualifies as GC the way I thought of it.

Darren New

unread,
Dec 15, 2007, 1:33:15 PM12/15/07
to
Alan Anderson wrote:
> My essential confusion is as strong as it always was. I'm still waiting
> for an example where automatic garbage collection is a better answer
> than simply not creating garbage in the first place.

Stop looking at the program as a unified perfect whole, and look instead
at how a library writer might work things.

Your confusion is that you're saying to yourself "I can implement
garbage collection manually, so why would I need to automate it?"

You don't need to automate it. It's just easier in some cases.
Especially when things get very complicated, or they get written by a
variety of people in different libraries that use each others services
in non-heirarchical ways, or when a mistake is made in the program.

tom.rmadilo

unread,
Dec 15, 2007, 1:35:30 PM12/15/07
to
On Dec 14, 7:46 pm, Alan Anderson <arand...@insightbb.com> wrote:

> "tom.rmadilo" <tom.rmad...@gmail.com> wrote:
> I think the terminology confusion here arises from your giving things
> like Jacl and TCOM the label "Tcl application". I'd have expected that
> to mean an application written in Tcl. I can see where it might be
> appropriate to use it to refer to a program like wish, but that would be
> using "Tcl" to name what the program *is* rather than what it is written
> in.

The reason I pointed out the my sources for using the term 'Tcl
application' to apply to an extension written in C is that most
programmers, even Tcl'ers don't recognize that Tcl has a C API. Most
consider the scripting commands to be the whole thing, which it isn't.
The Tcl C API is an important part of the Tcl language. So again, my
two sources were:

1. Every Tcl manpage index I have ever seen which lists tclsh and wish
as Tcl Applications.
2. Ousterhout's original book.

So, when I point out that Tcl scripting doesn't have GC, and someone
says, welllll, TclBlend, etc. have GC. What do you say? GC is not part
of the Tcl scripting language. The only way to get GC into Tcl
scripting is to write a new application using the Tcl C API.

But I rarely use tclsh or wish, instead using AOLserver, which is a
Tcl application. And over many years I have come to understand one
thing very clearly: every resource issue is due to bugs in C code, and
most solutions to resource management are also in C code. Of course,
if you only write Tcl scripts and execute them with tclsh and never
load a buggy library, you will never need to know anything about this
other level of Tcl...until you want to figure out if a channel needs
to be closed and you can't do it from Tcl scripting.

In Tcl scripting, the most you can do is to bracket code with [catch],
although the new form of catch makes this much easier. A big thanks to
the authors of this new syntax! However, the new syntax still requires
a [catch] and potential handling and analysis of return codes, and
catch may not be what you want. The result is potentially complex
code which is too tightly connected.

This is why I am interested in any references to GC strategies which
might be possible in pure Tcl scripting. And by strategy, I mean an
algorithm which can be effectively applied to the problem.

Off the top of my head here are some ideas:

1. For memory management.

This applies to things in memory which might be used by reference,
probably as a way to save RAM.

The issue here is a system issue, an application level issue. It is
easy to solve if certain constraints can be applied. The biggest
constraint is the source of the data itself. Any data worth worrying
about should have some persistent storage associated with it. It may
be expensive to get the data, so putting it in a shared 'cache' might
be a smart way to speed up access and not make too many copies into
memory. But basically, the data exists somewhere else and is copied
into the process memory.

Now the only issue is running a cache. Data can go in with a
timestamp, and data can be referred to many times. If some object
needs the data, it gets it from the cache, maybe asking that it be
only so old. There are many options using a cache, so nothing is set
in stone.

Even if you don't use a cache, the constraint here is that a data
object can have multiple references if this data object has some
persistent storage which can be refreshed in case it gets deleted from
memory. Maybe the constraint is just this: don't store important stuff
in memory, everything in memory is going to become garbage at some
point.

Another way of looking at this: consider data objects which can be
shared as self managing. You use an interface to point to the data and
let the object worry about the details.

2. Borrowed references.

I'm making up this term, basically this is a shared resource that can
only have one valid reference at a time. The reference is borrowed
from the owner, then returned so it can be reused (thus shared). Maybe
it is a semaphore with a count of 1.

A database connection is a good example, or a mutex which protects
multiple files, so you don't have to maintain a per-file mutex/lock.

The problem is that you have two things here. One is the token or
mutex, which is either in use or not. The other is the actual open
file, or database connection, active or not.

It is impossible to know if either part is garbage, and really neither
is garbage. How these two-part resources are managed is so variable
and particular to the situation that it can't be handled by GC.

In particular with open(ing) channels of any kind, you can never let
these objects self manage. You always use an explicit surrounding
controller because failure must lead to cleanup and error reporting
immediately. Channel errors are just like references to non-existing
memory. You don't want to go on your merry way at this point.

Darren New

unread,
Dec 15, 2007, 1:39:09 PM12/15/07
to
Óscar Fuentes wrote:
> You are confusing GC with finalizers. A GC system just manages
> memory.

Well, most GC systems nowadays only manage memory, because most modern
operating systems aren't GCed. There are a number of operating systems
where things other than memory participate in the GC system.

Consider, for example, deleting a file that's still open in a process.
The data of the file doesn't go away, because there are still references
to it. If you fork, you now have even more references. If the parent
then dies, the child still has a reference, so the space for the file
still isn't collected. Only when the last file goes away does the file
get collected.

Now store a file full of capabilities that way.

Indeed, in an OS based on capabilities, generally everything is GCed,
not just memory. (Or, alternately, you could describe everything in the
system as "just memory".)

> Someone thought that executing a data-type-dependent procedure just
> before the data is marked as free would be useful, and so finalizers
> was invented.

More like, most operating systems aren't designed with GC in mind, so
they don't track how many references to stuff outside the program are
still around.

(All the snipped part is all good stuff, IMO. :-)

Alan Anderson

unread,
Dec 15, 2007, 1:55:18 PM12/15/07
to
Mark Janssen <mpc.j...@gmail.com> wrote:

> Of course creating the memory leak in the
> first place is a programming bug, but that's not the point.

Does GC mask such bugs? If so, I'd call it a very bad solution. Bugs
should not be swept under the rug. They should be exposed, exposed
quickly, and exposed with the least possible politeness. :)

> The point
> of GC is that you don't have to do the bookkeeping anymore allowing
> you to focus on more pressing tasks like actually creating a
> functional application.

I don't understand how I need to do any less bookkeeping. If I don't
keep track of when I'm finished with a resource, how am I to trust that
the system isn't going to "garbage collect" it out from under me before
I'm ready? How am I to trust that the system *is* going to collect it
when I'm through? If the system is to know accurately whether or not
the resource is still in use, there's got to be something specific I'm
doing when I'm done with it. I'd be doing that thing whether or not I'm
taking responsibility for freeing it myself. So why is it any less work
on my part to keep track of it?

It still feels very, very strange to me. I don't use a lot of dynamic
memory allocation in C -- my low-level programming is generally
targeting PIC processors in specialized embedded applications -- but it
just seems wrong to take a chunk of memory and not give it back
explicitly when I'm done with it.

Darren New

unread,
Dec 15, 2007, 2:11:05 PM12/15/07
to
Alan Anderson wrote:
> Mark Janssen <mpc.j...@gmail.com> wrote:
>
>> Of course creating the memory leak in the
>> first place is a programming bug, but that's not the point.
>
> Does GC mask such bugs?

No. It stops it from being a bug. Does virtual memory mask allocating
too much address space?

>> The point
>> of GC is that you don't have to do the bookkeeping anymore allowing
>> you to focus on more pressing tasks like actually creating a
>> functional application.
>
> I don't understand how I need to do any less bookkeeping. If I don't
> keep track of when I'm finished with a resource, how am I to trust that
> the system isn't going to "garbage collect" it out from under me before
> I'm ready?

Because that's how garbage collectors work. You trust it for the same
reason you trust that a file's space won't get reused until you delete it.

> How am I to trust that the system *is* going to collect it
> when I'm through?

Because that's how garbage collectors work. You trust it for the same
reason you trust that a file's space *will* get reused *after* you
delete it.

> If the system is to know accurately whether or not
> the resource is still in use, there's got to be something specific I'm
> doing when I'm done with it.

Nope. That's the point.

{ char * x = malloc(10000); exit(0); }

What did you have to do with the malloc'ed space to return it to the
system? Does it change depending on how much space you malloc'ed?

> just seems wrong to take a chunk of memory and not give it back
> explicitly when I'm done with it.

It's what you're used to. What do you do to explicitly free up registers
in your C code when you're done with them? How does your code know when
it's OK to reuse AX, and that you don't have something you need in there
you'll use 10 instructions later?

It is loading more messages.
0 new messages