-- John Wiersba
Anyway, there are several mature OO extensions to Tcl that work very
well. XOTcl and Snit (my fav) to name a couple.
There is a TIP to add oo to the core (hoping for 8.5 but you never
know) but I have no idea if that will change how Tcl does gc.
I am sure someone more knowledgable than I will answer here as well.
Robert
All the extensions I've looked at force you to write and call explicit
destructors. This is inconvenient and extremely error prone -- the
same reasons for automatic garbage collection of string storage.
> Anyway, there are several mature OO extensions to Tcl that work very
> well. XOTcl and Snit (my fav) to name a couple.
Do they force you to call a destructor in order to release the memory
for an object?
Otherwise, I'm not aware of any OO framework for Tcl that does
automatic garbage collection. For the benefit of people who may
stumble across this discussion in the future, I think it's worth
pointing out that people still successfully program in these
frameworks, although it sounds like they may not suit your particular
tastes.
In response to your question about references, I don't expect to see
them in Tcl anytime soon; certainly, nobody has proposed a TIP dealing
with references. I think we get along fine without them, but it could
be interesting to have them just the same.
I find your criteria for "fully higher-level languages" a bit
arbitrary, but to each his own. I'm personally fond of Tcl's style and
it's "cool features", as you said... But when it comes to programming
languages, one size doesn't necessarily fit all programming problems or
all programmers.
Best regards,
Aric
A lot of people who like the feel of Tcl really like the feel of Snit.
It is not especially complicated but it is rather elegant and extremely
"Tclish". Snit does not have any kind of garbage collection.
Short answer: No. Tcl does not support garbage collection of stateful
objects yet.
As others have answered, Tcl does do ref-counting GC of its values,
which works fine for strings as they can't contain circular references
and are stateless/immutable. Adding support for garbage collection of
stateful references is much harder. Basically, it's hard to distinguish
a reference (to an object) from any other string, which makes it
difficult to know when it is safe to delete something. Therefore, mostly
people either explicitly destroy objects, or tie them to the lifespan of
a variable using traces, so that when the var goes out of scope the
object is destroyed (similar to techniques used in C++). However, if you
want to return an object then this is a pain. There isn't really a good
solution for this yet. My TOOT experiment, mentioned elsewhere in this
thread, allows for "objects" which are GC'ed by Tcl, but these are
stateless (functional) objects. TOOT is also very slow at present.
You *can* implement GC'ed references in Tcl (see e.g. Salvatore
Sanfilippo's work [1]), so long as you take care. For instance, if you
tried the following:
set a [ref ...]
set b [string range $a 1 end]
set c [string index $a 0]
unset a
set d $c$b
puts "Value of reference is [ref get $d]"
Then it would fail if the GC ran before d was created. By breaking up
the reference you violate the guarantee needed by the GC that any valid
references be recognisable as such when the GC runs. Other languages
enforce this guarantee by not allowing references to be decomposed in
this way to begin with. Tcl however, doesn't support such enforced
guarantees, preferring instead to leave this down to programmer
discretion, i.e., Don't Do That! I think that's fair enough -- it seems
unlikely that anyone would accidentally use string operations on a
reference.
However, such a GC mechanism potentially requires scanning lots of
strings for matching references, which is inefficient. I *think* an
efficient implementation could be made without violating Tcl's
semantics, but it would require clever tinkering on the C level (perhaps
using some of Paul Duffin's old Feather ideas [2]). Probably would have
to wait for the mythical Tcl 9 though, if it is feasible at all.
[1] "Tcl References in Tcl": http://wiki.tcl.tk/9549
[2] "Feather: Teaching Tcl Objects to Fly":
http://www.usenix.org/publications/library/proceedings/tcl2k/duffin.html
-- Neil
Example
Class A
Class B
set aInstance [A new]
set bNested [B new -childof A]
$aInstance destroy
puts "Test: [Object isobject $bNested]"
Working quite long with GC-languages I can say: Also GC-Systems has
memory leaks
(circular references), which are even worse to fix and discover as in
no GC-Systems.
Anyway Qt C++-Library also use this kind of mamory managment (nested
objects)
and it is quite fast and easy to use.
The another question is: Is it possible to have GC in OO-Tcl?
I think that it is not possible without loosing of one of main Tcl
characteristic (everything is string).
One can image that we can bind a lifetime of some XOTcl object to
lifetime of internal Tc_Obj structure. It would work but not for all
scenarios.
There are also some experimental implementation of it.
But Tcl converts a lot from string to internal representation and vice
versa.
It would be very tricky and out of control to know when your object is
destroyed. There are many
places in Tcl where scripts are strings: "Tk -command "script"
set obj [A new]
button .b -text "Hallo" -command [list $obj invoke]
unset obj
# GC based on Tcl_Obj would destroy your object. Button whould send the
message in nirvana
The Jacl (or Tcl-Java bridge) implemntation use such GC (like) object
managment.
Artur
Java does *not* have problems with circular references.
Java does some bookkeeping of what objects hold references
to what other objects. Once the GC happens, it will detect
all objects that have a live (not just any) reference on it,
and any clusters of circular referenced objects that are not
also referenced from outside are swept away.
This kind of GC does require a deep knowledge of all
involved types, and may be somewhat expensive (more than
ref-count based GC), but it definitely doesn't leak for
circular refs.
-- Neil
No, but you can still have memory leaks, of course. The problem shifts
from forgetting to explicitly delete an object to forgetting to remove a
reference from some long-lived container.
The deep point here is that every indexed container creates a new way of
creating references. For instance, if I create a hashtable then the keys
of that table are "references" to the objects in the table. But how do I
tell if an object is no longer reachable by any key? Well, I can write a
special GC to track hashtable keys, or I can require that people
explicitly remove objects that are no longer needed (i.e. require them
to explicitly destroy the reference).
-- Neil
Yes, unless you do a trick to bind the object lifetime to that of
something that is automatically collected. The automatically collected
things that matter in this sense are call stack entries (primarily
procedure scopes, though namespaces also count) and Tcl_Obj values. The
handling of the former is pretty easy to code up for yourself using an
unset trace on an otherwise-unused local variable, and the handling of
the latter is pretty easy to do from C code. There are down-sides to
both though, in that binding the lifetime to a procedure scope causes
problems if you decide that you want to transfer the object out of the
scope, and the internal representations of Tcl_Objs are known to be
fragile. Mind you, I've done OO code in the past that used Tcl_Obj
lifetimes to control object lifetimes, and provided you write your code
in the "right way" (whose exact nature I forget right now) it works
really well as a technique.
If an object is given an explicit name and operates as a command (the
way both Itcl and XOTcl work) I think it is entirely reasonable to
require explicit deletion, since commands can be looked up using [info
commands] of course. The "reference" held by the core doesn't ever get
lost implicitly. But this is an area which might change in the future.
Donal.
I certainly hope it does not make it to the core. Extensions fine but not in
the core. The core is nice and lean and I would want it to stay that way.
OO should be an option.
Ian
Do you have more precise objections? I think it is premature to judge
whether OO will bloat the core until there is an actual reference
implementation of proposed OO-in-the-core.
-- Neil
> The deep point here is that every indexed container creates a new way of
> creating references. For instance, if I create a hashtable then the keys
> of that table are "references" to the objects in the table. But how do I
> tell if an object is no longer reachable by any key?
There are two answers:
The simple one: every collection defines iterators, so you
can always find all objects that are referenced by the
Hashtable, even if you had no other ref to some Key-object.
The complex one: There are so-called Soft- and Weak-References
in Java (This I only learned between my last posting and this :-)
They are a bit differently used from normal (strong) references,
but allow you to keep some Objects available "as long as reasonable"
without preventing them from being ever gc'ed. An interesting concept.
Anyway, this is irrelevant to Tcl.
If a Program fills up collections with objects it doesn't need,
this is not a leak, but rather a memory-wasting application.
All Tcl-oo-systems that I'm currently aware of manifest their
objects in some way e.g. as commands, as namespaces, as
array-variables, and each of these types of manifestation
could be searched for leftover objects. So, objects don't
really leak.
In C/C++ (in contrast) any malloc/new'ed block of memory whose
address is "forgotten" is really lost and thus leaked. (Leaving
aside special bookkeeping overridings of new/malloc)
i.e. you can implement your own GC. Note though that this doesn't tell
you what you need to know. You don't need to know which objects are
referenced by the hashtable, but rather which hashtable entries are
referenced by keys in the application. For instance, if you have string
keys, then you potentially need to scan every string in your application
to see if it matches a hashtable entry -- sound familiar?
>
> The complex one: There are so-called Soft- and Weak-References
> in Java (This I only learned between my last posting and this :-)
> They are a bit differently used from normal (strong) references,
> but allow you to keep some Objects available "as long as reasonable"
> without preventing them from being ever gc'ed. An interesting concept.
Soft/weak references don't solve the problem. They allow references to
be collected in some more cases, but then run the risk of being too eager.
>
> Anyway, this is irrelevant to Tcl.
It's not irrelevant to a discussion of GC in Tcl.
>
> If a Program fills up collections with objects it doesn't need,
> this is not a leak, but rather a memory-wasting application.
What's the difference between filling up collections with unwanted
objects and filling up memory with unwanted objects?
>
> All Tcl-oo-systems that I'm currently aware of manifest their
> objects in some way e.g. as commands, as namespaces, as
> array-variables, and each of these types of manifestation
> could be searched for leftover objects. So, objects don't
> really leak.
This is not the same thing -- being able to reclaim memory is not the
same as actually doing it.
Now, all that said, I'm very much in favour of having *some* reference
type that can be garbage collected. We should just be clear that GC
isn't a silver bullet that magically eliminates memory leaks. It's very
hard to recognise what a reference is in general because the property of
being a reference is not inherent but is a property of some
interpretation. So new forms of reference (and self-reference, c.f.
Gödel) crop up all the time in applications and so garbage collection is
always a best-attempt. Better than nothing, but never perfect.
-- Neil
Robert
I am not particularly concerned about bloat. I object to the OO paradigm
itself. Tcl is a jobbing language and IMHO the OO paradigm is totally
inappropriate. OK as an extension for those who want to do more with the
language but the core not need it.
Ian
Ian
What is the definition of a "jobbing language"?
> and IMHO the OO paradigm is totally
> inappropriate. OK as an extension for those who want to do more with the
> language but the core not need it.
If "oo in the core" meant you had to do "package require oo" before it
was available -- much like you have to do "package require Tk" -- would
that be acceptable?
--
Bryan Oakley
http://www.tclscripting.com
Robert
The problem is that then you get multiple OO systems in use by
extensions. Indeed, it's quite common for extensions to implement their
own private OO for what they're doing, and to do so badly.[*] A core OO
system will allow people to start clearing out this particular thicket
of half-baked code.
OO doesn't solve all problems. But it does some things quite well, and
there's no reason why everyone should be reinventing this particular
wheel all the time. And there's no reason why core OO should force you
to change your scripts at all. Tcl will still be Tcl.
Donal.
[* I'm not thinking about OO extensions like Itcl or XOTcl here, but
rather many of the packages in tcllib. ]
As I was saying a few days ago, it's interesting how one comes to see
the way he uses Tcl as the "normal" way, and other uses appear to
be weird, if not invalid.
--
| Don Porter Mathematical and Computational Sciences Division |
| donald...@nist.gov Information Technology Laboratory |
| http://math.nist.gov/~DPorter/ NIST |
|______________________________________________________________________|
I agree with this. I've long abandoned the OO paradigm for DO (data
oriented) which languages line Lisp and Scheme (and not surprisingly
Tcl) are good at. For a language like Tcl, where one can simply eval or
subst data, I find that thinking in OO is often the wrong way of doing
things.
> ... You don't need to know which objects are referenced by the
> hashtable, but rather which hashtable entries are referenced by
> keys in the application. For instance, if you have string
> keys, then you potentially need to scan every string in your
> application to see if it matches a hashtable entry -- sound familiar?
No, we obviously do not agree on the definition of "garbage"
to start from.
My definition: garbage is, what the programm cannot access anymore.
Your (appearant) definition: anything the program doesn't need anymore.
I'm bold enough to claim that your definition is flakey, because
it is not determinable without a deeper knowledge of the program.
References are "not Tcl" in about the same way as dynamic code
or accessing caller's local variables is "not Java".
Apparently not.
> My definition: garbage is, what the programm cannot access anymore.
> Your (appearant) definition: anything the program doesn't need anymore.
Fair enough. The OP was asking about garbage collection of "objects"
though. It is generally not possible for an object to become unreachable
in Tcl in the same way that one can become unreachable in Java. So your
definition would seem to rule out GC for objects in Tcl.
> References are "not Tcl" in about the same way as dynamic code
> or accessing caller's local variables is "not Java".
You'd have to define what you mean by reference here. Certainly Tcl has
means to reference things -- variables etc.
-- Neil
> It is generally not possible for an object to become unreachable
> in Tcl in the same way that one can become unreachable in Java.
> So your definition would seem to rule out GC for objects in Tcl.
Yes, indeed.
GC would only become important, if we had objects that are not
tied into the usual places (cmd,var,nsp). I don't want to go
that route, but that alone doesn't mean it might not happen some
day...
>> References are "not Tcl" in about the same way as dynamic code
>> or accessing caller's local variables is "not Java".
> You'd have to define what you mean by reference here. Certainly
> Tcl has means to reference things -- variables etc.
I thought it was clear from the context, that "References" in
Tcl *would* be like those of jim or hecl: somewhat like anonymous
variables.
Anonymous variables are (until someone convinces me of the opposite)
a contradiction to the "everything is a string": as soon as a
reference can be stringified(and later restored), it can no
longer be determined at any point in time whether any reference
exists to a given object or not, thus thwarting GC.
>
>> and IMHO the OO paradigm is totally
>> inappropriate. OK as an extension for those who want to do more with the
>> language but the core not need it.
I agree.
When I have the luxury of poring over a specification and architectural
diagrams for a month then I might use OO. On the other
hand, poring a few days longer has often provided me with a way of
using a couple more tools (web server, database) and get away with
a much smaller program, which Tcl's keep-it-simple approach is
perfectly suited for.
> If "oo in the core" meant you had to do "package require oo" before it
> was available -- much like you have to do "package require Tk" -- would
> that be acceptable?
IMHO we've got that already. If what you mean is, some core support that
makes it possible to implement OO packages easier then maybe. Depends on
how many users want to use that. Count me out.
Just out of curiosity, what is the case pro GC?
I mean, there are lots of programming languages that don't have
GC and don't leak either (all unix shells for instance). When I used OO,
objects typically had a lot of state (like license tokens) and I particularly
want to control the lifetime of such objects. Having a destructor called 15min
after the object goes out of scope is not a good idea in those circumstances. So,
I end up writing "finish" procedures and calling them explicitly. Great.
Debugging *that* is like debugging malloc/free's without purify.
Lots of Greetings!
Volker
Oh, well Tcl is well suited for much more than just that! I've worked on
programs that had 10's of 1000's of lines of Tcl code, separate QA
departments, etc. Tcl is a marvelous language for all sorts of tasks,
both large and small.
Don't limit yourself to thinking Tcl is only good for such a small
domain of problems. Think spacecraft flight path visualization, oil rig
control, assisting 24x7 broadcasting ...
>>
>>> and IMHO the OO paradigm is totally
>>> inappropriate. OK as an extension for those who want to do more with the
>>> language but the core not need it.
>
> I agree.
> When I have the luxury of poring over a specification and architectural
> diagrams for a month then I might use OO.
Maybe we're differing on how an OO extension would work. You don't need
a OO based specification and architectual diagram to need objects.
Create a single megawidget and an OO extension becomes very handy
indeed. Any time you need to pass around a stateful block of data, doing
it as an object makes sense.
There are lots of places where objects would be useful to solve a
problem without resorting to a fully object oriented architecture.
>> If "oo in the core" meant you had to do "package require oo" before it
>> was available -- much like you have to do "package require Tk" --
>> would that be acceptable?
>
> IMHO we've got that already. If what you mean is, some core support that
> makes it possible to implement OO packages easier then maybe. Depends on
> how many users want to use that. Count me out.
What I meant was, it comes with the core much as Tk comes with the core.
You don't need a third party distribution to get it. There is no OO
package that comes with the core today (unless your thining of "core" is
starkit or ActiveTcl).
And, as someone else pointed out, having it in the core means we don't
keep inventing the wheel all the time. For example, if you download my
ages-old combobox, it has its own object emulation so that the combobox
works almost like a real widget. Probably 50% of that combobox code is
that mini object system. If there was was OO in the core I could count
on, my code would be cut in half. Any time you can throw away 50% of
your code, it's a good day.
Likewise, bwidgets has its own object system. Many modules in tcllib
either use snit or some other object system. And so on.
Gee, I very rarely (maybe twice a year, for personal use only) write such
small little beast in Tcl/Tk and that has been true since I've started using
Tcl/Tk.
Tcl is not *just* a jobbing language, come to the 2006 Tcl/Tk Conference
(http://www.tcl.tk/community/tcl2006/) and broaden your horizons!
--
+--------------------------------+---------------------------------------+
| Gerald W. Lester |
|"The man who fights for his ideals is the man who is alive." - Cervantes|
+------------------------------------------------------------------------+
having usability as a jobbing language is no disability.
in contrast to a language were you are busy for ages to
set up the scaffolding to get "hello World" in a barmy little window.
there _is_ one detrimental effect though:
while you set up the scaffolding there is time in abundace
to churn design-variants around in you head.( which will be helpfull later)
the same goes for a lot of "steep initial learning curve" stuff.
You don't have that with tcl/tk.
uwe
Wow. I've run several successful businesses based on "jobs" then. :-)
You just have to pick the right problem to solve.
--
Darren New / San Diego, CA, USA (PST)
"I think these anchovies are spoiled.
They're not flat."
Precisely, and I see no reason to hobble it with a paradigm more suited to
the 'big' applications.
Ian
>>
> Tcl is not *just* a jobbing language, come to the 2006 Tcl/Tk Conference
> (http://www.tcl.tk/community/tcl2006/) and broaden your horizons!
>
Let me rephrase; for ME, tcl is a jobbing language. I don't write mega
applications. I write small ones and Tcl/Tk is pretty much fine as it is
for that. For what I do OO is inappropriate so I would not want it to be
part of the core. Have it as an extension for mega projects by all means
but keep OO away from the core.
ian
I don't follow. Precisely because Tcl is good for large projects, we
shouldn't include tools suited for large projects? Is that what you just
said?
Tcl isn't being turned into a OO language and we're not adding any new
paradigms. We're just wanting to add a new command that creates objects.
It's really not any more complicated than that.
>
> I don't follow. Precisely because Tcl is good for large projects, we
> shouldn't include tools suited for large projects? Is that what you just
> said?
>
No, precisely because it is good for big AND small projects. As the small
projects requirements are a subset of the large project ones, the core
should not be cluttered with the additional stuff necessary only for the
big ones. Make it an extension.
> Tcl isn't being turned into a OO language and we're not adding any new
> paradigms. We're just wanting to add a new command that creates objects.
>
Why? What is the reason for moving this functionality from extensions into
the core.
Ian
> Why? What is the reason for moving this functionality from extensions into
> the core.
Because a large body of Tcl programmers want it, and many people agree
it's something Tcl needs for a variety of reasons.
If you don't use it, the only effect you'll likely see is a slighly
larger download of the core. That, and any extensions you use might
actually run faster and/or have a smaller footprint if they make use of it.
having oo "in the core will NOT affect you.
it is NOT changing the Tcl syntax or language itself
it is just adding some commands which you are free to totally ignore
Bruce
Robert
Virtually every modern, high-level, scripting language, such as perl,
ruby, or python, has some form of GC. Many modern, non-scripting
languages such as java also have GC. Even older languages such as lisp
and smalltalk have GC. One broad GC classification divides these
languages by the style of GC: either reference-counting or mark-sweep
and its variants. Reference counting has the benefit of immediately
"finalizing" unreachable referents, but it has some drawbacks, too.
All these languages have some form of anonymous references, where a
data structure can be referred to without being named. Sometimes these
references are implicit, other times they're explicit, but one thing is
for sure: they're darn handy. With them, you can easily build
multilevel, nested data structures very easily. Sometimes the data
structure can be thought of as an "object" of some sort. Other times,
it's just a record or an array or hashtable. These anonymous
references are managed by GC because it's very hard to accurately keep
track of them by hand.
Tcl seems to have avoided adding something resembling references to the
language, but IMHO this causes programmers to jump through unnatural
hoops to simulate them with other means, e.g. naming every object and
storing the name in some hashtable somewhere. And because objects end
up with "names" as their "references", it's hard to implement GC
because its impossible to know when an object is really dead.
Consequently, whenever someone implements a tcl object system, they end
up forcing the caller to track all his objects and destroy them
explicitly or use other tricks (such as tracing?) to get the objects'
memory reclaimed.
Part of the "problem" seems to be that tcl wants everything to be a
string or at least reconstructable from a string. This causes issues
which other languages have solved by negating the premise: references
cannot be reconstituted from a string.
For me, the bottom line is that, although tcl has many, very elegant,
clean abstractions, such as vfs, that lend themselves to simple,
streamlined solutions to various problems, it appears (to me) that when
it comes to building multi-level nested data structures, the techniques
available are all less than elegant or streamlined. In particular,
objects or nested data structures shouldn't have to be explicitly
destroyed and a procedure should be able to return an object or nested
data structure to its caller without naming every piece of it.
I know that many smaller programs do not need GC or references, but
it's hard for me to imagine writing a larger program without
references. Certainly GC (of data structures, not just strings) is
preferable for larger programs. And even smaller programs would gain
in clarity and expressiveness if data structures didn't have to be
named, tracked and destroyed.
I hope this clarifies the reason why I wrote the original post and what
I was hoping for. BTW, my primary experience with tcl is through a
medium size client-server tcl application and GUI I wrote which has
been used heavily on a daily basis for about 7 years now.
-- John Wiersba
> Ian Bell wrote:
>
>> Why? What is the reason for moving this functionality from extensions
>> into the core.
>
> Because a large body of Tcl programmers want it,
Are you suggesting there is going to be some sort of democratic vote on it?
> and many people agree it's something Tcl needs for a variety of reasons.
>
A bit vague IMHO. Which people and what reasons? Are there threads here that
have discussed it?
Ian
Does it change the way any of the current commands are implemented?
> it is just adding some commands which you are free to totally ignore
>
So, to be clear, these are completely separate from and have no influence at
all on the current command set?
Ian
Yes. It's been a topic of discussion around here for at least as long as
I've been using Tcl.
Donal.
That's an open question, but it's probably not going to affect any of
the basic Tcl commands. It might well end up affecting the http package
though; it has its own dodgy pseudo-object system, and it would be very
nice to scratch that cruft. But that should be strictly extensive; code
using the published APIs ought to be unaffected.
> So, to be clear, these are completely separate from and have no influence at
> all on the current command set?
Well, it'll add to the current command set if you do a [package require] :-)
Donal.
> It's very hard to recognise what a reference is in general because
> the property of being a reference is not inherent but is a property
> of some interpretation. So new forms of reference (and
> self-reference, c.f. Gödel) crop up all the time in applications and
> so garbage collection is always a best-attempt. Better than nothing,
> but never perfect.
I was wondering if there's any facility to put a trace on a Tcl_Obj from
within a TCL procedure. On a number of occasions, I've found myself
using reference counting in non-OO contexts (well, I suppose they could
actually be interpreted as very basic OO mechanisms), to accomplish
something akin to garbage collection, and it's been a pain having to
constantly increment and decrement the reference count. Even more
troublesome is where values are returned from a procedure; my general
solution was to set an idle event to destroy the object, instead of
destroying it immediately when the reference count drops to zero. The
idle event would simply let it be if the count is not zero when it
checks. But it's an ugly solution to an unnecessary problem.
As I understand it, any time you set a value to a variable you create a
Tcl_Obj. And as that value moves around, that one Tcl_Obj gets shared
around rather than the string being copied. As long as you don't do
anything that "might" modify the content of the Tcl_Obj (such as most
string or list operations). So you can pass it into procedures, return
it from procedures, set it to local variables, global variables, pluck
it from previous stack frames, whatever. And it's still the same
Tcl_Obj you originally started with. (A mechanism to avoid
accidentally changing such a variable might come in handy, but probably
not a nice thing to bother the core with - you'd have to create a kind
of "read only value" which can be replaced entirely, but not modified
- the difference, however, is a very fine one)
So, if you could put a "variable unset trace" on that Tcl_Obj, rather
than on the variable it was stored in, you could make that Tcl_Obj
itself one reference to a reference counted object. It should even
survive being stored in an array just fine. You'd "mark" the Tcl_Obj
with the unset trace, and then set an increment on the object.
Storing it in a list or a dict, however, remains a problem. But that
just means you have to go back to the older ways of manual reference
counting again. Perhaps a kind of "pure list" could be created at some
stage that would store Tcl_Obj's intact? It'd probably be a whole lot
faster as well for many kinds of list operation.
It's certainly not a complete solution, but I'm guessing it would fit a
fairly large number of use cases quite well.
Fredderic
See the Feather paper I referenced from another post in this thread,
which contains a way of doing this. Generally though, I don't think it
is a good idea, as it exposes too many implementation details at the
script level, and subtly breaks Tcl's semantics. It's best not to try
and solve these problems by looking at how Tcl works at the C level.
Instead, come up with a (maybe slow) solution that would work in pure
Tcl, so that you know that it behaves correctly, and then think about
how you would optimize this with C cleverness.
> Storing it in a list or a dict, however, remains a problem. But that
> just means you have to go back to the older ways of manual reference
> counting again. Perhaps a kind of "pure list" could be created at some
> stage that would store Tcl_Obj's intact? It'd probably be a whole lot
> faster as well for many kinds of list operation.
There is no problem with storing a Tcl_Obj in a (pure) list or dict --
elements are already stored as intact Tcl_Objs, and yes, it's very fast.
There are however, still many places where Tcl_Objs are copied or
destroyed. Having to track this behaviour when writing a script would be
unfortunate to say the least. The places where this occurs are not
guaranteed to remain the same either.
A Tcl_Obj represents a *value*, not a variable. From the script level,
you should never be aware of a Tcl_Obj being created, copied, or
destroyed. (OK, values might be constructed vs existing as Platonic
entities, but lets not get into that).
-- Neil
Of course, using packages extended this considerably for me.
Lots of Greetings!
Volker
Tcl has reference counting GC of values (where every value is
representable as a string).
>
> All these languages have some form of anonymous references, where a
> data structure can be referred to without being named. Sometimes these
> references are implicit, other times they're explicit, but one thing is
> for sure: they're darn handy. With them, you can easily build
> multilevel, nested data structures very easily. Sometimes the data
> structure can be thought of as an "object" of some sort. Other times,
> it's just a record or an array or hashtable. These anonymous
> references are managed by GC because it's very hard to accurately keep
> track of them by hand.
You can certainly pass around data structure *values* in Tcl, without
having to name them. For instance, using dicts (new in 8.5, but
available in 8.4 too):
set me [dict create name "Neil" age 25]
callSomeCmd $me
dict incr me age
You can nest dicts and lists etc, and pass them around without caring
about GC -- Tcl will take care of it. However, these data structures are
values: if you want to change them you need to take a copy.
[...]
> Part of the "problem" seems to be that tcl wants everything to be a
> string or at least reconstructable from a string. This causes issues
> which other languages have solved by negating the premise: references
> cannot be reconstituted from a string.
Variables aren't reconstructable from their string representation
either. I don't think that is the problem.
> For me, the bottom line is that, although tcl has many, very elegant,
> clean abstractions, such as vfs, that lend themselves to simple,
> streamlined solutions to various problems, it appears (to me) that when
> it comes to building multi-level nested data structures, the techniques
> available are all less than elegant or streamlined. In particular,
> objects or nested data structures shouldn't have to be explicitly
> destroyed and a procedure should be able to return an object or nested
> data structure to its caller without naming every piece of it.
As I pointed out above, nested data structures are not a problem in Tcl.
It is *stateful* nested data structures which are a (minor) problem. If
you really want GC of stateful entities, then there isn't a good
solution at present. In practice, I don't find this to be a problem very
often.
I don't imagine that the need to name stateful entities is going to
disappear in Tcl at any time in the future, as that would definitely
violate everything-is-a-string. To me, that is Tcl's defining
characteristic, for better or worse. My ideal would be to have something
like ML's references in Tcl, so that you can put a reference into a data
structure instead of having to make the whole data structure a
reference. i.e. I'd like to do:
set person [dict create name "Neil" age [ref 25]]
incr [dict get $person age]
puts "Age: [dict get $person age]" ;# prints 26
The advantage here would be that state is confined to just those bits
that need it, and is managed in a uniform way. You can do this now with
auto-generated fully-qualified variables, which has the advantage that
normal variable commands (set, incr, append, lappend, trace etc etc)
work on them. The only biggy missing is how to GC these variables. I
think this can be solved, but not soon.
-- Neil
> All these languages have some form of anonymous references, where a
> data structure can be referred to without being named.
You are right here, Tcl/Tk doesn't have this.
> Tcl seems to have avoided adding something resembling references to the
> language, but IMHO this causes programmers to jump through unnatural
> hoops to simulate them with other means, e.g. naming every object and
> storing the name in some hashtable somewhere.
I never ran into this. Maybe it's in the mental approach? Or did I just
use Tcl for the other kind of problems?
> Part of the "problem" seems to be that tcl wants everything to be a
> string or at least reconstructable from a string. This causes issues
> which other languages have solved by negating the premise: references
> cannot be reconstituted from a string.
IMHO it's a tradeoff. For me it solved a lot of other problems and I
can't recall ever running into a problem with this approach.
> For me, the bottom line is that, although tcl has many, very elegant,
> clean abstractions, such as vfs, that lend themselves to simple,
> streamlined solutions to various problems, it appears (to me) that when
> it comes to building multi-level nested data structures, the techniques
> available are all less than elegant or streamlined.
I agree. This is the price to pay for the no-references approach.
OTOH that's precisely what makes Tcl useful as a "jobbing" or "hacking"
language which is what I do all the time. I simply *LOVE* the way I can
throw any string at tcl throught a file or a gui and tell tcl to calculate
on it, or execute it or do whatever on it.
We've had programs with custom control structures here. We've had programs
which when run didn't execute in the conventional sense but wrote out
themselves as VHDL code. It would have been way beyond my timeframe if I
had to do it in a more conventional language.
Given that there a loads of languagees around with objects and references
but just one moderately powerful one without, I'm perhaps a bit particular
about having this one disappear into the objects/references crowd too.
Also when I need complex data structures in my shop here I use oracle for
data management and computing and tcl as a client. Then referential integrity
is taken care of by the database.
If I needed to sell this stuff I'd probably try to make this in-process-mysql
database work as a tcl extension and do my data handling in there.
Lot of Greetings!
Volker
> set person [dict create name "Neil" age [ref 25]]
> incr [dict get $person age]
> puts "Age: [dict get $person age]" ;# prints 26
Sounds interesting.
Only problem is what ref is supposed to return and what
happens if you puts this across a socket.
Maybe there are commands that resolve the reference and
those could be documented, kind of like side effects?
Lots of Greetings!
Volker
It should return a fully-qualified variable name. e.g. an implementation
right now is:
namespace eval ::ref { variable id 0 }
proc ::ref value {
set name ::ref::[incr ::ref::id]
set $name $value
return $name
}
I forgot that the puts should have been:
puts "Age: [set [dict get $person age]]"
($ won't work in this situation, unfortunately).
Obviously, if you send this structure across a socket, you will end up
with an unresolved reference at the other side, i.e. something that
looks like: {age ::ref::1 name Neil}. This isn't great, but is no worse
than any of the other ways of handling state which all require
serialisation. You could provide some [ref::serialise] command which
substituted all refs (strings that look like refs) in a data structure
with their current values. Conceptually this would have to operate like
a [string map], but internally it could know about dicts and lists etc
to try and speed things up/avoid shimmering (or you could pass a
description of the structure: {dict {name string age {ref integer}}}).
-- Neil
> > So, if you could put a "variable unset trace" on that Tcl_Obj,
> Generally though, I don't think it is a good idea, as it exposes too
> many implementation details at the script level, and subtly breaks
> Tcl's semantics.
Out of curiosity, which semantics are you referring to?
> A Tcl_Obj represents a *value*, not a variable. From the script
> level, you should never be aware of a Tcl_Obj being created, copied,
> or destroyed. (OK, values might be constructed vs existing as
> Platonic entities, but lets not get into that).
Exactly. It's the value that I'm interested in. I know there are
plenty cases where the reference link will be lost. But apart from
transferring the value from one variable or array element to another,
there's not much else you should be doing with them, and I'd be happy
to see that stated clearly in the documentation. It's in the simple
cases that I see the most problems; returning a reference from a
function, and passing the object as a procedure argument. Any use of
the object handle in a non-simple situation (packing it into a list with
other data, for example) should be wrapped by manual reference
counting; you'll NEVER be able to automatically deal with the case of
the handle text being combined with other text in a simple string using
a non-list separator (such as a comma), for example.
I personally suspect that in practise, your "hunt for references"
garbage collection scheme (perhaps I'm just tired, but that's how it
seemed to me when I looked at the code) would do a whole lot better
than tracking Tcl_Obj's. Looks on the surface to me like a fair bit of
overhead for only a small gain in areas where the person who wrote the
script generally has very good control, and equivalent functionality at
best in the areas where they don't.
But then I've had this kind of argument before. I always prefer to use
what's already there, rather than add more bulk that duplicates existing
functionality in a less elegant mannor. *shrug*
Fredderic
This:
>>A Tcl_Obj represents a *value*, not a variable. From the script
>>level, you should never be aware of a Tcl_Obj being created, copied,
>>or destroyed. (OK, values might be constructed vs existing as
>>Platonic entities, but lets not get into that).
>
> Exactly. It's the value that I'm interested in. I know there are
> plenty cases where the reference link will be lost. But apart from
> transferring the value from one variable or array element to another,
> there's not much else you should be doing with them, and I'd be happy
> to see that stated clearly in the documentation. It's in the simple
> cases that I see the most problems; returning a reference from a
> function, and passing the object as a procedure argument. Any use of
> the object handle in a non-simple situation (packing it into a list with
> other data, for example) should be wrapped by manual reference
> counting; you'll NEVER be able to automatically deal with the case of
> the handle text being combined with other text in a simple string using
> a non-list separator (such as a comma), for example.
Sorry, you've lost me. Are you arguing for tying the lifetime of
stateful objects to that of a Tcl_Obj or against? When you say "there's
not much else you should be doing with them..." do you mean Tcl_Objs,
values, or references?
>
> I personally suspect that in practise, your "hunt for references"
> garbage collection scheme (perhaps I'm just tired, but that's how it
> seemed to me when I looked at the code)
Which code are you referring to?
> would do a whole lot better
> than tracking Tcl_Obj's. Looks on the surface to me like a fair bit of
> overhead for only a small gain in areas where the person who wrote the
> script generally has very good control, and equivalent functionality at
> best in the areas where they don't.
>
> But then I've had this kind of argument before. I always prefer to use
> what's already there, rather than add more bulk that duplicates existing
> functionality in a less elegant mannor. *shrug*
Again, what are you referring to? What is already there and what is bulk?
-- Neil
Nope. A small body -- the Tcl Core Team. But they listen to the community.
>
>>and many people agree it's something Tcl needs for a variety of reasons.
>>
>
>
> A bit vague IMHO. Which people and what reasons? Are there threads here that
> have discussed it?
Over the years there have been many, many threads. I'm sure google can
point them all out.
Well, that's the rub. The problem comes when you do something like
set user [ref [dict create name "Neil" age 25]]
set tracker($channel,$user) [...]
Now what's the value of $user, and how does it get tracked?
How would you define "GC" differently from "automatic memory
management"? Sure, you can automatically manage memory using a stack.
It's just not an especially useful memory model in a lot of cases.
> I agree. This is the price to pay for the no-references approach.
> OTOH that's precisely what makes Tcl useful as a "jobbing" or "hacking"
> language which is what I do all the time. I simply *LOVE* the way I can
> throw any string at tcl throught a file or a gui and tell tcl to calculate
> on it, or execute it or do whatever on it.
Except maybe use it as the argument to [read], etc. File handles *are*
references, in some sense.
Think of the whole GC thing as "how can I make the file close when I no
longer have the return value of [open] available", and you'll see the
problem. Now, for "file" substitute "http page", or "mime part", or ...
Heck, without any packages, only pure Tcl, you can implement a simple
web server in only 40 lines of code: http://wiki.tcl.tk/4333
Of course the semantics of values is not the semantics of references.
If I have a huge hashtable, I don't want to create a copy of the entire
table just to change one value. References are the key. I want to
have a reference to a hashtable, not the hashtable itself.
> You can nest dicts and lists etc, and pass them around without caring
> about GC -- Tcl will take care of it. However, these data structures are
> values: if you want to change them you need to take a copy.
Exactly the problem.
> > Part of the "problem" seems to be that tcl wants everything to be a
> > string or at least reconstructable from a string. This causes issues
> > which other languages have solved by negating the premise: references
> > cannot be reconstituted from a string.
>
> Variables aren't reconstructable from their string representation
> either. I don't think that is the problem.
It seems to me that references (a la Jim References, maybe?) would be
more palatable if they weren't strings which could be decomposed
(hiding the reference) and then recomposed later.
-- John Wiersba
You also said:
>>> I'm bold enough to claim that your definition is flakey, because
>>> it is not determinable without a deeper knowledge of the program.
Which is exactly my point! The fact that GC cannot know when a hashtable
entry (for instance) is no longer going to be used is why "you can still
have memory leaks" (from my first post in this sub-thread).
You've argued that a memory leak is only such when the memory cannot be
reclaimed, because it is unreachable. That misses the underlying point
that I was trying to make: having GC doesn't eliminate the need to
manually manage resources, it just reduces the cases where you have to
do so. If I forget to delete an object from a hashtable then that is
some memory that I have leaked, whether it is reachable or not --
neither I nor any other process can make use of that memory. So I must
manage hashtable entries manually, or write my own special-purpose
hashtable GC.
My definition of garbage is not "flakey", but the one that practically
makes most sense. Those items which are no longer needed *should* be
reclaimed, either manually or automatically. How many of those a
particular GC actually manages to collect is a separate issue. Defining
garbage as just those items which can be collected by a certain
collector, by contrast, seems circular. For instance, if Java provided a
means to traverse all objects that had been allocated with "new" would
that then mean that GC was unnecessary (the objects are still
reachable)? Of course not.
> I thought it was clear from the context, that "References" in
> Tcl *would* be like those of jim or hecl: somewhat like anonymous
> variables.
>
> Anonymous variables are (until someone convinces me of the opposite)
> a contradiction to the "everything is a string": as soon as a
> reference can be stringified(and later restored), it can no
> longer be determined at any point in time whether any reference
> exists to a given object or not, thus thwarting GC.
We agree here -- anonymous references would be a contradiction to EIAS.
The only way to reference something with a string is by naming, so a
nameless name doesn't make sense. Uniquely named references do make
sense in some situations however, and being able to GC them would be useful.
-- Neil
Reference counting is a form of GC.
> Also, I'm still out on the "why gc?". For me, GC is something someone
> did because he was too lazy to implement real memory management for the
> language he was designing.
> Very likely I'm wrong and I'd love to be enlightened.
> Is there a reason why it's not possible to do automatic memory
> management without GC?
Would you want to code a call to a destructor function for every string
value that you no longer need? Aren't you happy that tcl does that for
you? It's the same for structured objects and references.
> > All these languages have some form of anonymous references, where a
> > data structure can be referred to without being named.
> You are right here, Tcl/Tk doesn't have this.
>
> > Tcl seems to have avoided adding something resembling references to the
> > language, but IMHO this causes programmers to jump through unnatural
> > hoops to simulate them with other means, e.g. naming every object and
> > storing the name in some hashtable somewhere.
> I never ran into this. Maybe it's in the mental approach? Or did I just
> use Tcl for the other kind of problems?
After having used other languages which have GC'ed references to
anonymous values, it's a real pain to have to deal with the memory
management issues which come up when dealing with named values which
have to be tracked via a hashtable. It just feels very clunky. Like
having to code parallel arrays in fortran (where an index is used to
look up corresponding values in multiple arrays rather than having a
pointer to a structured value). Once you have used
pointers/references, you just never want to go back to the old way of
doing things.
> > Part of the "problem" seems to be that tcl wants everything to be a
> > string or at least reconstructable from a string. This causes issues
> > which other languages have solved by negating the premise: references
> > cannot be reconstituted from a string.
> IMHO it's a tradeoff. For me it solved a lot of other problems and I
> can't recall ever running into a problem with this approach.
I think tcl references would have to be an exception to the
"everything's a string" rule.
> > For me, the bottom line is that, although tcl has many, very elegant,
> > clean abstractions, such as vfs, that lend themselves to simple,
> > streamlined solutions to various problems, it appears (to me) that when
> > it comes to building multi-level nested data structures, the techniques
> > available are all less than elegant or streamlined.
> I agree. This is the price to pay for the no-references approach.
> OTOH that's precisely what makes Tcl useful as a "jobbing" or "hacking"
> language which is what I do all the time. I simply *LOVE* the way I can
> throw any string at tcl throught a file or a gui and tell tcl to calculate
> on it, or execute it or do whatever on it.
> We've had programs with custom control structures here. We've had programs
> which when run didn't execute in the conventional sense but wrote out
> themselves as VHDL code. It would have been way beyond my timeframe if I
> had to do it in a more conventional language.
I believe you can also do all these things in other dynamic languages
like perl, ruby, python, lisp. I think what sets tcl apart is it's
emphasis on clean and minimal abstractions. Maybe I'm wrong, but I
haven't heard of any other language having an abstraction layer like
vfs.
> Given that there a loads of languagees around with objects and references
> but just one moderately powerful one without, I'm perhaps a bit particular
> about having this one disappear into the objects/references crowd too.
I think the capability of using references would not affect any program
which didn't use them.
-- John Wiersba
As I pointed out in another post, I made a mistake, the last line above
should be:
puts "Age: [set [dict get $person age]]"
Which implies that the string rep of a reference should be a variable
name (e.g. something like ::ref::1).
If by tracked you mean for GC purposes, then you would have to scan for
strings that match ::ref::*. That's clearly expensive. At present, in
code where I've used such references I manually delete them rather than
relying on GC (or just leak the memory in short-lived scripts). It's not
great, but it is no worse than any other approach I've seen so far in
Tcl. It does have many advantages however:
* I can use normal variable commands: [set], [lappend], [incr] etc.
* I get a whole framework for change notifications (trace) for free.
* I can use members of TOOT objects and dictionaries as -textvariables
etc easily, e.g. .age configure -variable [dict get $person age].
It really makes a lot of sense to me. References are just mutable boxes
for values, which is exactly what variables are. To satisfy
everything-is-a-string references must be named. Variables are already
named. Thus, make references be uniquely named variables. The basics
work. Adding efficient GC would be a bonus, but I don't think it's a
show-stopper, and even without it such references are no worse than any
other way this is done at present, IMHO.
-- Neil
Well, behind the scenes Tcl passes values as references, it just doesn't
allow them to be mutated (at least not if such a change would be
observable to a script). Strictly, issues of reference are separate from
those of mutability (although you could argue that mutation cannot
happen without reference, the converse is not true). For instance,
Haskell doesn't allow any mutation at all, but e.g. see the work on
Zipper (http://okmij.org/ftp/Computation/Continuations.html#zipper):
"Zipper is a construction that lets us replace an item deep in a complex
data structure, e.g., a tree or a term, without any mutation. The result
will share as much of its components with the old structure as possible."
Tcl also does tricks to share as much of data structures as possible.
E.g. if you do:
set a [dict create name "Neil" age 25]
set b [dict replace $a age 26]
Then I *think* you will get back a new dict, but most of the objects
(those representing "name" "Neil" and "age") in that dict will be shared
(Donal Fellows could say for sure). IIRC, Joe English has some ideas for
a construct similar to Zipper that would allow such maximum-sharing
non-mutating changes to be done on deeply nested structures (e.g. dicts
of lists of dicts).
>
> It seems to me that references (a la Jim References, maybe?) would be
> more palatable if they weren't strings which could be decomposed
> (hiding the reference) and then recomposed later.
Yes and no. Having strings which can't be decomposed would be useful to
avoid cases where someone splits up a reference and then puts it back
together and wonders why their object has been collected. I don't see
that as a big problem as it would presumably only happen by accident
(i.e. you've probably got a bug in your program anyway if you are
chopping up references). The bigger problem comes from observable
differences that this could bring in. For instance:
set a [ref ...] ;# $a cannot be split
puts $a ;# lets say it prints "ref0x1"
set b "ref0x1"
use-reference $a
use-reference $b
Now the question is whether that last line works. If yes then making our
references not split-upable hasn't solved anything, as we can still
manufacture references to existing objects (so our GC still can't be
sure if it is safe to collect). If no, then we have two objects at the
Tcl level whose string representations are identical and yet are
considered not equivalent in this situation -- something that is
generally disallowed in Tcl so far.
It's a hard problem. I think the most sane thing to do would be to
optimise for the common case (where people don't chop up references, and
don't try and manufacture them) -- perhaps by marking some Tcl_Objs as
being totally immutable, even when copied -- and then just live with the
observable side-effects this creates at the Tcl level, in the
expectation that most sane code won't ever notice. I'm not sure what Jim
does here, but probably worth a look.
-- Neil
Sure. And a number of packages work that way.
> If by tracked you mean for GC purposes, then you would have to scan for
> strings that match ::ref::*. That's clearly expensive.
Plus, it doesn't work reliably, as seen by the example I gave. :-)
And the ability to not have to track things manually is what people are
complaining is missing. Personally, I've not had that problem, but then
I don't do a whole lot of OO-style Tcl.
> Ian Bell wrote:
>> Bryan Oakley wrote:
>>
>>
>>>Ian Bell wrote:
>>>
>>>
>>>>Why? What is the reason for moving this functionality from extensions
>>>>into the core.
>>>
>>>Because a large body of Tcl programmers want it,
>>
>>
>> Are you suggesting there is going to be some sort of democratic vote on
>> it?
>>
>
> Nope. A small body -- the Tcl Core Team. But they listen to the community.
>
Good. This member of the community says keep OO in any form out of the Tcl
core. Anyone else who feels the same can append here.
Ian
> Ian Bell wrote:
>> Bryan Oakley wrote:
>>
>>
>>>Ian Bell wrote:
>>>
>>>
>>>>Why? What is the reason for moving this functionality from extensions
>>>>into the core.
>>>
>>>Because a large body of Tcl programmers want it,
>>
>>
>> Are you suggesting there is going to be some sort of democratic vote on
>> it?
>>
>
> Nope. A small body -- the Tcl Core Team. But they listen to the community.
>
While we are on this topic, can you confirm is it intended that the Tcl core
will continue to be written in C? There are no plans to use C++ for
example?
Ian
I think it's reasonably safe to assume the tcl core will not start using
C++. The core developers are too smart to do that :-)
How can you be sooooo backwards!
uwe
No,no,no don't beat me nooooooo
> But everybody else does it!
> Even the Fiasco _Microkernel_ is written in C++.
Why, do you think, is it called 'Fiasco'?
> http://os.inf.tu-dresden.de/fiasco/
uwe
But do you have any reasons and/or evidence to go with these opinions?
From what I can tell reading back through this thread you object to OO
as a paradigm, and you object to adding new functionality to Tcl except
that which is useful for small projects. But I can't seem to find any
actual reasoning for these decisions. You have a desire to keep Tcl
small (if it is small now) and suitable for small projects. However, you
don't know how much size OO will add to the core, and you don't say why
this extra size will make it suddenly unsuitable for small projects. Are
these small projects small in the embedded sense? If not, then why do
you care how big the core is?
TIP 257 contains the actual proposal being considered. I would hope
objections would be phrased in terms of concrete objections to the TIP
or its details, along with some reasoning for these objections.
-- Neil
> Ian Bell wrote:
>> While we are on this topic, can you confirm is it intended that the Tcl
>> core will continue to be written in C? There are no plans to use C++ for
>> example?
>
> I think it's reasonably safe to assume the tcl core will not start using
> C++. The core developers are too smart to do that :-)
>
Thank heaven for small mercies ;-)
Ian
> Ian Bell wrote:
>> Bryan Oakley wrote:
>>
>>>Nope. A small body -- the Tcl Core Team. But they listen to the
>>>community.
>>
>> Good. This member of the community says keep OO in any form out of the
>> Tcl core. Anyone else who feels the same can append here.
>
> But do you have any reasons and/or evidence to go with these opinions?
> From what I can tell reading back through this thread you object to OO
> as a paradigm, and you object to adding new functionality to Tcl except
> that which is useful for small projects. But I can't seem to find any
> actual reasoning for these decisions. You have a desire to keep Tcl
> small (if it is small now) and suitable for small projects. However, you
> don't know how much size OO will add to the core, and you don't say why
> this extra size will make it suddenly unsuitable for small projects. Are
> these small projects small in the embedded sense? If not, then why do
> you care how big the core is?
I object to OO as a paradigm, yes.
I never objected to adding new functionality per se to the core. I see no
reason to add functionality to the core which is not useful to ALL sizes of
projects. Anything else rightly belongs in an extension. That's what they
are for.
I never said the size of the core bothered me. In fact I said bloat in the
core was not a concern.
Ian
I don't see anything about garbage collection in that TIP. In fact, I
can't find it in any TIP. Are there no proposals which say that if an
OO subsystem is added to tcl that it would be able to automatically
release the memory for those objects when they're no longer accessible?
-- John Wiersba
No, there are no such proposals that I know of. Other than possibly
linking the lifetime of an object to a scope, I wouldn't expect an OO
proposal to include a proposal for GC. From the rest of this thread it
should be clear that GC of objects is not a solved problem in Tcl yet.
-- Neil
> > Exactly. It's the value that I'm interested in. I know there are
> > plenty cases where the reference link will be lost. But apart from
> > transferring the value from one variable or array element to
> > another, there's not much else you should be doing with them
> Sorry, you've lost me. Are you arguing for tying the lifetime of
> stateful objects to that of a Tcl_Obj or against? When you say
> "there's not much else you should be doing with them..." do you mean
> Tcl_Objs, values, or references?
Values, really...
My original take, was an "object" of some description that returns an
access token. Like the return value of [after], or a file, for
example. Tokens that are passed to an accessor function to perform
operations on the object. Specifically, where such an "object" has
reference counting...
The token will be held in a Tcl_Obj, such as those that represent a
value being stored in a TCL variable. Tying the lifetime of the object
to the variable itself isn't much use, since the value (and hence the
Tcl_Obj) will move through several variables in its lifetime.
The closest component to a "value" in TCL that I can think of, is the
Tcl_Obj. So if you bind one reference count to the object, to that
Tcl_Obj, you should have a pretty good guide to lifetime through most
simple uses. It'd only be when you want to get tricky by storing the
token amongst other data (lists or dicts, for example), that you have
to watch the reference count yourself.
And like I said, one of the main areas where I think this would be of
most use of all, is in argument passing, both into procedures and back
out as return values.
> > I personally suspect that in practise, your "hunt for references"
> > garbage collection scheme (perhaps I'm just tired, but that's how it
> > seemed to me when I looked at the code)
> Which code are you referring to?
I forget the URL now... It was posted on the same message as the
"feather paper" thing.
> > But then I've had this kind of argument before. I always prefer to
> > use what's already there, rather than add more bulk that duplicates
> > existing functionality in a less elegant mannor. *shrug*
> Again, what are you referring to? What is already there and what is
> bulk?
heh Tcl_Obj's are reference counted internally. They hold a "value",
on behalf of one or more entities such as TCL variables. As I
understand it, as long as the value remains intact, singular, and
unmodified, the Tcl_Obj should too. Idealy you'd want the trace to be
called if the value ceases to exist, or gets modified in any way. But
an unset on the Tcl_Obj would probably get you most of the way there.
Fredderic
> Yes and no. Having strings which can't be decomposed would be useful
> to avoid cases where someone splits up a reference and then puts it
> back together and wonders why their object has been collected. I
> don't see that as a big problem as it would presumably only happen by
> accident (i.e. you've probably got a bug in your program anyway if
> you are chopping up references). The bigger problem comes from
> observable differences that this could bring in. For instance:
You're going over some of the things I've been trying (somewhat
unsuccessfully, I gather) to say in my other post... I'd not have the
lifetime of a TCL "object" totally controlled by GC. There are too many
ways where normal-ish TCL activities could cause a reference to be
forgotten.
What I WOULD like to see, is the capability to link automatic GC
techniques to an objects reference counting capacity. The value
wouldn't be directly bound to the object in any way, but a
[::my_object::make_ref] type function would return a Tcl_Obj with an
unset (modify) trace to call the objects un-reference function.
Fredderic
Not true. See itcl::local
> While we are on this topic, can you confirm is it intended that the Tcl core
> will continue to be written in C? There are no plans to use C++ for
> example?
Wasn't Jeff Hobbs working on a complete rewrite in Ada?
--
David N. Welton
- http://www.dedasys.com/davidw/
Linux, Open Source Consulting
- http://www.dedasys.com/
PS - kidding! For anyone without a humor detector.
> Ian Bell wrote:
>
>> While we are on this topic, can you confirm is it intended that the Tcl
>> core will continue to be written in C? There are no plans to use C++ for
>> example?
>
> Wasn't Jeff Hobbs working on a complete rewrite in Ada?
>
What, deterministic Tcl, whatever next ;-)
Ian
All the common programming paradigms, ? imperative, object-oriented,
functional, and logic programming ? are based on a fundamental model of
computation in which the program design predetermines what will occur when
the program is run.
The event-driven programming paradigm turns this world inside out.
Event-driven programs do not predict the control sequence that will occur;
they are written to run reasonably to any particular sequence of events
that may occur once execution begins.
TCl and Tk are designed principally for event driven systems. OO is
therefore inappropriate in the core. It seems to me to be more relevant to
integrate expect into the core than OO.
OAm
Yes, they're shared. :-) (And, FWIW, if you put them in Tcl arrays, the
"name" and the "age" wouldn't even be objects, let alone shared.)
Donal.
What evidence do you have that OO techniques can't be used in an object
oriented environment?
If anything, I think that Tk proves that objects and an event driven
model work exceedingly well together.
Because, as we all know, there are so few event driven systems that use
OO to model things like windows, menus, and so on.
> Ian Bell wrote:
>>
>> TCl and Tk are designed principally for event driven systems. OO is
>> therefore inappropriate in the core.
>
> What evidence do you have that OO techniques can't be used in an object
> oriented environment?
I did not say they cannot be used. You can use a sledge hammer to crack a
nut. It's just inappropriate. You can write PC code in assembler, it's just
inappropriate.
What evidence do you have that OO techniques are beneficial in event driven
systems?
>
> If anything, I think that Tk proves that objects and an event driven
> model work exceedingly well together.
>
I agree that probably the most widespread event driven program is the GUI.
However, Tk however is not objected oriented.
Ian
> Ian Bell wrote:
>> TCl and Tk are designed principally for event driven systems. OO is
>> therefore inappropriate in the core.
>
> Because, as we all know, there are so few event driven systems that use
> OO to model things like windows, menus, and so on.
>
The most common event driven application is certainly the GUI. The assertion
that OO techniques are commonly used for GUIs demonstrates only that they
are commonly used for GUIs which is just ONE type of event driven program.
It says nothing about the appropriateness of OO techniques for event driven
systems in general.
Ian
Oh shut up! This shouldn't have gone
public until start of next month...
I thought about it, and decided not to put that as core functionality at
this stage. Too many subtle, surprising and nasty edge-cases. I'd hope
that it would be possible to add some form of GC as an "after market
product"; for example, binding an object to a scope's lifetime is pretty
easy to do in a script library. Deeper things (such as binding to
reference lifetime or doing reachability searching) require more complex
changes to the core to make happen nicely though, so I'm not planning to
do those in 8.5.
Donal.
We won't be using C++ - it's got too many nasty surprises when you come
to deployment.
Donal.
Oh, but yes it is. It just doesn't support subclassing of widgets and
other stuff like that. But then (with plenty of experience with using
X11 and Java toolkits) subclassing is vastly overrated when it comes to
doing GUI development.
Donal.
>>> Tcl seems to have avoided adding something resembling references to the
>>> language, but IMHO this causes programmers to jump through unnatural
>>> hoops to simulate them with other means, e.g. naming every object and
>>> storing the name in some hashtable somewhere.
>> I never ran into this. Maybe it's in the mental approach? Or did I just
>> use Tcl for the other kind of problems?
>
> After having used other languages which have GC'ed references to
> anonymous values, it's a real pain to have to deal with the memory
> management issues which come up when dealing with named values which
> have to be tracked via a hashtable.
Never had any problem with arrays either. I do a lot of transformational
programs. Parse a text file into a big complicated array, do something
to it, write out another text file. Or read several files and do something.
I never delete much stuff anyway, and if, I use procedures for doing that
and they take care of consistency. Or I do the data management in a database
and do the consistency stuff with referential constraints.
It just feels very clunky. Like
> having to code parallel arrays in fortran (where an index is used to
> look up corresponding values in multiple arrays rather than having a
> pointer to a structured value). Once you have used
> pointers/references, you just never want to go back to the old way of
> doing things.
I come from C++ and the STL. Never want to go back.
>>> Part of the "problem" seems to be that tcl wants everything to be a
>>> string or at least reconstructable from a string. This causes issues
>>> which other languages have solved by negating the premise: references
>>> cannot be reconstituted from a string.
>> IMHO it's a tradeoff. For me it solved a lot of other problems and I
>> can't recall ever running into a problem with this approach.
>
> I think tcl references would have to be an exception to the
> "everything's a string" rule.
So how can you tell whether a string contains a reference? Or
produces one when executed?
>> Given that there a loads of languagees around with objects and references
>> but just one moderately powerful one without, I'm perhaps a bit particular
>> about having this one disappear into the objects/references crowd too.
>
> I think the capability of using references would not affect any program
> which didn't use them.
Hopefully...
Lots of Greetings!
Volker
>> I agree. This is the price to pay for the no-references approach.
>> OTOH that's precisely what makes Tcl useful as a "jobbing" or "hacking"
>> language which is what I do all the time. I simply *LOVE* the way I can
>> throw any string at tcl throught a file or a gui and tell tcl to
>> calculate
>> on it, or execute it or do whatever on it.
>
> Except maybe use it as the argument to [read], etc. File handles *are*
> references, in some sense.
> Think of the whole GC thing as "how can I make the file close when I no
> longer have the return value of [open] available", and you'll see the
> problem. Now, for "file" substitute "http page", or "mime part", or ...
Since in tcl everything is a string I can (and want to) do all sorts
of things with return values, including parsing, copying to clipboard,
retrieving from there and so on. I wouldn't want tcl to accidentally delete
them because tcl and I disagree on when a handle becomes unreachable.
Lots of Greetings!
Volker
Of course it is! Widgets are objects, and these objects have methods.
Widgets also have a class, and the class defines the behavior. Tk
doesn't support inheritance and subclassing but that's merely one
feature of OO.
MFC? Smalltalk? .NET? You don't think it's significant that the same
system that introduced windows, mice, and pointers was also the first
interactive OO system?
Um. If you clean up afterwards, it's not leaking, right? And if you're
using reference counting, you're cleaning up right away. So I'm not sure
what your distinction is.
> Since in tcl everything is a string I can (and want to) do all sorts
> of things with return values, including parsing, copying to clipboard,
> retrieving from there and so on. I wouldn't want tcl to accidentally delete
> them because tcl and I disagree on when a handle becomes unreachable.
Yes. That's probably the biggest reason that GC isn't already in Tcl.
But my point is that putting references in a file and then reading them
back in a later process, or sending them over a socket, just doesn't
make sense in the usual modern OSes, for the same reason that putting
"file7" over a socket doesn't give the receiver access to the open file.
Why wouldn't they be helpful in event-driven systems? OO techniques are
helpful in event-driven systems for the same reason that they're helpful
in linear systems. Database servers are event-driven and many are
OO-based. "Web services" are usefully event-driven and OO. Why would OO
be *less* useful in a large event-driven system than in a large linear
system?
> the internal representations of Tcl_Objs are known to be fragile.
> Mind you, I've done OO code in the past that used Tcl_Obj lifetimes
> to control object lifetimes, and provided you write your code in the
> "right way" (whose exact nature I forget right now) it works really
> well as a technique.
For better or for worse, I would like to be able to set an "unset"-type
trace on a Tcl_Obj, from within TCL code. To improve its usefulness, it
should also force "shared" semantics, as though the trace itself has a
reference to the Tcl_Obj. Perhaps it would even be as simple as that;
add an extra reference for the unset trace.
if { refs <= has_unset_trace } then collect
(assuming has_unset_trace assumes boolean values of 0 or 1)
> If an object is given an explicit name and operates as a command (the
> way both Itcl and XOTcl work) I think it is entirely reasonable to
> require explicit deletion, since commands can be looked up using [info
> commands] of course. The "reference" held by the core doesn't ever get
> lost implicitly. But this is an area which might change in the future.
Personally, I think that's a fine way to represent an "object class",
but having an "object instance" as a command is a little strange.
Though if all methods of an object take a variable name (into which the
objects contents are stored), then an "instance command" could easily
be created using an interpreter alias. Also works particularly well if
there's a "new" method that generates an object name for you (eg.
the next variable with a specific naming pattern within the objects
namespace), and returns it after calling the objects constructor to set
it up.
I think that pattern of OO matches most of the newer parts of the TCL
core, without sacrificing functionality for those who like making
commands of object instances. Myself, I'd only want to see that happen
if it were possible to create a Tcl_Obj that acted like a Tcl_Obj. ie.
whenever the interpreter fails to find a command, check to see whether
it's a variable containing a Tcl_Obj marked as an "alias", and then
evaluate it along the same lines as interpreter aliases. That could be
an interesting capability. Perhaps an {alias} token to go along with
the new {expand} token? ;)
Fredderic
[...]
> But my point is that putting references in a file and then reading
> them back in a later process, or sending them over a socket, just
> doesn't make sense in the usual modern OSes, for the same reason
> that putting "file7" over a socket doesn't give the receiver access
> to the open file.
Well no, but in Tcl there's a limited number of places to conveniently
stick references that *ought* to work, since it has no closures. Most
scripting languages seem to have some syntactic way of saying that a
blob is code, allowing (and requiring, I imagine) the interpreter to
parse it to obtain variable and object references. Tcl doesn't.
I guess for some commands it might do this. So things passed as the
value of -command in Tk widgets could be scanned like that, to make
sure variables and commands got remembered properly. Seems complex
and unlikely to be reliable, though, and (unlike the astonishing
bytecode compiler work) would probably need to be understood by users.
Hmm, or you could have objects which represent code (and they could
nest to support loops and things), but that doesn't feel very like
Tcl.
So overall, I suspect the right answer for people who want proper
garbage collection is just to use some language which has it (along
with the things that support it). There's no shortage nowadays of
lightweight scripting languages with easy to use C interfaces, after
all.
> Ian Bell wrote:
>> I agree that probably the most widespread event driven program is the
>> GUI. However, Tk however is not objected oriented.
>
> Oh, but yes it is. It just doesn't support subclassing of widgets and
> other stuff like that.
Then we must agree to differ. While it may have *some* aspects of OO it
certainly does not have all those commonly considered necessary to be
considered OO.
Ian
Precisely, and it doesn't support operator overriding either - just what
subset of OO do you consider still to be OO??
Ian
> Ian Bell wrote:
>> What evidence do you have that OO techniques are beneficial in event
>> driven systems?
>
> Why wouldn't they be helpful in event-driven systems? OO techniques are
> helpful in event-driven systems for the same reason that they're helpful
> in linear systems. Database servers are event-driven and many are
> OO-based. "Web services" are usefully event-driven and OO. Why would OO
> be *less* useful in a large event-driven system than in a large linear
> system?
>
So you cannot cite reasons why it would be beneficial other than to ask why
wouldn't it? I thought there were supposed to be overpowering reasons for
including OO. Obviously I was wrong.
Ian