eval is evil (for spreading list-arguments)

50 views
Skip to first unread message

Andreas Leitgeb

unread,
Mar 7, 2003, 8:38:19 AM3/7/03
to
Yes, it's a frequently recurring topic.

Just now, reading the docs for MetaKit, I stumbled over the
following example:
puts [eval format {%-20s %d} [mk::get db.phonebook!2 name date]]

See the mistake ?
At first glance, or just because I indicated that there is one ?


The format-specifier also gets torn apart by eval.
It would require doubled brace-quoting for use with eval.

--
Nichts ist schlimmer als zu wissen, wie das Unheil sich entwickelt,
und voll Ohnmacht zusehn muessen - das macht mich voellig krank...
-- (Musical Elisabeth; "Die Schatten werden laenger")

Ralf Fassel

unread,
Mar 7, 2003, 9:09:38 AM3/7/03
to
* Andreas Leitgeb <Andreas...@siemens.at>

| The format-specifier also gets torn apart by eval.
| It would require doubled brace-quoting for use with eval.

But is `eval' to blame for that? Or rather the programmer who does
not do his homework?

R'

Jeffrey Hobbs

unread,
Mar 7, 2003, 11:19:41 AM3/7/03
to
Andreas Leitgeb wrote:
> Yes, it's a frequently recurring topic.
>
> Just now, reading the docs for MetaKit, I stumbled over the
> following example:
> puts [eval format {%-20s %d} [mk::get db.phonebook!2 name date]]
>
> See the mistake ?
> At first glance, or just because I indicated that there is one ?

Well, I saw the mistake immediately, which is why I always write my
evals now pedantically, like so:

eval [list format {%-20s %d}] [mk::get get db.phonebook!2 name date]

Anything that should not be broken apart should be list'ed.

--
Jeff Hobbs The Tcl Guy
Senior Developer http://www.ActiveState.com/
Tcl Support and Productivity Solutions

Andreas Leitgeb

unread,
Mar 7, 2003, 12:03:18 PM3/7/03
to

Are pitfalls to blame, or just those people who fall into them ?
(And it shows, that it's not only newbies who fall, but also
long-term tcl-seniors, who just happen to have been less than
hard-concentrating while using "eval")

A few years ago, a proposal has been made to alleviate this particular
pitfall, by adding some new syntax to the Tcl-core (backwards-compatibly).
Not only would that offer an obvious way *around* this pitfall
(compared to the "eval-way" leading *straight over* it, expecting you
to jump over it), it would also offer object-safety in many cases where
eval still does not. Also, using it adds less visual "line-noise"
than [list ...]-wrapping.

There have been long threads about it, in the past. From time to time,
if some sighting of someone falling into that pitfall triggers me, I
cast another poll here to see, if time is finally mature for improving
tcl-syntax.

PS: there are also safe and useful uses for "eval".
Expansion of lists, however, is a dangerous one.

Ralf Fassel

unread,
Mar 7, 2003, 2:43:56 PM3/7/03
to
* Andreas Leitgeb <Andreas...@siemens.at>

| Are pitfalls to blame, or just those people who fall into them ?

If you fall once, blame the pitfall, and remember the pit. If you
fall often into the same pit, well... ;-)

| A few years ago, a proposal has been made to alleviate this
| particular pitfall, by adding some new syntax to the Tcl-core
| (backwards-compatibly).

Would be fine with me. Just make it obvious enough so that someone
who is used to the current behaviour does not easily fall into *that*
pit. If I have to add extra arguments or simply use a new name, that
should be enough to trigger my attention to not double-list the
`new-eval' arguments, for example.

| Also, using it adds less visual "line-noise" than [list
| ...]-wrapping.

Well, in this particular example,
eval format {{%d foo}} [some_other_func]
is not too much line-noise IMHO. But of course, there are other cases
where you have to use an extra `list'.

| PS: there are also safe and useful uses for "eval".
| Expansion of lists, however, is a dangerous one.

Expanding lists is the primary use of `eval' for me :-/
Why would I use `eval' otherwise?

R'

Andreas Leitgeb

unread,
Mar 7, 2003, 4:00:49 PM3/7/03
to
Ralf Fassel <ral...@gmx.de> wrote:
> * Andreas Leitgeb <Andreas...@siemens.at>
>| Are pitfalls to blame, or just those people who fall into them ?
> If you fall once, blame the pitfall, and remember the pit. If you
> fall often into the same pit, well... ;-)
Actually it's not so much about me falling myself, but more about me
watching many others (and surely not only newbies) fall in.
Doing Tcl for almost 10 years now, I'm kind of used to it, too.
I still find eval-lines ugly, however.

>| A few years ago, a proposal has been made to alleviate this
>| particular pitfall,

> should be enough to trigger my attention to not double-list the
> `new-eval' arguments, for example.

You seem to be relatively new to this group :-)

Based on the observation that with eval, programmers *really often*
protect too little rather than too much, the proposal was primarily
about "doing something" with the args that should be *expanded*, rather
than protecting all that is to be left as is.

Then, as it is not allowed in current tcl to have trailing characters
after a close-brace, compatibility is guaranteed, if the special
behaviour is triggered by something like {}$ or {}[...].
(Other suggestions were to use e.g. &varname as the "expanding
version of $varname", but that would have caused lots of
migration-headaches) Thus, the first proposal was to add
new syntax {}[command_returning_a_list] and {}$list_variable
to expand the values to multiple arguments.
Various commands that take an arbitrary number of arguments could
then be directly passed a list, spread out into several separate
arguments.
Further discussions showed a general preference to something less
noisy than three special-chars in a row (which seemed too perl'ish),
which led to the still pending proposal of {expand}$varname and
{expand}[command...]

instead of
eval [list command $all [the] $non-expanding [arguments]] $list
one would write:
command $all [the] $non-expanding [arguments] {expand}$list
and gain the following advantages:

a) the "real" command is back at the start of the line, rather than
obscured from sight by eval&list
b) in many cases conversion to and from string can be avoided
(keyword: object-safety). Meanwhile the "eval" has evolved
to avoid this conversion, too, but special care must be taken
to not accidentally thwart it. (That's why the old example uses
[list ...] rather than curlies.)
c) it's much more *un*likely to make a mistake by specifying
{expand} too often, than to list-protect to little.

Most often, in practise you'll find above example written as:
eval command $all [the] $non-expanding [arguments] $list
under the weak assumption, that each of the other vars or commands
evaluate to "nice" values immune to eval's reparsing. Such mistakes
happen easily, and are Tcl's weak point just like wild pointers
are C-programs' #1-cause for random failure.

Recently, TIP #103 has gone some way in the right direction,
but failed to provide advantage "a)" (see above).

>| PS: there are also safe and useful uses for "eval".
>| Expansion of lists, however, is a dangerous one.
> Expanding lists is the primary use of `eval' for me :-/
> Why would I use `eval' otherwise?

For interpreting real scripts. Tk-widget bindings make frequent
use of eval. Outside these uses, I cannot currently think of
any other, but surely there exist more.
It is used for argument expansion, because, as of now, no other
way exists to do argument expansion.

se...@fishpool.com

unread,
Mar 7, 2003, 2:40:45 PM3/7/03
to
Andreas Leitgeb <Andreas...@siemens.at> wrote:
>
> A few years ago, a proposal has been made to alleviate this particular
> pitfall, by adding some new syntax to the Tcl-core (backwards-compatibly).
> Not only would that offer an obvious way *around* this pitfall
> (compared to the "eval-way" leading *straight over* it, expecting you
> to jump over it), it would also offer object-safety in many cases where
> eval still does not. Also, using it adds less visual "line-noise"
> than [list ...]-wrapping.

Who was it who suggested the following form for expand?

expand <list>

I think the idea was that this works partly like eval, but no variable
subtitution or anything is done for <list>. Ie. one level is dropped
off and that's it. So the following would always work as espected:

set foo {puts blah}
expand $foo

Seemed to take care of most issues.

--
/ http://www.fishpool.com/~setok/

Andreas Leitgeb

unread,
Mar 7, 2003, 4:32:28 PM3/7/03
to
se...@fishpool.com <se...@fishpool.com> wrote:
> Ie. one level is dropped off and that's it.
> Seemed to take care of most issues.
Nope, it didn't.

While argument-expansion really is not much more than "dropping
off a level of quoting", the important difference is the scope.

To be really useful, the flattening must only happen to those
arguments, for which it is explicitly "requested", rather than
to all that aren't explicitly protected.

Dan Smart

unread,
Mar 7, 2003, 6:47:25 PM3/7/03
to
Andreas Leitgeb <Andreas...@siemens.at> wrote in
news:slrnb6hk1s.ddm....@pcc524.gud.siemens.at:
> There have been long threads about it, in the past. From time to
> time, if some sighting of someone falling into that pitfall triggers
> me, I cast another poll here to see, if time is finally mature for
> improving tcl-syntax.
<pedant>
Actually you are looking for a significant change in TCL's *semantics*.
</pedant>
And the proposed syntax (constrained as it is by backwards compatibility)
is an ugly wart, not an "improvement".

>
> PS: there are also safe and useful uses for "eval".
> Expansion of lists, however, is a dangerous one.

All uses of eval require care, even with a perfectly well formed list, if
the first word is 'exec', you may still not be safe.

Dan "eval [list [file delete [info script]]]" Smart
--
Dan Smart. C++ Programming and Mentoring.
cpp...@dansmart.com
ADDvantaged

Donald Arseneau

unread,
Mar 7, 2003, 8:43:45 PM3/7/03
to
se...@fishpool.com writes:

> Andreas Leitgeb <Andreas...@siemens.at> wrote:
> >
> > A few years ago, a proposal has been made to alleviate this particular
> > pitfall, by adding some new syntax to the Tcl-core (backwards-compatibly).
> > Not only would that offer an obvious way *around* this pitfall
> > (compared to the "eval-way" leading *straight over* it, expecting you
> > to jump over it), it would also offer object-safety in many cases where
> > eval still does not. Also, using it adds less visual "line-noise"
> > than [list ...]-wrapping.
>
> Who was it who suggested the following form for expand?
>
> expand <list>

There has been much discussion (use google/groups), and a minimalist
solution proposed by Peter Spjuth and Donal K. Fellows:
TIP #103: Argument Expansion Command,
http://www.tcl.tk/cgi-bin/tct/tip/103.html

Please have a look at how that would affect the present pitfall, and
provide some feedback for the Tip. I think it would be written

puts [expand {format {%-20s %d} @[mk::get db.phonebook!2 name date]} ]

to work successfully.


Donald Arseneau as...@triumf.ca

Suchenwi

unread,
Mar 9, 2003, 4:31:53 PM3/9/03
to
Andreas Leitgeb wrote:
>PS: there are also safe and useful uses for "eval".
> Expansion of lists, however, is a dangerous one.

Well, but before we have extended the Tcl syntax with an "expand" construct,
it's the only half-ways readable way...
Consider the great idiom:
eval pack [winfo children $w]

This can only be done with explicit eval. I think one avoids most pitfalls by
remembering that [eval] just like [join] removes one level of grouping, so just
add one level with [list ...] where necessary.

Best regards, Richard Suchenwirth

se...@fishpool.com

unread,
Mar 9, 2003, 9:45:40 PM3/9/03
to
Andreas Leitgeb <Andreas...@siemens.at> wrote:
> se...@fishpool.com <se...@fishpool.com> wrote:
>> Ie. one level is dropped off and that's it.
>> Seemed to take care of most issues.
> Nope, it didn't.
>
> While argument-expansion really is not much more than "dropping
> off a level of quoting", the important difference is the scope.
>
> To be really useful, the flattening must only happen to those
> arguments, for which it is explicitly "requested", rather than
> to all that aren't explicitly protected.

And how does the explained not do that? Only the first list level is
opened up. No other expansion was done in that model.

expand [list button .b] $options

The above should do what we expect in all situations. The $options list
is opened up once as arguments, but nothing inside it is ever expanded.
Surely it's a matter of taste whether it's better to explicitly mark
areas for expansion or for protection? The above system avoids having
new magic characters.

--
/ http://www.fishpool.com/~setok/

se...@fishpool.com

unread,
Mar 9, 2003, 9:49:24 PM3/9/03
to
Donald Arseneau <as...@triumf.ca> wrote:
>>
>> Who was it who suggested the following form for expand?
>>
>> expand <list>
>
> There has been much discussion (use google/groups), and a minimalist
> solution proposed by Peter Spjuth and Donal K. Fellows:
> TIP #103: Argument Expansion Command,
> http://www.tcl.tk/cgi-bin/tct/tip/103.html

Yes, I've seen the TIP and followed the discussion ;-)


>
> Please have a look at how that would affect the present pitfall, and
> provide some feedback for the Tip. I think it would be written
>
> puts [expand {format {%-20s %d} @[mk::get db.phonebook!2 name date]} ]

With the other expand proposal that would work:

puts [expand [list format {%-20s %d}] [mk::get db.phonebook!2 name date]]

Not quite sure what you wanted commented in the TIP. I could add the above
suggestion there, but I'd prefer if the original inventor did that so I
don't get it wrong. I just can't remember who was talking about it on
IRC.

--
/ http://www.fishpool.com/~setok/

Andreas Leitgeb

unread,
Mar 10, 2003, 4:50:55 AM3/10/03
to
se...@fishpool.com <se...@fishpool.com> wrote:

> Andreas Leitgeb <Andreas...@siemens.at> wrote:
>> To be really useful, the flattening must only happen to those
>> arguments, for which it is explicitly "requested", rather than
>> to all that aren't explicitly protected.
This paragraph was intended to answer your questions below.
Was it really that unintellegible ?

The misinterpretations might stem from my programmer's view to
it, as opposed to the Tcl-interpreter's view to it:

For Tcl, the first argument to eval in
% eval [list destroy] [winfo children .]
is just plain the result of the list-command. Nothing special.
One level of *everything* is dropped, so what finally gets to
work is the original "destroy" and each of the returned child-widget
names separately.

For a programmer, otoh, this [list ...] is a protection-wrapper
around the string "destroy", to make eval see it as a list.
For the programmer, this [list ...] construct is not part of
the program's logic. It's some magic salt that he has learnt
to add, to make things work.

In the programmer's mind there is the command "destroy", and
the command that returns the list of .'s childs. To tie these
together, he now has to add one list-level to all those parts
that already were at the "correct" quoting-level before, and then
let eval remove one level of *all* the arguments, merely undoing
the "protection" for the "destroy".

For the programmer it would be like "avoiding a detour" to be able
to express: "I want only the widget-list spreaded, the rest untouched."

This part of the whole story fortunately got addressed with
TIP 103 as in: expand {destroy @[winfo children .]}

The other part, namely getting the "destroy" command to
the start of the line, is something, TIP 103 does not
address.

As it seems, TIP 103 is at least the best we can get to for the time.

> No other expansion was done in that model.
> expand [list button .b] $options

This would be practically equivalent to the "eval"-approach.
The pitfall would be indeed a tiny bit narrower, but still large enough
for the example in the first posting of this thread to still fall in.
Btw., that example was not that of a newbie, but taken from the
documentation of "MK4tcl", the Tcl-bindings to metakit.

> Surely it's a matter of taste whether it's better to explicitly mark
> areas for expansion or for protection?

Not a matter of taste, but a matter of narrowing a pitfall,
... of making Tcl programmer-friendlier.
And that's the whole point of this thread, by the way.

> The above system avoids having new magic characters.

Braces were "magic" even before :-)
Otoh, the "@" as used in TIP 103 wasn't magic in Tcl before.
The association of "@" with "list" is entirely perl'ish ;-)

Andreas Leitgeb

unread,
Mar 10, 2003, 5:37:49 AM3/10/03
to
Donald Arseneau <as...@triumf.ca> wrote:
> There has been much discussion (use google/groups), and a minimalist
> solution proposed by Peter Spjuth and Donal K. Fellows:
> TIP #103: Argument Expansion Command,
> http://www.tcl.tk/cgi-bin/tct/tip/103.html
>
> puts [expand {format {%-20s %d} @[mk::get db.phonebook!2 name date]} ]
It's *much* better than the current status quo.

> Please have a look at how that would affect the present pitfall,

Yes, it would definitely fix it.

> and provide some feedback for the Tip.

I'd even prefer {}$var instead of @$var inside the braced argument
of expand, for it's "*forward*-compatibility" (see TIP), but with
"@" it is still much better than nothing.

Any estimates about when this TIP will undergo the judgement of
the TCT ?

Bryan Oakley

unread,
Mar 10, 2003, 9:09:24 AM3/10/03
to
Andreas Leitgeb wrote:
> Donald Arseneau <as...@triumf.ca> wrote:
>
>>There has been much discussion (use google/groups), and a minimalist
>>solution proposed by Peter Spjuth and Donal K. Fellows:
>>TIP #103: Argument Expansion Command,
>> http://www.tcl.tk/cgi-bin/tct/tip/103.html
>>
>>puts [expand {format {%-20s %d} @[mk::get db.phonebook!2 name date]} ]
>
> It's *much* better than the current status quo.
>
>
>>Please have a look at how that would affect the present pitfall,
>
> Yes, it would definitely fix it.

How would it fix it? It would still be available for people who choose
to use it. The proposal simply provides an alternative to the "pitfall";
it doesn't remove or fix the pitfall.

Donal K. Fellows

unread,
Mar 10, 2003, 9:16:06 AM3/10/03
to
Andreas Leitgeb wrote:
> There have been long threads about it, in the past. From time to time,
> if some sighting of someone falling into that pitfall triggers me, I
> cast another poll here to see, if time is finally mature for improving
> tcl-syntax.

Having thought about this long and hard (including looking through the possible
alternatives) my favoured alternative is a single extra rule added to Tcl(n):

If the first character of a word is back-quote (```'') then the rest of
the word (as determined by the other rules, including the double-quote
and open brace rules) is interpreted as a list and each of the words is
appended to the list of words to be passed to the command as a separate
word. No further expansion is performed on those words, and nor is the
back-quote character part of any word.

The idea is that this allows for expansion to be performed exactly where the
programmer wants, and nowhere else, as well as putting the emphasis clearly on
what is being executed for real instead of the machinery to make everything be
expanded as we want. It's also much prettier than the {} hack. In the current
example of the hour, this would allow things to be written as:

format {%-20s %d} `[mk::get get db.phonebook!2 name date]

The choice of back-quote? This is enforced by the fact that most other
characters are far more common in Tcl scripts (e.g. @ is used for a number of
things throughout Tk, as well as being commonplace in places like email
addresses...)

Bet someone's going to hate this idea though. C'est la vie.

> PS: there are also safe and useful uses for "eval".
> Expansion of lists, however, is a dangerous one.

Minor correction: It's safe, but only if you're seriously paranoid.

Donal.
--
Donal K. Fellows http://www.cs.man.ac.uk/~fellowsd/ donal....@man.ac.uk
"If RedHat are so purblind that they think computing is about ever fancier
'desktop themes', they are in the interior design business, and as everyone
knows, if you can piss you can paint." -- Steve Blinkhorn

Andreas Leitgeb

unread,
Mar 10, 2003, 9:41:16 AM3/10/03
to
Bryan Oakley <br...@bitmover.com> wrote:
>> Yes, it would definitely fix it.
> How would it fix it? It would still be available for people who choose
> to use it. The proposal simply provides an alternative to the "pitfall";
> it doesn't remove or fix the pitfall.

You're right. This requires overloading of the verb "fix" as:

to fix a pitfall :=
to offer an obvious alternative way around the pitfall.

;-)

Roy Terry

unread,
Mar 10, 2003, 9:58:20 AM3/10/03
to
"Donal K. Fellows" wrote:

> If the first character of a word is back-quote (```'') then the rest of
> the word (as determined by the other rules, including the double-quote
> and open brace rules) is interpreted as a list and each of the words is
> appended to the list of words to be passed to the command as a separate
> word. No further expansion is performed on those words, and nor is the
> back-quote character part of any word.
>

The following ought to be a high priority for
any programming language that aspires to ease of use
and certainly Tcl qualifies:


> The idea is that this allows for expansion to be performed exactly where the
> programmer wants, and nowhere else,

> format {%-20s %d} `[mk::get get db.phonebook!2 name date]


>
> The choice of back-quote? This is enforced by the fact that most other
> characters are far more common in Tcl scripts (e.g. @ is used for a number of
> things throughout Tk, as well as being commonplace in places like email
> addresses...)
>

Hhmm. At first I didn't like this but on second look it gets better
and better. Grepping 100K lines of nearby Tcl code turns up only a one
backquotes and not at word beginning. It's definitely way better than
adding a new command, which is merely a crutch and a bandaid,
and it really assists writing concise code.

IOW: A big win.

Roy
(Cheers and waves from sidelines ;*)

se...@fishpool.com

unread,
Mar 10, 2003, 10:56:21 AM3/10/03
to
Andreas Leitgeb <Andreas...@siemens.at> wrote:

> For Tcl, the first argument to eval in
> % eval [list destroy] [winfo children .]
> is just plain the result of the list-command. Nothing special.
> One level of *everything* is dropped, so what finally gets to
> work is the original "destroy" and each of the returned child-widget
> names separately.

Just in case it wasn't clear, with the proposed [expand] you would
not need a [list] around the destroy there. I don't find it particularly
confusing to have the idea "remove one list level and execute command".
I do, however, accept your reasoning for the reverse. I just have a natural
fear of adding new special characters (even though they are only
command-specific) -- can end up in nasty mistakes when you're not expecting
it: a variable starts with @ and the programmer doesn't know it.

--
/ http://www.fishpool.com/~setok/

lvi...@yahoo.com

unread,
Mar 10, 2003, 12:38:41 PM3/10/03
to

According to Donal K. Fellows <donal.k...@man.ac.uk>:
:Having thought about this long and hard (including looking through the possible

:alternatives) my favoured alternative is a single extra rule added to Tcl(n):

Hmm - along with adding this rule, some code has to be added to the interpreter
to deal with the back quote, right?

--
Join us at the Tenth Annual Tcl/Tk Conference <URL: http://mini.net/tcl/6274 >
Even if explicitly stated to the contrary, nothing in this posting
should be construed as representing my employer's opinions.
<URL: mailto:lvi...@yahoo.com > <URL: http://www.purl.org/NET/lvirden/ >

Bryan Oakley

unread,
Mar 10, 2003, 12:58:07 PM3/10/03
to
Donal K. Fellows wrote:
> Andreas Leitgeb wrote:
>
>>There have been long threads about it, in the past. From time to time,
>>if some sighting of someone falling into that pitfall triggers me, I
>>cast another poll here to see, if time is finally mature for improving
>>tcl-syntax.
>
>
> Having thought about this long and hard (including looking through the possible
> alternatives) my favoured alternative is a single extra rule added to Tcl(n):
>
> If the first character of a word is back-quote (```'') then the rest of
> the word (as determined by the other rules, including the double-quote
> and open brace rules) is interpreted as a list and each of the words is
> appended to the list of words to be passed to the command as a separate
> word. No further expansion is performed on those words, and nor is the
> back-quote character part of any word.

Is there a reason you chose a single leading character rather than a
pair of characters, such as a pair of single quotes, or pair of
backquotes (eg: command foo '$bar' baz) There's precidence for either ($
is a single char, "", {} and [] are used in pairs).

Tom Wilkason

unread,
Mar 10, 2003, 2:05:49 PM3/10/03
to
"Donal K. Fellows" <donal.k...@man.ac.uk> wrote in message
news:3E6C9E26...@man.ac.uk...
I like the approach of specifying lists that should be passed in as multiple
arguments rather than the other way around (i.e. specifying the ones that
shouldn't be).

Has anyone looked into the core to see if something like this a relatively
easy solution to implement? When I first thought about this problem I
thought a good way to handle it would be something like:

format {%-20s %d} [unlist [mk::get get db.phonebook!2 name date]]

But from an implementation point of view I don't think it is feasable since
all function calls are structured to return a single object.

Another consideration, what should be the result of:
set A `[mk::get get db.phonebook!2 name date]
Should it throw a syntax error such as "set a 1 2 3" would?

Tom Wilkason


Suchenwi

unread,
Mar 10, 2003, 4:00:36 PM3/10/03
to
Tom Wilkason wrote:

>Another consideration, what should be the result of:
>set A `[mk::get get db.phonebook!2 name date]
>Should it throw a syntax error such as "set a 1 2 3" would?

Yes, I strongly think so - substitution is a separate step from command
invocation, so it should react (with the same error message) as
eval [list set A] [mk::get get db.phonebook!2 name date]
does now.
--
Best regards, Richard Suchenwirth

Donal K. Fellows

unread,
Mar 11, 2003, 8:06:00 AM3/11/03
to
lvi...@yahoo.com wrote:
> Hmm - along with adding this rule, some code has to be added to the
> interpreter to deal with the back quote, right?

No, the english specification in Tcl(n) just magically manifests itself as
implementation. :^D

-- Thanks, but I only sleep with sentient lifeforms. Anything else is merely
a less sanitary form of masturbation.
-- Alistair J. R. Young <avatar...@arkane.demon.co.uk>

Donal K. Fellows

unread,
Mar 11, 2003, 8:10:06 AM3/11/03
to
Bryan Oakley wrote:
> Is there a reason you chose a single leading character rather than a
> pair of characters, such as a pair of single quotes, or pair of
> backquotes (eg: command foo '$bar' baz) There's precidence for either ($
> is a single char, "", {} and [] are used in pairs).

Personal preference and the fact that I think of ` as flagging the word for
different treatment (as opposed to `` containing different treatment of a word.)
In fact, if you have something that "brackets" things, then you have to state
what happens if you have more than one word inside.

*thinks*

I suppose you could make `` act like double quotes except with expand-to-words
behaviour. That'd work...

Peter.DeRijk

unread,
Mar 11, 2003, 9:52:42 AM3/11/03
to
Jeffrey Hobbs <Je...@activestate.com> wrote:
> Andreas Leitgeb wrote:
>> Yes, it's a frequently recurring topic.
>>
>> Just now, reading the docs for MetaKit, I stumbled over the
>> following example:
>> puts [eval format {%-20s %d} [mk::get db.phonebook!2 name date]]
>>
>> See the mistake ?
>> At first glance, or just because I indicated that there is one ?

> Well, I saw the mistake immediately, which is why I always write my
> evals now pedantically, like so:

> eval [list format {%-20s %d}] [mk::get get db.phonebook!2 name date]

> Anything that should not be broken apart should be list'ed.

Is there a reason for not doing it this way:
eval {format {%-20s %d}} [mk::get get db.phonebook!2 name date]
This has less noise, and also seems to work nicely to protect i
variables from expanding, eg.
set format {%-20s %d}
eval {format $format} [mk::get get db.phonebook!2 name date]

--
Peter De Rijk der...@uia.ua.ac.be
<a href="http://rrna.uia.ac.be/~peter/">Peter</a>
To achieve the impossible, one must think the absurd.
to look where everyone else has looked, but to see what no one else has seen.

Jeffrey Hobbs

unread,
Mar 11, 2003, 11:56:31 AM3/11/03
to
Peter.DeRijk wrote:

> Jeffrey Hobbs wrote:
>>Well, I saw the mistake immediately, which is why I always write my
>>evals now pedantically, like so:
>> eval [list format {%-20s %d}] [mk::get get db.phonebook!2 name date]
>>Anything that should not be broken apart should be list'ed.

> Is there a reason for not doing it this way:
> eval {format {%-20s %d}} [mk::get get db.phonebook!2 name date]
> This has less noise, and also seems to work nicely to protect i

The solution with lists is more pedantically correct, but just using {}s
works just as well (that's Brent Welch's preferred method, BTW). The
small added advantage with [list] is that special internal optimizations
will make it a bit faster in some cases than using {}s.

Andreas Leitgeb

unread,
Mar 13, 2003, 6:08:25 AM3/13/03
to
Donal K. Fellows <donal.k...@man.ac.uk> wrote:
> I suppose you could make `` act like double quotes except with expand-to-words
> behaviour. That'd work...

You mean something like the following ?
command $arg `$arglist`

There are some issues & caveats that need to be dealt with:
`"$arglist"` ? (does not make too much sense, anyway, does it ?)
`"$arglist`" ? (should be a syntax-error. Likely the only
bad thing is the trailing non-whitespace
after the closing backtick, as doublequotes
wouldn't have a special meaning inside)
`[command `$arglist`]` ? (unquoted nesting via [], as with "'s)
Under which limitations is object-safety possible ?
(I'd guess, if no concatenation of values happens between the
backticks, but then, would `$arg ` thwart objects ?)

One thing not to forget (it occurred to me while in an email-discussion
with setok): if ` becomes a special character, then [list] will have to
protect it. (it currently doesn't)
Even if TIP 103 (with @ as the special char only for expand) gets done
instead of this, it would be a good thing to have list protect @-chars.

I'd really love to see this latest suggestion added to Tcl.
With backticks it looks cleaner and more consistent (Tcl'ish)
than my previous {}$ and {}[-approaches.

Cameron Laird

unread,
Mar 13, 2003, 9:01:49 AM3/13/03
to
In article <slrnb6i1v5.ddm....@pcc524.gud.siemens.at>,
Andreas Leitgeb <a...@logic.at> wrote:
.
.

.
>> Expanding lists is the primary use of `eval' for me :-/
>> Why would I use `eval' otherwise?
>For interpreting real scripts. Tk-widget bindings make frequent
> use of eval. Outside these uses, I cannot currently think of
> any other, but surely there exist more.
>It is used for argument expansion, because, as of now, no other
> way exists to do argument expansion.
.
.
.
Yes.

I just want to add my own emphasis to this accurate description.
Many, many Tcl applications do NOT need [eval], or only need it
for argument expansion, as we've been discussing. This deserves
repetition, as, during Tcl's early history, it was more widely
thought that "interpretation of synthesized scripts" was a good
and necessary Tcl style. This turns out to have been largely a
"dead end".

So, yes, it's for "interpreting real scripts". If we understand
'real' in sufficient generality, that's it's only proper use;
there are no others.

Tangential point: eval is formally redundant; it's only an abbre-
viation for [uplevel 0 ...]
--

Cameron Laird <Cam...@Lairds.com>
Business: http://www.Phaseit.net
Personal: http://phaseit.net/claird/home.html

lvi...@yahoo.com

unread,
Mar 13, 2003, 10:55:33 AM3/13/03
to

According to Cameron Laird <cla...@phaseit.net>:
:Many, many Tcl applications do NOT need [eval], or only need it

:for argument expansion, as we've been discussing.

Cameron - is the need for argument expansion some artifact of Tcl?
If not, then what do other languages do about the issue?

Volker Hetzer

unread,
Mar 13, 2003, 11:12:39 AM3/13/03
to

<lvi...@yahoo.com> wrote in message news:b4q9ll$8ne$2...@srv38.cas.org...

>
> According to Cameron Laird <cla...@phaseit.net>:
> :Many, many Tcl applications do NOT need [eval], or only need it
> :for argument expansion, as we've been discussing.
>
> Cameron - is the need for argument expansion some artifact of Tcl?
No, the need is always there.

> If not, then what do other languages do about the issue?

They demand everything expanded into complex data structures.

Greetings!
Volker


Darren New

unread,
Mar 13, 2003, 1:40:48 PM3/13/03
to
lvi...@yahoo.com wrote:
> is the need for argument expansion some artifact of Tcl?
> If not, then what do other languages do about the issue?

Some languages distinguish argument expansion from evaluation. So you'd
pass a list of arguments and it would be assigned to the variables, but
not via evaluation. Formally, this is known as the "apply" operation.


Joe English

unread,
Mar 13, 2003, 1:37:13 PM3/13/03
to
lvirden wrote:
>According to Cameron Laird:

>:Many, many Tcl applications do NOT need [eval], or only need it
>:for argument expansion, as we've been discussing.
>
>Cameron - is the need for argument expansion some artifact of Tcl?
>If not, then what do other languages do about the issue?

The need for argument expansion can arise for any command
that takes a variable number of arguments instead of a
single list-valued argument. Some of these could have
been avoided by using a better-designed interface (e.g.,
[file join]), but not all.

Many (most?) other languages don't even support variadic procedures.
Those that do often also provide either an "apply" procedure
(Scheme, Lisp), or an alternate fixed-argument entry point
for all variadic procedures (e.g., sprintf/vsprintf in C).

Another common use of [eval] is to expand command _prefixes_:
[eval $callback [list $arg1 $arg2 $arg3]], where $callback
is a user-supplied script with a command name and optional
initial arguments.

Other languages handle this use case with closures (Lisp-family FPLs),
curried functions (ML/Haskell-family FPLs), or idiomatically
(e.g., callback functions that take an additional 'void
*ClientData' argument in C).

--Joe English

jeng...@flightlab.com

Donald Arseneau

unread,
Mar 13, 2003, 4:53:44 PM3/13/03
to
lvi...@yahoo.com writes:

> According to Cameron Laird <cla...@phaseit.net>:
> :Many, many Tcl applications do NOT need [eval], or only need it
> :for argument expansion, as we've been discussing.
>
> Cameron - is the need for argument expansion some artifact of Tcl?
> If not, then what do other languages do about the issue?

The worlds cleanest programming language uses \expandafter ):->

Donald Arseneau as...@triumf.ca


lvi...@yahoo.com

unread,
Mar 14, 2003, 7:03:24 AM3/14/03
to

According to Joe English <jeng...@flightlab.com>:

:lvirden wrote:
:>According to Cameron Laird:
:>:Many, many Tcl applications do NOT need [eval], or only need it
:>:for argument expansion, as we've been discussing.
:>
:>Cameron - is the need for argument expansion some artifact of Tcl?
:>If not, then what do other languages do about the issue?
:
:The need for argument expansion can arise for any command
:that takes a variable number of arguments

:Many (most?) other languages don't even support variadic procedures.

The reason I am trying to couch this branch of the thread in terms of
other languages is to see if some other language's solution might fit
Tcl. It may be that it is not possible. I'm hoping that it is.

ksh and perl, for instance, allow one to create functions which take variable
number of arguments.

:Those that do often also provide either an "apply" procedure


:(Scheme, Lisp), or an alternate fixed-argument entry point
:for all variadic procedures (e.g., sprintf/vsprintf in C).

I don't know how either perl or ksh solve the problem - perl probably has
the apply type procedure.

:Other languages handle this use case with closures (Lisp-family FPLs),


:curried functions (ML/Haskell-family FPLs), or idiomatically
:(e.g., callback functions that take an additional 'void
:*ClientData' argument in C).

So, of the at least 5 solutions you mention Joe, are any of these ones
that might be a fit for Tcl?

Andreas Leitgeb

unread,
Mar 14, 2003, 9:36:46 AM3/14/03
to
lvi...@yahoo.com <lvi...@yahoo.com> wrote:
> The reason I am trying to couch this branch of the thread in terms of
> other languages is to see if some other language's solution might fit
> Tcl. It may be that it is not possible. I'm hoping that it is.
The reasoning is good, but I fear, Tcl is too unique in this regard
to allow for a solution of another language to be applied to it.

> ksh
ksh has the proposed backtick-semantics automatical. You need to
double-quote variable- or command-substitutions to avoid having
the substituted values be torn apart.
x="1 2 3"; argcount() { echo $#; }
argcount $x -> 3
argcount "$x" -> 1

> and perl,
perl flattens all lists (references to lists are not lists)
@x=(1,2,3)
print 0,@x,4,5 -> print sees only one flat list, namely (0,1,2,3,4,5)

> So, of the at least 5 solutions you mention Joe, are any of these ones
> that might be a fit for Tcl?

The 6th one, suggested by Donal K. Fellows :-)
% set x {1 2 3}
% argcount `$x` -> 3
% argcount $x -> 1

Bryan Oakley

unread,
Mar 14, 2003, 10:56:07 AM3/14/03
to
Andreas Leitgeb wrote:

> lvi...@yahoo.com <lvi...@yahoo.com> wrote:
>
>>So, of the at least 5 solutions you mention Joe, are any of these ones
>>that might be a fit for Tcl?
>
>
> The 6th one, suggested by Donal K. Fellows :-)
> % set x {1 2 3}
> % argcount `$x` -> 3
> % argcount $x -> 1
>

Didn't Donal suggest a single leading backquote (eg: `$x)? I asked why
he chose a single leading quote rather than a pair. Personally, I'm
partial to the pair, but could live with either.

I like this approach _much_ better than the {}$x solution proposed a
year or two ago. I feel silly saying that, though. Really, why should it
matter that `$x` is "prettier" than {}$x? And yet, to me it does matter.
In my head I dismissed the {}$x solution as a hack, yet consider `$x or
`$x` to be good solutions. Go figure.

Perhaps it has to do with the fact I perceive {}$foo as an exception to
the normal {} processing rules, whereas `$foo` is a new, simple(ish)
rule. I like the fact that, generally speaking, Tcl has no exceptions to
it's rules.

bo...@aol.com

unread,
Mar 14, 2003, 2:58:16 PM3/14/03
to
You did a very good job of pointing up the difficulties of using a
*pair* of backticks. To me it makes better sense to use just one and
consider it as a modifier to $ and [ substitution. There is already a
precedent for this since we do $<var-name> not $<var-name>$. In a
context where substitution is done, `$ calls for the list elements to
be substituted and $ substitutes the value. This is a minimal
solution and does only what is needed and nothing more.

Using a pair of backticks does more than is needed (not the 'tcl way')
and introduces unwanted behaviors more rules to be aware of. Making
the result look like some other language should be a non-goal. In
this case the pair of backquotes looks like a ksh feature, but does
something entirely different (the worst kind of similarity!).

bob

Andreas Leitgeb <Andreas...@siemens.at> wrote in message news:<slrnb70pf3.ddm....@pcc524.gud.siemens.at>...

Roy Terry

unread,
Mar 14, 2003, 3:47:13 PM3/14/03
to
Yes the single back tick has some appeal. It also is virtually
guaranteed to break some existing scripts. The {}$ {}[ solution was
specifically proposed because it used a previously illegal construct.

Frankly, it is a puzzle to me that we all seem to be lining up behind
an incompatible change as opposed to a basically compatible change
(unless
you consider *not* causing an error a *basic* imcompatibility)

As to making the backquotes come in a pair, I think that opens a
huge can of worms re escaping embedded backquotes and throws open
the meaning of "" and {} inside backquotes. We absolutely don't want
to go there! Anyway, the idea of expansion is not to introduce a new
quoting mechanism but to mark conventional words for the parser to
expand. For that purpose the single leading ` is both easer to use and
easier to implement and explain I think.

I hope the TCT will have the forward-looking courage to let this
change in and solve a fundamental weakness in the Tcl language.
( Either {} or ` would do and the sooner the better)

Cheers,
Roy

PS: I don't have any (known) args with leading ` so the incompability
doesn't touch my code. How about others?

John E. Perry

unread,
Mar 14, 2003, 5:15:23 PM3/14/03
to

"Roy Terry" <royt...@earthlink.net> wrote in message
news:3E724019...@earthlink.net...

> Bryan Oakley wrote:
> >
> > Andreas Leitgeb wrote:
> > > lvi...@yahoo.com <lvi...@yahoo.com> wrote:
> > >
> > >>...etc wrote

> >
> > Didn't Donal suggest a single leading backquote (eg: `$x)? I asked why
> > he chose a single leading quote rather than a pair. Personally, I'm
> > partial to the pair, but could live with either.
> >

Am I the only one reading this list who feels _very_ uncomfortable with
unmatched quote marks -- front, back, or whatever? Aren't there some
characters available that don't almost _mandate_ matching co-characters?

John Perry
AS&M


Donald Arseneau

unread,
Mar 14, 2003, 6:42:13 PM3/14/03
to
Bryan Oakley <br...@bitmover.com> writes:

> he chose a single leading quote rather than a pair. Personally, I'm
> partial to the pair, but could live with either.

I think a great strength of Tcl is its use of matching { } for
verbatim quoting (where "shell" uses ' ') and [ ] for execution
(like ` `). Tcl does have " " for weak quoting though. I
wouldn't want to add another case of pairing identical
characters wih ` ` for expansion. Moreover, in the argument
expansion proposal, the argument is clearly delimited already
by the variable name or matched [ ]. A lone ` prefix is
preferable.

As for the "backwards compatible" {} prefix, that isn't really
backwards compatible for the reason stated: removing an error
condition *is* an incompatible change. This falls into the
common trap for language extenders -- fitting new features into
previously illegal syntax. Doing that corrupts the regularity
of the language; and regularity is the prime ingrediant for
ease of use (for all but the most expert). Also, a good language
should have a sparse syntax, so errors can be recognized as such,
and not masquerade as some unintended command. A {} prefix is
the epitome of this mistake by making an easy typgraphic error
({} [foo] typed as {}[foo]) into legal syntax.

If I were voting, I'd vote for the syntax change with ` prefix.

Donald Arseneau as...@triumf.ca

Bryan Oakley

unread,
Mar 14, 2003, 9:23:29 PM3/14/03
to
Roy Terry wrote:

> Frankly, it is a puzzle to me that we all seem to be lining up behind
> an incompatible change as opposed to a basically compatible change
> (unless
> you consider *not* causing an error a *basic* imcompatibility)

Weird, isn't it? I agree with your observation -- {}$foo provides less
(well, no) backwards compatibility issues, yet I still personally would
go for the backtick.

Sometimes beauty wins over truth :-)

> ...


>
> I hope the TCT will have the forward-looking courage to let this
> change in and solve a fundamental weakness in the Tcl language.

I wouldn't go so far as to say this is a fundamental weakness. A mild
inconvenience perhaps. This feature is mostly just syntactic sugar to
make certain constructs a bit more palatable. Still, it's very nice sugar.

Jean-Claude Wippler

unread,
Mar 15, 2003, 10:04:35 AM3/15/03
to
"Donal K. Fellows" <donal.k...@man.ac.uk> wrote ...

> If the first character of a word is back-quote (```'') then the rest of
> the word (as determined by the other rules, including the double-quote
> and open brace rules) is interpreted as a list and each of the words is
> appended to the list of words to be passed to the command as a separate
> word. No further expansion is performed on those words, and nor is the
> back-quote character part of any word.
>
> The idea is that this allows for expansion to be performed exactly where the
> programmer wants, and nowhere else, as well as putting the emphasis clearly on
> what is being executed for real instead of the machinery to make everything be
> expanded as we want. [...]

I admit that {}blah, and especially `blah both look appealing.

But I wonder more and more if it's the best way forward - long term...

(Warning: very rough & raw ponderings ahead...)

In Tcl, "everything is a string" (EIAS). Types are introduced by
context-of-use, and cached. With all the tricky consequences of
shimmering, and all the benefits of dual object representations. Both
shimmering and caching are about performance, conceptually everything
is *still* a string.

Aggregation in Tcl is treated in the same way, whether something
consists of N things or not is a matter of using list operators such
as llength, lindex, and lset.

There is a good reason for this: by sticking to the EIAS mantra, we
get the ability to display/store/transmit aggregate data in the same
uniform way as everything else. It's why one could argue that Tcl
also follows the "everything is an object" model.

But it comes at a price. I dare state that Tcl's weakest side is the
way it deals with non-trivial data structures, starting with lists,
but also hashes/maps/arrays/associaitions, all the way up to nested
combinations thereof. Because of the lack of even the distiction
between values and aggregation of values, we *always* have to apply
list commands to lists. Hence all the code with zillions of lindex,
lappend, lsearch, etc commands.

In theory, EIAS is a brilliant simplification. In practice, I find
EIAS one step too much. There are far, *far*, *FAR* more cases where
one treats data as a scalar (string, int, float, handle, whatever)
*or* as an aggregate. Sure, it's extremely useful to have lossless
mappings between those two categories, but that's of the same order as
having a way to display all values in text form. Great for data
interchange - i.e. UI's as well as for transport and storage

The whole EIAS concept has some arbitrariness to it btw - a string is,
after all, an aggregate of characters. So we already have a built-in
concept of aggregation. In fact it was altered in a very fundamental
way not so long ago, when Unicode was introduced. Now, strings have a
length (number of chars) as well as a distinct representation (UTF-8),
which in turn has a "bytelength" (an octet count).

So in a way, Tcl also follows the "everything is an aggregate" model.
After all, you can use the "string length" command on anything. And
there are explicit conversions between strings and lists, in the form
of [split ... {}] and [join ... { }].

Back to ticks, eh, I mean backticks...

In today's Tcl model, since aggregation or arbitrary elements is not
supported as fundamental operation, when a function returns something,
the caller has no way to find out whether the returned data is a
scalar or an aggregate. We need to decide at the *caller* location
what to do with the result and how to treat it.

Hence the proposal to introduce a new convenient notation such as:
foo 1 2 `[bar] 5 6
With bar returning data perhaps using this:
return [list 3 4]

But what if Tcl 9 were to break with this tradition and introduce a
distinct non-shimmering aspect of data, being its dimension?

It would make the backtick issue go away, allowing:
foo 1 2 [bar] 5 6
The reasoning being that a return value is either a scalar (no
dimension) or a list (with a length).

There are some biiiig issues with such a change. For one, eval would
not be needed much (allowing much better bytecode compilation, I
expect). The "interp alias" trick to introduce args ("currying") can
be replaced by list:
set lsi [list lsearch -integer $mylist]
... [$lsi $key] ...
If the "namespaces are ensembles" TIP is adopted, this would make the
"::" namespace separator redundant ("a::b::c" becomes [list a b c]).

There is also a lot of trouble ahead: the "list" command needs to
return its value as a 1-element list of a list, to avoid nullifying
its effect. The "foreach" command, and tons of other commands would
need to be reviewed, to see how automatic expansion affects them.

Despite the extreme impact of such a change, I think it's very much
worth trying to think this through. Because the underlying issue
still holds: aggregation is IMO not a datatype, as ints and floats and
dates are. By introducing aggregation as separate concept (which it
already is, but only for unicode chars), a *lot* of software
complexity might just go away.

Maybe Tcl can evolve to "everything is an aggregate", of chars *and*
of other things? It might just turn out to unify length, index,
append, range enough to make the distinction between strings and lists
go away. Arrays (i.e. hashes/maps) can easily be built on top, which
means this would open the door to arrays-as-first-class values. And
quite a bit more, but I feel that this ramble is already over the
top... :)

-jcw

PS. None of the above need affect the core concepts of Tcl IMO, such
as copy-on-write and the ability to treat everything as a string for
display, storage, and transport.

PPS. One could consider introducing "cardinality" even, i.e. dealing
with 0..N dimensions, which would open the door to making Tcl a
data-manipulation powerhouse. But that's of less importance, since it
can be simulated with uni-dimensional aggregation.

Joe English

unread,
Mar 15, 2003, 12:47:06 PM3/15/03
to
Roy Terry wrote:

>Yes the single back tick has some appeal. It also is virtually
>guaranteed to break some existing scripts. The {}$ {}[ solution was
>specifically proposed because it used a previously illegal construct.
>
>Frankly, it is a puzzle to me that we all seem to be lining up behind
>an incompatible change as opposed to a basically compatible change

My main objection to {}$ is aesthetic. Specifically,
I think the principle of extending a language's syntax
by using currently-illegal constructs is a Really Bad Idea.
Witness C++ :-)

I prefer the leading backtick syntax, but not until Tcl 9.


--Joe English

jeng...@flightlab.com

Derk Gwen

unread,
Mar 16, 2003, 7:19:36 AM3/16/03
to
# Despite the extreme impact of such a change, I think it's very much
# worth trying to think this through. Because the underlying issue

Alternatively, define a script language which can be loaded with
Tcl and which can Tcl procs and is callable from Tcl procs.

--
Derk Gwen http://derkgwen.250free.com/html/index.html
But I do believe in this.

Lars Hellström

unread,
Mar 17, 2003, 5:37:12 AM3/17/03
to
Roy Terry wrote:

>Bryan Oakley wrote:
>
>>Andreas Leitgeb wrote:
>>
>>>lvi...@yahoo.com <lvi...@yahoo.com> wrote:
>>>
>>>>So, of the at least 5 solutions you mention Joe, are any of these ones
>>>>that might be a fit for Tcl?
>>>>
>>>
>>>The 6th one, suggested by Donal K. Fellows :-)
>>>% set x {1 2 3}
>>>% argcount `$x` -> 3
>>>% argcount $x -> 1
>>>
>>Didn't Donal suggest a single leading backquote (eg: `$x)? I asked why
>>he chose a single leading quote rather than a pair. Personally, I'm
>>partial to the pair, but could live with either.
>>
>>I like this approach _much_ better than the {}$x solution proposed a
>>year or two ago. I feel silly saying that, though. Really, why should it
>>matter that `$x` is "prettier" than {}$x? And yet, to me it does matter.
>>In my head I dismissed the {}$x solution as a hack, yet consider `$x or
>>`$x` to be good solutions. Go figure.
>>

My aestethic feeling about these methods are actually quite opposite. In
{}$x, the braces look a bit like arrows that aim to break up the $x in
smaller pieces, which is precisely what the expansion does. In `$x, the
quote mainly looks as if it is there by error. But Donal has a point
that {}$x may prevent certain typos from being caught by the interpreter.

>>
>>Perhaps it has to do with the fact I perceive {}$foo as an exception to
>>the normal {} processing rules, whereas `$foo` is a new, simple(ish)
>>rule. I like the fact that, generally speaking, Tcl has no exceptions to
>>it's rules.
>>
>Yes the single back tick has some appeal. It also is virtually
>guaranteed to break some existing scripts. The {}$ {}[ solution was
>specifically proposed because it used a previously illegal construct.
>
>Frankly, it is a puzzle to me that we all seem to be lining up behind
>an incompatible change as opposed to a basically compatible change
>(unless
>you consider *not* causing an error a *basic* imcompatibility)
>

Allow me to present another syntax suggestion for this. The fundamental
problem with ` is that it makes another character special which didn't
use to be, and therefore creates a danger where there didn't use to be
any. It is however perfectly possible to introduce a single character
expand syntax without "activating" another character, since the space of
$ substitutions hasn't been fully claimed. The Tcl syntax rules say

[7] If a word contains a dollar-sign (``$'') then Tcl
performs variable substitution: the dollar-sign
and the following characters are replaced in the
word by the value of a variable.

but the rest of that rule only defines what happens when the $ is
followed by a a letter, digit, underscore, left brace, or left
parenthesis; nothing is said about what happens when the next char is
e.g. another $ or a left bracket. Currently Tcl won't substitute on such
a $, but I'm pretty sure Tcl programmers anyway escape the dollar in
those cases where there wouldn't be any substitution (writing [set a
\$$b] rather than [set a $$b]) because they feel "$ signs which aren't
to be substituted must be escaped."

Hence it shouldn't be a large conceptual difference if $$ and $[ were
defined to mean $ and [ respectively, but expanded. Then one could write
the four [eval] examples

eval lappend someList $moreItems
eval destroy [winfo children .]
eval button .b $stdargs -text \$mytext -bd \$border
eval exec \$prog $opts1 [getMoreopts] \$file1 \$file2

as

lappend someList $$moreItems
destroy $[winfo children .]
button .b $$stdargs -text $mytext -bd $border
exec $prog $$opts1 $[getMoreopts] $file1 $file2

Lars Hellström

Andreas Leitgeb

unread,
Mar 17, 2003, 7:35:33 AM3/17/03
to
While in prior discussions about argument-expansion the usefulness
of the change was principially frowned upon by most, I'm very
glad now to see, that it is generally accepted.

Discussion has now shifted from "why do we need it?" to
"how shall we make it?"

Since not everyone reads every posting, I'll post a small
summary about what I've gathered in the thread:

Suggestions:
- {}$varname and {}[command]
(my "original" proposal from years ago.)
pro: theoretically 100% compatible
con: looks ugly ("like a hack")
con: {}$ can easily be a typo, caused by a forgotten blank.
- `$varname and `[command]
(Donal K. Fellows' first followup to this thread)
pro: virtually no compatibility-issues, as backtick
almost never appears as first char of a word.
pro: looks better than {}$...
con: some classify the backtick as a pairwise-operator (see next)
- `$varname` and `[command]` and even more: `whatever ...`
(Donal K. Fellows' considered alternative in a second followup)
pro: <like above.>
con: some classify the backtick as a non-pairwise-operator (see prev.)
- $$varname and $[command]
pro: currently rarely used. (hardly any compatibility-issues)
pro: prefix-operator with no pairing/enclosing associations.
con: $$ implies double-dereference (as in perl).
- @$varname and @[command]
(as suggested in TIP 103 limited to the argument of expand)
con: @ appears too often in existing tcl-scripts.
(not an issue for non-global syntax, as with TIP 103)
pro: ... no pairing/enclosing associations.

- big change having real lists automatically spread themselves,
con: this is a really fundamental change. It's no longer Tcl then :-/


The apostrophe is used as a singleton quite often in english language,
isn't it ?-)
I'm not sure about singleton backtick-occurrences, but I for myself
would classify apostrophe and backtick equally, thus considering
the backtick also as a valid prefix-operator.
"Quoting" chars apostrophe('), backtick(`) and double-quote("), that
are paired up with themselves are often the source of all kinds
of quoting-hell, especially if they can appear nested. With double-
quotes we're mostly used to it, but it's probably not the kind of
thing we'd newly introduce.

Donald Arseneau <as...@triumf.ca> wrote:
> If I were voting, I'd vote for the syntax change with ` prefix.

Me, too.

And I would also be happy with any of these solutions, whichever
gets implemented in the core, as long as at least one of them does.

Apropos voting: does there still exist the regular quick-poll ?

Shall we update TIP 103 to reflect the dismissal of the expand-command
and change of the prefix-char ?

Donal K. Fellows

unread,
Mar 17, 2003, 9:50:17 AM3/17/03
to
Joe English wrote:
> I prefer the leading backtick syntax, but not until Tcl 9.

I concur with that, though I'm not currently sure whether to go for a single
(leading) backtick or a double (enclosing) backtick proposal. Both would work
about equally well.

Donal.
--
"Understanding leads to tolerance, which in turn leads to acceptance. And from
there, it's just a quick hop to speeding in Ohio, chewing peyote, and
frottage in the woods with a family of moose. And I just want to claim my
part of the credit." -- bunnythor <bunn...@uswest.net>

Donal K. Fellows

unread,
Mar 17, 2003, 9:54:23 AM3/17/03
to
Andreas Leitgeb wrote:
> Even if TIP 103 (with @ as the special char only for expand) gets done
> instead of this, it would be a good thing to have list protect @-chars.

Two points.

Firstly, rest assured that [list] will protect any new special characters. I
think that's pretty much implicit in everyone's suggestions...

Secondly, leading @ has problems with cursor specifications in Tk, and widget
configuration is one of the areas where you might realistically expect this sort
of thing to be found.

Donal K. Fellows

unread,
Mar 17, 2003, 10:56:18 AM3/17/03
to
Jean-Claude Wippler wrote:
> There is also a lot of trouble ahead: the "list" command needs to
> return its value as a 1-element list of a list, to avoid nullifying
> its effect. The "foreach" command, and tons of other commands would
> need to be reviewed, to see how automatic expansion affects them.
>
> Despite the extreme impact of such a change, I think it's very much
> worth trying to think this through. Because the underlying issue
> still holds: aggregation is IMO not a datatype, as ints and floats and
> dates are. By introducing aggregation as separate concept (which it
> already is, but only for unicode chars), a *lot* of software
> complexity might just go away.

No, you've just turned everything on its head in effect; i.e. you'd need to add
something like [list] whenever you wanted to stop expansion. I'm not really
convinced that'd be much better, and it would definitely be deeply incompatible
with a large fraction of existing scripts.

Programmers need control of when aggregation and deaggregation happen, and how
they happen. I don't want to take that away; I just want it to be easier (i.e.
shorter and sweeter) to write down.

se...@fishpool.com

unread,
Mar 17, 2003, 10:45:02 AM3/17/03
to
Donal K. Fellows <donal.k...@man.ac.uk> wrote:
> Joe English wrote:
>> I prefer the leading backtick syntax, but not until Tcl 9.
>
> I concur with that, though I'm not currently sure whether to go for a single
> (leading) backtick or a double (enclosing) backtick proposal. Both would work
> about equally well.

Are thre no other chars we could use to do that? Somehow I find the
backtick, at least a single one, not quite as natural as say "<" or "^".
Unfortunately [expr] has stolen far too many nice chars.

I think backticks should probably come in pairs. Define it so that
if they contain many words, they are all expanded.

--
/ http://www.fishpool.com/~setok/

Tom Wilkason

unread,
Mar 17, 2003, 1:12:20 PM3/17/03
to
<se...@fishpool.com> wrote in message
news:24mda.154$143....@reader1.news.jippii.net...

> Donal K. Fellows <donal.k...@man.ac.uk> wrote:
> > Joe English wrote:
> >> I prefer the leading backtick syntax, but not until Tcl 9.
> Are thre no other chars we could use to do that? Somehow I find the
> backtick, at least a single one, not quite as natural as say "<" or "^".
> Unfortunately [expr] has stolen far too many nice chars.
>
> I think backticks should probably come in pairs. Define it so that
> if they contain many words, they are all expanded.
>
> --
> / http://www.fishpool.com/~setok/

As an alternative to `, I can't think of a reason why the | char won't work,
it is used in expressions and exec/open commands for different things
already.

lappend thelist |$manyItems $lastitem

Tom Wilkason


Don Porter

unread,
Mar 18, 2003, 12:44:30 AM3/18/03
to
Andreas Leitgeb wrote:
> - {}$varname and {}[command]
> (my "original" proposal from years ago.)
> pro: theoretically 100% compatible
> con: looks ugly ("like a hack")
> con: {}$ can easily be a typo, caused by a forgotten blank.

This has been my favorite, because "100% compatible" means "can
be accomplished in a Tcl 8.X release".

However...

After some recent forays into Tcl's parse and substitution routines,
I must report that the accurate description must be "100% script compatible"
and that's not the only compatibility to be concerned about.

Without going into all the details right now (it's late) trust me
that this cannot be accomplished without introducing incompatibilities
in Tcl's public C API (notably the Tcl_Parse* routines), so if we
observe strict compatibility constraints (which we haven't always), even
this cannot be done for an 8.X release.

So, I'm pulling away from {}$ and its cousin {expand}[]. Instead, I'm
resigning to not having this as a real syntax change until Tcl 9. Given
that, the current argument about the right syntax among a freer set
of choices is the right one to have.

Once we *know* what Tcl 9 will have, then I think TIP 103's proposal
to have a command for Tcl 8 that simulates it will be helpful for
bridging the gap.

--
| Don Porter Mathematical and Computational Sciences Division |
| donald...@nist.gov Information Technology Laboratory |
| http://math.nist.gov/~DPorter/ NIST |
|______________________________________________________________________|

Don Porter

unread,
Mar 18, 2003, 12:48:29 AM3/18/03
to
Donal K. Fellows wrote:
> I concur with that, though I'm not currently sure whether to go for a single
> (leading) backtick or a double (enclosing) backtick proposal. Both would work
> about equally well.

I don't think so.

Paired characters are good for delimiting things. Here is the beginning;
here is the end.

But this whole thread is not about new ways to delimit things. It is
about indicating a boolean choice: expand or do not expand a word
into multiple words.

Whatever character (sequence) is chosen to be that indicator, it should
be a single, leading one. No pairs.

If the backtick makes people uncomfortable without using it as a pair,
that indicates the backtick is a poor choice for the expansion indicator.

Donal K. Fellows

unread,
Mar 18, 2003, 4:34:57 AM3/18/03
to
Andreas Leitgeb wrote:
> - @$varname and @[command]
> (as suggested in TIP 103 limited to the argument of expand)
> con: @ appears too often in existing tcl-scripts.
> (not an issue for non-global syntax, as with TIP 103)
> pro: ... no pairing/enclosing associations.

The problem is that one of the non-global syntax places is in the syntax of Tk
cursor specifications. And an area that it is expected that will make heavy use
of expansion will be Tk's widget configuration. This means that we have an
*expected* conflict, even if @ remains just local syntax...

> - big change having real lists automatically spread themselves,
> con: this is a really fundamental change. It's no longer Tcl then :-/

It makes my head ache trying to figure out the consequences. Not a good sign...

> The apostrophe is used as a singleton quite often in english language,
> isn't it ?-)

Yes, but that's also heavily used in Tcler's Wiki syntax. :^)

> "Quoting" chars apostrophe('), backtick(`) and double-quote("), that
> are paired up with themselves are often the source of all kinds
> of quoting-hell, especially if they can appear nested. With double-
> quotes we're mostly used to it, but it's probably not the kind of
> thing we'd newly introduce.

That's why I'm currently preferring backtick as a singleton. Singleton marks
are probably the easiest solution, especially if we don't permit nesting of
singletons.

> Shall we update TIP 103 to reflect the dismissal of the expand-command
> and change of the prefix-char ?

Maybe, maybe not. Syntactic updates will, frankly, not have a chance of getting
into the core except at a major version change. Or at least I can say for sure
that *I* will vote against them. And any modification of 103 will keep the fact
that it is an expansion command that asks to go in at a minor version, which is
fine for a new command, but not an update to the Endecalogue. I believe that
the other TCT members will agree with me on this.

A better solution would be a new TIP that obsoletes 103.

Donal.
--
"Everyone else is envisioning a future of 32-bit deep bump-mapped lizard-skin
scrollbars that adapt to the lusers mood by imperceptibly alpha-blending
through a range of chromatic combinations not repeating in 3^18 years. Needs
hardware acceleration, anaglyptic glasses, and 2 aspirin." -- Phil Ehrens

Donal K. Fellows

unread,
Mar 18, 2003, 4:43:24 AM3/18/03
to
Lars Hellström wrote:
[...]

> but the rest of that rule only defines what happens when the $ is
> followed by a a letter, digit, underscore, left brace, or left
> parenthesis; nothing is said about what happens when the next char is
> e.g. another $ or a left bracket. Currently Tcl won't substitute on such
> a $, but I'm pretty sure Tcl programmers anyway escape the dollar in
> those cases where there wouldn't be any substitution (writing [set a
> \$$b] rather than [set a $$b]) because they feel "$ signs which aren't
> to be substituted must be escaped."

My only objection to this is that there are a substantial body of people coming
to Tcl who think that $$ does double dereferencing. OK, neither of us are
proposing that it should do, but I think that the current behaviour (the first $
is literal) is slightly less surprising, and as the people who have this
misconception are newcomers to the language, we shouldn't frighten them off.

The main thing I like about the backtick proposal is that I can also define what
happens if a backtick precedes some other kind of word (i.e. a "bareword", a
double-quoted word, or a brace-quoted word) whereas these have fundamental
differences in the $ proposal (and must continue to do so, for backward
compatability reasons.) This makes the rules for $ more complex and harder to
learn, whereas ` has a single rule whose effects can be stated simply.

Andreas Leitgeb

unread,
Mar 18, 2003, 7:31:43 AM3/18/03
to
Tom Wilkason <tom.wi...@cox.net> wrote:
> <se...@fishpool.com> wrote in message

>> Are thre no other chars we could use to do that? Somehow I find the
>> backtick, at least a single one, not quite as natural as say "<" or "^".
>> Unfortunately [expr] has stolen far too many nice chars.
expr is not that much of an issue, imho:
You're advised and expected to place the whole expression
into braces anyway, so any, even leading "^" wouldn't be affected.
Those, who write [expr 0xff ^ $x] are and always were on the
dangerous side.
The ^ is instead "killed" by regexp/regsub, as ^ is quite common at
the start of a regular expression.

> As an alternative to `, I can't think of a reason why the | char won't work,
> it is used in expressions and exec/open commands for different things
> already.

"exec" and "open" are much more of a "magic-char killer":
exec whatever </dev/null >/dev/null &
open |whatever r

I always write the arguments to open in doublequotes, as this looks
better for my eyes, but I cannot claim this to be general practise.
These (relatively common) examples kind of lock out <,>,| and & as
candidates for the new magic character, in order to open as few
compatibility issues as possible.

PS: As of now, the single backtick is still my favourite...
Just like the apostrophe, it's not too bound to be used pairwise.

PPS: it's not that we'd lock out every character, that can appear at
word-start, but only those that have a higher tendency to do so.

Hemang Lavana

unread,
Mar 18, 2003, 8:18:18 AM3/18/03
to
Don Porter wrote:

> Whatever character (sequence) is chosen to be that indicator, it should
> be a single, leading one. No pairs.
>
> If the backtick makes people uncomfortable without using it as a pair,
> that indicates the backtick is a poor choice for the expansion indicator.

How about ~ (tilda) character? It can give the impression of expansion,
but it may be currently used as leading char for filenames (on unix) and
also in expr. I don't know how often it occurs in the scripts though.

Hemang.

Volker Hetzer

unread,
Mar 18, 2003, 8:34:04 AM3/18/03