eval is evil (for spreading list-arguments)

50 views
Skip to first unread message

Andreas Leitgeb

unread,
Mar 7, 2003, 8:38:19 AM3/7/03
to
Yes, it's a frequently recurring topic.

Just now, reading the docs for MetaKit, I stumbled over the
following example:
puts [eval format {%-20s %d} [mk::get db.phonebook!2 name date]]

See the mistake ?
At first glance, or just because I indicated that there is one ?


The format-specifier also gets torn apart by eval.
It would require doubled brace-quoting for use with eval.

--
Nichts ist schlimmer als zu wissen, wie das Unheil sich entwickelt,
und voll Ohnmacht zusehn muessen - das macht mich voellig krank...
-- (Musical Elisabeth; "Die Schatten werden laenger")

Ralf Fassel

unread,
Mar 7, 2003, 9:09:38 AM3/7/03
to
* Andreas Leitgeb <Andreas...@siemens.at>

| The format-specifier also gets torn apart by eval.
| It would require doubled brace-quoting for use with eval.

But is `eval' to blame for that? Or rather the programmer who does
not do his homework?

R'

Jeffrey Hobbs

unread,
Mar 7, 2003, 11:19:41 AM3/7/03
to
Andreas Leitgeb wrote:
> Yes, it's a frequently recurring topic.
>
> Just now, reading the docs for MetaKit, I stumbled over the
> following example:
> puts [eval format {%-20s %d} [mk::get db.phonebook!2 name date]]
>
> See the mistake ?
> At first glance, or just because I indicated that there is one ?

Well, I saw the mistake immediately, which is why I always write my
evals now pedantically, like so:

eval [list format {%-20s %d}] [mk::get get db.phonebook!2 name date]

Anything that should not be broken apart should be list'ed.

--
Jeff Hobbs The Tcl Guy
Senior Developer http://www.ActiveState.com/
Tcl Support and Productivity Solutions

Andreas Leitgeb

unread,
Mar 7, 2003, 12:03:18 PM3/7/03
to

Are pitfalls to blame, or just those people who fall into them ?
(And it shows, that it's not only newbies who fall, but also
long-term tcl-seniors, who just happen to have been less than
hard-concentrating while using "eval")

A few years ago, a proposal has been made to alleviate this particular
pitfall, by adding some new syntax to the Tcl-core (backwards-compatibly).
Not only would that offer an obvious way *around* this pitfall
(compared to the "eval-way" leading *straight over* it, expecting you
to jump over it), it would also offer object-safety in many cases where
eval still does not. Also, using it adds less visual "line-noise"
than [list ...]-wrapping.

There have been long threads about it, in the past. From time to time,
if some sighting of someone falling into that pitfall triggers me, I
cast another poll here to see, if time is finally mature for improving
tcl-syntax.

PS: there are also safe and useful uses for "eval".
Expansion of lists, however, is a dangerous one.

Ralf Fassel

unread,
Mar 7, 2003, 2:43:56 PM3/7/03
to
* Andreas Leitgeb <Andreas...@siemens.at>

| Are pitfalls to blame, or just those people who fall into them ?

If you fall once, blame the pitfall, and remember the pit. If you
fall often into the same pit, well... ;-)

| A few years ago, a proposal has been made to alleviate this
| particular pitfall, by adding some new syntax to the Tcl-core
| (backwards-compatibly).

Would be fine with me. Just make it obvious enough so that someone
who is used to the current behaviour does not easily fall into *that*
pit. If I have to add extra arguments or simply use a new name, that
should be enough to trigger my attention to not double-list the
`new-eval' arguments, for example.

| Also, using it adds less visual "line-noise" than [list
| ...]-wrapping.

Well, in this particular example,
eval format {{%d foo}} [some_other_func]
is not too much line-noise IMHO. But of course, there are other cases
where you have to use an extra `list'.

| PS: there are also safe and useful uses for "eval".
| Expansion of lists, however, is a dangerous one.

Expanding lists is the primary use of `eval' for me :-/
Why would I use `eval' otherwise?

R'

Andreas Leitgeb

unread,
Mar 7, 2003, 4:00:49 PM3/7/03
to
Ralf Fassel <ral...@gmx.de> wrote:
> * Andreas Leitgeb <Andreas...@siemens.at>
>| Are pitfalls to blame, or just those people who fall into them ?
> If you fall once, blame the pitfall, and remember the pit. If you
> fall often into the same pit, well... ;-)
Actually it's not so much about me falling myself, but more about me
watching many others (and surely not only newbies) fall in.
Doing Tcl for almost 10 years now, I'm kind of used to it, too.
I still find eval-lines ugly, however.

>| A few years ago, a proposal has been made to alleviate this
>| particular pitfall,

> should be enough to trigger my attention to not double-list the
> `new-eval' arguments, for example.

You seem to be relatively new to this group :-)

Based on the observation that with eval, programmers *really often*
protect too little rather than too much, the proposal was primarily
about "doing something" with the args that should be *expanded*, rather
than protecting all that is to be left as is.

Then, as it is not allowed in current tcl to have trailing characters
after a close-brace, compatibility is guaranteed, if the special
behaviour is triggered by something like {}$ or {}[...].
(Other suggestions were to use e.g. &varname as the "expanding
version of $varname", but that would have caused lots of
migration-headaches) Thus, the first proposal was to add
new syntax {}[command_returning_a_list] and {}$list_variable
to expand the values to multiple arguments.
Various commands that take an arbitrary number of arguments could
then be directly passed a list, spread out into several separate
arguments.
Further discussions showed a general preference to something less
noisy than three special-chars in a row (which seemed too perl'ish),
which led to the still pending proposal of {expand}$varname and
{expand}[command...]

instead of
eval [list command $all [the] $non-expanding [arguments]] $list
one would write:
command $all [the] $non-expanding [arguments] {expand}$list
and gain the following advantages:

a) the "real" command is back at the start of the line, rather than
obscured from sight by eval&list
b) in many cases conversion to and from string can be avoided
(keyword: object-safety). Meanwhile the "eval" has evolved
to avoid this conversion, too, but special care must be taken
to not accidentally thwart it. (That's why the old example uses
[list ...] rather than curlies.)
c) it's much more *un*likely to make a mistake by specifying
{expand} too often, than to list-protect to little.

Most often, in practise you'll find above example written as:
eval command $all [the] $non-expanding [arguments] $list
under the weak assumption, that each of the other vars or commands
evaluate to "nice" values immune to eval's reparsing. Such mistakes
happen easily, and are Tcl's weak point just like wild pointers
are C-programs' #1-cause for random failure.

Recently, TIP #103 has gone some way in the right direction,
but failed to provide advantage "a)" (see above).

>| PS: there are also safe and useful uses for "eval".
>| Expansion of lists, however, is a dangerous one.
> Expanding lists is the primary use of `eval' for me :-/
> Why would I use `eval' otherwise?

For interpreting real scripts. Tk-widget bindings make frequent
use of eval. Outside these uses, I cannot currently think of
any other, but surely there exist more.
It is used for argument expansion, because, as of now, no other
way exists to do argument expansion.

se...@fishpool.com

unread,
Mar 7, 2003, 2:40:45 PM3/7/03
to
Andreas Leitgeb <Andreas...@siemens.at> wrote:
>
> A few years ago, a proposal has been made to alleviate this particular
> pitfall, by adding some new syntax to the Tcl-core (backwards-compatibly).
> Not only would that offer an obvious way *around* this pitfall
> (compared to the "eval-way" leading *straight over* it, expecting you
> to jump over it), it would also offer object-safety in many cases where
> eval still does not. Also, using it adds less visual "line-noise"
> than [list ...]-wrapping.

Who was it who suggested the following form for expand?

expand <list>

I think the idea was that this works partly like eval, but no variable
subtitution or anything is done for <list>. Ie. one level is dropped
off and that's it. So the following would always work as espected:

set foo {puts blah}
expand $foo

Seemed to take care of most issues.

--
/ http://www.fishpool.com/~setok/

Andreas Leitgeb

unread,
Mar 7, 2003, 4:32:28 PM3/7/03
to
se...@fishpool.com <se...@fishpool.com> wrote:
> Ie. one level is dropped off and that's it.
> Seemed to take care of most issues.
Nope, it didn't.

While argument-expansion really is not much more than "dropping
off a level of quoting", the important difference is the scope.

To be really useful, the flattening must only happen to those
arguments, for which it is explicitly "requested", rather than
to all that aren't explicitly protected.

Dan Smart

unread,
Mar 7, 2003, 6:47:25 PM3/7/03
to
Andreas Leitgeb <Andreas...@siemens.at> wrote in
news:slrnb6hk1s.ddm....@pcc524.gud.siemens.at:
> There have been long threads about it, in the past. From time to
> time, if some sighting of someone falling into that pitfall triggers
> me, I cast another poll here to see, if time is finally mature for
> improving tcl-syntax.
<pedant>
Actually you are looking for a significant change in TCL's *semantics*.
</pedant>
And the proposed syntax (constrained as it is by backwards compatibility)
is an ugly wart, not an "improvement".

>
> PS: there are also safe and useful uses for "eval".
> Expansion of lists, however, is a dangerous one.

All uses of eval require care, even with a perfectly well formed list, if
the first word is 'exec', you may still not be safe.

Dan "eval [list [file delete [info script]]]" Smart
--
Dan Smart. C++ Programming and Mentoring.
cpp...@dansmart.com
ADDvantaged

Donald Arseneau

unread,
Mar 7, 2003, 8:43:45 PM3/7/03
to
se...@fishpool.com writes:

> Andreas Leitgeb <Andreas...@siemens.at> wrote:
> >
> > A few years ago, a proposal has been made to alleviate this particular
> > pitfall, by adding some new syntax to the Tcl-core (backwards-compatibly).
> > Not only would that offer an obvious way *around* this pitfall
> > (compared to the "eval-way" leading *straight over* it, expecting you
> > to jump over it), it would also offer object-safety in many cases where
> > eval still does not. Also, using it adds less visual "line-noise"
> > than [list ...]-wrapping.
>
> Who was it who suggested the following form for expand?
>
> expand <list>

There has been much discussion (use google/groups), and a minimalist
solution proposed by Peter Spjuth and Donal K. Fellows:
TIP #103: Argument Expansion Command,
http://www.tcl.tk/cgi-bin/tct/tip/103.html

Please have a look at how that would affect the present pitfall, and
provide some feedback for the Tip. I think it would be written

puts [expand {format {%-20s %d} @[mk::get db.phonebook!2 name date]} ]

to work successfully.


Donald Arseneau as...@triumf.ca

Suchenwi

unread,
Mar 9, 2003, 4:31:53 PM3/9/03
to
Andreas Leitgeb wrote:
>PS: there are also safe and useful uses for "eval".
> Expansion of lists, however, is a dangerous one.

Well, but before we have extended the Tcl syntax with an "expand" construct,
it's the only half-ways readable way...
Consider the great idiom:
eval pack [winfo children $w]

This can only be done with explicit eval. I think one avoids most pitfalls by
remembering that [eval] just like [join] removes one level of grouping, so just
add one level with [list ...] where necessary.

Best regards, Richard Suchenwirth

se...@fishpool.com

unread,
Mar 9, 2003, 9:45:40 PM3/9/03
to
Andreas Leitgeb <Andreas...@siemens.at> wrote:
> se...@fishpool.com <se...@fishpool.com> wrote:
>> Ie. one level is dropped off and that's it.
>> Seemed to take care of most issues.
> Nope, it didn't.
>
> While argument-expansion really is not much more than "dropping
> off a level of quoting", the important difference is the scope.
>
> To be really useful, the flattening must only happen to those
> arguments, for which it is explicitly "requested", rather than
> to all that aren't explicitly protected.

And how does the explained not do that? Only the first list level is
opened up. No other expansion was done in that model.

expand [list button .b] $options

The above should do what we expect in all situations. The $options list
is opened up once as arguments, but nothing inside it is ever expanded.
Surely it's a matter of taste whether it's better to explicitly mark
areas for expansion or for protection? The above system avoids having
new magic characters.

--
/ http://www.fishpool.com/~setok/

se...@fishpool.com

unread,
Mar 9, 2003, 9:49:24 PM3/9/03
to
Donald Arseneau <as...@triumf.ca> wrote:
>>
>> Who was it who suggested the following form for expand?
>>
>> expand <list>
>
> There has been much discussion (use google/groups), and a minimalist
> solution proposed by Peter Spjuth and Donal K. Fellows:
> TIP #103: Argument Expansion Command,
> http://www.tcl.tk/cgi-bin/tct/tip/103.html

Yes, I've seen the TIP and followed the discussion ;-)


>
> Please have a look at how that would affect the present pitfall, and
> provide some feedback for the Tip. I think it would be written
>
> puts [expand {format {%-20s %d} @[mk::get db.phonebook!2 name date]} ]

With the other expand proposal that would work:

puts [expand [list format {%-20s %d}] [mk::get db.phonebook!2 name date]]

Not quite sure what you wanted commented in the TIP. I could add the above
suggestion there, but I'd prefer if the original inventor did that so I
don't get it wrong. I just can't remember who was talking about it on
IRC.

--
/ http://www.fishpool.com/~setok/

Andreas Leitgeb

unread,
Mar 10, 2003, 4:50:55 AM3/10/03
to
se...@fishpool.com <se...@fishpool.com> wrote:

> Andreas Leitgeb <Andreas...@siemens.at> wrote:
>> To be really useful, the flattening must only happen to those
>> arguments, for which it is explicitly "requested", rather than
>> to all that aren't explicitly protected.
This paragraph was intended to answer your questions below.
Was it really that unintellegible ?

The misinterpretations might stem from my programmer's view to
it, as opposed to the Tcl-interpreter's view to it:

For Tcl, the first argument to eval in
% eval [list destroy] [winfo children .]
is just plain the result of the list-command. Nothing special.
One level of *everything* is dropped, so what finally gets to
work is the original "destroy" and each of the returned child-widget
names separately.

For a programmer, otoh, this [list ...] is a protection-wrapper
around the string "destroy", to make eval see it as a list.
For the programmer, this [list ...] construct is not part of
the program's logic. It's some magic salt that he has learnt
to add, to make things work.

In the programmer's mind there is the command "destroy", and
the command that returns the list of .'s childs. To tie these
together, he now has to add one list-level to all those parts
that already were at the "correct" quoting-level before, and then
let eval remove one level of *all* the arguments, merely undoing
the "protection" for the "destroy".

For the programmer it would be like "avoiding a detour" to be able
to express: "I want only the widget-list spreaded, the rest untouched."

This part of the whole story fortunately got addressed with
TIP 103 as in: expand {destroy @[winfo children .]}

The other part, namely getting the "destroy" command to
the start of the line, is something, TIP 103 does not
address.

As it seems, TIP 103 is at least the best we can get to for the time.

> No other expansion was done in that model.
> expand [list button .b] $options

This would be practically equivalent to the "eval"-approach.
The pitfall would be indeed a tiny bit narrower, but still large enough
for the example in the first posting of this thread to still fall in.
Btw., that example was not that of a newbie, but taken from the
documentation of "MK4tcl", the Tcl-bindings to metakit.

> Surely it's a matter of taste whether it's better to explicitly mark
> areas for expansion or for protection?

Not a matter of taste, but a matter of narrowing a pitfall,
... of making Tcl programmer-friendlier.
And that's the whole point of this thread, by the way.

> The above system avoids having new magic characters.

Braces were "magic" even before :-)
Otoh, the "@" as used in TIP 103 wasn't magic in Tcl before.
The association of "@" with "list" is entirely perl'ish ;-)

Andreas Leitgeb

unread,
Mar 10, 2003, 5:37:49 AM3/10/03
to
Donald Arseneau <as...@triumf.ca> wrote:
> There has been much discussion (use google/groups), and a minimalist
> solution proposed by Peter Spjuth and Donal K. Fellows:
> TIP #103: Argument Expansion Command,
> http://www.tcl.tk/cgi-bin/tct/tip/103.html
>
> puts [expand {format {%-20s %d} @[mk::get db.phonebook!2 name date]} ]
It's *much* better than the current status quo.

> Please have a look at how that would affect the present pitfall,

Yes, it would definitely fix it.

> and provide some feedback for the Tip.

I'd even prefer {}$var instead of @$var inside the braced argument
of expand, for it's "*forward*-compatibility" (see TIP), but with
"@" it is still much better than nothing.

Any estimates about when this TIP will undergo the judgement of
the TCT ?

Bryan Oakley

unread,
Mar 10, 2003, 9:09:24 AM3/10/03
to
Andreas Leitgeb wrote:
> Donald Arseneau <as...@triumf.ca> wrote:
>
>>There has been much discussion (use google/groups), and a minimalist
>>solution proposed by Peter Spjuth and Donal K. Fellows:
>>TIP #103: Argument Expansion Command,
>> http://www.tcl.tk/cgi-bin/tct/tip/103.html
>>
>>puts [expand {format {%-20s %d} @[mk::get db.phonebook!2 name date]} ]
>
> It's *much* better than the current status quo.
>
>
>>Please have a look at how that would affect the present pitfall,
>
> Yes, it would definitely fix it.

How would it fix it? It would still be available for people who choose
to use it. The proposal simply provides an alternative to the "pitfall";
it doesn't remove or fix the pitfall.

Donal K. Fellows

unread,
Mar 10, 2003, 9:16:06 AM3/10/03
to
Andreas Leitgeb wrote:
> There have been long threads about it, in the past. From time to time,
> if some sighting of someone falling into that pitfall triggers me, I
> cast another poll here to see, if time is finally mature for improving
> tcl-syntax.

Having thought about this long and hard (including looking through the possible
alternatives) my favoured alternative is a single extra rule added to Tcl(n):

If the first character of a word is back-quote (```'') then the rest of
the word (as determined by the other rules, including the double-quote
and open brace rules) is interpreted as a list and each of the words is
appended to the list of words to be passed to the command as a separate
word. No further expansion is performed on those words, and nor is the
back-quote character part of any word.

The idea is that this allows for expansion to be performed exactly where the
programmer wants, and nowhere else, as well as putting the emphasis clearly on
what is being executed for real instead of the machinery to make everything be
expanded as we want. It's also much prettier than the {} hack. In the current
example of the hour, this would allow things to be written as:

format {%-20s %d} `[mk::get get db.phonebook!2 name date]

The choice of back-quote? This is enforced by the fact that most other
characters are far more common in Tcl scripts (e.g. @ is used for a number of
things throughout Tk, as well as being commonplace in places like email
addresses...)

Bet someone's going to hate this idea though. C'est la vie.

> PS: there are also safe and useful uses for "eval".
> Expansion of lists, however, is a dangerous one.

Minor correction: It's safe, but only if you're seriously paranoid.

Donal.
--
Donal K. Fellows http://www.cs.man.ac.uk/~fellowsd/ donal....@man.ac.uk
"If RedHat are so purblind that they think computing is about ever fancier
'desktop themes', they are in the interior design business, and as everyone
knows, if you can piss you can paint." -- Steve Blinkhorn

Andreas Leitgeb

unread,
Mar 10, 2003, 9:41:16 AM3/10/03
to
Bryan Oakley <br...@bitmover.com> wrote:
>> Yes, it would definitely fix it.
> How would it fix it? It would still be available for people who choose
> to use it. The proposal simply provides an alternative to the "pitfall";
> it doesn't remove or fix the pitfall.

You're right. This requires overloading of the verb "fix" as:

to fix a pitfall :=
to offer an obvious alternative way around the pitfall.

;-)

Roy Terry

unread,
Mar 10, 2003, 9:58:20 AM3/10/03
to
"Donal K. Fellows" wrote:

> If the first character of a word is back-quote (```'') then the rest of
> the word (as determined by the other rules, including the double-quote
> and open brace rules) is interpreted as a list and each of the words is
> appended to the list of words to be passed to the command as a separate
> word. No further expansion is performed on those words, and nor is the
> back-quote character part of any word.
>

The following ought to be a high priority for
any programming language that aspires to ease of use
and certainly Tcl qualifies:


> The idea is that this allows for expansion to be performed exactly where the
> programmer wants, and nowhere else,

> format {%-20s %d} `[mk::get get db.phonebook!2 name date]


>
> The choice of back-quote? This is enforced by the fact that most other
> characters are far more common in Tcl scripts (e.g. @ is used for a number of
> things throughout Tk, as well as being commonplace in places like email
> addresses...)
>

Hhmm. At first I didn't like this but on second look it gets better
and better. Grepping 100K lines of nearby Tcl code turns up only a one
backquotes and not at word beginning. It's definitely way better than
adding a new command, which is merely a crutch and a bandaid,
and it really assists writing concise code.

IOW: A big win.

Roy
(Cheers and waves from sidelines ;*)

se...@fishpool.com

unread,
Mar 10, 2003, 10:56:21 AM3/10/03
to
Andreas Leitgeb <Andreas...@siemens.at> wrote:

> For Tcl, the first argument to eval in
> % eval [list destroy] [winfo children .]
> is just plain the result of the list-command. Nothing special.
> One level of *everything* is dropped, so what finally gets to
> work is the original "destroy" and each of the returned child-widget
> names separately.

Just in case it wasn't clear, with the proposed [expand] you would
not need a [list] around the destroy there. I don't find it particularly
confusing to have the idea "remove one list level and execute command".
I do, however, accept your reasoning for the reverse. I just have a natural
fear of adding new special characters (even though they are only
command-specific) -- can end up in nasty mistakes when you're not expecting
it: a variable starts with @ and the programmer doesn't know it.

--
/ http://www.fishpool.com/~setok/

lvi...@yahoo.com

unread,
Mar 10, 2003, 12:38:41 PM3/10/03
to

According to Donal K. Fellows <donal.k...@man.ac.uk>:
:Having thought about this long and hard (including looking through the possible

:alternatives) my favoured alternative is a single extra rule added to Tcl(n):

Hmm - along with adding this rule, some code has to be added to the interpreter
to deal with the back quote, right?

--
Join us at the Tenth Annual Tcl/Tk Conference <URL: http://mini.net/tcl/6274 >
Even if explicitly stated to the contrary, nothing in this posting
should be construed as representing my employer's opinions.
<URL: mailto:lvi...@yahoo.com > <URL: http://www.purl.org/NET/lvirden/ >

Bryan Oakley

unread,
Mar 10, 2003, 12:58:07 PM3/10/03
to
Donal K. Fellows wrote:
> Andreas Leitgeb wrote:
>
>>There have been long threads about it, in the past. From time to time,
>>if some sighting of someone falling into that pitfall triggers me, I
>>cast another poll here to see, if time is finally mature for improving
>>tcl-syntax.
>
>
> Having thought about this long and hard (including looking through the possible
> alternatives) my favoured alternative is a single extra rule added to Tcl(n):
>
> If the first character of a word is back-quote (```'') then the rest of
> the word (as determined by the other rules, including the double-quote
> and open brace rules) is interpreted as a list and each of the words is
> appended to the list of words to be passed to the command as a separate
> word. No further expansion is performed on those words, and nor is the
> back-quote character part of any word.

Is there a reason you chose a single leading character rather than a
pair of characters, such as a pair of single quotes, or pair of
backquotes (eg: command foo '$bar' baz) There's precidence for either ($
is a single char, "", {} and [] are used in pairs).

Tom Wilkason

unread,
Mar 10, 2003, 2:05:49 PM3/10/03
to
"Donal K. Fellows" <donal.k...@man.ac.uk> wrote in message
news:3E6C9E26...@man.ac.uk...
I like the approach of specifying lists that should be passed in as multiple
arguments rather than the other way around (i.e. specifying the ones that
shouldn't be).

Has anyone looked into the core to see if something like this a relatively
easy solution to implement? When I first thought about this problem I
thought a good way to handle it would be something like:

format {%-20s %d} [unlist [mk::get get db.phonebook!2 name date]]

But from an implementation point of view I don't think it is feasable since
all function calls are structured to return a single object.

Another consideration, what should be the result of:
set A `[mk::get get db.phonebook!2 name date]
Should it throw a syntax error such as "set a 1 2 3" would?

Tom Wilkason


Suchenwi

unread,
Mar 10, 2003, 4:00:36 PM3/10/03
to
Tom Wilkason wrote:

>Another consideration, what should be the result of:
>set A `[mk::get get db.phonebook!2 name date]
>Should it throw a syntax error such as "set a 1 2 3" would?

Yes, I strongly think so - substitution is a separate step from command
invocation, so it should react (with the same error message) as
eval [list set A] [mk::get get db.phonebook!2 name date]
does now.
--
Best regards, Richard Suchenwirth

Donal K. Fellows

unread,
Mar 11, 2003, 8:06:00 AM3/11/03
to
lvi...@yahoo.com wrote:
> Hmm - along with adding this rule, some code has to be added to the
> interpreter to deal with the back quote, right?

No, the english specification in Tcl(n) just magically manifests itself as
implementation. :^D

-- Thanks, but I only sleep with sentient lifeforms. Anything else is merely
a less sanitary form of masturbation.
-- Alistair J. R. Young <avatar...@arkane.demon.co.uk>

Donal K. Fellows

unread,
Mar 11, 2003, 8:10:06 AM3/11/03
to
Bryan Oakley wrote:
> Is there a reason you chose a single leading character rather than a
> pair of characters, such as a pair of single quotes, or pair of
> backquotes (eg: command foo '$bar' baz) There's precidence for either ($
> is a single char, "", {} and [] are used in pairs).

Personal preference and the fact that I think of ` as flagging the word for
different treatment (as opposed to `` containing different treatment of a word.)
In fact, if you have something that "brackets" things, then you have to state
what happens if you have more than one word inside.

*thinks*

I suppose you could make `` act like double quotes except with expand-to-words
behaviour. That'd work...

Peter.DeRijk

unread,
Mar 11, 2003, 9:52:42 AM3/11/03
to
Jeffrey Hobbs <Je...@activestate.com> wrote:
> Andreas Leitgeb wrote:
>> Yes, it's a frequently recurring topic.
>>
>> Just now, reading the docs for MetaKit, I stumbled over the
>> following example:
>> puts [eval format {%-20s %d} [mk::get db.phonebook!2 name date]]
>>
>> See the mistake ?
>> At first glance, or just because I indicated that there is one ?

> Well, I saw the mistake immediately, which is why I always write my
> evals now pedantically, like so:

> eval [list format {%-20s %d}] [mk::get get db.phonebook!2 name date]

> Anything that should not be broken apart should be list'ed.

Is there a reason for not doing it this way:
eval {format {%-20s %d}} [mk::get get db.phonebook!2 name date]
This has less noise, and also seems to work nicely to protect i
variables from expanding, eg.
set format {%-20s %d}
eval {format $format} [mk::get get db.phonebook!2 name date]

--
Peter De Rijk der...@uia.ua.ac.be
<a href="http://rrna.uia.ac.be/~peter/">Peter</a>
To achieve the impossible, one must think the absurd.
to look where everyone else has looked, but to see what no one else has seen.

Jeffrey Hobbs

unread,
Mar 11, 2003, 11:56:31 AM3/11/03
to
Peter.DeRijk wrote:

> Jeffrey Hobbs wrote:
>>Well, I saw the mistake immediately, which is why I always write my
>>evals now pedantically, like so:
>> eval [list format {%-20s %d}] [mk::get get db.phonebook!2 name date]
>>Anything that should not be broken apart should be list'ed.

> Is there a reason for not doing it this way:
> eval {format {%-20s %d}} [mk::get get db.phonebook!2 name date]
> This has less noise, and also seems to work nicely to protect i

The solution with lists is more pedantically correct, but just using {}s
works just as well (that's Brent Welch's preferred method, BTW). The
small added advantage with [list] is that special internal optimizations
will make it a bit faster in some cases than using {}s.

Andreas Leitgeb

unread,
Mar 13, 2003, 6:08:25 AM3/13/03
to
Donal K. Fellows <donal.k...@man.ac.uk> wrote:
> I suppose you could make `` act like double quotes except with expand-to-words
> behaviour. That'd work...

You mean something like the following ?
command $arg `$arglist`

There are some issues & caveats that need to be dealt with:
`"$arglist"` ? (does not make too much sense, anyway, does it ?)
`"$arglist`" ? (should be a syntax-error. Likely the only
bad thing is the trailing non-whitespace
after the closing backtick, as doublequotes
wouldn't have a special meaning inside)
`[command `$arglist`]` ? (unquoted nesting via [], as with "'s)
Under which limitations is object-safety possible ?
(I'd guess, if no concatenation of values happens between the
backticks, but then, would `$arg ` thwart objects ?)

One thing not to forget (it occurred to me while in an email-discussion
with setok): if ` becomes a special character, then [list] will have to
protect it. (it currently doesn't)
Even if TIP 103 (with @ as the special char only for expand) gets done
instead of this, it would be a good thing to have list protect @-chars.

I'd really love to see this latest suggestion added to Tcl.
With backticks it looks cleaner and more consistent (Tcl'ish)
than my previous {}$ and {}[-approaches.

Cameron Laird

unread,
Mar 13, 2003, 9:01:49 AM3/13/03
to
In article <slrnb6i1v5.ddm....@pcc524.gud.siemens.at>,
Andreas Leitgeb <a...@logic.at> wrote:
.
.

.
>> Expanding lists is the primary use of `eval' for me :-/
>> Why would I use `eval' otherwise?
>For interpreting real scripts. Tk-widget bindings make frequent
> use of eval. Outside these uses, I cannot currently think of
> any other, but surely there exist more.
>It is used for argument expansion, because, as of now, no other
> way exists to do argument expansion.
.
.
.
Yes.

I just want to add my own emphasis to this accurate description.
Many, many Tcl applications do NOT need [eval], or only need it
for argument expansion, as we've been discussing. This deserves
repetition, as, during Tcl's early history, it was more widely
thought that "interpretation of synthesized scripts" was a good
and necessary Tcl style. This turns out to have been largely a
"dead end".

So, yes, it's for "interpreting real scripts". If we understand
'real' in sufficient generality, that's it's only proper use;
there are no others.

Tangential point: eval is formally redundant; it's only an abbre-
viation for [uplevel 0 ...]
--

Cameron Laird <Cam...@Lairds.com>
Business: http://www.Phaseit.net
Personal: http://phaseit.net/claird/home.html

lvi...@yahoo.com

unread,
Mar 13, 2003, 10:55:33 AM3/13/03
to

According to Cameron Laird <cla...@phaseit.net>:
:Many, many Tcl applications do NOT need [eval], or only need it

:for argument expansion, as we've been discussing.

Cameron - is the need for argument expansion some artifact of Tcl?
If not, then what do other languages do about the issue?

Volker Hetzer

unread,
Mar 13, 2003, 11:12:39 AM3/13/03
to

<lvi...@yahoo.com> wrote in message news:b4q9ll$8ne$2...@srv38.cas.org...

>
> According to Cameron Laird <cla...@phaseit.net>:
> :Many, many Tcl applications do NOT need [eval], or only need it
> :for argument expansion, as we've been discussing.
>
> Cameron - is the need for argument expansion some artifact of Tcl?
No, the need is always there.

> If not, then what do other languages do about the issue?

They demand everything expanded into complex data structures.

Greetings!
Volker


Darren New

unread,
Mar 13, 2003, 1:40:48 PM3/13/03
to
lvi...@yahoo.com wrote:
> is the need for argument expansion some artifact of Tcl?
> If not, then what do other languages do about the issue?

Some languages distinguish argument expansion from evaluation. So you'd
pass a list of arguments and it would be assigned to the variables, but
not via evaluation. Formally, this is known as the "apply" operation.


Joe English

unread,
Mar 13, 2003, 1:37:13 PM3/13/03
to
lvirden wrote:
>According to Cameron Laird:

>:Many, many Tcl applications do NOT need [eval], or only need it
>:for argument expansion, as we've been discussing.
>
>Cameron - is the need for argument expansion some artifact of Tcl?
>If not, then what do other languages do about the issue?

The need for argument expansion can arise for any command
that takes a variable number of arguments instead of a
single list-valued argument. Some of these could have
been avoided by using a better-designed interface (e.g.,
[file join]), but not all.

Many (most?) other languages don't even support variadic procedures.
Those that do often also provide either an "apply" procedure
(Scheme, Lisp), or an alternate fixed-argument entry point
for all variadic procedures (e.g., sprintf/vsprintf in C).

Another common use of [eval] is to expand command _prefixes_:
[eval $callback [list $arg1 $arg2 $arg3]], where $callback
is a user-supplied script with a command name and optional
initial arguments.

Other languages handle this use case with closures (Lisp-family FPLs),
curried functions (ML/Haskell-family FPLs), or idiomatically
(e.g., callback functions that take an additional 'void
*ClientData' argument in C).

--Joe English

jeng...@flightlab.com

Donald Arseneau

unread,
Mar 13, 2003, 4:53:44 PM3/13/03
to
lvi...@yahoo.com writes:

> According to Cameron Laird <cla...@phaseit.net>:
> :Many, many Tcl applications do NOT need [eval], or only need it
> :for argument expansion, as we've been discussing.
>
> Cameron - is the need for argument expansion some artifact of Tcl?
> If not, then what do other languages do about the issue?

The worlds cleanest programming language uses \expandafter ):->

Donald Arseneau as...@triumf.ca


lvi...@yahoo.com

unread,
Mar 14, 2003, 7:03:24 AM3/14/03
to

According to Joe English <jeng...@flightlab.com>:

:lvirden wrote:
:>According to Cameron Laird:
:>:Many, many Tcl applications do NOT need [eval], or only need it
:>:for argument expansion, as we've been discussing.
:>
:>Cameron - is the need for argument expansion some artifact of Tcl?
:>If not, then what do other languages do about the issue?
:
:The need for argument expansion can arise for any command
:that takes a variable number of arguments

:Many (most?) other languages don't even support variadic procedures.

The reason I am trying to couch this branch of the thread in terms of
other languages is to see if some other language's solution might fit
Tcl. It may be that it is not possible. I'm hoping that it is.

ksh and perl, for instance, allow one to create functions which take variable
number of arguments.

:Those that do often also provide either an "apply" procedure


:(Scheme, Lisp), or an alternate fixed-argument entry point
:for all variadic procedures (e.g., sprintf/vsprintf in C).

I don't know how either perl or ksh solve the problem - perl probably has
the apply type procedure.

:Other languages handle this use case with closures (Lisp-family FPLs),


:curried functions (ML/Haskell-family FPLs), or idiomatically
:(e.g., callback functions that take an additional 'void
:*ClientData' argument in C).

So, of the at least 5 solutions you mention Joe, are any of these ones
that might be a fit for Tcl?

Andreas Leitgeb

unread,
Mar 14, 2003, 9:36:46 AM3/14/03
to
lvi...@yahoo.com <lvi...@yahoo.com> wrote:
> The reason I am trying to couch this branch of the thread in terms of
> other languages is to see if some other language's solution might fit
> Tcl. It may be that it is not possible. I'm hoping that it is.
The reasoning is good, but I fear, Tcl is too unique in this regard
to allow for a solution of another language to be applied to it.

> ksh
ksh has the proposed backtick-semantics automatical. You need to
double-quote variable- or command-substitutions to avoid having
the substituted values be torn apart.
x="1 2 3"; argcount() { echo $#; }
argcount $x -> 3
argcount "$x" -> 1

> and perl,
perl flattens all lists (references to lists are not lists)
@x=(1,2,3)
print 0,@x,4,5 -> print sees only one flat list, namely (0,1,2,3,4,5)

> So, of the at least 5 solutions you mention Joe, are any of these ones
> that might be a fit for Tcl?

The 6th one, suggested by Donal K. Fellows :-)
% set x {1 2 3}
% argcount `$x` -> 3
% argcount $x -> 1

Bryan Oakley

unread,
Mar 14, 2003, 10:56:07 AM3/14/03
to
Andreas Leitgeb wrote:

> lvi...@yahoo.com <lvi...@yahoo.com> wrote:
>
>>So, of the at least 5 solutions you mention Joe, are any of these ones
>>that might be a fit for Tcl?
>
>
> The 6th one, suggested by Donal K. Fellows :-)
> % set x {1 2 3}
> % argcount `$x` -> 3
> % argcount $x -> 1
>

Didn't Donal suggest a single leading backquote (eg: `$x)? I asked why
he chose a single leading quote rather than a pair. Personally, I'm
partial to the pair, but could live with either.

I like this approach _much_ better than the {}$x solution proposed a
year or two ago. I feel silly saying that, though. Really, why should it
matter that `$x` is "prettier" than {}$x? And yet, to me it does matter.
In my head I dismissed the {}$x solution as a hack, yet consider `$x or
`$x` to be good solutions. Go figure.

Perhaps it has to do with the fact I perceive {}$foo as an exception to
the normal {} processing rules, whereas `$foo` is a new, simple(ish)
rule. I like the fact that, generally speaking, Tcl has no exceptions to
it's rules.

bo...@aol.com

unread,
Mar 14, 2003, 2:58:16 PM3/14/03
to
You did a very good job of pointing up the difficulties of using a
*pair* of backticks. To me it makes better sense to use just one and
consider it as a modifier to $ and [ substitution. There is already a
precedent for this since we do $<var-name> not $<var-name>$. In a
context where substitution is done, `$ calls for the list elements to
be substituted and $ substitutes the value. This is a minimal
solution and does only what is needed and nothing more.

Using a pair of backticks does more than is needed (not the 'tcl way')
and introduces unwanted behaviors more rules to be aware of. Making
the result look like some other language should be a non-goal. In
this case the pair of backquotes looks like a ksh feature, but does
something entirely different (the worst kind of similarity!).

bob

Andreas Leitgeb <Andreas...@siemens.at> wrote in message news:<slrnb70pf3.ddm....@pcc524.gud.siemens.at>...

Roy Terry

unread,
Mar 14, 2003, 3:47:13 PM3/14/03
to
Yes the single back tick has some appeal. It also is virtually
guaranteed to break some existing scripts. The {}$ {}[ solution was
specifically proposed because it used a previously illegal construct.

Frankly, it is a puzzle to me that we all seem to be lining up behind
an incompatible change as opposed to a basically compatible change
(unless
you consider *not* causing an error a *basic* imcompatibility)

As to making the backquotes come in a pair, I think that opens a
huge can of worms re escaping embedded backquotes and throws open
the meaning of "" and {} inside backquotes. We absolutely don't want
to go there! Anyway, the idea of expansion is not to introduce a new
quoting mechanism but to mark conventional words for the parser to
expand. For that purpose the single leading ` is both easer to use and
easier to implement and explain I think.

I hope the TCT will have the forward-looking courage to let this
change in and solve a fundamental weakness in the Tcl language.
( Either {} or ` would do and the sooner the better)

Cheers,
Roy

PS: I don't have any (known) args with leading ` so the incompability
doesn't touch my code. How about others?

John E. Perry

unread,
Mar 14, 2003, 5:15:23 PM3/14/03
to

"Roy Terry" <royt...@earthlink.net> wrote in message
news:3E724019...@earthlink.net...

> Bryan Oakley wrote:
> >
> > Andreas Leitgeb wrote:
> > > lvi...@yahoo.com <lvi...@yahoo.com> wrote:
> > >
> > >>...etc wrote

> >
> > Didn't Donal suggest a single leading backquote (eg: `$x)? I asked why
> > he chose a single leading quote rather than a pair. Personally, I'm
> > partial to the pair, but could live with either.
> >

Am I the only one reading this list who feels _very_ uncomfortable with
unmatched quote marks -- front, back, or whatever? Aren't there some
characters available that don't almost _mandate_ matching co-characters?

John Perry
AS&M


Donald Arseneau

unread,
Mar 14, 2003, 6:42:13 PM3/14/03
to
Bryan Oakley <br...@bitmover.com> writes:

> he chose a single leading quote rather than a pair. Personally, I'm
> partial to the pair, but could live with either.

I think a great strength of Tcl is its use of matching { } for
verbatim quoting (where "shell" uses ' ') and [ ] for execution
(like ` `). Tcl does have " " for weak quoting though. I
wouldn't want to add another case of pairing identical
characters wih ` ` for expansion. Moreover, in the argument
expansion proposal, the argument is clearly delimited already
by the variable name or matched [ ]. A lone ` prefix is
preferable.

As for the "backwards compatible" {} prefix, that isn't really
backwards compatible for the reason stated: removing an error
condition *is* an incompatible change. This falls into the
common trap for language extenders -- fitting new features into
previously illegal syntax. Doing that corrupts the regularity
of the language; and regularity is the prime ingrediant for
ease of use (for all but the most expert). Also, a good language
should have a sparse syntax, so errors can be recognized as such,
and not masquerade as some unintended command. A {} prefix is
the epitome of this mistake by making an easy typgraphic error
({} [foo] typed as {}[foo]) into legal syntax.

If I were voting, I'd vote for the syntax change with ` prefix.

Donald Arseneau as...@triumf.ca

Bryan Oakley

unread,
Mar 14, 2003, 9:23:29 PM3/14/03
to
Roy Terry wrote:

> Frankly, it is a puzzle to me that we all seem to be lining up behind
> an incompatible change as opposed to a basically compatible change
> (unless
> you consider *not* causing an error a *basic* imcompatibility)

Weird, isn't it? I agree with your observation -- {}$foo provides less
(well, no) backwards compatibility issues, yet I still personally would
go for the backtick.

Sometimes beauty wins over truth :-)

> ...


>
> I hope the TCT will have the forward-looking courage to let this
> change in and solve a fundamental weakness in the Tcl language.

I wouldn't go so far as to say this is a fundamental weakness. A mild
inconvenience perhaps. This feature is mostly just syntactic sugar to
make certain constructs a bit more palatable. Still, it's very nice sugar.

Jean-Claude Wippler

unread,
Mar 15, 2003, 10:04:35 AM3/15/03
to
"Donal K. Fellows" <donal.k...@man.ac.uk> wrote ...

> If the first character of a word is back-quote (```'') then the rest of
> the word (as determined by the other rules, including the double-quote
> and open brace rules) is interpreted as a list and each of the words is
> appended to the list of words to be passed to the command as a separate
> word. No further expansion is performed on those words, and nor is the
> back-quote character part of any word.
>
> The idea is that this allows for expansion to be performed exactly where the
> programmer wants, and nowhere else, as well as putting the emphasis clearly on
> what is being executed for real instead of the machinery to make everything be
> expanded as we want. [...]

I admit that {}blah, and especially `blah both look appealing.

But I wonder more and more if it's the best way forward - long term...

(Warning: very rough & raw ponderings ahead...)

In Tcl, "everything is a string" (EIAS). Types are introduced by
context-of-use, and cached. With all the tricky consequences of
shimmering, and all the benefits of dual object representations. Both
shimmering and caching are about performance, conceptually everything
is *still* a string.

Aggregation in Tcl is treated in the same way, whether something
consists of N things or not is a matter of using list operators such
as llength, lindex, and lset.

There is a good reason for this: by sticking to the EIAS mantra, we
get the ability to display/store/transmit aggregate data in the same
uniform way as everything else. It's why one could argue that Tcl
also follows the "everything is an object" model.

But it comes at a price. I dare state that Tcl's weakest side is the
way it deals with non-trivial data structures, starting with lists,
but also hashes/maps/arrays/associaitions, all the way up to nested
combinations thereof. Because of the lack of even the distiction
between values and aggregation of values, we *always* have to apply
list commands to lists. Hence all the code with zillions of lindex,
lappend, lsearch, etc commands.

In theory, EIAS is a brilliant simplification. In practice, I find
EIAS one step too much. There are far, *far*, *FAR* more cases where
one treats data as a scalar (string, int, float, handle, whatever)
*or* as an aggregate. Sure, it's extremely useful to have lossless
mappings between those two categories, but that's of the same order as
having a way to display all values in text form. Great for data
interchange - i.e. UI's as well as for transport and storage

The whole EIAS concept has some arbitrariness to it btw - a string is,
after all, an aggregate of characters. So we already have a built-in
concept of aggregation. In fact it was altered in a very fundamental
way not so long ago, when Unicode was introduced. Now, strings have a
length (number of chars) as well as a distinct representation (UTF-8),
which in turn has a "bytelength" (an octet count).

So in a way, Tcl also follows the "everything is an aggregate" model.
After all, you can use the "string length" command on anything. And
there are explicit conversions between strings and lists, in the form
of [split ... {}] and [join ... { }].

Back to ticks, eh, I mean backticks...

In today's Tcl model, since aggregation or arbitrary elements is not
supported as fundamental operation, when a function returns something,
the caller has no way to find out whether the returned data is a
scalar or an aggregate. We need to decide at the *caller* location
what to do with the result and how to treat it.

Hence the proposal to introduce a new convenient notation such as:
foo 1 2 `[bar] 5 6
With bar returning data perhaps using this:
return [list 3 4]

But what if Tcl 9 were to break with this tradition and introduce a
distinct non-shimmering aspect of data, being its dimension?

It would make the backtick issue go away, allowing:
foo 1 2 [bar] 5 6
The reasoning being that a return value is either a scalar (no
dimension) or a list (with a length).

There are some biiiig issues with such a change. For one, eval would
not be needed much (allowing much better bytecode compilation, I
expect). The "interp alias" trick to introduce args ("currying") can
be replaced by list:
set lsi [list lsearch -integer $mylist]
... [$lsi $key] ...
If the "namespaces are ensembles" TIP is adopted, this would make the
"::" namespace separator redundant ("a::b::c" becomes [list a b c]).

There is also a lot of trouble ahead: the "list" command needs to
return its value as a 1-element list of a list, to avoid nullifying
its effect. The "foreach" command, and tons of other commands would
need to be reviewed, to see how automatic expansion affects them.

Despite the extreme impact of such a change, I think it's very much
worth trying to think this through. Because the underlying issue
still holds: aggregation is IMO not a datatype, as ints and floats and
dates are. By introducing aggregation as separate concept (which it
already is, but only for unicode chars), a *lot* of software
complexity might just go away.

Maybe Tcl can evolve to "everything is an aggregate", of chars *and*
of other things? It might just turn out to unify length, index,
append, range enough to make the distinction between strings and lists
go away. Arrays (i.e. hashes/maps) can easily be built on top, which
means this would open the door to arrays-as-first-class values. And
quite a bit more, but I feel that this ramble is already over the
top... :)

-jcw

PS. None of the above need affect the core concepts of Tcl IMO, such
as copy-on-write and the ability to treat everything as a string for
display, storage, and transport.

PPS. One could consider introducing "cardinality" even, i.e. dealing
with 0..N dimensions, which would open the door to making Tcl a
data-manipulation powerhouse. But that's of less importance, since it
can be simulated with uni-dimensional aggregation.

Joe English

unread,
Mar 15, 2003, 12:47:06 PM3/15/03
to
Roy Terry wrote:

>Yes the single back tick has some appeal. It also is virtually
>guaranteed to break some existing scripts. The {}$ {}[ solution was
>specifically proposed because it used a previously illegal construct.
>
>Frankly, it is a puzzle to me that we all seem to be lining up behind
>an incompatible change as opposed to a basically compatible change

My main objection to {}$ is aesthetic. Specifically,
I think the principle of extending a language's syntax
by using currently-illegal constructs is a Really Bad Idea.
Witness C++ :-)

I prefer the leading backtick syntax, but not until Tcl 9.


--Joe English

jeng...@flightlab.com

Derk Gwen

unread,
Mar 16, 2003, 7:19:36 AM3/16/03
to
# Despite the extreme impact of such a change, I think it's very much
# worth trying to think this through. Because the underlying issue

Alternatively, define a script language which can be loaded with
Tcl and which can Tcl procs and is callable from Tcl procs.

--
Derk Gwen http://derkgwen.250free.com/html/index.html
But I do believe in this.

Lars Hellström

unread,
Mar 17, 2003, 5:37:12 AM3/17/03