Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

Exception to the rule of "if"

142 views
Skip to first unread message

Alexandru

unread,
Jul 11, 2018, 10:10:25 PM7/11/18
to
Hi,

If I take the rule from the "if" manpage:

"The if command evaluates expr1 as an expression (in the same way that [expr] evaluates its argument) [...]"

Then I can do this:

if {3 > 2} {puts 1}

But what if I have put the boolean operator ">" into a variable?

set op ">"

if {3 $op 2} {puts 1}

returns an error:
missing operator at _@_
in expression "(3 _@_$op 2)"

If instead I do:

if {[expr 3 $op 2]} {puts 1}

it works like a charm.

So, it looks like the manual is not completly correct. There is an exception to the rule. Are there any other exceptions there? How can I safely use this rule. Is it safer to always use "expr" in "if", just to be sure that all potential exceptions are covered?

Regards
Alexandru

Rich

unread,
Jul 11, 2018, 10:24:00 PM7/11/18
to
Alexandru <alexandr...@meshparts.de> wrote:
> Hi,
>
> If I take the rule from the "if" manpage:
>
> "The if command evaluates expr1 as an expression (in the same way that [expr] evaluates its argument) [...]"
>
> Then I can do this:
>
> if {3 > 2} {puts 1}
>
> But what if I have put the boolean operator ">" into a variable?
>
> set op ">"
>
> if {3 $op 2} {puts 1}
>
> returns an error:
> missing operator at _@_
> in expression "(3 _@_$op 2)"
>
> If instead I do:
>
> if {[expr 3 $op 2]} {puts 1}
>
> it works like a charm.
>
> So, it looks like the manual is not completly correct.

No, the two statements are not semantically identical. You did not
brace the argument to 'expr' in your second command. So without the
braces, the $op is expanded as part of preparing to call the 'expr'
command by the command substitution []. So when 'expr' finally runs,
it sees three operands, 3, >, and 2, and proceeds to process the
comparison.

In the first, expr receives one operand, the string "2 $op 3" (without
$op being expanded). And, from the looks of the result, expr's
internal parser for strings is not programmed to perform variable
expansion on items that should be operators, only on items that should
be operands.

Note that you get the same result without the 'if', just from plain
expr, if you brace expr's operand so it gets the unaltered string:


$ rlwrap tclsh
% set op >
>
% expr {3 $op 2}
missing operator at _@_
in expression "3 _@_$op 2"
%

> There is an exception to the rule.

No, it is not, because the two were not identical.

> Are there any other exceptions there?

Doubtful.

> How can I safely use this rule.

There is no new rule, and 'safely' would mean gaining a fuller
understanding of how Tcl expands operands, and when Tcl expands
operands.

> Is it safer to always use "expr" in "if", just to be sure that all
> potential exceptions are covered?

The second operand to 'if' is an expression, but you were comparing
apples to oranges so your comparison was flawed.

Alexandru

unread,
Jul 11, 2018, 10:52:50 PM7/11/18
to
Thanks for the clarification. I guess I'll never be so deep in understanding how Tcl handles those special cases. For me, the interesting point here is that "expr" does variable substitutions only on operands but not on operators. Would that be an improvement, to also perform substitutions on the operators?

Gerald Lester

unread,
Jul 11, 2018, 11:35:56 PM7/11/18
to
On 07/11/2018 09:52 PM, Alexandru wrote:
> Am Donnerstag, 12. Juli 2018 04:24:00 UTC+2 schrieb Rich:
>> Alexandru <alexandr...@meshparts.de> wrote:
>>> ...
> Thanks for the clarification. I guess I'll never be so deep in understanding how Tcl handles those
special cases. ...

There are no "special cases"! You are refusing to believe the simple
truth of the Endekalogue specifies *ALL* of the syntax rules of Tcl
(NOTE -- [expr] and regular expressions are mini-languages)

--
+----------------------------------------------------------------------+
| Gerald W. Lester, President, KNG Consulting LLC |
| Email: Gerald...@kng-consulting.net |
+----------------------------------------------------------------------+

Brad Lanam

unread,
Jul 12, 2018, 1:08:44 AM7/12/18
to
% set op >
>
% expr "3 $op 2"
1
%

In your braced expression, $op is never expanded, it is a literal string.

Brad Lanam

unread,
Jul 12, 2018, 1:16:00 AM7/12/18
to
And...

% if {[expr {3 $op 2}]} { puts "yes" }
missing operator at _@_
in expression "3 _@_$op 2"
% if {[expr 3 $op 2]} { puts "yes" }
yes

Alexandru

unread,
Jul 12, 2018, 1:46:04 AM7/12/18
to
Yes... I get it now... No exception, just the normal rules.
I'm just saying, that it's very easy for me to program errors like this. I don't know how other people see this.
Besides, I was wondering if it would be an improvement for "expr" to also perform substitutions on the operators.

Rich

unread,
Jul 12, 2018, 7:04:52 AM7/12/18
to
Alexandru <alexandr...@meshparts.de> wrote:
> Besides, I was wondering if it would be an improvement for "expr" to
> also perform substitutions on the operators.

There is really no need, since for those, likely very few, times you
want operators in variables, you have the tools necessary already
available in Tcl:

$ rlwrap tclsh
% set op >
>
% expr [subst -nocommands -nobackslashes {3 $op 2}]
1
%

Alexandru

unread,
Jul 12, 2018, 11:31:42 AM7/12/18
to
I wasn't aware of the existance of the command. Very usefull. Thanks!

Mike Griffiths

unread,
Jul 12, 2018, 5:03:14 PM7/12/18
to
You're comparing
if {3 $op 2}
with
expr "3 $op 2"

If you used if "3 $op 2" instead, you'd see the same behaviour as with your quoted (not braced) expression.

I think part of the reason that this expansion isn't done is that, in the vast majority of cases it isn't desirable, and it would prevent the expression from being bytecode-compiled, which would dramatically slow it down.

briang

unread,
Jul 13, 2018, 10:29:02 PM7/13/18
to
> Yes... I get it now... No exception, just the normal rules.
> I'm just saying, that it's very easy for me to program errors like this. I don't know how other people see this.
> Besides, I was wondering if it would be an improvement for "expr" to also perform substitutions on the operators.

This is a very interesting statement and question. Having spent >20 years programming in Tcl, writing 100's of thousands of lines of code, and having to maintain it, I have long ago learned that it never pays to get tricky or cute with coding. Writing stuff like [expr {3 $op 4}] is not useful, it's just a bug waiting to happen. So no, it would be to the detriment of Tcl to allow substations like this.

Code should always be clear and simple. If it's not, a future you will not be able to understand what the present you wrote, and will curse you six ways to Sunday for writing such a mess. (been there, done that.)

-Brian

Alexandru

unread,
Jul 14, 2018, 7:42:38 AM7/14/18
to
I know what you mean. I also like to code things clearly and without complications. Here is the procedure, where I use $op. What do think? How would you write it?

## Get first element in list matching the expression giving by the operator "op" with respect to the referrence value "ref".
# \param l list of values or sublists of values
# \param op operator as in Tcl (e.g. >, <, == etc.)
# \param ref referrence value to compare list elements to
# \param index index for sublists
# \return list index of first matching list element
proc ListFirst {l op ref {index ""}} {
set idx 0
foreach e $l {
if {$index!=""} {
set e [lindex $e $index]
}
# Don't remove "expr" from "if"! See:
# https://groups.google.com/forum/#!topic/comp.lang.tcl/jbMtF4heTIs
if {[expr ($e $op $ref)]} {
return $idx
}
incr idx
}
return -1
}

Gerald Lester

unread,
Jul 14, 2018, 9:25:06 AM7/14/18
to
Have "index" default to 0 and always to the "set e [lindex $e $index]"
-- note time it both ways, I think always doing it will win out.

You have one of the "special cases" where you really, really, really
want to substitute in the operator -- so either do what you did or do:
if "\$e $op \$ref" {
again, time it and see if there is a timing difference.

Robert Heller

unread,
Jul 14, 2018, 10:19:14 AM7/14/18
to
At Sat, 14 Jul 2018 04:42:35 -0700 (PDT) Alexandru <alexandr...@meshparts.de> wrote:

>
> Am Samstag, 14. Juli 2018 04:29:02 UTC+2 schrieb briang:
> > > Yes... I get it now... No exception, just the normal rules.
> > > I'm just saying, that it's very easy for me to program errors like this=
> . I don't know how other people see this.=20
> > > Besides, I was wondering if it would be an improvement for "expr" to al=
> so perform substitutions on the operators.
> >=20
> > This is a very interesting statement and question. Having spent >20 year=
> s programming in Tcl, writing 100's of thousands of lines of code, and havi=
> ng to maintain it, I have long ago learned that it never pays to get tricky=
> or cute with coding. Writing stuff like [expr {3 $op 4}] is not useful, i=
> t's just a bug waiting to happen. So no, it would be to the detriment of T=
> cl to allow substations like this.
> >=20
> > Code should always be clear and simple. If it's not, a future you will n=
> ot be able to understand what the present you wrote, and will curse you six=
> ways to Sunday for writing such a mess. (been there, done that.)
> >=20
> > -Brian
>
> I know what you mean. I also like to code things clearly and without compli=
> cations. Here is the procedure, where I use $op. What do think? How would y=
> ou write it?
>
> ## Get first element in list matching the expression giving by the operator=
> "op" with respect to the referrence value "ref".
> # \param l list of values or sublists of values
> # \param op operator as in Tcl (e.g. >, <, =3D=3D etc.)
> # \param ref referrence value to compare list elements to
> # \param index index for sublists
> # \return list index of first matching list element
> proc ListFirst {l op ref {index ""}} {
> set idx 0
> foreach e $l {
> if {$index!=3D""} {
> set e [lindex $e $index]
> }
> # Don't remove "expr" from "if"! See:
> # https://groups.google.com/forum/#!topic/comp.lang.tcl/jbMtF4heTIs
> if {[expr ($e $op $ref)]} {
> return $idx
> }
> incr idx
> }
> return -1
> }

This is something of a variation of lsort, where there -increasing (>),
-decreasing (<), and -command ("etc.").

Rather than limiting it to a simple scalar compare, I would follow lsort's
lead and just allow a random predicate:

proc ListFirst {l predicate ref} {
set idx 0
foreach e $l {
set flag [uplevel #0 [list $predicate $e $ref]]
if {$flag} {
return $idx
}
incr idx
}
return -1
}

This allows predicate to be anything and allows the elements of l to be
anything (including sub-lists (of *arbitary* depth), or other things, like
keys into an array, SNIT objects, canvas graphic element indexes, etc.) and
the predicate can do whatever test you need, beyond the *limited* comparison
ops available in expr. The predicate can handle whatever structure you need to
handle. And you don't need index, since you would embed its functionallity
into the predicate. This *simplfies* the function and makes it more general
purpose.

>

--
Robert Heller -- 978-544-6933
Deepwoods Software -- Custom Software Services
http://www.deepsoft.com/ -- Linux Administration Services
hel...@deepsoft.com -- Webhosting Services

Robert Heller

unread,
Jul 14, 2018, 11:31:56 AM7/14/18
to
Special cases are often bad news, sooner or later, you will have another
special case, eventually leading to a mess of special cases. See my other
post, which gets rid of all the special cases and results in a general purpose
function that handles all cases.

briang

unread,
Jul 14, 2018, 12:22:05 PM7/14/18
to
I agree with Robert's solution. I would have suggested something very similar.
This does move any "questionable" code to $predicate. However, it also becomes easy to evaluate the predicate on it's own merits, bringing sanity back to the equation.

-Brian

Alexandru

unread,
Jul 14, 2018, 12:48:47 PM7/14/18
to
So how would I call your procedure now? What forms should have "predicate"? Somehow I can't link this to "lsort"...

briang

unread,
Jul 14, 2018, 1:12:46 PM7/14/18
to
(* note subtle tweak to Robert's function *)

proc ListFirst {l predicate ref} {
set idx 0
foreach e $l {
set flag [uplevel #0 [list {*}$predicate $e $ref]]
if {$flag} {
return $idx
}
incr idx
}
return -1
}

set lst {12 4 15 301 21}

set answer [ListFirst $lst {apply {{a b} {expr {$a > $b}}}} 100]
puts "apply 100 -> $answer"

proc sqrtOf {a b} {
expr {($a * $a) == $b}
}

set answer [ListFirst $lst sqrtOf 16]
puts "sqrtOf 16 -> $answer"

Alexandru

unread,
Jul 14, 2018, 2:51:14 PM7/14/18
to
Thanks for the example. Now I get it.
I have timed the old an new procedure with the result, that the new one is 4x slower than the old one.

Robert Heller

unread,
Jul 14, 2018, 3:28:04 PM7/14/18
to
predicate is a function with two arguments, the element of l and ref:

proc lessthan {e r} {
return [expr {$e < $r}]
}

ListFirst {2 3 4 1 42 7 0} lessthan 1

returns 6

ListFirst {2 3 4 1 42 7 0} lessthan 2

returns 3

proc greaterthan {e r} {
return [expr {$e > $r}]
}

ListFirst {2 3 4 1 42 7 0} greaterthan 40

returns 4

proc equals {e r} {
return [expr {$e == $r}]
}

ListFirst {2 3 4 1 42 7 0} equals 42

returns 4

package require snit
snit::type complex {
variable real 0.0
variable imaginary 0.0
constructor {{r 0.0} {i 0.0} args} {
set real $r
set imaginary $i
}
method Real {} {return $real}
method Imaginary {} {return $imaginary}
method Real= {r} {set real $r}
method Imaginary= {i} {set imaginary $i}
typemethod validate {o} {
if {[catch {$o info type} thetype]} {
error "Not a $type"
} elseif {$thetype ne $type} {
error "Not a $type"
}
}
proc isComplexP {o} {
if {[catch {complex validate $o}]} {
return false
} else {
return true
}
}
typemethod == {a b} {
if {[isComplexP $a] && [isComplexP $b]} {
return [expr {[$a Real] == [$b Real] &&
[$a Imaginary] == [$b Imaginary]}]
} elseif {[isComplexP $a] && [string is double -strict $b]} {
return [$type == $a [$type $b 0.0]]
} elseif {[string is double -strict $a] && [isComplexP $b]} {
return [$type == [$type $a 0.0] $b]
} elseif {[string is double -strict $a] && [string is double -strict $b]} {
return [expr {$a == $b}]
} else {
error "Cannot compare non-numbers"
}
}
}

set list [list 3.14159 1.0 2.0 42 [complex 3.14159 1.0]]

proc checkComplex {e r} {
if {[catch {complex validate $e}]} {
return false
} else {
return true
}
}

ListFirst $list checkComplex {}

returns 4

ListFirst $list "complex ==" [complex 3.14159 1.0]

also returns 4

(Yes, these latter examples are probably beyond what you were thinking of, but
it all works...)

Robert Heller

unread,
Jul 14, 2018, 3:31:01 PM7/14/18
to
How were you coding the predicates?

Donal K. Fellows

unread,
Jul 14, 2018, 3:52:11 PM7/14/18
to
On 14/07/2018 12:42, Alexandru wrote:
> if {[expr ($e $op $ref)]} {

I'd make that be this instead:

if {[{*}$op $e $ref]} {

since then I'd be able to pass ::tcl::mathop::== or any other operation
or other command or even a command prefix (such as a partially applied
lambda term) and the whole thing is more powerful and less restrictive.
The ::tcl::mathop namespace contains command versions of all Tcl
expression operators (except for &&, || and ?: as they do short-circuit
evaluation).

Donal.
--
Donal Fellows — Tcl user, Tcl maintainer, TIP editor.

Christian Gollwitzer

unread,
Jul 14, 2018, 4:23:09 PM7/14/18
to
Am 14.07.18 um 15:25 schrieb Gerald Lester:
> On 07/14/2018 06:42 AM, Alexandru wrote:
>>      if {$index!=""} {
>>        set e [lindex $e $index]
>>      }

>
> Have "index" default to 0 and always to the "set e [lindex $e $index]"

That's not the same, I think you misunderstood what it does. If you set
index to "0", it'll operate on the first sublist and check these
elements. If you set it to "" it'll work on the whole list. There is,
however, a way to do this and more:

set e [lindex $e {*}$index]

This will now work even arbitrarily deep nested lists.

> You have one of the "special cases" where you really, really, really
> want to substitute in the operator -- so either do what you did or do:
>    if "\$e $op \$ref" {

maybe it's better to set this string to a variable, which can then be
bytecompiled into an expression?

Christian

Alexandru

unread,
Jul 15, 2018, 2:12:25 AM7/15/18
to
Am Samstag, 14. Juli 2018 22:23:09 UTC+2 schrieb Christian Gollwitzer:
> Am 14.07.18 um 15:25 schrieb Gerald Lester:
> > On 07/14/2018 06:42 AM, Alexandru wrote:
> >>      if {$index!=""} {
> >>        set e [lindex $e $index]
> >>      }
>
> >
> > Have "index" default to 0 and always to the "set e [lindex $e $index]"
>
> That's not the same, I think you misunderstood what it does. If you set
> index to "0", it'll operate on the first sublist and check these
> elements. If you set it to "" it'll work on the whole list. There is,
> however, a way to do this and more:
>
> set e [lindex $e {*}$index]

Good point! I have modified this line in my procedure according to your suggestion. Thanks.

>
> This will now work even arbitrarily deep nested lists.
>
> > You have one of the "special cases" where you really, really, really
> > want to substitute in the operator -- so either do what you did or do:
> >    if "\$e $op \$ref" {
>
> maybe it's better to set this string to a variable, which can then be
> bytecompiled into an expression?

I'm afraid I don't understand this point.

>
> Christian

Christian Gollwitzer

unread,
Jul 15, 2018, 3:22:38 AM7/15/18
to
Am 15.07.18 um 08:12 schrieb Alexandru:
This condition is evaluated over and over in the loop, right? But the
string is built every time afresh. Try:

set test "\$e $op \$ref"

..

foreach {
if $test {
}
}

Then, the first time "test" is used as an expression, it is
byte-compiled and the code is stored in the internal representation of
test. Next time, the code will be executed instead of being string-parsed.

Christian

Alexandru

unread,
Jul 15, 2018, 3:35:23 AM7/15/18
to
I get you now, but I dont't think this is gona work, since $test is different in each loop.

Christian Gollwitzer

unread,
Jul 15, 2018, 3:46:51 AM7/15/18
to
Am 15.07.18 um 09:35 schrieb Alexandru:
Try before you complain ;) Notice the \ in front of the $. e and ref are
different, but the expression needs not to be different. The trick is
exactly to build one single expression which works. And of course, no
braces to the if!

Christian

Alexandru

unread,
Jul 15, 2018, 3:54:46 AM7/15/18
to
Oh, missed that. Now it makes sense of course. Sorry for complaining to early.
The performace increase is also icredible: 3x faster!

proc ::meshparts::ListFirst {l op ref {index ""}} {
set idx 0
# Precompile the comparisson expression (procedure runs 3x faster!)
set test "\$e $op \$ref"
foreach e $l {
if {$index!=""} {
set e [lindex $e {*}$index]
}
if $test {

heinrichmartin

unread,
Jul 16, 2018, 4:01:08 AM7/16/18
to
On Saturday, July 14, 2018 at 4:19:14 PM UTC+2, Robert Heller wrote:
> set flag [uplevel #0 [list $predicate $e $ref]]

On Saturday, July 14, 2018 at 9:52:11 PM UTC+2, Donal K. Fellows wrote:
> if {[{*}$op $e $ref]} {

I was about to reply the same. Personally, I'd go for [uplevel 1] assuming that the predicate should be resolved in the callers context. Or use [namespace code] and friends.

Shouldn't ref be part of the predicate (or allow any number of args to the predicate)?

heinrichmartin

unread,
Jul 16, 2018, 8:31:36 AM7/16/18
to
On Monday, July 16, 2018 at 10:01:08 AM UTC+2, heinrichmartin wrote:
> Shouldn't ref be part of the predicate (or allow any number of args to the predicate)?

Btw, a coroutine is way faster than mathop in my timings; apply is halfway between mathop and coroutine.

::tcl::mathop::< 0 $v => 686.506 microseconds per iteration
apply {v {expr {0 < $v}}} $v => 456.086 microseconds per iteration
apply {{r v} {expr {$r < $v}}} 0 $v => 468.614 microseconds per iteration
::namespace inscope ::test {greaterThan 0} $v => 3141.482 microseconds per iteration
greaterThan0 $v => 370.73 microseconds per iteration

(and another use case is even faster: $v => 351.144 microseconds per iteration)

Code for reference and comments:
#!/usr/bin/env tclsh

proc lsearchP {l predicate} {
set idx 0
foreach e $l {
if {[{*}$predicate $e]} {
return $idx
}
incr idx
}
return -1
}

# time with many non-matches
variable misses 1000
variable L [lrepeat $misses 0]
lappend L 1

# predicate can be empty, if the list contains command names
# - a different use case!
proc 1 {} {return true}
proc 0 {} {return false}

# what if predicate is not in the context of lsearchP
namespace eval test {
proc greaterThan {r v} {
return [expr {$r < $v}]
}
}

# why not try coroutines
proc greaterThan {r} {
for {set v [yield]} {true} {
set v [yield [expr {$r < $v}]]
} {}
}
coroutine greaterThan0 greaterThan 0

# mathop
variable p1 [list ::tcl::mathop::< 0]
# lambda
variable p2 [list apply {v {expr {0 < $v}}}]
variable p3 [list apply {{r v} {expr {$r < $v}}} 0]
# keep context
variable p4 [namespace eval test {namespace code {greaterThan 0}}]
# coroutine
variable p5 greaterThan0
# list with command names
variable p6 [list]

proc doTest {} {
variable L
variable misses
while true {
incr p
variable p$p
if {![info exists p$p]} break
puts stderr [string trimleft "[set p$p] \$v => [expr {
$misses != [lsearchP $L [set p$p]]
? "Wrong result: expected $misses, but got [lsearchP $L [set p$p]]"
: [time {lsearchP $L [set p$p]} 1000]
}]"]
}
}

doTest

Alexandru

unread,
Jul 16, 2018, 8:41:53 AM7/16/18
to
weird, but I get

time {::tcl::mathop::< 0 $v} 1000
0.218 microseconds per iteration

which is stellar faster than your measurement.

And:
time {apply {v {expr {0 < $v}}} $v} 1000
0.565 microseconds per iteration

I'm not sure I can use your timings as a reference due to the exremly large differences. Do you have another process running in parallel?

heinrichmartin

unread,
Jul 16, 2018, 10:04:25 AM7/16/18
to
On Monday, July 16, 2018 at 2:41:53 PM UTC+2, Alexandru wrote:
> > ::tcl::mathop::< 0 $v => 686.506 microseconds per iteration
> > apply {v {expr {0 < $v}}} $v => 456.086 microseconds per iteration
> > apply {{r v} {expr {$r < $v}}} 0 $v => 468.614 microseconds per iteration
> > ::namespace inscope ::test {greaterThan 0} $v => 3141.482 microseconds per iteration
> > greaterThan0 $v => 370.73 microseconds per iteration
>
> weird, but I get
>
> time {::tcl::mathop::< 0 $v} 1000
> 0.218 microseconds per iteration
>
> which is stellar faster than your measurement.
>
> And:
> time {apply {v {expr {0 < $v}}} $v} 1000
> 0.565 microseconds per iteration
>
> I'm not sure I can use your timings as a reference due to the exremly large differences. Do you have another process running in parallel?

Hmmm, my environment's load was not volatile and the ratio of my timings was about reproducible several times (and again right now). I am using Tcl 8.6.4.

The output you pasted suggests that you did not run my very script. In my case, one iteration contains 1001 calls to the predicate (one iteration is one call to lsearchP/ListFirst). This is different from just calling the predicate 1000 times.
0 new messages