resolve barewords as variable names in expr

[heinrichmartin]

Hi,
On Friday, May 4, 2018 at 9:49:56 PM UTC+2, briang wrote:
> On Friday, May 4, 2018 at 1:25:02 AM UTC-7, heinrichmartin wrote:
> > out of curiosity: is there a reason for not accepting barewords as strings in expressions (expr/if).
>
> The reason is that the [expr] language is not Tcl, it is it's own language. Different syntax, different semantics.

in the light of recent conversations here, I wondered why not try to interpret "invalid barewords" in expr as variable names.

PRO: math problems look much cleaner[1]
CON: it's not so tcl'ish anymore (but expr has its own language anyway)

The implementation (in 8.6.4) was easier than expected[2].

What do you think about this idea?

Regards
Martin

[1]
On Tuesday, June 12, 2018 at 1:12:59 PM UTC+2, Arjen Markus wrote:
> Here is some code to walkt the ellipse:
>
> set t 0.0
> while { $t < 6.29 } { ;# 2pi would probably be better ...
> set x [expr {$a * cos($t)}]
> set y [expr {$b * sin($t)}]
> set t [expr {$t + 0.01}]
> }

set t 0.0
while { t < 6.29 } { ;# 2pi would probably be better ...
set x [expr {a * cos(t)}]
set y [expr {b * sin(t)}]
set t [expr {t + 0.01}]
}

[2] didn't put effort in compliance with Tcl maintenance rules before receiving comments on the idea:

vvvvvv
--- tclCompExpr.c.orig 2018-06-12 20:45:01.696018614 +0200
+++ tclCompExpr.c 2018-06-12 23:22:59.299016471 +0200
@@ -732,6 +732,12 @@
} else if (Tcl_GetBooleanFromObj(NULL,literal,&b) == TCL_OK) {
lexeme = BOOLEAN;
} else {
+// instead of complaining (try to) resolve as variable name
+lexeme = VARIABLE;
+// VARIABLE expects a $ as the first character, but it does not touch it - fake it
+start--;
+numBytes++;
+break;
Tcl_DecrRefCount(literal);
msg = Tcl_ObjPrintf("invalid bareword \"%.*s%s\"",
(scanned < limit) ? scanned : limit - 3, start,
^^^^^^

[Christian Gollwitzer]

Am 12.06.18 um 23:25 schrieb heinrichmartin:

> in the light of recent conversations here, I wondered why not try to interpret "invalid barewords" in expr as variable names.
>
> PRO: math problems look much cleaner[1]
> CON: it's not so tcl'ish anymore (but expr has its own language anyway)
>
> The implementation (in 8.6.4) was easier than expected[2].
>
> What do you think about this idea?

You could even go one step further and move assignment and simple
control structures into the expression. That is already possible using
VecTcl:

> set t 0.0
> while { t < 6.29 } { ;# 2pi would probably be better ...
> set x [expr {a * cos(t)}]
> set y [expr {b * sin(t)}]
> set t [expr {t + 0.01}]
> }

package require vectcl
set t 0.0
vectcl::vexpr {
while t < 6.29 {
x = a*cos(t)
y = b*sin(t)
t = t + 0.01
}
}

...or even better, using the vector features of VecTcl:

vectcl::vexpr {
t = linspace(0, 6.29, 100)
x = a*cos(t)
y = b*sin(t)
}

This results in x and y being vectors (aka lists of doubles) for the
coordinates on the ellipse.

Christian

[stefan]

> CON: it's not so tcl'ish anymore (but expr has its own language anyway)

I think, there is general agreement that barewords for operators were not necessarily a good idea (see also the discussion in TIP #461), but there is also a shared understanding that the built-in expr-sublanguage will remain stuck with barewords for a while (and a "while" in the Tcl-verse counts decades, empirically), won't it?

I am not sure that adding an additional piece of unconditional substitution will help here, but rather add to the confusion, as long as there are just barewords (true, false, t, f, ...):

set t 1
set f 1

expr {t != f}
expr t != f; # not recommended, but still

vs.

expr {$t != $f}
expr $t != $f; # not recommended, but still


What is the expected outcome under your suggestion for the above example?

Alternatives that render the ambiguity explicit, at least:

- Adding a switch to [expr]?
- Unknown handler for "invalid" then "unknown" barewords? An unknown handler may then resort to variable substitution, as one strategy?

Someone else might come with more compelling examples, but I hope I could convey my message somehow.

Stefan

[Mike Griffiths]

This has the potential to create horrible bugs, especially since there are already "eq" and "ne" operators. Is:

expr {ne}

Checking of the variable $ne contains a true value? Or is it missing the two values to compare?

Is:

expr {ne foo}

missing the first value (to compare to $foo), or is it trying to compare $ne and $foo in some way but missing the operator?

You could special-case "eq" and "ne" so they are always assumed to be operators, but that kind of special treatment is a) horrible, and b) not at all forwards-compatible, because you're preventing yourself from ever being able to add new operators in a backwards-compatible manner.

A lot of potential pitfalls for extremely little gain, IMHO.

[heinrichmartin]

On Wednesday, June 13, 2018 at 2:08:51 PM UTC+2, stefan wrote:
> I think, there is general agreement that barewords for operators were not necessarily a good idea (see also the discussion in TIP #461), but there is also a shared understanding that the built-in expr-sublanguage will remain stuck with barewords for a while (and a "while" in the Tcl-verse counts decades, empirically), won't it?

On Tuesday, June 5, 2018 at 9:51:58 AM UTC+2, Andreas Otto wrote:
> example: the logical operator *= and ~= in "expr" (!*= !~= is the reverse)

Not using letters for operators would yield weird assemblies of special characters for operators.

> I am not sure that adding an additional piece of unconditional substitution will help here, but rather add to the confusion, as long as there are just barewords (true, false, t, f, ...):

Haha, I didn't know all of them: 0, false, no, or off; 1, true, yes, or on (Tcl_GetBoolean)

> What is the expected outcome under your suggestion for the above example?

The real example that triggered my idea was [::math::constants::constants pi].

expr sin($pi) ;# 1.2246467991473532e-16(sic!)
expr sin(pi) ;# would be nice

On Thursday, June 14, 2018 at 12:45:06 AM UTC+2, Mike Griffiths wrote:
> This has the potential to create horrible bugs, especially since there are already "eq" and "ne" operators. Is:
>
> expr {ne}
>
> Checking of the variable $ne contains a true value? Or is it missing the two values to compare?
>
> Is:
>
> expr {ne foo}
>
> missing the first value (to compare to $foo), or is it trying to compare $ne and $foo in some way but missing the operator?
>
> You could special-case "eq" and "ne" so they are always assumed to be operators, but that kind of special treatment is a) horrible, and b) not at all forwards-compatible, because you're preventing yourself from ever being able to add new operators in a backwards-compatible manner.
>
> A lot of potential pitfalls for extremely little gain, IMHO.

I see the point.

My intention was to get rid of the rather useless error "invalid bareword". Disregarding that they share the namespace with operators, I first thought that it would great to interpret them as strings (like all Tcl words).

This conversation was about variable names instead of strings. Again, just in case of no other possible interpretation; I like the term "unknown barewords" by Stefan!

Redirecting the lexer's output was simple; other ideas would require more code changes...

Thanks anyway for sharing your thoughts.
Regards
Martin

[two...@gmail.com]

In addition to this suggestion, I would like to see a change to the set command so that one could do something like this without having to specifically code the expr command itself. When promoting tcl/tk, most of my friends object to the need to use [expr ...] since it's kinda clunky looking to people not used to tcl.

set a := b+10

For example, here's how one can do it inefficiently (but not the barewords, which if expr could handle it, should then also work):

catch {console show}
rename ::set ::realset

proc ::set args {
if { [lindex $args 1] == ":=" && [llength $args] > 2} {
uplevel 1 ::realset [lindex $args 0] [expr [lrange $args 2 end]]
} else {
uplevel 1 ::realset $args
}
}

set foo 10
set bar := ($foo*2 + 2)/2
set xxx ":="

puts "foo= |$foo| bar= |$bar| xxx= |$xxx|"

[heinrichmartin]

On Thursday, June 14, 2018 at 9:04:19 PM UTC+2, two...@gmail.com wrote:
> In addition to this suggestion, I would like to see a change to the set command so that one could do something like this without having to specifically code the expr command itself. When promoting tcl/tk, most of my friends object to the need to use [expr ...] since it's kinda clunky looking to people not used to tcl.

Sometimes I feel the same.

> set a := b+10

I'd go for another command name, e.g. assign. Otherwise, = is sorter than :=.

> For example, here's how one can do it inefficiently
>

> proc ::set args {
> if { [lindex $args 1] == ":=" && [llength $args] > 2} {
> uplevel 1 ::realset [lindex $args 0] [expr [lrange $args 2 end]]
> } else {
> uplevel 1 ::realset $args
> }
> }
>
> set foo 10
> set bar := ($foo*2 + 2)/2

Upvar should do it. And you definitely need uplevel around expr; your example works only because it is resolved *before* the call.

[two...@gmail.com]

On Thursday, June 14, 2018 at 12:52:51 PM UTC-7, heinrichmartin wrote:

> Sometimes I feel the same.
>
> > set a := b+10
>
> I'd go for another command name, e.g. assign. Otherwise, = is sorter than :=.
>

Actually, the test probably only needs to be for more than 2 arguments to the set command, and the assignment operator could be anything you want. There's a wikipedia page on assignment operators,

https://en.wikipedia.org/wiki/Assignment_(computer_science)

I've always disliked the = alone, because of the problems it can create in C code if you meant == and I've had that bug enough times to not like it. I also used to program in modula-2.

This concept is already in use with the "if" command, where an expression without "expr" is ok already. I'm surprised tcl hasn't done this for assignments; maybe there's a gotcha I'm not seeing, other than you would need to have whitespace around the assignment operator, and that's not totally natural.

I thought of using another command, like let, or assign, but then why add an extra command.

The downside to doing it in tcl is the extra processing. But if set is implemented in C code (which I don't really know), then it must already be checking for the number of arguments, since "set a" vs. "set a b" already would have to count the arguments.

Unless I'm missing something, this would seem to be an easy addition to the language and shouldn't break any existing code, where adding a new command could do so.

[jda...@gmail.com]

> > > set a := b+10

how about:

set a [= $a + $b]

or

set a [= {$a*3 + $b*7}]

all it takes to set this up is:

interp alias {} = {} expr

Dave B

[heinrichmartin]

On Thursday, June 14, 2018 at 12:45:06 AM UTC+2, Mike Griffiths wrote:

> On Tuesday, 12 June 2018 22:25:20 UTC+1, heinrichmartin wrote:
> > in the light of recent conversations here, I wondered why not try to interpret "invalid barewords" in expr as variable names.
> >
> > PRO: math problems look much cleaner[1]
> > CON: it's not so tcl'ish anymore (but expr has its own language anyway)
>

> This has the potential to create horrible bugs, especially since there are already "eq" and "ne" operators. Is:
>
> expr {ne}
>
> Checking of the variable $ne contains a true value? Or is it missing the two values to compare?
>

> You could special-case "eq" and "ne" so they are always assumed to be operators, but that kind of special treatment is a) horrible, and b) not at all forwards-compatible, because you're preventing yourself from ever being able to add new operators in a backwards-compatible manner.
>
> A lot of potential pitfalls for extremely little gain, IMHO.

On Wednesday, June 13, 2018 at 2:08:51 PM UTC+2, stefan wrote:

> - Unknown handler for "invalid" then "unknown" barewords? An unknown handler may then resort to variable substitution, as one strategy?

Not trying to push hard on this topic, just adding a comment for the archive:

Tcl already features behavior that is not forward compatible. So, resolving as either string or variable name in the last resort is not a new concept for Tcl:

tclsh
% man n unknown ;# opens the man pages
% oo::class create man {}
::man
% man n unknown
unknown method "n": must be create, destroy or new

This is my reoccurring real-world example:

expect:~$ ;# work a while, and then sometimes forget that I am not in bash ...
expect:~$ df -h ;# still not recognizing
expect:~$ ll
wrong # args: should be "llength list"
while evaluating ll
expect:~$ ;# wtf - I see
expect:~$ ls -l

[stefan]

> tclsh
> % man n unknown ;# opens the man pages
> % oo::class create man {}
> ::man
> % man n unknown
> unknown method "n": must be create, destroy or new

This is more of a particularity of the Tcl shell (tclsh) in interactive mode only, rather than a case in point regarding compat in Tcl itself.

Stefan

[heinrichmartin]

On Thursday, June 14, 2018 at 12:45:06 AM UTC+2, Mike Griffiths wrote:

> This has the potential to create horrible bugs, [...]

True for this very case, but maybe we can think of more examples:

package require my::hexdump
# forget to [namespace import my::hexdump::hexdump]
hexdump [file join $basedir $filename]

This will work just fine until you (a) run into performance problems or (b) change OS or distro. "unknown" is not a particularity of the interactive mode.

Different story, but still a bug introduced by ambiguity plus sloppiness (happened recently):

string is boolean -strict ;# forgot to type last parameter $value

... and wondered why non of my input record contained valid data.

I tried to underline my point with an example, and "man" was not the perfect one. But I am still sure that Tcl (as well as other languages) already has pitfalls.