tcl ?missing? expr operator…

[Andreas Otto]

Hi, if you work with TCL … sometimes you have the feeling… something is missing

example: the logical operator *= and ~= in "expr" (!*= !~= is the reverse)

*= → glob match
~= → regular expression

so the following code is possible…

================================
if ($first == yes && $prefix in {…} && $name ~= {^Mq\w+§§}} {
… do something
}
================================

mfg AO

[Arjen Markus]

You can easily extend the set of functions that is recognised by [expr]. Unfortunately extending the set of operations requires more work. The above example could be:

if {$first == "yes" && $prefix in {...} && matchre( $name, {^Mq\w+§§} ) } {
...
}

matchre should be a procedure residing in the (relative) namespace tcl::mathfunc.

Regards,

Arjen

[Arjen Markus]

Just an aside: I like functions better than such operations, as the names of the functions can be chosen to match (no pun intended) their purpose. I do not connect "*=" with glob matching, but that may be my limited imagination.

Regards,

Arjen

[Rich]

Arjen Markus <arjen.m...@gmail.com> wrote:
> On Tuesday, June 5, 2018 at 12:01:07 PM UTC+2, Arjen Markus wrote:
>> On Tuesday, June 5, 2018 at 9:51:58 AM UTC+2, Andreas Otto wrote:

>> > Hi, if you work with TCL ? sometimes you have the feeling?

>> > something is missing
>> >
>> > example: the logical operator *= and ~= in "expr" (!*= !~= is the
>> > reverse)
>> >

>> > *= ? glob match
>> > ~= ? regular expression
>> >
>> > so the following code is possible?
>> >
>> > ================================
>> > if ($first == yes && $prefix in {?} && $name ~= {^Mq\w+งง}} {
>> > ? do something

>> > }
>> > ================================
>> >
>> > mfg AO
>>
>> You can easily extend the set of functions that is recognised by
>> [expr]. Unfortunately extending the set of operations requires more
>> work. The above example could be:
>>
>> if {$first == "yes" && $prefix in {...} && matchre( $name,

>> {^Mq\w+งง} ) } {

>> ...
>> }
>>
>> matchre should be a procedure residing in the (relative) namespace
>> tcl::mathfunc.
>>
>> Regards,
>>
>> Arjen
>
> Just an aside: I like functions better than such operations, as the
> names of the functions can be chosen to match (no pun intended) their
> purpose. I do not connect "*=" with glob matching, but that may be
> my limited imagination.

The OP also appears to be coming from a Perl background, as I think *=
and ~= are perl'isms. They certianly look like perl'isms at first
glance.

[Arjen Markus]

On Tuesday, June 5, 2018 at 1:01:16 PM UTC+2, Rich wrote:

> >
> > Just an aside: I like functions better than such operations, as the
> > names of the functions can be chosen to match (no pun intended) their
> > purpose. I do not connect "*=" with glob matching, but that may be
> > my limited imagination.
>
> The OP also appears to be coming from a Perl background, as I think *=
> and ~= are perl'isms. They certianly look like perl'isms at first
> glance.

Yes, and even more confusing: *= and ~= are also valid C operators, with completely different meanings.

Regards,

Arjen

[Rolf Ade]

All well and right - I had the same two thoughts ("if you want Perl you
know where you find it", "that bites with C meaning") - and the
canonical answer - use a custom expr function - aleady provided.

But the OP feels a need. The only answer we can give is "roll your own
solution". Why can't we point to a package ("exprFunctionPlus") and say:
"Beside tons of other good stuff this package includes this somethimes
handy machtre(<string>,<re>) function"?

Or do I miss something and there are collections of helpful, "nice"
tcl::mathfunc?

rolf

[Arjen Markus]

Yes, OP's wishes are clear enough and understandable. It is simply that currently Tcl's expr can be extended with functions, not with operators. I know of no package that does this sort of things, but it should not be too hard to add (note to self: a good idea for some of the mathematical stuff in Tcllib?)

Regards,

Arjen

[Andreas Otto]

after thinking about. I would call these operators as follow…

= → native
* → glob
~ → regexp

== → compare with native
=* → compare with glob
=~ → compare with regexp

!= → inverse
!* → inverse
!~ → inverse

[Andreas Otto]

>
> You can easily extend the set of functions that is recognised by [expr]. Unfortunately extending the set of operations requires more work. The above example could be:
>
> if {$first == "yes" && $prefix in {...} && matchre( $name, {^Mq\w+§§} ) } {
> ...
> }
>
> matchre should be a procedure residing in the (relative) namespace tcl::mathfunc.
>

yes… If done it after your recommendation…

proc tcl::mathfunc::regexp {var rx} {
::regexp $rx $var
}

set p1 ""
if {$name eq "${prefix}Dup" && $ret eq $class} {
set ty "cob"
} elseif {$ret eq "void" && $type0 eq "$class§P" && $argvL == 1 && regexp($name,"Delete")} {
set ty "dst"
} elseif {$class eq $type0N && regexp($varE,"^out")} {
set ty "ort"
} elseif {$class eq $type0N} {
set ty "obj"
set argv [lrange $argv 1 end]
} elseif {$ret eq $class && $class eq "${prefix}C" } {
set ty "cst"
set p1 " static"
} elseif {$ret eq "void" && $type0N eq "$class§P"} {
set ty "dst"
} else {
set ty "ukn"
}

[heinrichmartin]

I was thinking the very same: introducing 3-character-operators did not look right.

When someone points out that == is not a string comparison, the others are:
We have nice options for positive ops: gl and re (note that they are also used as switches in Expect: -ex, -gl, -re). But reverse to negate? Or ng and nr???

[Gerald Lester]

TIP it with a sample implementation.

That being said, I think I'd rather functions since those symbol
combinations for operators do not speak to me of what they will do.


--
+----------------------------------------------------------------------+
| Gerald W. Lester, President, KNG Consulting LLC |
| Email: Gerald...@kng-consulting.net |
+----------------------------------------------------------------------+

[two...@gmail.com]

When using non-numeric arguments to math procs one will have to place the entire expression in a pair of {}'s or one will get an error. This is not the case with math functions with numerical arguments. This is because the value of a variable which is numeric can be dynamically understood, whereas text strings will throw a bareword error:

() 3 % proc tcl::mathfunc::regexpfunc {var rx} {
> return [::regexp $rx $var]
> }

() 4 % set f "test"
test

() 5 % expr regexpfunc( $f, {t..t} )
invalid bareword "test"
in expression "regexpfunc( test, t..t )";
should be "$test" or "{test}" or "test(...)" or ...


() 6 % expr {regexpfunc( $f, {t..t} )}
1
() 7 % set x 10
10
() 8 % expr min($x,5)
5
() 9 % expr {min($x,5)}
5

[Donal K. Fellows]

On 07/06/2018 23:20, Gerald Lester wrote:
> TIP it with a sample implementation.

That's going to be quite tricky as it significantly increases the
complexity of the expression parser. Even just adding simple assignment
would add virtually all that effort. IIRC, the issue is to do with
determining whether something is a function call or an array assignment.
(One can resolve this stuff, sure, but doing so makes the parser quite a
bit more complex.)

Donal.
--
Donal Fellows — Tcl user, Tcl maintainer, TIP editor.

[Rich]

two...@gmail.com wrote:
> When using non-numeric arguments to math procs one will have to place
> the entire expression in a pair of {}'s or one will get an error.

Already known. See these portions of the Tcl expr man page:

As discussed below, it is usually best to enclose expressions in
braces to prevent the command parser from performing substitutions on
the contents.

...

As mentioned above, expressions are substituted twice: once by the
Tcl parser and once by the expr command. For example, the commands

set a 3
set b {$a + 2}
expr $b*4

return 11, not a multiple of 4. This is because the Tcl parser will
first substitute $a + 2 for the variable b, then the expr command
will evaluate the expression $a + 2*4.

Most expressions do not require a second round of
substitutions. Either they are enclosed in braces or, if not, their
variable and command substitutions yield numbers or strings that do
not themselves require substitutions.


It is generally accepted that unless you 1) actually know what you are
doing, 2) *really* need the two rounds of substitution that occur with
unbraced expr expressions, one should *always* brace your expressions.

[Andreas Leitgeb]

Donal K. Fellows <donal.k...@manchester.ac.uk> wrote:
> On 07/06/2018 23:20, Gerald Lester wrote:
>> TIP it with a sample implementation.
> That's going to be quite tricky as it significantly increases the
> complexity of the expression parser. Even just adding simple assignment
> would add virtually all that effort. IIRC, the issue is to do with
> determining whether something is a function call or an array assignment.

I'm not too much in favour of the awk-like ~= and the (new to me) *=,
but for sure they are NOT meant as assignments here (unlike their C
counterparts with entirely different meaning), and thus would operate
on values, and thus not come even remotely near to the can of worms
that assignment in expr turned out to be.

[florent...@gmail.com]

About this : i got the idea to change the array syntax, renewing the way tcl handle access to the part of a variable, then to change the function call in expr.

$Dict«$key» -> dict key $Dict $key
$Array“$index” -> set Array($index)
$String‛$index’-> string index $String $index
$list‹$index› -> lindex $list $index

Example of function call :

expr {x“$i”=(sin,$a)*$l‹$i›}

Means the product of the sinus of the angle $a by the $ith element of the list l should be assign to the $ith component of the array x.

This way, parenthesis have only one meaning in expr (ie grouping), and a meaning only in expr, so that there is no ambiguity any more.

But i don't have any idea about the cost of using unicode operator in the tcl parser. Would it be a problem in matter of performance ?

[Donal K. Fellows]

On 06/06/2018 05:39, Arjen Markus wrote:
> it should not be too hard to add

Extending parsers is trickier than extending functions; in the latter
case, you're just introducing another name, whereas in the former case
you have to also add a token recogniser and establish how that token is
understood in relation to all the other tokens. Sometimes that is fairly
easy, sometimes that's very hard (which is why we don't have assignment
operators in [expr]; semantically they're trivial, but syntactically
they're a PITA since we don't have “lvalues” as concepts in Tcl's
expression sublanguage).

Matching operators would be simpler to add than assignment operators,
since they would not require making complex new classes in the parse
tree, but rather plugging in more siblings to == and eq.

Donal (I'd like comments in expressions).

[florent...@gmail.com]

On changing the way function are called, i don't see any good reason to write : expr {f(x)} except this is very traditionnal in mathematics.

The tcl way is to write a list whose first argument is taken as a command.

Transposed in expr, where commas are the delimiters of arguments, we get (ex):
expr {sin,$x},
where first bareword (sin) is the command.

[Rich]

If you want to do this (or at least something similar) it is already
possible:

$ rlwrap tclsh
% namespace path {::tcl::mathop ::tcl::mathfunc}
% set x [sin .5]
0.479425538604203
% set y [+ [* 3 4] [/ 8.0 3]]
14.666666666666666
%

[florent...@gmail.com]

Yes but
Set x($i+1) [Sin $a/2]
Could never work
You should write
Set x([expr {$i+1}]) [sin [expr {$a/2}]
instead.

Wheras
Expr {x“$i+1”=sin,$a/2}
Would work.

[jda...@gmail.com]

set x([+ $i 1]) [sin [/ $a 2]]

[Donal K. Fellows]

On 04/11/2018 14:06, florent...@gmail.com wrote:
> The tcl way is to write a list whose first argument is taken as a command.

If you want to do it that way, just put the tcl::mathop and
tcl::mathfunc namespaces on your [namespace path] and use everything as
full commands:

namespace eval example {

namespace path {::tcl::mathop ::tcl::mathfunc}

variable x1 123
variable y1 234
variable x2 321
variable y2 432
variable r [hypot [- $y1 $y2] [- $x1 $x2]]
variable theta [atan2 [- $y1 $y2] [- $x1 $x2]]
# etc...
}

Only the &&, || and ?: operators aren't available that way, but you've
got [if] instead, which lets you do the moral equivalent.

Donal.

[florent...@gmail.com]

My problem is to have "orthogonal" grammars between plain Tcl and expr. By "orthogonal" grammar, i mean something analog with a basis for linear algebra : each grammar has his own "dimension". Well, this analogy is not totaly correct, because we are in a hierarchy : plain tcl is "higher" than expr.

In principle, the significant operators of the sublanguage expr must have no signification into plain Tcl (so we can build an expression by pieces), whereas significant operators of plain Tcl should have an identical signification into expr (so we can use Tcl inside expr).

But Parenthesis have a meaning either in expr (to group or to introduce argument), either in tcl (to denote an array key), and thoses meanings differ. Thus, it exists some ambiguities that the expr parser can't resolve.

[Donal K. Fellows]

On 05/11/2018 00:45, florent...@gmail.com wrote:
> But Parenthesis have a meaning either in expr (to group or to introduce argument), either in tcl (to denote an array key), and thoses meanings differ. Thus, it exists some ambiguities that the expr parser can't resolve.

And yet Tcl currently manages to distinguish them. How? It's because in
an expression, when you are reading from an array (or other variable)
you prefix it with ‘$’. When you are using a function, you don't. That
is entirely formally unambiguous. Tcl's expressions do not refer to
variables other than to read from them (as [this…] is an embedding of a
different language — standard Tcl — within the expression).

The potential ambiguity you highlight would be a problem if we had
assignment operators… and in fact this the core of why we don't. We
simply have not managed to design an unambiguous grammar for doing this
yet that we have been happy with. I *think* it is possible to make one,
but it is quite difficult to do so, and the TCT certainly won't revisit
the assignment operators (at least as a body) until someone sorts out
the grammar.

[Andreas Leitgeb]

Donal K. Fellows <donal.k...@manchester.ac.uk> wrote:

> The potential ambiguity you highlight would be a problem if we had
> assignment operators… and in fact this the core of why we don't. We
> simply have not managed to design an unambiguous grammar for doing this
> yet that we have been happy with. I *think* it is possible to make one,
> but it is quite difficult to do so, and the TCT certainly won't revisit
> the assignment operators (at least as a body) until someone sorts out
> the grammar.

If we eventually (read Tcl 9.0) sanitize array key syntax (to require
backslash-escaping certain literal characters before the close-paren),
then it would once again be possible to tackle the assignments.

Alternatively, if [expr {'arr(...) := ...}] *were* considered acceptable
expr-syntax (which it wasn't last time it was up), then the ' would give
the same syntactic unambiguity as $arr(...) currently does on value-side.
Such-tagged array-elements or even blank names could also make nice
arguments for e.g. some "incr" imported into tcl::mathfunc.

[two...@gmail.com]

On Monday, November 5, 2018 at 8:39:14 AM UTC-8, Andreas Leitgeb wrote:


> Such-tagged array-elements or even blank names could also make nice
> arguments for e.g. some "incr" imported into tcl::mathfunc.

Early on the plethora of operators in the C language made sense and arose for 2 reasons (IMO).

1. The assignment and auto-increment operators could be compiled into single instructions, especially on the pdp-11, the most popular Unix system.

2. The Unix developers were not especially fond of typing long commands, especially when using teletypes for a terminal.

When I moved to using modula-2 instead of C, I never missed these operators. And it was wonderful to not have to be wary of some C traps, such as, if (a = 1) …. which could be very difficult to spot. Later when C became C++ many of these gotchas remained. With modern pipelined computer architectures, and advances in compilers, less typed code does not predictably translate to faster programs.

I would hope that the Tcl community would not wish to add these operators to the language by changing the syntax of the expr command.

On the other hand, it would be nice to have the set command be a little more like the if command, in that the test argument does not require one to explicitly use expr. Since the set statement currently has only 1 or 2 arguments, it would be easy to implement a form of assignment as in,

set a = cos($b) + 1

by having more than 2 arguments. I’ve experimented with this using the following,

rename ::set ::realset
proc ::set args {
if { [llength $args] > 2} {
uplevel 1 ::realset [lindex $args 0] [expr [lrange $args 2 end]]
} else {
uplevel 1 ::realset $args
}
}

This should be compatible since currently more than 2 args is an error. One could have the “=” be significant instead of just taking up an argument though I haven’t come up with anything yet.

[heinrichmartin]

On Monday, November 5, 2018 at 8:04:51 PM UTC+1, two...@gmail.com wrote:
> set a = cos($b) + 1

set b {0) + (40} ;# ;-)

> uplevel 1 ::realset [lindex $args 0] [expr [lrange $args 2 end]]

My guts feeling suggests [expr {*}[lrange $args 2 end]]. I haven't looked for relevant cases though...

[Andreas Leitgeb]

two...@gmail.com <two...@gmail.com> wrote:
> On Monday, November 5, 2018 at 8:39:14 AM UTC-8, Andreas Leitgeb wrote:
>> Such-tagged array-elements or even blank names could also make nice
>> arguments for e.g. some "incr" imported into tcl::mathfunc.

First of all, my comment was *not* about adding incr or any other
specific operator to expr. It is already possible to import commands/
procedures into tcl::mathfunc (or create them there), and then call
them from within an expression.

E.g. string length can be imported(*) as tcl::mathfunc::slen
and then used like [expr {(slen($str)+1)/2}] and it is
also possible to import commands that take a var name as an
argument, and incr is just one example of such a function.

Assuming that one has imported incr into tcl::mathfunc, then the
following is already legal syntax: [expr {incr("varname",42)}]
Similarly, one can also already now apply it on an array-element:
[expr {incr("arr($key)",42}]

It is also already possible to import the "set" command, and have
assignments like [expr {set("x",42)}]

Nothing new so far.

But when the discussion arose to add some assignment operator to expr,
(and iirc preferrence was for ":=" to avoid repeating C's mistake there)
they wanted a syntax that was "less ugly" than the one already possible -
so, under all circumstances anything than having the variable name bare
of any quoting would be worthy on left side of such an operator. Well,
quoted strings would still be allowed as well, but unless one could have
a bare varname or a bare array-element on left side of :=, then majority
of TCT would rather not have assignment, than require quoting; but
bare array-elements just would make expr syntax ambiguous.

While I agree that expr {"foo($key)" := ...} would indeed be pretty
strange to look at, I seemed to be alone in the world to consider
expr {'foo($key) := ...} an acceptable compromise.

> When I moved to using modula-2 instead of C, I never missed these
> operators.

Well, I did, and I was soo glad when that one course using modula-2
was finished and I would no longer have to talk to the compiler like
to a baby - I was used to ":=" already from Pascal, but not having a
polymorphic output procedure was awkward. I can see your point, but
it's really just a matter of personal preference.

> I would hope that the Tcl community would not wish to add these

> operators to the language ...

That was certainly not my point to suggest.

[two...@gmail.com]

On Monday, November 5, 2018 at 2:13:25 PM UTC-8, heinrichmartin wrote:

>
> My guts feeling suggests [expr {*}[lrange $args 2 end]]. I haven't looked for relevant cases though...

I wondered about that {*} too, however, uplevel says it concats all it's arguments, and then evaluates the result. All the usual tests I've done seem to work either way.

But it would be certainly better if the set code which I assume is written in C were to do this, instead of my hack.

[two...@gmail.com]

On Monday, November 5, 2018 at 3:21:16 PM UTC-8, Andreas Leitgeb wrote:
> I seemed to be alone in the world to consider expr {'foo($key) := ...} an acceptable compromise.

If you one uses my set replacement example, one could write:

% set a = [set b = 3 + 1] + 2
6
% set a
6
% set b
4

And since the "=" is used only to use up an argument, it could also be written as:

set a := [set b := 3 + 1] + 2

which is, in my opinion, not all that bad to look at :)

>
> That was certainly not my point to suggest.

I think I misunderstood your comment, my bad!

[florent...@gmail.com]

Quote

While I agree that expr {"foo($key)" := ...} would indeed be pretty
strange to look at, I seemed to be alone in the world to consider
expr {'foo($key) := ...} an acceptable compromise.

/quote

But then how should we compute the array key ? I mean, would we have to write :
Expr {'foo($key +1) := ...}
Or
Expr {'foo(($key + 1)) := ... }

To do so ?

That doesn't remove the ambiguity of the parenthesis... which still stand as a potentiel problem.

That's why i'd like to change the array syntax :

Let's say :

Array“$key“
String‘$numbyte’
List‹$index›
Dict«$key»

Now the parenthesis are free, and we can use it as an expr shorthand...

Set a“($key+1)“ (l‹($index+1)›*3)
Or
Expr {a“($key+1)“=l‹$index+1›*3}

[Andreas Leitgeb]

florent...@gmail.com <florent...@gmail.com> wrote:

> Andreas Leitgeb <a...@logic.at> wrote:
>> While I agree that expr {"foo($key)" := ...} would indeed be pretty
>> strange to look at, I seemed to be alone in the world to consider
>> expr {'foo($key) := ...} an acceptable compromise.

> But then how should we compute the array key?

There exist different opinions about how important the usecase
of arithmetically computed array indices is. Keys for arrays
in Tcl are most typically already stored in a var as in arr($key),
or result from concatenation as in arr($x,$y), or are barewords
as in ::env(PATH) . Almost each of these common uses would need to
be changed to enable arithmetically computed indices.

The usecase of arr([expr {$x+$y}]) is - in practice - so marginal
that you're having a hard time trying to convince others of the
benefits of a special syntax for it, and even more so at the
cost of overthrowing substantial parts of tcl's syntax for it.

[heinrichmartin]

On Tuesday, November 6, 2018 at 2:37:45 AM UTC+1, two...@gmail.com wrote:
> On Monday, November 5, 2018 at 2:13:25 PM UTC-8, heinrichmartin wrote:
>
> >
> > My guts feeling suggests [expr {*}[lrange $args 2 end]]. I haven't looked for relevant cases though...
>
> I wondered about that {*} too, however, uplevel says it concats all it's arguments, and then evaluates the result. All the usual tests I've done seem to work either way.

This is (another?) issue in your code: expr is called in the wrong context;
set a foo
set b = {$a eq "bar"}

> But it would be certainly better if the set code which I assume is written in C were to do this, instead of my hack.

Definitely.

[two...@gmail.com]

On Wednesday, November 7, 2018 at 8:29:42 AM UTC-8, heinrichmartin wrote:

>
> This is (another?) issue in your code: expr is called in the wrong context;
> set a foo
> set b = {$a eq "bar"}

Yes, this seems to be an issue with expr and some expressions (in particular the string operators with quoted strings), since

expr $a eq "bar"

gives an error, while,

expr {$a eq "bar"}

does not, while the following uses of expr are both ok,

set x 1
set y 2
expr $x > $y
expr {$x > $y}

As was pointed out to me above, the expr manual page does speak about a need for extra {}'s at times, though I haven't been able to figure out how to make that work all the time. And using {*}, which was implemented to get around these sorts of problems doesn't seem to help in this case.

However, the if command does seem to do something to get around this, since,

if {$a eq "bar"} {puts yes} else {puts no}

works, but

if {[expr $a eq "bar"]} {puts yes} else {puts no}

does not, while the manual pages says,

"The if command evaluates expr1 as an expression (in the same way that expr evaluates its argument)"

and that would appear to not be entirely true.

So, as we agree, it seems this would need to be something done in the set or expr commands to make it work in all cases. I'd write a tip, but I don't see all that much interest, and I'm not able to make the changes myself. So, I just play with my partial implementation.

Still, I think it would make tcl appear more elegant if it could be made to work in all cases. After all, somebody must have thought similarly when the if command was built. If you had to explicitly use expr in all if statements, things would look kinda ugly.

[jda...@gmail.com]

> However, the if command does seem to do something to get around this, since,
>
> if {$a eq "bar"} {puts yes} else {puts no}
>
> works, but
>
> if {[expr $a eq "bar"]} {puts yes} else {puts no}
>
> does not, while the manual pages says,
>
> "The if command evaluates expr1 as an expression (in the same way that expr evaluates its argument)"

[if] handles the expression argument differently than [expr]. However they could be the same if you required [expr] to take only a single argument.

This is what happens when the [expr] expression is enclosed in {} or has no embeded spaces (etc).

It is the extra flexibility in [expr] argument handling, due to it only having a single logical argument (the expression), that makes it different.

[two...@gmail.com]

On Wednesday, November 7, 2018 at 11:23:49 AM UTC-8, jda...@gmail.com wrote:

> [if] handles the expression argument differently than [expr]. However they could be the same if you required [expr] to take only a single argument.
>
> This is what happens when the [expr] expression is enclosed in {} or has no embeded spaces (etc).

The following fails too:

expr $a=="foo"

as well as

if $a=="foo" ...

so {}'s are required even though there are no embedded spaces. It is apparently the substitution rules that cause some issues with the string operators and/or string literals that require extra "quoting" tricks. I guess that's why it's sometimes called "quoting hell".

[heinrichmartin]

I was not aware how basic this would become.

On Wednesday, November 7, 2018 at 7:37:04 PM UTC+1, two...@gmail.com wrote:
> On Wednesday, November 7, 2018 at 8:29:42 AM UTC-8, heinrichmartin wrote:
>
> >
> > This is (another?) issue in your code: expr is called in the wrong context;
> > set a foo
> > set b = {$a eq "bar"}
>
> Yes, this seems to be an issue with expr and some expressions (in particular the string operators with quoted strings), since

No, this is a matter of understanding the syntax. All of expr, if, and while use the same parser for their expressions.

> expr $a eq "bar"

is the same as [expr "foo eq bar"] which contains two invalid barewords.

> expr {$a eq "bar"}

allows expr to optimize the byte-code and evaluates $a inside expr. It is documented under "performance considerations".

> As was pointed out to me above, the expr manual page does speak about a need for extra {}'s at times, though I haven't been able to figure out how to make that work all the time.

With or without {} are two different things - there is a need to consider the difference.
Also compare [set a 1; while $a {set a 0}] vs [set a 1; while {$a} {set a 0}].
\$a is equivalent to {$a} here.

> And using {*}, which was implemented to get around these sorts of problems doesn't seem to help in this case.

I did not think it through in my first response; and that comment is obsolete now. Just be aware that it is important whether expr sees (the string representation of) a single list or whether it joins the elements of a list with space as a separator.

> if {$a eq "bar"} {puts yes} else {puts no}
>
> works, but
>
> if {[expr $a eq "bar"]} {puts yes} else {puts no}
>
> does not, while the manual pages says,

These are not the same: you are using [if {}] but [expr ...] without {}.

> uplevel 1 ::realset [lindex $args 0] [expr [lrange $args 2 end]]

Iirc, the implementation with best performance would be sth like

# untested
rename ::set ::realset ;# caution if you're not alone in the global namespace
proc ::set {var args} {
upvar 1 $var target
if { [llength $args] > 1} {
# now uplevel should indeed do the right thing
::realset target [uplevel 1 expr [lreplace $args 0 0]]
} else {
# this includes set with a single arg
::realset target {*}$args
}
# note how the return value bubbles from ::realset
}

[two...@gmail.com]

On Wednesday, November 7, 2018 at 2:10:47 PM UTC-8, heinrichmartin wrote:
> Iirc, the implementation with best performance would be sth like
>
> # untested
> rename ::set ::realset ;# caution if you're not alone in the global namespace
> proc ::set {var args} {
> upvar 1 $var target
> if { [llength $args] > 1} {
> # now uplevel should indeed do the right thing
> ::realset target [uplevel 1 expr [lreplace $args 0 0]]
> } else {
> # this includes set with a single arg
> ::realset target {*}$args
> }
> # note how the return value bubbles from ::realset
> }

Well, I knew someone better than me could figure this out. I don't pretend to know why yours works where mine didn't, but I think I know how to use it:

If the expression uses any string comparisons, then it must be enclosed in {}'s. Otherwise they are not required.

I did a lot of bizarre looking tests and it worked. Anything that expr by itself didn't like, resulted in the same error, so I think your code nailed it.

So, thanks, I've replaced mine with your version in my toolkit.

[Christian Gollwitzer]

Am 08.11.18 um 05:31 schrieb two...@gmail.com:

> Well, I knew someone better than me could figure this out. I don't pretend to know why yours works where mine didn't, but I think I know how to use it:
>
> If the expression uses any string comparisons, then it must be enclosed in {}'s. Otherwise they are not required.

Actually I advise *against* expr with more than one argument. IMHO expr
with more than one argument should become an error in Tcl 9. It has been
long deprecated with a style rule like "brace your expressions", but
sometimes you want to compute an expressino given in a string, so

expr $e

is fine in some (special!) cases. That braces "are not required", or
more precisely, expr does some tricks and jumps through hoops to support

expr $a * 3

was just introduced for backwards compatibility with Tcl 7, that is
almost 20 years ago. Besides performance, there are various other
problems with it, including a security problem. In other cases it is
called "double substitution bug" or "injection". Consider

set a 3+4
expr $a*5 # 23
expr {$a*5} # error

set a [expr 3+4]
expr $a*5 # 35, i.e. only this works without braces

set a {[exec format C:]} # no problem
expr $a * 3 # formats your hard drive
expr {$a * 3} # error, works in the intended cases

Christian

[Donal K. Fellows]

On 08/11/2018 06:15, Christian Gollwitzer wrote:
> IMHO expr with more than one argument should become an error in Tcl 9.

I think that's a tremendous suggestion.

[Donal K. Fellows]

On 08/11/2018 06:15, Christian Gollwitzer wrote:

> IMHO expr with more than one argument should become an error in Tcl 9.

I've just written a TIP on this:

https://core.tcl.tk/tips/doc/trunk/tip/526.md

I would welcome your input on this, and for you to be listed as a
co-author for your insight this this would be a good idea.

[florent...@gmail.com]

Quote

The usecase of arr([expr {$x+$y}]) is - in practice - so marginal
that you're having a hard time trying to convince others of the
benefits of a special syntax for it, and even more so at the
cost of overthrowing substantial parts of tcl's syntax for it.

/quote

Year after year, you are right, but century after century, you will be wrong.

The ambiguity of the grammar, depending on the context, lead to so many corner cases...

It s not like there still exists only the ascii charset.
Ok, it is very understandable, that in the late 80's, we had to reuse some char, with distinct meaning, in many places, since the charset was very limited.

But now, there is 1 million of (wide)chars... why keep all those historical limitations ?

Yes, it will be a different Tcl. The same library, but a different scripting syntax.

But the same philosophy, a toplevel parser, with a simple and generic syntax, glueing many sublevel parsers, with richer and specialized syntax.

[two...@gmail.com]

On Thursday, November 8, 2018 at 2:18:52 AM UTC-8, Donal K. Fellows wrote:
>
> > IMHO expr with more than one argument should become an error in Tcl 9.
>
> I've just written a TIP on this:
>

Wow, no sooner than someone helps me to create a more easy to read assignment statement than someone writes a tip which would break that. LOL

Fortunately, the example with [exec format c:] won’t really format your c: drive but I do understand the concern.

The developers of [if] and [while] clearly wished to hide the syntactic noise of [expr], and that is what I wanted to do with [set]. It doesn’t actually have to be [set], It could be a [let] command, as in:

let a = sqrt(sin($x)**2 + cos($y)**2)

over

set a [expr {sqrt(sin($x)**2 + cos($y)**2)}]

One of tcl’s strengths is it’s ability to craft (as Brian Kernighan would say) “little languages”. Perhaps tcl could use something (if its not already there) to get control before a command line is tokenized.

Perhaps something like an inverse of {*}.

Say we have this,

proc foo * {…}

Where inside of foo, the * argument could mean the entire command line, before it gets tokenized into words. Maybe with something like $* that could then be parsed locally and tokenized when ready. Then something like [let] could be coded to add the bracketing before the first scan of the command line.

I have to read programs much more than I write them. I already have too many floaters and other squiggles in my vision, and any fewer braces and brackets would be much friendlier to my eyes, especially all those )}] closers.

[heinrichmartin]

On Friday, November 9, 2018 at 7:50:41 PM UTC+1, two...@gmail.com wrote:
> I have to read programs much more than I write them. I already have too many floaters and other squiggles in my vision

> let a = sqrt(sin($x)**2 + cos($y)**2)

Would you expect that?

expect [~]proc echo args {puts [join $args \n]}
expect [~]set x 1
1
expect [~]set y 0
0
expect [~]echo sqrt(sin($x)**2 + cos($y)**2)
sqrt(sin(1)**2
+
cos(0)**2)

That's why it's also clearer to use {}.

> One of tcl’s strengths is it’s ability to craft (as Brian Kernighan would say) “little languages”. Perhaps tcl could use something (if its not already there) to get control before a command line is tokenized.
>
> Perhaps something like an inverse of {*}.

The inverse of {*} is [list].

> Say we have this,
>
> proc foo * {…}
>
> Where inside of foo, the * argument could mean the entire command line, before it gets tokenized into words. Maybe with something like $* that could then be parsed locally and tokenized when ready. Then something like [let] could be coded to add the bracketing before the first scan of the command line.

Undoing Tcl parser's job sounds not very Tcl'ish to me; imo, {} is the way to go.

[two...@gmail.com]

On Friday, November 9, 2018 at 12:27:10 PM UTC-8, heinrichmartin wrote:
> > let a = sqrt(sin($x)**2 + cos($y)**2)
>
> Would you expect that?

> echo sqrt(sin($x)**2 + cos($y)**2)
> sqrt(sin(1)**2
> +
> cos(0)**2)
>
>That's why it's also clearer to use {}.

Yes, clearer for the parser, but to my eyes, not clearer to read.

I was talking about adding {} automatically, not eliminating them.

If the rule is bracket everything, then it makes more sense to me to make that the default, as long as there's a way around that, like using "".

This reminds me of the semi-colon requiring languages. The most common error during a C program compile is, ".... or perhaps you forgot a semi-colon". Except for forgetting the $, leaving out a bracket is my most common tcl error.

> Undoing Tcl parser's job sounds not very Tcl'ish to me; imo, {} is the way to go.

Like the previous poster said, I use tcl/tk for it’s great library, easy sockets, the FANTASTIC twapi package, interactive console, tclkits, androwish etc.

But I'm not a fan of the substitution rules.

The problem with understanding tcl’s few, but tricky rules is that you really need to completely understand the parsing/substitution algorithm - which I obviously still don’t even after 20 years of using tcl/tk.

Contrast that with most other languages, which hide those details. One doesn’t need to understand the python parser algorithm to write python code.

[Rich]

two...@gmail.com wrote:
> On Friday, November 9, 2018 at 12:27:10 PM UTC-8, heinrichmartin wrote:
>> > let a = sqrt(sin($x)**2 + cos($y)**2)
>>
>> Would you expect that?
>
>> echo sqrt(sin($x)**2 + cos($y)**2)
>> sqrt(sin(1)**2
>> +
>> cos(0)**2)
>>
>>That's why it's also clearer to use {}.
>
> Yes, clearer for the parser, but to my eyes, not clearer to read.
>
> I was talking about adding {} automatically, not eliminating them.
>
> If the rule is bracket everything, then it makes more sense to me to
> make that the default, as long as there's a way around that, like
> using "".

The reason the curly brackets are recommend is due to how the Tcl
parser parses a command.

Part of the first phase of parsing a command call is to evaluate the
arguments, performg dollar-sign and square bracket replacements. [1]

After that evaluation is performed, only then does the command being
called get access to its arguments as part of the call.

The curly brackets are the defined Tcl way to turn off the dollar and
square bracket expansion [2] for any strings given to any commands,
including to [if], [while], [expr], etc. so that dollar signs and
square bracets are not substituted out before any of those commands
receives the string. You can achive the identical results of "surround
with {}" by adding backslashes before each dollar sign and each open
square bracket, but this:

expr \$a + (\$b - \[some-command])

is harder to type and more error prone (you might forget one backslash
somewhere in a complicated formula) than this:

expr {$a +($b - [some-command])}

> But I'm not a fan of the substitution rules.
>

> The problem with understanding tcl?s few, but tricky rules is that

> you really need to completely understand the parsing/substitution

> algorithm - which I obviously still don?t even after 20 years of
> using tcl/tk.

Read, very carefully, the rules on the page I pointed towards. And it
is not so much understanding the full algorithm as it is fully
understanding the ordering of what occurs when. The ordering is
different than what occurs in other languages, and trying to view Tcl
with a viewpoint of "language X does it this way" is what creates most
of the difficulties in understanding Tcl.

[1] https://wiki.tcl-lang.org/page/Dodekalogue - item [2] in the list
[2] https://wiki.tcl-lang.org/page/Dodekalogue - item [6] in the list

[two...@gmail.com]

On Friday, November 9, 2018 at 3:39:19 PM UTC-8, Rich wrote:
> Read, very carefully, the rules on the page I pointed towards. And it
> is not so much understanding the full algorithm as it is fully
> understanding the ordering of what occurs when.

I think we agree, just are using different terms. If you don’t know the right order, then you don’t understand the algorithm.

But speaking of order…. When a paragraph begins with “… 2 steps… First, the tcl intrepter…” I immediately look for the word “second” to try to understand the structure of the paragraph. I don’t see the word “second” anywhere in rule 2. I might be anal, but I get stuck when that happens.

> Tcl with a viewpoint of "language X does it this way" is what creates most of the difficulties in understanding Tcl.

I have no problem with procs, if statements, sequential down the page execution, etc. These are all very common programming idioms that transfer well from prior experience.

I recall first reading Brian Kernighan’s famous bell labs articles on the C language. His technique was to show how knowledge of other languages like fortran had counterparts in the C language. That is how I was able to learn C. Prior to that, I thought C was so strange, what with all those “APL” like operators.

The problem is I can’t transfer anything to Tcl in the area of substitution. Tcl seems to be unique here. That’s what makes tricky substitution impenetrable to me. When I have to, I just fire up a console and use trial and error.

And I’m afraid the average age of a Tcl programmer must be climbing; I know of no young people who want to learn tcl. Their languages of choice seem to be python, c/c++ java and ruby. According to github, tcl isn’t even on their top 20 list.

So, this is why I created my version of [set]. Then I could show a young person that tcl wasn’t so very different from other programming languages. Only then would they be receptive to the other great features of the language, like the tk toolkit, which I remind them is also part of python.

[heinrichmartin]

On Saturday, November 10, 2018 at 2:27:12 AM UTC+1, two...@gmail.com wrote:
> > Tcl with a viewpoint of "language X does it this way" is what creates most of the difficulties in understanding Tcl.
>
> I have no problem with procs, if statements, sequential down the page execution, etc. These are all very common programming idioms that transfer well from prior experience.

Now I understand that you are not looking for syntax, but for native expression support in Tcl. This in turn contradicts your liking of the general Tcl parser that hosts other small languages.

[expr] is nothing special in Tcl, not even [set] or [proc].

> The problem is I can’t transfer anything to Tcl in the area of substitution. Tcl seems to be unique here. That’s what makes tricky substitution impenetrable to me. When I have to, I just fire up a console and use trial and error.

Tcl's EIAS and substitutions are the comfort I like - a scripting language with amazing performance. Yes, there are a few pitfalls, but once I understood the underlying mechanisms, it was quite clear.

On top, you don't need printf a lot :-)

> So, this is why I created my version of [set]. Then I could show a young person that tcl wasn’t so very different from other programming languages. Only then would they be receptive to the other great features of the language, like the tk toolkit, which I remind them is also part of python.

Btw, you can use arbitrary whitespace, including line breaks, in expressions (expr, if, while), to help your eyes to apply the correct mini-language.

expect [~]expr {
> [string is integer -strict $a] &&
> 0 < $a
>}
1

[two...@gmail.com]

On Saturday, November 10, 2018 at 2:15:59 PM UTC-8, heinrichmartin wrote:
> Now I understand that you are not looking for syntax, but for native expression support in Tcl. This in turn contradicts your liking of the general Tcl parser that hosts other small languages.

I plead guilty of being of two minds here. I do like the power of tcl’s substitution (like a macro) while also recognizing that it can produce strange gotchas if you’re not really good at it. And I’m not.

Mainly I would like it if tcl had a cleaner expression mechanism. So, I dabble with ideas, like my [set] example above. I think [expr {…}] is just too noisy, and we probably could also use a string concatenation operator. I would like something like say,

set h “hello ”
SomeCommand –AnOption ($x + 1) -Another ($h ++ “world”)

Where the opening ( would be like an opening { but equivalent to [expr { and the closing ) would then be a closing }] and ++ could be a string concatenation operator.

I doubt if the actual ()’s could ever be made to work in tcl, but maybe if there were some other pair of left/right nestable bracketers as mentioned by florent above.

> [expr] is nothing special in Tcl, not even [set] or [proc].

I think [expr] is a bit special in tcl, since it has it’s own algebraic language all its own. And I might have this wrong, but doesn’t [expr] do it’s own substitution pass internally? Wasn’t that part of the point of the security issue mentioned previously.

[florent...@gmail.com]

Expr is special because it has his own parser.
If we could use () as a shorthand, the machinery is allready there. But we can't, because () are used by arrays.

That's why i was proposing to change the array syntax, in using some unicode caracter to denote the array element, and also to extend this sort of syntax with other unicode chars, to denote part of other kind of value (dict, list, string). Ex :

Set LIST‹($index*2) ⇿ ($index*2+1)› (list,$R*(cos,$a),$R*(sin,$a))

ie : The range between $index and $index+1 of the variable LIST should be set with the list of expressions ...
Equivalent to this actual code

set LIST [lreplace $LIST [expr {$index*2}] [expr {$index*2+1}] [list [expr {$R*cos($a)}] [expr {$R*sin($a)}]]]

[two...@gmail.com]

On Sunday, November 11, 2018 at 1:21:49 AM UTC-8, florent...@gmail.com wrote:
> Expr is special because it has his own parser.
> If we could use () as a shorthand, the machinery is allready there. But we can't, because () are used by arrays.

Intriguing, are you thinking of changing the parser or adding something analogous to the namespace machinery that would permit multiple parsers dynamically chosen?

Is the tcl bytecode compiler and interpretation engine independent of the parser, such that this would be possible?

But it sure would be something to leverage all the tcl/tk machinery but with a language facelift, AND do it without breaking all the existing tcl code. I can just see it,

language eval ::newtcl {
...
}

[heinrichmartin]

On Sunday, November 11, 2018 at 5:16:26 AM UTC+1, two...@gmail.com wrote:
> set h “hello ”
> SomeCommand –AnOption ($x + 1) -Another ($h ++ “world”)

Me too, I often liked extended index calculation (for lists or strings, which btw share the implementation of index parsing), e.g. [lindex $list $end-$start+1].

On a Tcl level, I suggest a proc to pre-process args, i.e.

proc ::args::expr args {
foreach var $args {
upvar 1 $var arg
set arg [uplevel 2 ::expr $arg]
}
}

proc SomeCommand {a b c d} {
# there is a TIP for named arguments (and implementations on the wiki)
::args::expr b
::args::cat d ;# tbd
# do it
}

SomeCommand –AnOption {$x + 1} -Another {$h ++ “world”)}

> > [expr] is nothing special in Tcl, not even [set] or [proc].
>
> I think [expr] is a bit special in tcl, since it has it’s own algebraic language all its own. And I might have this wrong, but doesn’t [expr] do it’s own substitution pass internally? Wasn’t that part of the point of the security issue mentioned previously.

It interprets its args like any other proc, including flow control and variable handling. You can even rename it and replace it with your own implementation.

Btw, I hesitated to prolong this thread, but I just had an encounter in Excel. The analogy would be: you are asking for formulae to support

=SUM(2:2-3:3) or even just
=2:2-3:3

instead of
=SUM(2:2)-SUM(3:3) ... sum of 2nd row minus sum of third row

[Christian Gollwitzer]

Am 12.11.18 um 21:48 schrieb heinrichmartin:

> On Sunday, November 11, 2018 at 5:16:26 AM UTC+1, two...@gmail.com wrote:
>>> [expr] is nothing special in Tcl, not even [set] or [proc].
>>
>> I think [expr] is a bit special in tcl, since it has it’s own algebraic language all its own. And I might have this wrong, but doesn’t [expr] do it’s own substitution pass internally? Wasn’t that part of the point of the security issue mentioned previously.
>
> It interprets its args like any other proc, including flow control and variable handling. You can even rename it and replace it with your own implementation.

Yes, you can write your own expr with its own rules, and some people did
that. Including myself - I wrote the "vexpr" command which is inspired
by the Matlab language and it supports setting variables and control
flow inside the expression. See here:

http://auriocus.github.io/VecTcl/tutorial.html

Array access is not currently supported, but the syntax could be
enhanced with ease. Another, very similar language is "xtal"

https://tarray.magicsplat.com/xtal_lang.html

It has more data oriented features, but less numeric functions.

Christian

[florent...@gmail.com]

Two.... wrote...

>Intriguing, are you thinking of changing the parser or adding something analogous to the namespace machinery that would permit multiple parsers dynamically chosen?

Well, there is the toplevel parser, the main tcl parser, that parse the lines, whose first word is taken as a command, the other words beeing taken as arguments. The commands are then free to parse their arguments beyond their needs. I was thinking about changing the main tcl parser.

> Is the tcl bytecode compiler and interpretation engine independent of the parser, such that this would be possible.

Yes bytecode compiler is independent.

> But it sure would be something to leverage all the tcl/tk machinery but with a language facelift, AND do it without breaking all the existing tcl code. I can just see it,

language eval ::newtcl {
...
}

A simple command is enough. If you don't want to break the existing code, you could imagine :

Oldversiontcl {
... code of the file...
}

This "Oldtcl" command would parse its only argument as it is parsed now.

[Andreas Leitgeb]

Christian Gollwitzer <auri...@gmx.de> wrote:
> Array access is not currently supported, but the syntax could be
> enhanced with ease.

How would vectcl do that?

(I'm wondering, if it would be easier in vectcl than in expr,
or whether it would be just as compromise-ridden)

[Donal K. Fellows]

On 09/11/2018 18:50, two...@gmail.com wrote:
> let a = sqrt(sin($x)**2 + cos($y)**2)

If you're happy to accept a little variation on your syntax, you can
have something close:

let a = {sqrt(sin($x)**2 + cos($y)**2)}

The braces inhibit base Tcl from substituting those variables so that
the [let] command can do what it wants with them. That's the Tcl Way.
Here's how you might implement a [let] command in that way with minimum
fuss:

proc let {varName = expression} {
if {${=} ni {= :=}} {
return -code error "syntax error"
}
upvar 1 $varName var
set var [uplevel 1 [list expr $expression]]
return
}

As you can see, it doesn't need to be complicated as long as you accept
the fundamental syntax restriction that forces you to put the braces in
(or deal with the consequences of that).

> One of tcl’s strengths is it’s ability to craft (as Brian Kernighan
> would say) “little languages”. Perhaps tcl could use something (if
> its not already there) to get control before a command line is
> tokenized.

Stick that awkward stuff in braces, then you've got the unparsed code
and can do anything you want with it. Requiring balanced braces turns
out to be an incredibly easy-to-handle restriction for most code. Insist
on being able to leave them out though, and you'll be fighting the
language itself in a system that says "ONE SET OF RULES FOR ALL" and
that's a way to get frustrated. (Tcl's own commands follow exactly the
same rules as all commands you make.)

[two...@gmail.com]

On Tuesday, November 13, 2018 at 11:59:33 AM UTC-8, Donal K. Fellows wrote:
> If you're happy to accept a little variation on your syntax, you can
> have something close

Thanks, that’s a pretty elegant solution. I’m going to save that in my toolbox. I’m coming around to needing that outer most set of braces here.

So, as long as we’re just brainstorming, what about the ( )’s that are used for arrays? Tcl appears to accept an array name of <null> i.e. the empty string, such that,


() 2 % set (name1) value1
value1
() 3 % set (name2) value2
value2
() 4 % set var $(name1)
value1
() 5 % array names {}
name1 name2
() 6 % array get {}
name1 value1 name2 value2

I’m not sure why anyone would actually need to use that for a variable name, and if they really wanted that, they could do this,

() 7 % set {(name1)} value1
value1
() 8 % set var $(name1)
value1

But if one exception was made (require {}s for the empty array name), then wouldn’t that free up ( )’s for another form of quoting, that could say, be only allowed if the ( were to start a word? Then one might be able to have more expressions, in many other places (without explicit use of [expr]) such as,

Command –option ($x + $y)

Where the ( )’s act like { }’s except they would really translate the above to

Command -option [expr {$x + $y}]

One could still use [expr] explicitly in say, “quoted [expr …]”, which provides something that few other languages have. But then tcl could also have a more standardized appearance for expressions.

Maybe we might have our cake and eat it too (:-)

[heinrichmartin]

On Wednesday, November 14, 2018 at 8:18:21 AM UTC+1, two...@gmail.com wrote:
> I’m coming around to needing that outer most set of braces here.

We were there ;-) "Wow, no sooner than someone helps me to create a more easy to read assignment statement than someone writes a tip which would break that. LOL"

> So, as long as we’re just brainstorming, what about the ( )’s that are used for arrays? Tcl appears to accept an array name of <null> i.e. the empty string, such that,

Honestly, I was sure it would require ${}(<index>) for the empty name.

Tcl accepts *any* name for a variable.

set \} foo
puts [set \}]

> () 2 % set (name1) value1
> value1

> () 7 % set {(name1)} value1
> value1

This makes no difference. Recall the dodekalogue. But I understand that it would make a difference when you introduce () for word boundaries. In which case (name1) results in "invalid bareword".

> But if one exception was made (require {}s for the empty array name), then wouldn’t that free up ( )’s for another form of quoting, that could say, be only allowed if the ( were to start a word? Then one might be able to have more expressions, in many other places (without explicit use of [expr]) such as,
>
> Command –option ($x + $y)

> Command -option [expr {$x + $y}]

You are mixing things up[1]: () was a problem inside [expr], not in Tcl. There were other reasons not to touch the Tcl syntax.

The way to go is a TIP and a reference implementation... I am not in the TCT and I do not track what is going on exactly, but I heard there are several TIPs to improve [expr]. So you might want to align your suggestion with those.

[1]
I just noticed this thread went from adding syntax to the language of expr ("Even just adding simple assignment
would add virtually all that effort. IIRC, the issue is to do with
determining whether something is a function call or an array assignment.") to implementing native expr support in the Tcl language. And I tend to like your idea now:

* ( is not yet given a special meaning as the first character of a word.
** This means it is free to use, great.
** This means it might break existing code. Not so good, but honestly, aren't these edge cases (to be fixed with "" quoting) - or error prone code like [Command –option ($x + $y)] with 5(!) words.

* You are suggesting a clear definition, great. My notes
** You are suggesting to translate it to a call to expr. What happens if expr was rewritten or removed?
** Compare: $varname does *not* translate to [set varname] so it is immune against re-writing.
** So you must choose: make expr a primus inter pares by raising its language to the Tcl syntax OR by relying on existence of [expr].
** The parser must generate code to invoke expr (or perform the calculation implicitly); it cannot do that while parsing (i.e. not before generating byte-code).

[Andreas Leitgeb]

heinrichmartin <martin....@frequentis.com> wrote:
> Honestly, I was sure it would require ${}(<index>) for the empty name.

There are two separate concepts in Tcl:
- a variable's name (*almost* arbitrary, except that seqences of
at least two consecutive colons have a special meaning, and
that var names that end in a ")" and do contain a "(" are
treated as array-elements)
- how $-substitution works. (Rule [8] of the dodekalogue)
Btw., ${...} is quite a hairy beast, and probably its only
good use is to separate a plain varname subst from an alpha
numeric literal, as in "${foo}bar". This trick is unnecessary
with arrays, as "$foo($x)bar" already works fine as is. Otoh,
using dollar-braces with arrays means that the index won't
undergo substitution: ${foo($x)} expects a key consisting
of literal "$" and "x".

In the case of ${}(<index>), the result would be the same as the
result of [string cat ${} (<index>)], not $(<index>).

[two...@gmail.com]

On Wednesday, November 14, 2018 at 6:39:18 AM UTC-8, heinrichmartin wrote:
>
> The way to go is a TIP and a reference implementation... I am not in the TCT and I do not track what is going on exactly, but I heard there are several TIPs to improve [expr]. So you might want to align your suggestion with those.
>

Thanks for your comments.

While I might be able to do a TIP (did one about 20 years ago), I doubt I would be able to provide an implementation. I did look as some code in the parser, and saw that there’s a char *src variable that moves down the text being parsed.

What I imagine doing would be to get to where it was processing the next word and implement some sort of “macro” replacement. There would still need to be a level count for nested ()’s and something done at the final ).

The way I’ve done macros in the past was based on to how Brian Kernighan implemented a “macro” processor in his famous Software Tools book (ratfor and pascal) where he had a pair of functions, getchar and putbackchar, along with a shared char array as a stack. But it has to be done everywhere input is scanned. I don’t see anything like that in tcl’s parser.

> ** You are suggesting to translate it to a call to expr. What happens if expr was rewritten or removed?

For sure, that would break what I had in mind. But how likely is it that [expr] would go away, and any changes probably wouldn’t matter to a fully braced input to [expr].

> ** Compare: $varname does *not* translate to [set varname] so it is immune against re-writing.

I didn’t know that. So, there’s special code to handle $. That might be what would be needed with my suggestion. That would be beyond my ability to code.

> ** The parser must generate code to invoke expr (or perform the calculation implicitly); it cannot do that while parsing (i.e. not before generating byte-code).

I’m not sure I understand this. If the source input was changed while scanning, like the above macro implementation, wouldn’t that work? I’m assuming there’s only the one place that the scanning is done, and so only one place would be needed to inject the [expr { and later seeing the final ) inject }] in its place.

But again, I don’t feel I could do this myself. So, I can’t tell if this is feasible or not. But it’s fun to think about.

[Christian Gollwitzer]

Am 13.11.18 um 18:20 schrieb Andreas Leitgeb:

Since vectcl is inspired by expression-oriented languages like Matlab or
Python, the expression inside the brackets of an array expression would
be evaluated and therefore, have a different meaning than in Tcl.
Ideally, the syntax would be

x = 5
A(x) = 3

means: set key 5 to 3 in the array A. Therefore, any wild literal key
wouldn't be much problematic, because it would need to be a string
expression (which also don't yet exist, but hey...)

set A(x) 3 ;# Tcl

would therefore be

A("x") = 3 # VecTcl

There is one drawback / compromise, though: to read an array, it would
be nice to do

y = A("hey") * 7

However, there wouldn't be a syntactic difference to function calls
then. Matlab does it that way, if you see a(3) you don't know if it is a
function call or an array index. And then it needs to be checked at
runtime if the function "a" exists, which prevents static compilation.
Therefore the compromise would be to introduce "$" - which is dropped or
optional for regular variables - for array, so

y = $A("hey") * 7 # array look up
y = A("hey") * 7 # function call

Christian

[Donal K. Fellows]

On 14/11/2018 07:18, two...@gmail.com wrote:
> I’m not sure why anyone would actually need to use that for a variable name

It's used by at least one package in Tcllib, stooop, which is one of the
many object systems that have been written for Tcl over the years. The
empty-named array is used by it for instance variables.


https://core.tcl.tk/tcllib/doc/trunk/embedded/www/tcllib/files/modules/stooop/stooop.html

As for why the syntax was allowed to start with? Probably because it was
easier to write the Tcl parser to allow it rather than forbid it.

[two...@gmail.com]

On Wednesday, November 14, 2018 at 11:36:42 AM UTC-8, Donal K. Fellows wrote:

> > I’m not sure why anyone would actually need to use that for a variable name
>
> It's used by at least one package in Tcllib, stooop, which is one of the
> many object systems that have been written for Tcl over the years. The
> empty-named array is used by it for instance variables.
>

Hmmm, interesting, I wonder how distasteful it would be to have a tcl_empty_array_name global variable that determined that behavior, default being to keep it the way it is. Or maybe another env variable.

It wouldn't be unprecedented, since there are a few tcl_* variables, like tcl_precision which affect all programs in a particular interpreter. But otherwise this would likely be a deal breaker for using ()'s as I suggested.

[florent...@gmail.com]

Donal fellows wrote :

> in a system that says "ONE SET OF RULES FOR ALL"

Except that parenthesis have several meanings :
- to denote a key for an array variable.
- to denote a subexpression in expr.
- to denote function arguments in expr.
- to denote a regular expression in regexp that is to be captured.

So, we can say : multiple rules for parenthesis.

Let's try a capturing regexp mixing array variable and expr... Quoting hell...


I would like if there would be "ONE SET OF RULES FOR ALL". But it's not totally true.

The variety of things to be coded implies that more significant symbol should be used. That's why i argue to use some unicode chats.

[Robert Heller]

At Wed, 14 Nov 2018 13:28:58 -0800 (PST) florent...@gmail.com wrote:

>
> Donal fellows wrote :
> > in a system that says "ONE SET OF RULES FOR ALL"
>
> Except that parenthesis have several meanings :
> - to denote a key for an array variable.
> - to denote a subexpression in expr.
> - to denote function arguments in expr.
> - to denote a regular expression in regexp that is to be captured.
>
> So, we can say : multiple rules for parenthesis.

Actually no. Both expr and regexp use completely different parsers from Tcl
proper. Tcl is a prefix language, just like LISP! regexp's parser is not
really part of Tcl, but is part of the regular expresion library. Expr uses a
*separate* in-fix parser for arithmetic/logical expressions.

>

> Let's try a capturing regexp mixing array variable and expr... Quoting
> hell...

I am not sure this even makes any sense. And I don't realy see any problem
(yes the '[' have to be quoted, but not the parens).

set mycharsets(alpha) {AaBbXxYyZz}
set mycharsets(nums) {1234590}
set pattern "^(\[$mycharsets(alpha)\]+)\[\[:space:\]\]+(\[$mycharsets(nums)\]+\)"
if {[regexp $pattern {abxz 1390} => a b] > 0} {
puts "a = $a, b = $b"

}

>
>

> I would like if there would be "ONE SET OF RULES FOR ALL". But it's not
> totally true.
>
> The variety of things to be coded implies that more significant symbol
> should be used. That's why i argue to use some unicode chats.
>
>
>

--
Robert Heller -- 978-544-6933
Deepwoods Software -- Custom Software Services
http://www.deepsoft.com/ -- Linux Administration Services
hel...@deepsoft.com -- Webhosting Services

[heinrichmartin]

On Wednesday, November 14, 2018 at 10:54:43 PM UTC+1, Robert Heller wrote:
> set mycharsets(alpha) {AaBbXxYyZz}
> set mycharsets(nums) {1234590}
> set pattern "^(\[$mycharsets(alpha)\]+)\[\[:space:\]\]+(\[$mycharsets(nums)\]+\)"

Looks like you proofed how ugly it is ;-) Escaping the last parenthesis is not wrong, but not needed either.

Generally, my eyes (and most of this thread was about readability) prefer regexps in braces:

set pattern [format {^([%s]+)\s+([%s]+)} $mycharsets(alpha) $mycharsets(nums)]

Btw, I am using a code style checker to enforce balanced pairs of ""{}[](). I know that this is not required in Tcl, but that's why I am not using string cat in such cases:

set pattern [string cat {^([} $mycharsets(alpha) {]+)\s+([} $mycharsets(nums) {]+)}]

[Robert Heller]

At Thu, 15 Nov 2018 00:45:02 -0800 (PST) heinrichmartin <martin....@frequentis.com> wrote:

>
> On Wednesday, November 14, 2018 at 10:54:43 PM UTC+1, Robert Heller wrote:
> > set mycharsets(alpha) {AaBbXxYyZz}
> > set mycharsets(nums) {1234590}
> > set pattern "^(\[$mycharsets(alpha)\]+)\[\[:space:\]\]+(\[$mycharsets(nums)\]+\)"
>
> Looks like you proofed how ugly it is ;-) Escaping the last parenthesis is
> not wrong, but not needed either.

But not really quoting hell. Compare to:

scp mail.deepsoft.com:'Mail/attachments/Some\ Stupid\ GUI\ File\ Name\ \[4\].docx' ./

Both quoted (with "'") AND escaped. If there is one thing Steve Jobs did that
*I* personally hate is the idea of a GUI file manager and making it SOP to use
spaces (and other random meta characters in file names...

>

> Generally, my eyes (and most of this thread was about readability) prefer
> regexps in braces:
>
> set pattern [format {^([%s]+)\s+([%s]+)} $mycharsets(alpha)
> $mycharsets(nums)]

Yes, that us another option.

>

> Btw, I am using a code style checker to enforce balanced pairs of ""{}[]().
I know that this is not required in Tcl, but that's why I am not using string
cat in such cases:

>
> set pattern [string cat {^([} $mycharsets(alpha) {]+)\s+([}
> $mycharsets(nums) {]+)}]

Yes, this is much more error prone.

[florent...@gmail.com]

Le mercredi 14 novembre 2018 22:54:43 UTC+1, Robert Heller a écrit :
> At Wed, 14 Nov 2018 13:28:58 -0800 (PST) florent...@gmail.com wrote:
>
> >
> > Donal fellows wrote :
> > > in a system that says "ONE SET OF RULES FOR ALL"
> >
> > Except that parenthesis have several meanings :
> > - to denote a key for an array variable.
> > - to denote a subexpression in expr.
> > - to denote function arguments in expr.
> > - to denote a regular expression in regexp that is to be captured.
> >
> > So, we can say : multiple rules for parenthesis.
>
> Actually no. Both expr and regexp use completely different parsers from Tcl
> proper. Tcl is a prefix language, just like LISP! regexp's parser is not
> really part of Tcl, but is part of the regular expresion library. Expr uses a
> *separate* in-fix parser for arithmetic/logical expressions.
>

It's surely true for regexp (Well, I never put my eyes into that part of the code), but not exactly for expr, which is using the Tcl_parseVariable machinery.

So we can say : No rules are shared by regexp, whereas allmost all the generic rules are shared by expr (Brace, quote, variable, bracket, backslash) - with the exception of 1° the expand syntax, that will not work fine, 2° the comment syntax, that is not handled, and 3° the space, that are not word spearator.

> >
>
> > Let's try a capturing regexp mixing array variable and expr... Quoting
> > hell...
>
> I am not sure this even makes any sense. And I don't realy see any problem
> (yes the '[' have to be quoted, but not the parens).
>
> set mycharsets(alpha) {AaBbXxYyZz}
> set mycharsets(nums) {1234590}
> set pattern "^(\[$mycharsets(alpha)\]+)\[\[:space:\]\]+(\[$mycharsets(nums)\]+\)"
> if {[regexp $pattern {abxz 1390} => a b] > 0} {
> puts "a = $a, b = $b"
> }
>

Sorry I made a mistake here. In regexp, the problem doesn't appear. We can't use the Tcl_Variable machinery inside, so the parenthesis are not a problem in this case : we have to build the regular expression before applying it. The problem of parenthesis arised into expr when it was imagined to implement variable assignation in it.

No one of the generic rules of Tcl can be apply to regexp.

Let's imagine, now, we want to be able to use those generics rules inside the regexp parser, to be able to write a regular expression using those generic rules :

regexp {^([$mycharsets(alpha)]+)[[:space:]]+([$mycharsets(nums)]+)} ...

There appears then ambiguities :
$ denote the end of line in regexp case, but a variable in Tcl case
[] denote a list of char to be match in regexp case, but command substitution in Tcl case.

Let's use the principles I proposed : One set of generic rules, those of the toplevel Tcl parser, used by every command, and sets of particular rules, used by particular commands that parses their argument (especially expr and regexp). Using unicode char to increase the amount of symboles

Denote a dict -> $D«key»
Denote an array -> $A“key”
Element of a list -> $L‹$index›
char of a string -> $S‘$index’
range of index : $L‹$i → $j›
...etc
Remove the ambiguities of bracket in regexp ->
Substituting [] by ⟦⟧ (\u27E6 \u27E7)
Deprecating the dollars sign as end of line

regexp {(⟦$mycharsets“alpha”⟧+)⟦⟦:space:⟧⟧+(⟦$mycharsets“nums”⟧+)} ...

OR

set keywords {incr dict expr set proc info ...etc}; # list of Tcl commands

regexp {([join $keywords "|"])} ...

Then we could use all the Tcl machinery into regexp.

[heinrichmartin]

On Saturday, November 17, 2018 at 2:18:31 PM UTC+1, florent...@gmail.com wrote:
> Let's use the principles I proposed : One set of generic rules, those of the toplevel Tcl parser, used by every command, and sets of particular rules, used by particular commands that parses their argument (especially expr and regexp). Using unicode char to increase the amount of symboles
>
> Denote a dict -> $D«key»
> Denote an array -> $A“key”
> Element of a list -> $L‹$index›
> char of a string -> $S‘$index’
> range of index : $L‹$i → $j›
> ...etc
> Remove the ambiguities of bracket in regexp ->
> Substituting [] by ⟦⟧ (\u27E6 \u27E7)
> Deprecating the dollars sign as end of line

I am unsure whether it was said earlier in this thread: I am strictly against syntax characters that are not available on a standard keyboard.

> regexp {(⟦$mycharsets“alpha”⟧+)⟦⟦:space:⟧⟧+(⟦$mycharsets“nums”⟧+)} ...

And imo, redefining well-known syntax is a bad idea.

> regexp {([join $keywords "|"])} ...

We had this already: the only problem is knowing the context of what you are reading. You want

regexp [join $subREs "|"] ... ;# so simple

or maybe

regexp [join [lmap keyword $keywords {regexpEscape $keyword}] "|"] ...

which should actually be (at least in production code) something like

# define & init
var usecaseRE
# apply or proc init to not pollute the namespace with temporary variables
apply {{} {
set usecaseKeywords [retrieveListOfKeywords]
set usecaseSubREs [lmap keyword $keywords {regexpEscape $keyword}]
var usecaseRE [join $usecaseSubREs "|"]
}}
# ...
# use it
regexp $usecaseRE ...

Tcl is mighty enough to handle that. It is not in a contest for shortest code. And I am sure your suggestion does not help readability.

I was straight - hopefully not offending - with my opinion. Nothing else intended.

[Donal K. Fellows]

On 17/11/2018 13:18, florent...@gmail.com wrote:
> It's surely true for regexp (Well, I never put my eyes into that part
> of the code), but not exactly for expr, which is using the

> Tcl_ParseVariable machinery.
But that's the category error: the syntax of expressions shares the
function because that sublanguage section is coincident, not the other
way round. There are other languages inside Tcl too; they share some
elements for familiarity, but are very much not the same. Some packages
add yet more (e.g., Critcl adds the C language!)

There's not going to be a lot of support for your proposed changes in
general, FWIW: they're simply going to be too awkward to type for most
programmers to want. It sounds like a silly concern, but it is a very
real one. You can write your own commands that parse their arguments in
the way you propose — that's totally supported — but don't expect others
to like it a whole lot.

[florent...@gmail.com]

Le samedi 17 novembre 2018 21:10:49 UTC+1, Donal K. Fellows a écrit :
... that sublanguage section is coincident

Very well formulated. I can reformulate my idea this way : Avoid coincidence between toplevel langage and sublangages. (Critcl is a bit special)

> There's not going to be a lot of support for your proposed changes in
> general, FWIW: they're simply going to be too awkward to type for most
> programmers to want. It sounds like a silly concern, but it is a very
> real one. You can write your own commands that parse their arguments in
> the way you propose — that's totally supported — but don't expect others
> to like it a whole lot.
>
> Donal.
> --
> Donal Fellows — Tcl user, Tcl maintainer, TIP editor.

Of course, It's never possible to force anybody to like what he's dislike.

My problem is to understand how Tcl can handle unicode chars as significant symbols. I saw in the source code a 384 component array [const char tclCharTypeTable) that classifies chars, according to their type (TYPE_NORMAL, TYPE_SPACE,... etc), so it's char or unside chars, used by the macro CHAR_TYPE of tclParse.h.

This kind of table for classifying unicode chars is not possible I think.