Tcl 8's expr: does anybody think it's acceptable?
Bryan Oakley
Just out of curiosity, is there anyone who thinks sacrificing the
fundamental parsing rules of tcl in favor of more speed in the compiler
is a Good Thing (tm), or are at the very least ambivalent? I don't think
I've seen any posts outside of Sun that favor the new expr (though my
news feed really sucks -- I probably only see half of what is posted).
I'd be interested in hearing some arguments for the change that come
from outside of Sun.
--
Bryan "Just Curious" Oakley mailto:oak...@healthcare.com
Software Engineer http://www1.clearlight.com/~oakley/
Healthcare Communications, Inc. http://www.healthcare.com/
Larry V. Streepy, Jr.
Bryan Oakley wrote:
>
> Just out of curiosity, is there anyone who thinks sacrificing the
> fundamental parsing rules of tcl in favor of more speed in the compiler
> is a Good Thing (tm), or are at the very least ambivalent? I don't think
> I've seen any posts outside of Sun that favor the new expr (though my
> news feed really sucks -- I probably only see half of what is posted).
> I'd be interested in hearing some arguments for the change that come
> from outside of Sun.
You may think we collaborated on this posting, but we didn't :-)
My first reaction after reading about the new internal compiler was "ALL
RIGHT"! Then I read the specifics in the release notes, and after
reading about the changes to "expr", my reacton was "oh shit". I
immediately thought of several places in our code that will break
horribly. This means additional development effort to move to the new
release!
I agree with Bryan's assessment 100%. You've changed the fundamentals
of the language. IMO, it would be more reasonable to either provide a
compatibility mode or implement the new expr processing as a new
command.
On the compatibility front, maybe provide a global variable that the
programmer can set which controls the compilation characteristics of the
exper command. You already have such a concept with regard to some
floating point operations, I think.
Or, provide "fast_expr" as a new command which has the fast compilation
characteristics and leave expr alone.
Personally, I prefer the compatibility option. It allows existing
applications to rapidly take advantage of the compiler without having to
go through a lengthy debug cycle to find all the expr's that break.
Then, when they want to squeeze every last drop of performance out of
their code, they enable the fast expr option and requal their
application.
--
Larry V. Streepy, Jr.
Chief Technical Officer, Healthcare Communications, Inc.
mailto:str...@healthcare.com
(214) 851-7033 (Dallas Main #)
(970) 626-5028 (My office #) (970) 626-4425 (Fax)
Dale.Parson
In article <32CD4C4D...@healthcare.com>,
Larry V. Streepy, Jr. <str...@healthcare.com> wrote:
>Bryan Oakley wrote:
>>
>> Just out of curiosity, is there anyone who thinks sacrificing the
>> fundamental parsing rules of tcl in favor of more speed in the compiler
>> is a Good Thing (tm), or are at the very least ambivalent? I don't think
>> I've seen any posts outside of Sun that favor the new expr (though my
>> news feed really sucks -- I probably only see half of what is posted).
>> I'd be interested in hearing some arguments for the change that come
>> from outside of Sun.
>
>You may think we collaborated on this posting, but we didn't :-)
>
>My first reaction after reading about the new internal compiler was "ALL
>RIGHT"! Then I read the specifics in the release notes, and after
>reading about the changes to "expr", my reacton was "oh shit". I
>immediately thought of several places in our code that will break
>horribly. This means additional development effort to move to the new
>release!
Perhaps I should exercise the same small degree of conscience that I
exercise the first Tuesday of November each year (election day for those
of you not rooted in the U.S.), and cast my vote of conscience on this
issue. This is meant only as constructive observation from the
perspective of the Bell Labs software team with which I work. We have
been using Tcl/Tk to gain very great leverage. I don't track comp.lang.tcl
threads & didn't follow the compiler plans closely. I am intimately familiar
with intermediate code compilation and related issues, so this isn't from
too far in left field.
Everything is a string.
Every string can be eval'ed with a simple, syntactically-minimal parse rule.
The first resolved token is a function name.
The minimal rules that new project members need to learn in order to form
valid Tcl programs has greatly accelerated useful acquisition of this
language compared with other languages that are NEW in the sense that
they must be newly acquired by project members. After grokking
minimal syntactic rules, acquisition of Tcl is incremental acquisition of
its library.
Furthermore it is hard to even guess the practical extent of problems
to be caused by special-case parsing. One commits a "fundamental" to one's
basic mental model of a programming language. When a "fundamental,"
like an axiom in a logical system, is found to be violated, the integrity
of the system is thrown into doubt.
The ease with which Tcl can be accurately acquired and applied to appropriate
subsets of our problem spaces has always been more important than run-time
performance. Acquisition speed and minimal maintenance time are more
important than run-time speed. Again this is not to be ungrateful for
compiler work already done, but to point out that there are other, greater
features about Tcl for which we are very grateful, and we would hate to see
them weakened for a feature we don't really need.
>
>I agree with Bryan's assessment 100%. You've changed the fundamentals
>of the language. IMO, it would be more reasonable to either provide a
>compatibility mode or implement the new expr processing as a new
>command.
>
>On the compatibility front, maybe provide a global variable that the
>programmer can set which controls the compilation characteristics of the
>exper command. You already have such a concept with regard to some
>floating point operations, I think.
I disagree that "moded" Tcl is a satisfactory way out of this bind.
A piece of code ought to mean something without carrying mode flags
in one's head.
>
>Or, provide "fast_expr" as a new command which has the fast compilation
>characteristics and leave expr alone.
Yes, an intrinsically cleaner alternative. Expr is only 1 of many functions.
Rather than force programmers with existing programs to consider every "expr,"
implement "fast_expr" and give them the choice of when & where to look
at "expr"s, not for mutant problems, but for potential replacement with
"fast_expr." When a profiler appears, IT can find hot "expr"s that are
candidates for hand-"fast_expr"ization.
The quest for performance is addictive.
Sometimes speed kills.
Tcl is fast approaching a discontinuity.
Time to slow down and pay attention.
Dale Parson, Bell Labs, dpa...@lucent.com
----------------------------------------------------------------------
| "These words are too solid, they don't move fast enough |
| to catch the blur in the brain that flies by, and is gone..." |
----------------------------------------------------------------------
| Suzanne Vega |
----------------------------------------------------------------------
Robin Becker
I have to agree with the language purists on this issue; however it's
not only a question of expr. The introduction of the two phase
compilation/execution has broken catch in a nontrivial way for many
commands.
Tryout
catch {if}
syntax error
eval catch {if}
syntax error
and under Win32 most devastating is
set z if
catch {$z}
which causes an exception (stack overflow), but this must be a bug.
Although
eval catch {$z}
results in the expected 1
This kind of behavour will cause problems for server style apps which
allow execution of client scripts since the server cannot guarantee the
code it will be executing. So effectively all code which originally read
catch {$x} must be changed.
Amusingly
set z "catch { if } msg1"
eval catch {$z} msg2
results in 0 as does
set z "catch { if } msg1"
catch "$z" msg2
even though the inner expression $z is syntactically incorrect.
--
Robin Becker
Joe English
My $.02 on the 'expr' issue:
I don't think the change is a good idea either,
but only because it introduces subtle, hard-to-track
down changes to the meaning of existing code.
That the new 'expr' parsing rules are different
from the rest of the language does not bother
me as much, because Tcl expressions are *currently*
parsed differently than everything else.
(I would prefer a collection of Lisp-style arithmetic
commands that could be used in place of 'expr'; it seems
to me that
if [<= [+ [* $x $x] [* $y $y]] $epsilon] { ... }
would be much more consistent with the rest of Tcl than
if {$x * $x + $y * $y <= $epsilon} { ... },
although I understand users' resistance to prefix
notation for arithmetic expressions...)
Better compile-time error checking would make the 'expr'
change more palatable, but since it's still possible
even with the new compiler to get syntax errors at
runtime, changing the syntax and semantics of 'if',
'while' and 'expr' is definitely a bad idea wrt
existing code.
--Joe English
Hume Smith
> That the new 'expr' parsing rules are different
> from the rest of the language does not bother
> me as much, because Tcl expressions are *currently*
> parsed differently than everything else.
no, you're confused; expression arguments are currently parsed and substituted
exactly the same as everything else. it's just that once expr gets ahold of
them, its processing does things that happen to look very like what the
evaluator does. you could think of it as another of Tcl's lexical collisions,
like the one it has with regular expressions.
> (I would prefer a collection of Lisp-style arithmetic
> commands that could be used in place of 'expr'; it seems
> to me that
>
> if [<= [+ [* $x $x] [* $y $y]] $epsilon] { ... }
>
> would be much more consistent with the rest of Tcl than
>
> if {$x * $x + $y * $y <= $epsilon} { ... },
uhh, yes it would be more consistent; but as much as i love Lisp and
Postscript i deSPISE trying to debug arithmetic in the things :)
> although I understand users' resistance to prefix
> notation for arithmetic expressions...)
no kidding. Asyst (a Forth/Postcript-style language for controlling
digital instruments and processing their output) had a command that
supported infix notation in addition to the usual stack commands,
just because prefix is so fugging uckly to work with.
--
Hume Smith <URL:mailto:hcls...@tallships.istar.ca>
<URL:http://dess.tallships.istar.ca/%7Ehclsmith/>
Dale.Parson
In article <5aq25l$2...@news.istar.ca>,
Hume Smith <hcls...@tallships.istar.ca> wrote:
>In article <5ap85t$6...@crl13.crl.com> jeng...@crl.com (Joe English) writes:
>
>> That the new 'expr' parsing rules are different
>> from the rest of the language does not bother
>> me as much, because Tcl expressions are *currently*
>> parsed differently than everything else.
>
>no, you're confused; expression arguments are currently parsed and substituted
>exactly the same as everything else. it's just that once expr gets ahold of
>them, its processing does things that happen to look very like what the
>evaluator does. you could think of it as another of Tcl's lexical collisions,
>like the one it has with regular expressions.
>
I posted 1 reservation to 8.0's expr this weekend, but when I got back to
work this morning I realized I had another, bigger reservation.
Our group at Bell Labs is using Tcl to do a processor-neutral assembly language
and C debugger. One of the functions we add for assembly debugging is "fxpr,"
which originally stood for fixed-point-expr (we mostly work with fixed-point
DSPs). We have since "lifted" fxpr so it consults its target processor
architecture to determine arithmetic semantics. Fxpr now does fixed-point
arithmetic for fixed-point DSPs, floating-point for floating point micro-
controllers, etc.
While fxpr has processor-specified semantics, it uses a superset (only minor
additions) to the syntax of expr. Furthermore, of course, fxpr is not a
"special form" in any sense. Its implementation in fact YACCs its string
arguments, then evaluates a parse tree in a processor-specified manner.
I don't know whether anybody else has copped expr syntax to do comparable
things, but now at a minimum I have to look a redefining fxpr arg syntax, and
at whether I can keep the two syntactically compatible at all.
This may not be a COMMON use of expr, but it points out the leverage gained
from the generality of Tcl, and the potential dangers of special cases.
Hume Smith
> >no, you're confused; expression arguments are currently parsed and substituted
> >exactly the same as everything else. it's just that once expr gets ahold of
> >them, its processing does things that happen to look very like what the
> >evaluator does. you could think of it as another of Tcl's lexical collisions,
> >like the one it has with regular expressions.
> >
> I don't know whether anybody else has copped expr syntax to do comparable
> things, but now at a minimum I have to look a redefining fxpr arg syntax, and
> at whether I can keep the two syntactically compatible at all.
i think i was unclear - there's no change to expressions lexically,
syntactically, or semantically as far as i know. the change is in
the evaluator; expression arguments are now special cases that don't
get eval substitutions. what i meant was, the actions of expr and
subst look so similar that the poster to whom i was responding had
mistakenly thought expr was already a special parsing case.
i don't think you'll have to change your fxpr at all. if you want it
to "look" *exactly* as expr is, though, you'll have to hack the evaluator
to make fxpr a special case.
Brian Lewis
Robin Becker (ro...@jessikat.demon.co.uk) wrote:
: I have to agree with the language purists on this issue; however it's
: not only a question of expr. The introduction of the two phase
: compilation/execution has broken catch in a nontrivial way for many
: commands.
: syntax error
Robin,
This was discussed in an earlier thread and I've added some paragraphs to
the http://www.sunlabs.com:80/research/tcl/compiler.html web page.
Having bugs found early is something that many people like. I've seen
at least two reports that people have found bugs in code that they
have been running for some time; they had been lucky so far, and had never
executed the bad code.
If you want catch to handle compile time errors, give it a script that is only
determined at runtime: for example,
set x {expr garbage}; catch $x
: and under Win32 most devastating is
: set z if
: catch {$z}
: which causes an exception (stack overflow), but this must be a bug.
This wasn't a problem with catch. Just "$z" by itself would provoke a
stack overflow on all platforms. This was an interesting and subtle bug!
Thanks for the report.
Brian
Bryan Oakley
Joe English wrote:
>
> My $.02 on the 'expr' issue:
>
> I don't think the change is a good idea either,
> but only because it introduces subtle, hard-to-track
> down changes to the meaning of existing code.
> from the rest of the language does not bother
> me as much, because Tcl expressions are *currently*
> parsed differently than everything else.
No. That is incorrect. Tcl expressions are parsed *exactly* as every
other tcl command. It's just that the expr command handles the data it
gets differently. The expr command itself does some parsing or
translation or substitutions or whatever you want to call it. But, when
expr is called it gets the exact same data as any other command. With
tcl8.0, presently, this is not true.
--
Bryan Oakley mailto:oak...@healthcare.com