TIP #90: Enable [return -code] in Control Structure Procs
Don Porter
TIP #90: ENABLE [RETURN -CODE] IN CONTROL STRUCTURE PROCS
===========================================================
Version: $Revision: 1.1 $
Author: Don Porter <dgp_at_users.sf.net>
State: Draft
Type: Project
Tcl-Version: 8.4
Vote: Pending
Created: Friday, 15 March 2002
URL: http://purl.org/tcl/tip/90.html
WebEdit: http://purl.org/tcl/tip/edit/90
Post-History:
-------------------------------------------------------------------------
ABSTRACT
==========
This TIP analyzes existing limitations on the coding of control
structure commands as _proc_s, and presents expanded forms of _catch_
and _return_ to remove those limitations.
BACKGROUND
============
It is a distinguishing feature of Tcl that everything is a command,
including control structure functionality that in many other languages
are part of the language itself, such as _if_, _for_, and _switch_. The
command interface of Tcl, including both a return code and a result,
allows extensions to create their own control structure commands.
Control structure commands have the feature that one or more of their
arguments is a script, often called a _body_, meant to be evaluated in
the caller's context. The control structure command exists to control
whether, when, in what context, or how many times that script is
evaluated. When the body is evaluated, however, it is intended to
behave as if it were interpreted directly in the place of the control
structure command.
The built-in commands of Tcl provide the ability for scripts themselves
to define new commands. Notably, the _proc_ command makes this
possible. In addition, other commands such as _catch_, _return_,
_uplevel_, and _upvar_ offer enough control and access to the caller's
context that it is possible to create new control structure commands
for Tcl, entirely at the script level.
Almost.
There is one limitation that separates control structure commands
created by _proc_ from those created in C by a direct call to
_Tcl_Create(Obj)Command_. It is most easily seen in the following
example that compares the built-in command _while_ to the command
_control::do_ created by _proc_ in the control package of tcllib.
% package require control
% proc a {} {while 1 {return -code error}}
% proc b {} {control::do {return -code error} while 1}
% catch a
1
% catch b
0
The control structure command _control::do_ fails to evaluate _return
-code error_ in such a way that it acts the same as if _return -code
error_ was evaluated directly within proc _b_.
ANALYSIS
==========
There are two deficiencies in Tcl's built-in commands that lead to this
incapacity in control structure commands defined by _proc_.
First, _catch_ is not able to capture the information. Consider:
% set code [catch {
return -code error -errorinfo foo -errorcode bar baz
} message]
After evaluation, _code_ contains "2" (_TCL_RETURN_), _message_
contains "baz", _::errorInfo_ contains "foo", and _::errorCode_
contains "bar". But there's is nothing accessible at the script level
that contains the value passed to the _-code_ option. That code is
stored internally in the _Tcl_Interp_ structure as
_interp->returnCode_, but scripts cannot access that value.
Second, even if the information were available, there is no built-in
command in Tcl that can be evaluated within the body of a proc to make
the proc itself act as if it were the command _return -code_. Stated
another way, it is not possible to create a command with _proc_ that
behaves exactly the same as _return -code_. Because of that, it is also
not possible to create a command with _proc_ that behaves exactly the
same as _while_, _if_, etc. - any command that evaluates any of its
arguments as a script in the caller's context.
This is a curious, and likely unintentional, limitation. Tcl goes to
great lengths to be sure I can create my own _break_ replacement with
_proc_.
proc myBreak {} {return -code break}
It would be a welcome completion of Tcl's built-in commands to be able
to create a replacement for every one of them using _proc_.
ALTERNATIVES
==============
For now, I will present a few alternatives, without proposing any one
of them in particular, to elicit comments. Later revisions of this
Draft TIP will select and propose one of the alternatives on the basis
of community discussion.
1. Do nothing.
We can maintain the status quo, and just accept that it is not
possible to create fully functional control structure commands
with _proc_. If you want to create such a command, it must be
coded in C.
Note that this limitation is one of the main reasons that [TIP
#89] proposes to add a _try_ command directly to Tcl, rather than
implementing it in a package. This limitation forces _try_ to be
coded in C, and then the lack of good distribution mechanisms for
C-coded packages motivates inclusion in Tcl itself. If we choose
to do nothing, we can expect more proposals to fill Tcl with more
control structure commands.
2. Extend _catch_
Dealing with the deficiency in _catch_ is not difficult, and may
be desirable even if we choose not to correct the other
deficiency. An optional additional argument to _catch_ can be a
variable name in which to store the _interp->returnCode_ value
that is currently not accessible. Syntax would be:
catch script ?msgVarName? ?returnCodeVarName?
Only when _catch_ is returning 2 (_TCL_RETURN_), and when the
_returnCodeVarName_ argument is provided, then _catch_ would
store the value of _interp->returnCode_ in the variable named
_$returnCodeVarName_.
3. Add a scalar-valued switch/argument to _return -code return_.
This would be in addition to alternative 2.
The built-in commands of Tcl enable _proc_ to create a
replacement for the _return_ command with no arguments or only
the result argument.
proc myReturn {{message {}}} {return -code return $message}
The _myReturn_ example could be further extended to silently
ignore any _-errorinfo_ or _-errorcode_ switches, since they have
no relevance when not returning an error. It is the _-code_
switch that _myReturn_ cannot emulate.
Since _return -code return_ solves most of the problem, we can
consider extending its syntax to solve the rest. Perhaps an
additional _-returncode_ switch:
return -code return -returncode "returnCode" ...
Or perhaps an additional optional argument:
return -code return ... ?"result"? ?"returnCode"?
An additional field within the _Tcl_Interp_ structure would need
to be added for storage of the value of the _returnCode_
argument, so that that value could be transferred into the
_interp->returnCode_ field by the caller.
Advantage: it's a simple change.
Disadvantage: it just pushes the incompleteness problem one down
level.
After this change, commands defined by _proc_ would be able to
behave exactly the same as the command _return -code $code_.
However, then commands created by _proc_ would not be able to
behave exactly as the command _return -code return -returncode
$code_. Said another way, this change will enable access to the
_returnCode_ field of the _Tcl_Interp_ structure, but does so
only by creating a new field in the _Tcl_Interp_ structure that
we do not have access to. So there would still be a fundamental
incompatibility between commands created by _proc_ and those
coded in C. But perhaps that change is enough to reduce the
incompatibility to the point that it is no longer important.
4. Add a list-valued switch/argument to _return -code return_.
The incompleteness remaining after the combination of
alternatives 2 and 3 could be solved by allowing the value of
_returnCode_ to be a list in both the extended _catch_ and the
extended _return -code return_. For example, after evaluation of:
set code [catch {
return -code return -returncodes $oldReturnCodes baz
} message returnCodes]
_code_ contains "2" (_TCL_RETURN_), _message_ contains "baz", and
_returnCodes_ contains [linsert $oldReturnCodes 0 2]. For other
arguments to _-code_:
set code [catch {
return -code $value ... baz
} message returnCodes]
_returnCodes_ contains [list $value]. The additional field of the
_Tcl_Interp_ structure would hold the list, and the caller would
pop the first element of that list and place it in the
_returnCode_ field of the _Tcl_Interp_ structure when handling a
_return -code return_ situation.
I believe this could work, but I'll need to prototype it to see
if and where it might lead to trouble.
Also, it appears that the first N-1 elements of the _returnCodes_
list might always be _TCL_RETURN_. In that case, a simple pair of
the last return code and the number of levels deep would be a
simpler way to describe the required information.
Advantage: appears to be a complete solution.
Disadvantage: still have to figure it out.
5. Add a new return code, _TCL_EVAL_.
The problem that one cannot cause a _proc_ to behave as if it
were the command _return -code_ can be viewed as a special case
of the general weakness that one cannot make a _proc_ behave as
if it were replaced by an arbitrary Tcl command. This general
power could be granted via a new return code, _TCL_EVAL_.
When detecting a _TCL_EVAL_ return code from a command, the
result of the command would be treated as a new command to be
evaluated in the same context as the first. The result and return
code of the new command would take the place of the original.
With that capability, combined with alternative 2, one could
easily take care of the problem posed in the analysis above:
set code [catch {uplevel 1 $body} message returnCode]
if {$code == 2} {
return -code eval [list return -code $returnCode $message]
}
Advantage: A complete solution that's easy to understand.
Disadvantage: Very powerful technique that might enable things we
don't want to enable.
SEE ALSO
==========
Documentation for tcllib's control package:
<URL:http://tcllib.sf.net/doc/control.html>
COPYRIGHT
===========
This document has been placed in the public domain.
-------------------------------------------------------------------------
TIP AutoGenerator - written by Donal K. Fellows
[[Send Tcl/Tk announcements to tcl-an...@mitchell.org
Send administrivia to tcl-announ...@mitchell.org
Announcements archived at http://groups.yahoo.com/group/tcl_announce/
The primary Tcl/Tk archive is ftp://ftp.neosoft.com/pub/tcl/ ]]
Don Porter
> TIP #90: ENABLE [RETURN -CODE] IN CONTROL STRUCTURE PROCS
> URL: http://purl.org/tcl/tip/90.html
Please see the URL for substantial updates to this proposal.
--
| Don Porter Mathematical and Computational Sciences Division |
| donald...@nist.gov Information Technology Laboratory |
| http://math.nist.gov/~DPorter/ NIST |
|______________________________________________________________________|
Andreas Leitgeb
> Don Porter wrote:
>> TIP #90: ENABLE [RETURN -CODE] IN CONTROL STRUCTURE PROCS
> ...
>> URL: http://purl.org/tcl/tip/90.html
>
> Please see the URL for substantial updates to this proposal.
First, I just want to contradict to your claim, that the problem solved by
this tip is "one of the main reasons that TIP #89 proposes to add a try
command directly to Tcl, ..."
The try keyword should be in the core mainly for performance-reasons,
and to give developers a cosier feeling for making use of it.
Second I want to think through the principials of
generalizing the deferral of return-codes:
If return now allows me to create a procedure that behaves
like return -code <whatever> ... , then it would also have
to allow me to create a procedure that makes its caller behave
like it. This goes arbitrary far.
But what would be stacked up is only "return"s and one specified
code, so instead of a list, we only had to carry on an integer
to show how many levels to defer the specified return code.
The command
return -code break -defer 5
would cause the fifth caller to behave like break, (just returning
from all the levels between) and as special cases:
return -code break -defer 0
would be identical to the break-command itself, and
return -code break -defer 1
would be the same as the current return -code break
Now for the other part, namely catch:
What is catch supposed to do, if it is hit by e.g. a
return -code break -defer 1
Without the catch, the function eval'ing or uplevel'ing the snippet,
would itself behave as if it were the break command. But is this
what the programmer, who put in the deferring return, wanted ?
Most likely not.
Lets assume, that the piece of code that contained the down-levelled
return was the body to a userdefined control-structure "forall", and
the programmer intended to let the procedure calling forall ... {...}
behave like the break command, if some condition inside the forall-loop
turned out true.
Now, inside forall, catch notices the deferred-break-return and what
shall it do now ?
The original deferral was chosen for the level in which forall was
called, not the level catch caught the return. There may be many levels
between.
Thus, at least, uplevel would have to re-adjust the defer-level by
the number of levels that it went up to reach the requested level.
Afterall, we shouldn't rely on the presence of a catch, when we talk
about correctly unrolling the correct number of levels.
catch will require an extra feature to report the defer-level,
so the same exception can be re-thrown later with the same
(or perhaps re-adjusted) defer-level.
What is catch supposed to return for a deferred code ?
For compatibility's sake it would have to be 2 (return),
as that is what was happening before, but then, if
a "catch {return -code whatever}" happened before, it was
bogus anyway.
So we could just as well choose something more sensible:
e.g. -1 or even "-<deferred returncode>" since 0 is not
deferrable, anyway, is it ? if we make it the latter, we
can more easily re-throw errors with the appropriate error-
options without passing these also to breaks and continues.
This may then lead to procedural control-structure that are
really indistinguishable from the builtins. (except for speed :-)
PS: On the other hand, clumsy eval & exec behaviour has bitten me
much harder in the past than this particular problem.
--
Newsflash: Sproingy made it to the ground !
read more ... <http://avl.enemy.org/sproingy/>
Harald Kirsch
> TIP #90: ENABLE [RETURN -CODE] IN CONTROL STRUCTURE PROCS
> ===========================================================
> ABSTRACT
> ==========
>
> This TIP analyzes existing limitations on the coding of control
> structure commands as _proc_s, and presents expanded forms of _catch_
> and _return_ to remove those limitations.
>
> ANALYSIS
> ==========
>
> There are two deficiencies in Tcl's built-in commands that lead to this
[deficiency of catch snipped]
> Second, even if the information were available, there is no built-in
> command in Tcl that can be evaluated within the body of a proc to make
> the proc itself act as if it were the command _return -code_. Stated
> another way, it is not possible to create a command with _proc_ that
> behaves exactly the same as _return -code_. Because of that, it is also
> not possible to create a command with _proc_ that behaves exactly the
> same as _while_, _if_, etc. - any command that evaluates any of its
> arguments as a script in the caller's context.
> ALTERNATIVES
> ==============
May I add a 5th one which would not only take care of the above
deficiency but would in general make it much easier to write new
control structures. Two observations support my argument:
a) The reason for the above mentioned deficiency is that the argument
to -code is always lost while *returning* from the proc.
b) Control structures are always expected to run in the scope of the
caller. On the contrary, proc always creates a new scope leading to
all kinds of hassles with upvar and uplevel.
Consequently I wonder if a proc-like thing (call it macro) would help
which, when called, does not create a new scope. Consider for example
macro loop {count body} {
while {count>0} {
eval $body ;# NOTE: for macro we don't need [uplevel]
incr count -1
}
}
When calling it like
loop 10 {return -code $someCode}
would the return work as proposed by this TIP when loop (as a macro)
does not constitute a new stack frame?
(Another question is of course what variable $count within the body is
supposed to see.)
Harald Kirsch.
--
----------------+------------------------------------------------------
Harald Kirsch | kirschh bei lionbioscience punkt com
*** Please do not send me copies of your posts. ***
miguel sofer
>
> Consequently I wonder if a proc-like thing (call it macro) would help
> which, when called, does not create a new scope. Consider for example
>
> macro loop {count body} {
> while {count>0} {
> eval $body ;# NOTE: for macro we don't need [uplevel]
> incr count -1
> }
> }
>
> When calling it like
>
> loop 10 {return -code $someCode}
>
> would the return work as proposed by this TIP when loop (as a macro)
> does not constitute a new stack frame?
>
> (Another question is of course what variable $count within the body is
> supposed to see.)
I agree, and have been thinking about these things for a while. The
show-stopper is indeed the issue of name resolution. The only solution I
found is to have a macro have all private variables, with the exception
of those whose name it received as an argument.
In that case, your code would have "count" refer to a variable which is
local to the macro; should you want to keep the count in the calling
proc, you'd write something like:
macro loop2 {countName body} {
upvar 0 count $countName
while {count>0} {
eval $body ;# NOTE: for macro we don't need [uplevel]
incr count -1
}
}
and call it like
count 10
loop count {return -code $someCode}
But it feels so clumsy ...
Miguel
Don Porter
> a) The reason for the above mentioned deficiency is that the argument
> to -code is always lost while *returning* from the proc.
No, it's lost as soon as [catch] intervenes:
set code [catch {return -code 17} msg]
There's no way to get "17" from $code or $msg.
> Consequently I wonder if a proc-like thing (call it macro)...
[macro] is a neat idea. Once the funciontality of [catch] and [return]
is complete according to one of the ideas in TIP 90, it will be simple
to create [macro] as a [proc].
Perhaps, one could work the other way as well. Provide [macro] as
a primitive, and then you might not need a complete [catch] and [return].
I'd still prefer strengthening existing commands over creating a new one.
I also like the fact that Tcl provides exactly one command for creating
commands, [proc]. I'd like to increase its power to cover all cases,
rather than have another built-in command to cover the corners.
> macro loop {count body} {
> while {count>0} {
> eval $body ;# NOTE: for macro we don't need [uplevel]
> incr count -1
> }
> }
Wouldn't this work better?
macro loop {count body} {
while {$count > 0} "$body; incr count -1"
}
> When calling it like
>
> loop 10 {return -code $someCode}
>
> would the return work as proposed by this TIP when loop (as a macro)
> does not constitute a new stack frame?
It really depends on how [macro] is implemented. It could work that
way, but variable context frames and call-stack levels are really
separate things.
Arjen Markus
>
> TIP #90: ENABLE [RETURN -CODE] IN CONTROL STRUCTURE PROCS
> ===========================================================
> Version: $Revision: 1.1 $
> Author: Don Porter <dgp_at_users.sf.net>
> State: Draft
> Type: Project
> Tcl-Version: 8.4
> Vote: Pending
> Created: Friday, 15 March 2002
> URL: http://purl.org/tcl/tip/90.html
> WebEdit: http://purl.org/tcl/tip/edit/90
> Post-History:
>
> 5. Add a new return code, _TCL_EVAL_.
>
> The problem that one cannot cause a _proc_ to behave as if it
> were the command _return -code_ can be viewed as a special case
> of the general weakness that one cannot make a _proc_ behave as
> if it were replaced by an arbitrary Tcl command. This general
> power could be granted via a new return code, _TCL_EVAL_.
>
> When detecting a _TCL_EVAL_ return code from a command, the
> result of the command would be treated as a new command to be
> evaluated in the same context as the first. The result and return
> code of the new command would take the place of the original.
>
> With that capability, combined with alternative 2, one could
> easily take care of the problem posed in the analysis above:
>
> set code [catch {uplevel 1 $body} message returnCode]
> if {$code == 2} {
> return -code eval [list return -code $returnCode $message]
> }
>
> Advantage: A complete solution that's easy to understand.
>
> Disadvantage: Very powerful technique that might enable things we
> don't want to enable.
>
One thing it seems to imply is a way (elegant, as far as I am
concerned) of doing tail-call recursion (there are already a number
of methods to achieve these effects, but this one looks quite
"natural").
Regards,
Arjen
Donal K. Fellows
> Don Porter <d...@users.sf.net> wrote:
> : 5. Add a new return code, _TCL_EVAL_.
>
> I support option 5. As you said, 3 still has a limitation and
> 4 is getting too complicated. 5 is elegant. I left 2 in there
> because I wasn't sure if it needs any changes in conjunction
> with 5.
The only problem is how to handle what happens when passing between security
domains (a.k.a. interpreters.) I've added a suggestion on this to the online
version of the TIP (which just says that TCL_EVAL shouldn't ever go across
boundaries, but should rather be evaluated immediately before the crossing.)
Hmm. That adds a nasty discontinuity though. Is it possible to somehow bind
TCL_EVAL to the interpreter which issued it, and allow [catch] in another interp
to either detect it (so it can be passed back to the caller in some way) or
handle it by evaluating in the issuing interpreter instead of the [catch]ing
one? I suppose you could add an extra -interp option to [return] to support
this...
Tangent: It would also be nice if you could give [catch] a stack level or an
interpreter to run in, instead of having to combine catch with some other
[eval]-like command, which is difficult to get right (handling stack traces is
particularly awkward.)
I think I want to think about this a bit more...
Donal.
--
Donal K. Fellows http://www.cs.man.ac.uk/~fellowsd/ fell...@cs.man.ac.uk
-- Anyone using MFC desperatly needs a nasal enigma.
-- David Steuber <tras...@david-steuber.com>
Andreas Leitgeb
article <slrna9ccq...@clover.cam.nist.gov>:
> [some comments on tip#90]
No one has yet commented on it ... maybe, it read too complicated ...
anyway, here's the gist:
1.) We'd only need to handle the number of the stack-levels to
go up before returning with the specified error-code.
The other discussions talk about lists of return-codes, which
is not necessary.
2.) We'd also need to change uplevel to make it possible to
get a deferred return-code propagate to the intended level
even without a catch
For more details please reread the previous article.
Don Porter
> The only problem is how to handle what happens when passing between security
> Hmm. That adds a nasty discontinuity though.
One thing to consider is how [uplevel] behaves in a similar context.
Might inspire a similar solution.
I don't plan on pursuing this option for TIP 90. It's a more powerful
change than I need to solve the immediate problem. My proposal will
not rule out introducing the TCL_EVAL return code later, though, and
it may indeed be useful for other things.
Don Porter
> article <slrna9ccq...@clover.cam.nist.gov>:
>> [some comments on tip#90]
>
> No one has yet commented on it ...
I did read it. It inspired, in part, a new approach I'm taking on
the proposal. Will update the TIP with details soon.
> 1.) We'd only need to handle the number of the stack-levels to
> go up before returning with the specified error-code.
> The other discussions talk about lists of return-codes, which
> is not necessary.
Yes, that's exactly correct. The -returncodes list was a bad fit
because it allowed specifying things that made no sense. The correct
inputs are a single return code (already the -code option) and a
single integer indicating the level in the call stack at which that
code should be returned (will propose -level for this).
> 2.) We'd also need to change uplevel to make it possible to
> get a deferred return-code propagate to the intended level
> even without a catch
Here's where I did get confused reading your original article, but I
think it's because you're imagining the problem to be harder than it
really is. As I'm drafting things, there's no need to change [uplevel]
itself. Maybe you're referring to the internal changes that need to
be made to TclUpdateReturnCode() ?
Andreas Leitgeb
>> 2.) We'd also need to change uplevel to make it possible to
>> get a deferred return-code propagate to the intended level
>> even without a catch
> Here's where I did get confused reading your original article, but I
> think it's because you're imagining the problem to be harder than it
> really is. As I'm drafting things, there's no need to change [uplevel]
> itself. Maybe you're referring to the internal changes that need to
> be made to TclUpdateReturnCode() ?
The reason, I think we will need to change uplevel is twofold: the
USER of a procedural control-structure shouldn't have to worry
about the implementation of the control-structure, and the PROGRAMMER
of a procedural control-structure shouldn't be forced to use catch
just to properly re-adjust eventual "return -level ..."-deferrals.
An example, of where it may hurt otherwise:
PROGRAMMER:
proc forall {ni body} { upvar $ni i
for {set i 0} {$i < $::GLOBALMAX} {incr i} {
uplevel 1 $body
}
}
USER:
proc mybreak {} {
forall n {
if {[check1 $n]} { return -code break };# let mybreak behave like break
if {[check2 $n]} { break } ;# no more checks, mybreak==noop
}
}
The USER wants mybreak to behave like break if check1 returns true,
and have the forall-loop finish and mybreak be a noop, if check2
returns true.
This is a prototype of the reason, why this TIP exists, isn't it ?
But, it turns out that the procedure "forall" itself behaves like
break (if check1 fires) and mybreak will throw an error, because it
encountered a break outside a loop.
What I meant is, that uplevel would have to recognize, when it's
body "posted" a break-exception to be thrown by exactly that level,
in which the body executed, namely by mybreak.
On the other hand, the not-deferred break (second line of the body)
will be expected by the USER to be handled by the control-structure,
thus it must not be re-targetted by uplevel.
It needs to be handled by forall from USER's pov, and would actually
be handled by the "for"-loop inside forall.
Now that I wrote it, the distinction between immediate and deferred
exceptions inside uplevel doesn't really comfort me ...
Are there any flaws ?
Don Porter
Andreas Leitgeb wrote:
> The reason, I think we will need to change uplevel is twofold: the
> USER of a procedural control-structure shouldn't have to worry
> about the implementation of the control-structure, and the PROGRAMMER
> of a procedural control-structure shouldn't be forced to use catch
> just to properly re-adjust eventual "return -level ..."-deferrals.
We disagree about the appropriate burden on the author of a control
structure proc.
I believe the revised TIP 90 proposal I'm cooking up will address this
matter satisfactorally. I'll include your [forall] example in my
revisions.
Thanks for the feedback.