TIP #280, extended callframe introspection and debugging
Eckhard Lehmann
Yesterday I saw that the new TIP #280 to which JH refered here
http://groups.google.com/group/comp.lang.tcl/msg/7ab0a2cbb4c9298a?hl=de&.
It looks great, hopefully the implementation is soon stable enough for
testing it, and a patch for 8.5 will be available... thanks.
A question in addition to that: is it planned or in work to have
builtin breakpoints, for stopping the execution of a script at a
certain position - or on errors?
I have an idea that will make Tcl an outstanding dynamic language (I
think this is available in LISP and Smalltalk, but not in Python, Ruby
and these "hype" languages):
It should be possible by default to break the execution of a script on
either a breakpoint (special command in the code) or on error. When the
execution is stopped, the Tcl interpreter goes into "console mode"
(open the console on Windows and Mac) so that the operator or developer
can start to inspect the running code at the position where it stopped.
He is able to examine the callframes, switch callframes and inspect
variables and objects in each callframe - just by typing the
appropriate commands, [info frame], etc. Tcl/Tk developers can extend
this by something more GUI'ish, if they want.
It would be really cool to have the program switch to debug mode after
an error occured this way. Imagine the following scenario: You deploy
your Tcl program to a customer who is in active contact to you (e.g. in
your company): (s)he encounteres an error and calls you when the
console appears with the error message. You go to her and debug the
program as it still runs, with callframes and variables still
available. In the best case, it is possible to fix the bug in the
running code and (s)he can resume her operation with the program,
without the need to even restart it. Otherwise it is at least much
easier to see what was going on before and fix it later. That's
always a problem if bug reports come later... the customer does not
remember what (s)he has done prior to the error.
What do you think?
Eckhard
Alex
On Oct 24, 8:15 pm, "Eckhard Lehmann" <eck...@web.de> wrote:
> A question in addition to that: is it planned or in work to have
> builtin breakpoints, for stopping the execution of a script at a
> certain position - or on errors?
It sure would be cool to have many bells and whistles, but I believe
your two proposals have a very different status:
1) Breaking at a certain position: it's doable today, just by
instrumenting the code, thanks to Tcl's introspective power ([info
body] et al).
2) Breaking "on error", on the other hand, lacks a precise definition,
because in many cases some kinds of error are fully expected and
properly caught at some level. Now you me want to jump to debug mode
only for errors creeping back to toplevel uncaught... However, as you
know, the stack is fully unwound at that point (the only trace of the
deeper context is in errorInfo, which has been built step by step while
popping back). So it's currently impossibke to arrange an access to the
locals of all the callers of the error-ignition culprit, and at the
same time know that you're back to toplevel without a catch...
Now, come to think of (2) again, one possibility would be to enrich the
"errorStack" mechanism I proposed on the wiki a few months ago, so that
the trace contains, not only the various [info level 0] (function and
args at each level), but also the [info locals]. Of course enabling
this would put a burden on the interpreter's memory footprint, because
many data would linger with a non-zero refcount until something resets
the trace list. But this would enable pretty cool debugging tools like
the one you outline.
-Alex
Eckhard Lehmann
Alex wrote:
> On Oct 24, 8:15 pm, "Eckhard Lehmann" <eck...@web.de> wrote:
> > A question in addition to that: is it planned or in work to have
> > builtin breakpoints, for stopping the execution of a script at a
> > certain position - or on errors?
>
> It sure would be cool to have many bells and whistles, but I believe
> your two proposals have a very different status:
> 1) Breaking at a certain position: it's doable today, just by
> instrumenting the code, thanks to Tcl's introspective power ([info
> body] et al).
No, that's not what I mean. That are introspection capabilities which
can be used once you *are* at the position where you want to be.
My proposal targets the question "how do you get at this position?".
And the answer is to simply stop the execution there. That is called
"breakpoint" - just think of breakpoints in a C debugger (gdb for
instance). It can be achieved by a command [breakpoint] as well, by
inserting this at the appropriate position in your code.
> 2) Breaking "on error", on the other hand, lacks a precise definition,
> because in many cases some kinds of error are fully expected and
> properly caught at some level. Now you me want to jump to debug mode
> only for errors creeping back to toplevel uncaught... However, as you
> know, the stack is fully unwound at that point (the only trace of the
> deeper context is in errorInfo, which has been built step by step while
> popping back). So it's currently impossibke to arrange an access to the
> locals of all the callers of the error-ignition culprit, and at the
> same time know that you're back to toplevel without a catch...
Right. Now imagine that the stack is *not* unwound at once, but the
execution is stopped just at the point where the error occured and you
are presented with a console to introspect the program. How cool would
that be? That's what I mean :-).
Part of the implementation for this is, that the execution is not
stopped on catched errors. This can probably be achieved by setting a
flag in the implementation of [catch], which triggers that errors in
one of the upwards following callframes don't lead to an execution
break.
> Now, come to think of (2) again, one possibility would be to enrich the
> "errorStack" mechanism I proposed on the wiki a few months ago, so that
> the trace contains, not only the various [info level 0] (function and
> args at each level), but also the [info locals]. Of course enabling
> this would put a burden on the interpreter's memory footprint, because
This is like the memory dump files that you can create with C programs
and later debug with gdb. But I think this is not the right way for a
dynamic language like Tcl... the right way IMO is to have dynamic
debugging.
Eckhard
Neil Madden
...
>>2) Breaking "on error", on the other hand, lacks a precise definition,
>>because in many cases some kinds of error are fully expected and
>>properly caught at some level. Now you me want to jump to debug mode
>>only for errors creeping back to toplevel uncaught... However, as you
>>know, the stack is fully unwound at that point (the only trace of the
>>deeper context is in errorInfo, which has been built step by step while
>>popping back). So it's currently impossibke to arrange an access to the
>>locals of all the callers of the error-ignition culprit, and at the
>>same time know that you're back to toplevel without a catch...
>
>
> Right. Now imagine that the stack is *not* unwound at once, but the
> execution is stopped just at the point where the error occured and you
> are presented with a console to introspect the program. How cool would
> that be? That's what I mean :-).
> Part of the implementation for this is, that the execution is not
> stopped on catched errors. This can probably be achieved by setting a
> flag in the implementation of [catch], which triggers that errors in
> one of the upwards following callframes don't lead to an execution
> break.
For Tcl-generated errors, this can be done already. Just override
[error] and [return -code error] to simply call a debug-console loop.
You can then use [uplevel], [info level], [info locals] etc to navigate
around the stack and change things. I wrote some code to do this once
[1]. You used [continue] to resume execution or [break] to rethrow the
original error. I think tkcon also contains something like this.
Smalltalk and Lisp tend to do this by capturing a "continuation" at the
point of the error and propagating up to some exception handler, which
can then examine it. You don't actually need continuations for this
though, as you can simply pass the exception-handler *down* the stack
(during normal execution) and then simply call it when an error occurs.
[1] http://lambda-the-ultimate.org/node/1544#comment-18446
-- Neil
Fredderic
"Alex" <alexandre...@gmail.com> wrote:
> On Oct 24, 8:15 pm, "Eckhard Lehmann" <eck...@web.de> wrote:
> > A question in addition to that: is it planned or in work to have
> > builtin breakpoints, for stopping the execution of a script at a
> > certain position - or on errors?
>
> It sure would be cool to have many bells and whistles, but I believe
> your two proposals have a very different status:
> 1) Breaking at a certain position: it's doable today, just by
> instrumenting the code, thanks to Tcl's introspective power ([info
> body] et al).
Sounds like an "error hook" type job. Allow it to run before the error
begins to be propagated away. Of course, not good to be doing this
from within a [catch]... And a [catch] that re-propagates an error
will fowl it up also.
On the other hand, if it's [catch] that's installing the hook, then it
can install a function which will selectively ignore errors it's
expecting, and either fix it, or allow it to continue on... These hook
functions could stack (unlike the [catch]s which are simply layered),
allowing a general debugging level to do whatever it wants; such as
storing the current call environment, or bringing up a debugging
console.
This would NOT be the recommended means of catching errors. But it
would allow for an error console or other debugging apparatus to jump
in, inspect the state of the program, and either allow it to continue
propagating the error (after taking a look around first), halt it right
there where it stands (don't pass go, don't collect $200), or clear the
error condition and continue on as though nothing'd happened (maybe
making it possible at some stage to introduce a kind of demand-based
functionality).
> Now, come to think of (2) again, one possibility would be to enrich
> the "errorStack" mechanism I proposed on the wiki a few months ago,
> so that the trace contains, not only the various [info level 0]
> (function and args at each level), but also the [info locals]. Of
> course enabling this would put a burden on the interpreter's memory
> footprint, because many data would linger with a non-zero refcount
> until something resets the trace list. But this would enable pretty
> cool debugging tools like the one you outline.
An error has to be caught, otherwise the program explodes. So when you
catch the error, it would be your responsibility to also deal with this
extra information. Something like a -preservestate option to [error]
allowing it to rethrow all that information instead allowing the
[catch] to ground it. (There's your "something that resets the trace
list", right there.)
Hmmm... The poor [error] and [catch] commands are getting a bit of a
work-out lately... ;)
Fredderic
Eckhard Lehmann
Neil Madden wrote:
> For Tcl-generated errors, this can be done already. Just override
> [error] and [return -code error] to simply call a debug-console loop.
> You can then use [uplevel], [info level], [info locals] etc to navigate
> around the stack and change things. I wrote some code to do this once
> [1]. You used [continue] to resume execution or [break] to rethrow the
> original error. I think tkcon also contains something like this.
[...]
> [1] http://lambda-the-ultimate.org/node/1544#comment-18446
Thank you - that is a very good starting point. It could be used as a
[breakpoint] command and for errors. The question remains, how errors
are processed in general, e.g. when they come from C commands via
"return TCL_ERROR;".
I think the best way is to have a command that hooks a procedure like
[debug-repl] into the Tcl interpreter which is called whenever Tcl is
about to throw an error. As arguments to this hooking command it could
be indicated whether caught or uncaught errors (or both) should be
processed.
Another thing is probably more complicated: How about stepwise
execution after a breakpoint? To have this, a command [step] (+ maybe
[next]) is needed which executes the next command in the script and
stops right afterwards... But how could this be realized?
Eckhard
Larry W. Virden
Eckhard Lehmann wrote:
> It looks great, hopefully the implementation is soon stable enough for
> testing it, and a patch for 8.5 will be available... thanks.
So are people working on this quickly enough to meet the Dec 1 deadline
for Tcl 8.5?
Don Porter
> Eckhard Lehmann wrote:
>
>> It looks great, hopefully the implementation is soon stable enough for
>> testing it, and a patch for 8.5 will be available... thanks.
hopefully? will be? Please see Tcl Patch 1571568.
Larry W. Virden wrote:
> So are people working on this quickly enough to meet the Dec 1 deadline
> for Tcl 8.5?
Andreas Kupries is working very hard on this, yes. It would help him, I
imagine, to have people testing his patch and offering their comments.
These matters are all spelled out in the TIP.
--
| Don Porter Mathematical and Computational Sciences Division |
| donald...@nist.gov Information Technology Laboratory |
| http://math.nist.gov/~DPorter/ NIST |
|______________________________________________________________________|
Eckhard Lehmann
Don Porter wrote:
> Larry W. Virden wrote:
> > Eckhard Lehmann wrote:
> >
> >> It looks great, hopefully the implementation is soon stable enough for
> >> testing it, and a patch for 8.5 will be available... thanks.
>
> hopefully? will be? Please see Tcl Patch 1571568.
To me it looks like this are patches for 8.4... I'll try to apply them
to 8.5a5, but I worry that this will not work if they are really made
against 8.4 (had such a scenario not long ago).
Eckhard
Jeff Hobbs
There will be a full 8.5 implementation when it is approved. The
current 8.4 patches exist because we have more 8.4-based sandboxes. I
think the TIP is nearly complete though.
--
Jeff Hobbs, The Tcl Guy, http://www.activestate.com/