Debugging Questions
Chris Perkins
the functions I have written. But, the function doesn't fail in all cases
and after careful examination of its code it looks correct, though obviously
there must be some mistake I'm overlooking.
Now what? I'd really like to run the function with the failing set of
arguments and watch it execute line by line to figure out where my mistake
is. But neither step nor trace seem to do this, that I can see.
After searching through the news archives of comp.lang.lisp and the docs for
LispWorks and Allegro CL, I understand that neither of these Lisp tools come
with any step-wise debuggers. If I'm incorrect, let me know.
So some questions:
1. Is there a way to watch line by line execution of a function? If so,
how?
2. If not, how does the average Lisp programmer proceed when faced with a
similar problem?
3. Are there any third party debuggers out there? I found some links to a
debugger from Marc Mertens, circa 1999, but they were dead ends. Any new
links?
I'll probably solve this problem in the short term by littering my code with
pprint and format statements, but I really hate to do this.
Chris Perkins
Friedrich Dominicus
> Using trace and step I've been able to trace a bad return value down to one
> the functions I have written. But, the function doesn't fail in all cases
> and after careful examination of its code it looks correct, though obviously
> there must be some mistake I'm overlooking.
>
> Now what? I'd really like to run the function with the failing set of
> arguments and watch it execute line by line to figure out where my mistake
> is. But neither step nor trace seem to do this, that I can see.
>
> After searching through the news archives of comp.lang.lisp and the docs for
> LispWorks and Allegro CL, I understand that neither of these Lisp tools come
> with any step-wise debuggers. If I'm incorrect, let me know.
For LispWorks see: step in the Common Lisp User Manual you can step
through forms form-by-form
manual/online/web/LWRM-U/html/LWRM_69.HTM#HEADING69-0
Here's a small except from such a stepping:
(defun fact (n)
(if (= n 0)
1
(* n (fact (1- n)))))
FACT
CL-USER 14 : 1 > (step (fact 10))
(FACT 10) -> :s
10 -> :s
10
(DECLARE (SPECIAL:SOURCE (LAMBDA (N) (DECLARE #) (BLOCK FACT #))) (LAMBDA-NAME FACT)) -> :s
NIL
(BLOCK FACT (IF (= N 0) 1 (* N (FACT #)))) -> :s
(IF (= N 0) 1 (* N (FACT (1- N)))) -> :s
(= N 0) -> :s
N -> :s
10
0 -> :s
0
NIL
(* N (FACT (1- N))) -> :s
N -> :s
10
(FACT (1- N)) -> :s
(1- N) -> :s
N -> :s
....
that are the available commands:
:s Step this form and all of its subforms (optional +ve integer arg)
:st Step this form without stepping its subforms
:si Step this form without stepping its arguments if it is a function call
:su Step up out of this form without stepping its subforms
:sr Return a value to use for this form
:sq Quit from the current stepper level
:bug-form <subject> &key <filename>
Print out a bug report form, optionally to a file.
:get <variable> <command identifier>
Get a command from the history list and put it in a variable.
:help Produce this list.
:his &optional <n1> <n2>
List the command history, optionally the last n1 or range n1 to n2.
:redo &optional <command identifier>
Redo a previous command, identified by its number or a substring.
:use <new form> <old form> &optional <command identifier>
Redo command after replacing old form with new form.
0 ->
you can simply ask what value a Variable has e.g
0 -> n
9
> So some questions:
> 1. Is there a way to watch line by line execution of a function? If so,
> how?
Line oriented debugging does not make much sense in Lisp. What you
want to know usually is what holds before a subform was evaluated and
what after that evaluation. So it seems to be a good idea to debug
"form" oriented.
> 2. If not, how does the average Lisp programmer proceed when faced with a
> similar problem?
Stripping down the problem as much as you can adding logging output,
even simple prints often help.
>
> I'll probably solve this problem in the short term by littering my code with
> pprint and format statements, but I really hate to do this.
Well what is so bad about it?
Regards
Friedrich
Tim Bradshaw
>
> So some questions:
> 1. Is there a way to watch line by line execution of a function? If so,
> how?
STEP should do this. You almost certainly need the function to be
interpreted, not compiled. This will also do form-by-form tracing not
line-by-line - line-by-line doesn't really make sense for Lisp.
Typically you'll also see all the macro expansions or step the
expanded code, which can make life harder work.
--tim
Chris Perkins
My code was still compiled. When I stepped through the uncompiled version
it worked.
And, as you mentioned, in the stepper all of my functions were expanded out,
making stepping very difficult. (I tried this in LispWorks).
So, anyway to watch my code execute form by form WITHOUT the massive
expansion? It makes it very difficult to work.
Chris
"Tim Bradshaw" <tfb+g...@tfeb.org> wrote in message
news:fbc0f5d1.01121...@posting.google.com...
Coby Beck
> the functions I have written. But, the function doesn't fail in all cases
> and after careful examination of its code it looks correct, though obviously
> there must be some mistake I'm overlooking.
>
> Now what? I'd really like to run the function with the failing set of
> arguments and watch it execute line by line to figure out where my mistake
> is. But neither step nor trace seem to do this, that I can see.
>
> After searching through the news archives of comp.lang.lisp and the docs for
> LispWorks and Allegro CL, I understand that neither of these Lisp tools come
> with any step-wise debuggers. If I'm incorrect, let me know.
>
> So some questions:
> 1. Is there a way to watch line by line execution of a function? If so,
> how?
> 2. If not, how does the average Lisp programmer proceed when faced with a
> similar problem?
I will usually create the input arguments at the top level or as commented out
lines in the source file and then evaluate the function form by form and see
where my expectations are not met. If it is a complex object(s) going in
perhaps insert a (break) and then grab the input args from the debugger. If
your function has flet's and/or label's this gets more complicated, also long
let lists make this less practical than using your implementation's debugger
but generally you can find out a lot doing that, especially if you have it
narrowed down to one bad function.
> I'll probably solve this problem in the short term by littering my code with
> pprint and format statements, but I really hate to do this.
>
This is not such a bad thing...it is easy to put them in and take them out
(just re-eval the defun) I often leave lines like:
(when *debug-this-pile* (format t "~&did this with ~A got ~A" arg1 arg2))
in delivered code.
--
Coby
(remove #\space "coby . beck @ opentechgroup . com")
Barry Margolin
Chris Perkins <cper...@medialab.com> wrote:
>So, anyway to watch my code execute form by form WITHOUT the massive
>expansion? It makes it very difficult to work.
Compile all the functions that you don't want to see the details of, and
interpret the ones that you want to step through. The stepper will treat a
call to a compiled function as a single step.
--
Barry Margolin, bar...@genuity.net
Genuity, Woburn, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.
Chris Perkins
with the exception of an accessor . Yet the expansions for it is well over
a half screen long. And as I step through I keep scrolling up large blocks
of expansion. The code has a let* statement, some setfs, some arithmetic,
a when clause - nothing fancy. Obviously, as I step the expansions become
smaller, but still, I have to look at every one to see "is this something I
can ignore?"
Maybe I'm doing something wrong (still), but its frustrating because
stepping through the code to find a bug seems to not really be a practical
everyday option. Though, still available if I really need it.
It's a little ironic, too. Lisp is great as a programming language because
it lets me code at a relatively high detail free level, but the tools seem
to force me into stepping at a low level.
Chris
P.S. I found the bug last night when re-examining the code and the output -
I was using 2pi in a place where I should have just used pi.
"Barry Margolin" <bar...@genuity.net> wrote in message
news:z8sR7.16$tk5.43135@burlma1-snr2...
Coby Beck
> The function I am trying to step doesn't call any of my other functions,
> with the exception of an accessor . Yet the expansions for it is well over
> a half screen long. And as I step through I keep scrolling up large blocks
> of expansion. The code has a let* statement, some setfs, some arithmetic,
> a when clause - nothing fancy. Obviously, as I step the expansions become
> smaller, but still, I have to look at every one to see "is this something I
> can ignore?"
>
> Maybe I'm doing something wrong (still), but its frustrating because
> stepping through the code to find a bug seems to not really be a practical
> everyday option. Though, still available if I really need it.
>
> It's a little ironic, too. Lisp is great as a programming language because
> it lets me code at a relatively high detail free level, but the tools seem
> to force me into stepping at a low level.
>
Well, when evaluating forms in your source file, you can do it at as high a
level as you like..a whole let form, an entire cond...that kind of thing. You
can also at any step bind a different value to one of you local vars. It is
quite versatile and practical (IMO) You don't have to worry about expansions
either..do a whole (with-open-file...) form etc.
> Chris
>
> P.S. I found the bug last night when re-examining the code and the output -
> I was using 2pi in a place where I should have just used pi.
>
I had a hunch that was it... ;-)
Kent M Pitman
> The function I am trying to step doesn't call any of my other functions,
> with the exception of an accessor . Yet the expansions for it is well over
> a half screen long.
Try stepping compiled code instead of interpreted code. The interpreter
probably calls tons of useless stuff that compiled code wouldn't. STEP
is supposed to work compiled. e.g.,
(funcall (compile (defun foo () (step ...your code here...))))
It's perhaps a little unfortunate that CL doesn't have DONT-STEP and DO-STEP
macros so that you could hide macro implementation by doing things like:
(let ((x 3)) (+ x x))
=> (dont-step ((lambda (x) (do-step (+ x x)))
(do-step 3)))
with all the intermediate generated code by the macro being put into a mode
where it was ordinarily hidden from the stepper as "subprimitive". If this
was the protocol, I think STEP would work a little better in interpreted code.
But I have to say that it's been almost 20 years since I used stepping for
debugging and I don't miss it really. I once in a while use TRACE and mostly
just put breakpoints or print statements in code at key points to get data
I need, and that works fine for me.
STEP seems to me to do nothing more than re-state what should be mostly
obvious from the source code in an well-written code. Sometimes when you
get unreadable, undocumented legacy code from someone else, it might be useful
for gaining a foothold. But it's really an admission of defeat, I think, in
code you maintain yourself... or, at least, that I maintain for myself.
Just my opinion.
Samir Sekkat
says...
> But I have to say that it's been almost 20 years since I used stepping for
> debugging and I don't miss it really. I once in a while use TRACE and mostly
> just put breakpoints or print statements in code at key points to get data
> I need, and that works fine for me.
>
> STEP seems to me to do nothing more than re-state what should be mostly
> obvious from the source code in an well-written code. Sometimes when you
> get unreadable, undocumented legacy code from someone else, it might be useful
> for gaining a foothold. But it's really an admission of defeat, I think, in
> code you maintain yourself... or, at least, that I maintain for myself.
>
> Just my opinion.
I agree, I also never use STEP :-)
Each time I have a problem I refactorize and write tests until the error
appears. Benefit is that you can keep the tests for later and your code
gets better.
Erik Naggum
| I'll probably solve this problem in the short term by littering my code
| with pprint and format statements, but I really hate to do this.
Have you considered using a macro around such debug output forms to
control the "debugging level" at which they emit output and to turn it
off entirely when you think you no longer need it, without removing the
debugging forms from the code? My prediction is that you really want to
keep those debugging forms in there when you deploy, because you may find
another bug and the ability to dump loads of debugging information by
tweaking a variable in a running image may be very valuable.
///
--
The past is not more important than the future, despite what your culture
has taught you. Your future observations, conclusions, and beliefs are
more important to you than those in your past ever will be. The world is
changing so fast the balance between the past and the future has shifted.
Tim Bradshaw
> It's a little ironic, too. Lisp is great as a programming language because
> it lets me code at a relatively high detail free level, but the tools seem
> to force me into stepping at a low level.
>
I think this is because Lisp is really such a low-level language -
much lower level than almost any other language that people write in
nowadays.
In most languages there's a great lot of transformation that goes on
between the source you type, and what the compiler actually ends up
converting into machine code, and all this transformation is hidden
away from you, and known only to the compiler, allowing you to write
in this nice high-level language. If transformations are exposed to
the user at all, they are often terribly simple and rigid, like
macros in C. Because the compiler encapsulates all the knowledge of
the language, it can leave traces in the object file which allow
things like stepping to work really nicely.
But Lisp really isn't like that, it's much lower level. The compiler
works on this awful low-level grut with only a few rudimentary control
constructs, like GO and so on. All the transformation from the human
language to what the compiler eats is done by what is essentially a
user program - macro expansion. Now fortunately there's a standard
macro library, which together with a fairly large function library and
the low-level grut is CL, and makes CL look kind of high level, even
though it's actually not. But this library is really just a
convention - you could write your own, or (more likely) extend the
standard library of macros: the system doesn't care because it only
eats the grut that results.
This is great, because it lets you pretend you are writing in a HLL
when really you're writing pretty much assembler, and even better you
can cook your own HLL on demand. But the cost is that things like
stepping are very very hard to do, other than by stepping at the grut
level. It's terribly hard for the stepper to know about the
transformations because they are, ultimately, arbitrary programs which
can do anything at all.
I think that the right approach would be to write a macro expander
which left secret hints in the code for the stepper to find, which
would allow it to step over things in a principled way. But I'm not
even sure that that's possible, really, since macros can expand to
completely arbitrary stuff. Genera had these things called `source
locators' which did something like this, I think. They never worked
well enough in the versions I had (8.3 I think, I don't think they
existed before 8) that I wanted to use them.
I wonder what other languages with hairy transformations do. C++
templates are prpobably a good example (although I think they are
typically much more predictable than Lisp macros).
--tim