how 'bout a debug-repl?

[George Jahad]

Every time I stick a println into some Clojure code to debug it, I
think to myself, "This is Lisp! I should be able to insert a repl
here!"

The problem is of course that Clojure's eval function doesn't know
about the surrounding lexical scope. So I started asking myself, what
is the simplest change I could make to Clojure to support an eval that
understands that scope? Then I tried to implement it.

Basically, here's what I came up with.

1. Modify the Clojure compiler so that when a flag is turned on, it
stores references to the lexical scope in a dynamic var. Thus, each
time the compiler creates a new lexical scope, it also emits the byte
code to push a hash-map with the details onto the var. When that scope
ends, the byte code for popping the hash-map off the var is emitted.

2. Then in Clojure proper, add a special version of eval that uses
that var. It wraps the form being eval'ed in a "let" that emulates the
original lexical scope, something like this:

`(eval
(let [~@(make-let-bindings
(:lexical-frames (var-get (resolve context))))]
~form))

With those two pieces, it's straight-forward creating a "debug-repl"
that understands the surrounding lexical scope.

I'm pretty pleased with the results and wanted to show them off. More
details here:
http://georgejahad.com/clojure/debug-repl.html

Thanks to the my coworkers, the "Sonian-Clojure Brain Trust", for
their support and encouragement!

[Richard Newman]

> I'm pretty pleased with the results and wanted to show them off. More
> details here:
> http://georgejahad.com/clojure/debug-repl.html

Pretty slick work, George! Thanks for sharing!

[Mike Douglas]

Very cool. Any chance we'd see this get merged into 'new'?

[Jeff Rose]

Awesome! I've googled high and low for exactly this functionality in
the past. +1 for getting this into core or contrib if it could work
there. Can you wrap a require to make a whole library debuggable?

(with-lexical-frames (require 'foo.bar))

It would be very handy to have a debug mode where exceptions
automatically drop you into a repl with all of the associated
environment for you to explore. I guess something ala Smalltalk
systems...

-Jeff

[Alex Osborne]

George Jahad <clo...@blackbirdsystems.net> writes:

> Every time I stick a println into some Clojure code to debug it, I
> think to myself, "This is Lisp! I should be able to insert a repl
> here!"
>
> The problem is of course that Clojure's eval function doesn't know
> about the surrounding lexical scope. So I started asking myself, what
> is the simplest change I could make to Clojure to support an eval that
> understands that scope? Then I tried to implement it.

Neat idea.

Unless I'm misunderstanding what your modifications do, I've come up
with a simple pure macro version that doesn't require any modifications
to Clojure. It also works fine in the middle of lets and such and you
can put a call to the (local-bindings) macro anywhere to get a map of
the locals to their symbols.

http://gist.github.com/252421

Cheers,

Alex

[George Jahad]

Brilliant. With such a simple change, I think we just revolutionized
the way
people debug Clojure. (They just don't realize it yet.)

[Konrad Hinsen]

On 9 Dec 2009, at 18:59, George Jahad wrote:

> Brilliant. With such a simple change, I think we just revolutionized
> the way people debug Clojure. (They just don't realize it yet.)

I for one do - this is an excellent improvement, and I hope that
Clojure IDEs will integrate similar tools into their debuggers. I'd
love to be able to single-step through my code and have a REPL in the
current environment at any time!

Konrad (used to WingIDE for Python, which has such functionality)

[Raoul Duke]

> Clojure IDEs will integrate similar tools into their debuggers. I'd
> love to be able to single-step through my code and have a REPL in the
> current environment at any time!

yes, catching up with what i assume CL can do would rock :-)

[kyle smith]

I'm having some trouble with Alex's macro. I can type in the debug-
repl, but when I hit enter, it just hangs and nothing happens.

[George Jahad]

are you using slime? Currently, you need to use a non-slime repl, (I
think because of how slime handles io redirection)

[kyle smith]

Yes, I just figured that out. Is there a way to use this with slime?

[George Jahad]

It's definitely on my todo list. But probably like yours, that list
ain't short.

[Konrad Hinsen]

On 10 Dec 2009, at 21:34, George Jahad wrote:

> are you using slime? Currently, you need to use a non-slime repl, (I
> think because of how slime handles io redirection)

I don't use slime, but I had a similar issue with Counterclockwise/
Eclipse, which doesn't quite understand that end-of-stream (Ctrl-D)
is meant to terminate not the stream, but just the portion of the
stream going to the debug repl.

Here is a modified version that permits quitting the debug REPL by
typing "()" (empty list):

http://gist.github.com/254110

It works fine with Counterclockwise, so perhaps it's also a solution
for Slime users.

Konrad.

[Laurent PETIT]

Great ! thanks Konrad !

Another, presumably more "general" solution, but implying modifications (by adding another optional attribute) to clojure.main/repl, would be to give to the REPL a particular instance whose meaning could be interpreted by the reader as "end the current reader".

This instance could be returned by a utility function provided by debug-repl as a global function.

This could be of general utility, not just for debugging purposes.

But then, we could even go one level deeper: not only provide a particular instance that would allow to quit the REPL, but a set of instances. And if the returned value of the call to the REPL returns one of the instances in the set, then quit.
This would allow to nest debug-repls (but is it interesting ?) calls, and to go back to the encapsulating repl by e.g. a call to debug-repl/quit(1), or to go up 2 levels in the debug-repls nesting by calling debug-repl/quit(2), ... or to go back to the main environment by e.g. a call to debug-repl/quit().

2009/12/11 Konrad Hinsen <konrad...@fastmail.net>

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en

[Stephen C. Gilardi]

On Dec 11, 2009, at 5:08 AM, Laurent PETIT wrote:

But then, we could even go one level deeper: not only provide a particular instance that would allow to quit the REPL, but a set of instances. And if the returned value of the call to the REPL returns one of the instances in the set, then quit.
This would allow to nest debug-repls (but is it interesting ?) calls, and to go back to the encapsulating repl by e.g. a call to debug-repl/quit(1), or to go up 2 levels in the debug-repls nesting by calling debug-repl/quit(2), ... or to go back to the main environment by e.g. a call to debug-repl/quit().

One possible implementation of this is to use keywords as commands to the debug-repl. Since evaluating a keyword (alone on a line by itself) is seldom interesting, we could use ones like :quit or :pop as commands intercepted by the debug repl before evaluation. We could even respond to a set of commands and send any unrecognized commands along to the evaluator.

If hijacking namespace-less keywords for that purpose is distasteful, we could also put the commands in a namespace, for example:

:dr/quit

--Steve

[Sean Devlin]

Wouldn't ::quit do the same thing?

>  smime.p7s
> 3KViewDownload

[Laurent PETIT]

namespacing symbols seems sufficient, indeed.

2009/12/11 Stephen C. Gilardi <sque...@mac.com>

[Laurent PETIT]

s/symbols/keywords/

2009/12/11 Laurent PETIT <lauren...@gmail.com>

[Dan Larkin]

On Dec 11, 2009, at 3:04 PM, Sean Devlin wrote:

Wouldn't ::quit do the same thing?

It wouldn't, because the repl is evaluating in the context of wherever you put the (debug-repl) call, so its namespace won't be "dr".

What about instead of using keywords for commands, we use functions for commands:

(debug-repl-quit)

[Sean Devlin]

Hmmm... functions as commands at a REPL. Now I feel silly for
considering the keyword approach :)

[Konrad Hinsen]

On 11 Dec 2009, at 23:26, Dan Larkin wrote:

> On Dec 11, 2009, at 3:04 PM, Sean Devlin wrote:
>
>> Wouldn't ::quit do the same thing?
>
> It wouldn't, because the repl is evaluating in the context of
> wherever you put the (debug-repl) call, so its namespace won't be
> "dr".
>
> What about instead of using keywords for commands, we use functions
> for commands:
>
> (debug-repl-quit)

I don't see any straightforward way to write a function that, when
executed inside a debug REPL, quits the REPL. The only way to get out
of a REPL is to have the its reader function return an object created
specifically for this purpose (and passed to the reader function).
Therefore any straightfoward method for quitting the REPL must be
based on objects returned by the reader, before evaluation. It is
probably possible to get around this by having a function that
modifies a global var which in turn is read by the REPL to decide when
to quit, but that's not what I'd call straightforward.

What *is* doable easily is quit whenever the reader sees the form
(debug-repl-quit). But it's the literal form and not its value. I
guess that would be good enough for the typical usage scenarios of a
debug REPL, but still it's not really a function call. Perhaps a
keyword would be a better choice to mark the difference.

Konrad.

[George Jahad]

What about using a function that throws an exception to quit?

The nice thing about using a function to quit is that then you can
return
values.

I've done so here:
http://gist.github.com/255883

In addition I've made it so the debug-repl call can wrap a form. If
you call quit-dr with no parameters, the debug-repl will just return
the value of the form wrapped. But you can also call quit with
another form, to replace the original wrapped value.

A couple of examples:

user=> (let [a 10] (debug-repl (* a a)))
dr-1-1006 => (quit-dr)
100

user=> (let [a 10] (debug-repl (* a a)))
dr-1-1007 => (quit-dr (* 9 a))
90

Finally, I've added a level and counter to the dr prompt so it is
easier to tell if you are in nested repls or if you quit one
debug-repl and enter another.

g