Common Lisp, the abstract syntax tree, introspection.
Berlin Brown
can I add code to common lisp code that will tell me when a function
is called and when has finished executing. I want to take any lisp
code and this particular modification to the code. I figure with
lisp's AST analysis, this should be possible.
For example, pseudo code, hello_world.lisp:
(defun hello-world ()
(format t "Hello World"))
(hello-world)
---- And then I have a utility to load hello_world.lisp and execute
the hello-world call.
At the command line:
#Inspect: hello-world function was called
#Hello World
#Inspect: hello-world has finished executing.
jos...@corporate-world.lisp.de
> With common lisp and I am assuming the introspection properties. How
> can I add code to common lisp code that will tell me when a function
> is called and when has finished executing. I want to take any lisp
> code and this particular modification to the code. I figure with
> lisp's AST analysis, this should be possible.
The questions is why would you do that and what would you like to
achieve?
Is it for debugging?
>
> For example, pseudo code, hello_world.lisp:
>
> (defun hello-world ()
> (format t "Hello World"))
>
> (hello-world)
>
> ---- And then I have a utility to load hello_world.lisp and execute
> the hello-world call.
In Common Lisp, I would just LOAD the file and TRACE the function.
(load "foo.lisp")
(trace hello-world)
(hello-world)
If you want to manipulate the file like it is, there are several
possibilities.
1) For example you can redefine DEFUN to to (progn (ORIGINAL-DEFUN
hello-world ...) (TRACE hello-world)) and then
load the file. You could have your own trace macro. You might need to
tell your Lisp implementation
something to be allowed to change a built-in macro like DEFUN, or just
use a different package.
2) You could (READ) each form, if it is a call to defun , add a TRACE
after it and evaluate each form.
3..n) ... lots of other possibilities...
Manipulating code like in 2) is easy. Similar to this:
? (read-from-string "(defun hello-world ()
(format t \"Hello World\"))")
(DEFUN HELLO-WORLD NIL (FORMAT T "Hello World"))
51
? `(progn ,* (trace ,(second *)))
(PROGN (DEFUN HELLO-WORLD NIL (FORMAT T "Hello World")) (TRACE HELLO-
WORLD))
>
> At the command line:
> #Inspect: hello-world function was called
> #Hello World
> #Inspect: hello-world has finished executing.
What is 'lisp's AST analysis'?
You don't need to look at source to do that. Compiled Lisp code also
has no source code at runtime.
Functions are first class objects. Functions are named by symbols. In
many cases functions are late-bound.
(symbol-function 'hello-world) gets the function object.
(setf (symbol-function 'hello-world) some-function) sets the function
object.
? (let ((f0 (symbol-function 'hello-world)))
(setf (symbol-function 'hello-world)
(lambda (&rest args)
(print "call to hello-world started")
(prog1 (apply f0 args)
(print "call to hello world ended")))))
#<COMPILED-LEXICAL-CLOSURE #x2A7C616>
? (hello-world)
"call to hello-world started" Hello World
"call to hello world ended"
Common Lisp has a TRACE macro:
? (trace hello-world)
NIL
? (hello-world)
Calling (HELLO-WORLD)
Hello World
HELLO-WORLD returned NIL
NIL
? (untrace)
(HELLO-WORLD)
? (hello-world)
Hello World
NIL
Most Common Lisp implementations have extended versions of the trace
macro that will do all kinds of fancy stuff.
http://www.lispworks.com/documentation/lw50/LWRM/html/lwref-107.htm#pgfId-1040564
Additionally there are extensions like DEFADVICE that can modify
functions.
http://www.lispworks.com/documentation/lw50/LWRM/html/lwref-279.htm#pgfId-1187582
You could also use DEFMETHOD instead of DEFUN. Generic functions
have :before and :after functionality built-in.
Alberto Riva
> With common lisp and I am assuming the introspection properties. How
> can I add code to common lisp code that will tell me when a function
> is called and when has finished executing. I want to take any lisp
> code and this particular modification to the code. I figure with
> lisp's AST analysis, this should be possible.
Yes, you can do that with macros, it's relatively easy (see below). We
don't even call it "introspection" because macros have access to their
body *as a Lisp data structure* (a list, in particular) and can use the
entire Lisp language to manipulate it. No introspection required.
> For example, pseudo code, hello_world.lisp:
>
> (defun hello-world ()
> (format t "Hello World"))
>
> (hello-world)
This is not pseudo-code, this is real Lisp (except that you would not
normally put the (hello-world) call inside the file: defining a function
and calling it are two different operations).
> ---- And then I have a utility to load hello_world.lisp and execute
> the hello-world call.
>
> At the command line:
> #Inspect: hello-world function was called
> #Hello World
> #Inspect: hello-world has finished executing.
Ok, first of all, this is already available in Common Lisp (it's called
TRACE rather than Inspect). Try this (after removing the hello-world
call from the file):
> (load "hello_world.lisp")
> (trace hello-world)
> (hello-world)
Implementing this feature yourself is relatively easy, as I was saying
above. For example, if you define the following macro:
(defmacro tracedefun (name arglist &body forms)
`(defun ,name ,arglist
(format t "~a called.~%" ',name)
,@forms
(format t "~a finished executing.~%" ',name)))
Then you can do the following:
> (tracedefun hello-world ()
(format t "hello world!~%"))
HELLO-WORLD
> (hello-world)
HELLO-WORLD called.
hello world!
HELLO-WORLD finished executing.
NIL
Note that this macro has the following problems:
1. It always returns NIL, instead of the value produced by the body of
the function (but this is easy to fix).
2. It's called TRACEDEFUN instead of DEFUN, but this is simply because
it's illegal to redefine symbols in the CL package.
3. It *always* prints the called/finished messages, while with TRACE you
can selectively turn tracing on or off for individual functions. Again,
it would be pretty easy to check against a list of traced functions to
make it work that way.
Alberto
Kaz Kylheku
> With common lisp and I am assuming the introspection properties. How
> can I add code to common lisp code that will tell me when a function
> is called and when has finished executing.
This is done by the standard macro TRACE.
Other than that, Lisp doesn't provide any standard mechanism for instrumenting
a function with advice. The building blocks are there, though.
You can use SYMBOL-FUNCTION to retrieve the function associated with a symbol.
A SYMBOL-FUNCTION form is a place, so you can assign a new function there.
(defun instrument-function (name pre post)
(let ((old (symbol-function name)))
(setf (symbol-function name)
(lambda (&rest args)
(prog2
(funcall pre name args)
(apply old args)
(funcall post name args))))))
A better version of this would save the old function in a way that you can
uninstrument it, analogous to UNTRACE. Also, the post-advice might have access
to the return value, rather than just the arguments. Etc. Everything can be
improved, extended, as usual.
Snippets from an session with CLISP:
The tool:
[10]>
(defun instrument-function (name pre post)
(setf (symbol-function name)
(lambda (&rest args)
(prog2
(let ((old (symbol-function name)))
(setf (symbol-function name)
(lambda (&rest args)
(prog2
(funcall pre name args)
(apply old args)
(funcall post name args))))))
INSTRUMENT-FUNCTION
The function we want to instrument:
[11]> (defun my-add (x y) (+ x y))
MY-ADD
Instrumenting helpers:
[12]> (defun pre-spy (name args) (format t "~s about to be called with ~s~%" name args))
PRE-SPY
[13]> (defun post-spy (name args) (format t "~s was called with ~s~%" name args))
POST-SPY
Now instrument MY-ADD:
[14]> (instrument-function 'my-add #'pre-spy #'post-spy)
#<FUNCTION :LAMBDA (&REST ARGS)
(PROG2 (FUNCALL PRE NAME ARGS) (APPLY OLD ARGS) (FUNCALL POST NAME ARGS))>
Call it, and watch the tracers do their job:
[15]> (my-add 2 3)
MY-ADD about to be called with (2 3)
MY-ADD was called with (2 3)
5
That looks like it.
Note the use of PROG2 to evaluate several forms in a row, and return the value
of the second one of them; that came in quite handy.
Berlin Brown
Cool, thanks.
And I was using that "hello world" example as just that. I wanted to
do more than just print when the function is invoked. For example,
how is trace implemented?
Berlin Brown
Kaz, why didn't you use macros in your example?
Barry Margolin
<ce425a3e-4976-4332...@v15g2000hsa.googlegroups.com>,
Berlin Brown <berlin...@gmail.com> wrote:
> And I was using that "hello world" example as just that. I wanted to
> do more than just print when the function is invoked. For example,
> how is trace implemented?
In most real implementations it makes use of implementation-specific
hooks. But you can get a good approximation with something like this:
(defun trace-function (func-name)
(let ((old-def (symbol-function func-name)))
(setf (gethash func-name *traced-functions*) old-def)
(setf (symbol-function func-name)
(lambda (&rest args)
(printf "~&Calling ~s~&" (cons func-name args))
(let ((results (multiple-value-list (apply old-def args)))
(printf "~&~s returns ~s~&" func-name results)
(values-list results))))))
--
Barry Margolin, bar...@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***
*** PLEASE don't copy me on replies, I'll read them in the group ***
Thomas A. Russ
> With common lisp and I am assuming the introspection properties. How
> can I add code to common lisp code that will tell me when a function
> is called and when has finished executing. I want to take any lisp
> code and this particular modification to the code. I figure with
> lisp's AST analysis, this should be possible.
...
> At the command line:
> #Inspect: hello-world function was called
> #Hello World
> #Inspect: hello-world has finished executing.
Isn't this just TRACE?
--
Thomas A. Russ, USC/Information Sciences Institute
Berlin Brown
And introspection is the wrong term, I believe. I think it is called
behavioral reflection or something along those lines?
Alberto Riva
It's called "code is data".
Alberto
Matthias Buelow
> And introspection is the wrong term, I believe. I think it is called
> behavioral reflection or something along those lines?
Cognitive behavioural therapy for buggy programs?
Or can we just throw in some pills?
Kenny
Sure, but with a name like that you'll never get the paper published.
You need a machine that goes ping in there somewhere.
hth,kenny
Alberto Riva
That's funny: at my previous job in Boston, I used to work on a project
called 'Ping'. Unfortunately they forced me to write it in Java, so it
never got anywhere. After I left they changed its name (and they also
rewrote most of it, to be fair ;) and now it's suddenly very popular...
It had nothing to do with behavioral reflection, anyway.
Alberto