Is using a macro the same as returning a form from a function
and using eval on that form?

Trivial example to show what I mean:

(defmacro madd ()
   (+ 5 10))

(defun dadd ()
   '(+ 5 10))

Are the following two forms semantically equivalent?

(madd)
(eval (dadd))

They return the same result, but is there some subtle difference
somewhere? What about in the general case?

And one more question. When is a macro expanded?

  (defmacro sample-macro (var)
    `(incf ,var))

  (defun sample-function (var)
    `(incf ,var))

  (let ((x 42))
    (sample-macro x))
  ;; == (let ((x 42))
  ;;      (incf x))
  ;; => 43

  (let ((x 42))
    (eval (sample-function 'x)))
  ;; == (let ((x 42))
  ;;      (eval '(incf x)))
  ;; error: the variable X is unbound

If the code containing the macro calls is not explicitly
compiled, then they will be expanded when they are evaluated,
or possibly earlier.  They may be expanded multiple times.

If you write for instance

  (setf *foo* 3)

which invokes the SETF macro, then indeed, the evaluation of the macro produces
a form, and that form is evaluated right away as if someone invoked
eval on it.

However, macros may occur in contexts in which their expansion are not
evaluated right away, for instance:

  (defun some-function ()
    (setf *foo* 3))

Is the macro expansion evaluated now? No, the macro expansion is evaluated
when you call the function. And it's not done in the same way as eval,
because the expansion has access to the lexical environment. Consider:

  (let ((lexical-variable 42))
    (eval 'lexical-variable))

  ==> error, lexical-variable not bound

When is the macro expanded? That depends; it may be expanded as late as
the first call to the function.

Lisp is also compiled as well as interpreted language; some-function could be
compiled, which means that the macro could be expanded in the environment of
the compiler.

Perhaps the single best way to understand macros is that they are invoked at
macro-expansion time, to produce a piece of code which is then substituted in
their place, and then in that place subject to the normal evaluation rules as
if it was written there.

Macro-expansion time can happen in a compiler, or the first time a macro form
is evaluated, or as the code is being loaded or read. This multitude is
a consequence of the various situations in which Lisp code can be processed.

  (defun some-func ()
    (eval (dadd)))

now there is the suspicion that the function dadd can be replaced with
another one, so this actually has to do the evaluation. It cannot
just be replaced by the constant 15. Adding a (declaim (inline dadd))
declaration to tell the compiler that the function may be inlined
would make that deduction permissible.

Now suppose you have these

  (defmacro madd () '(+ a b))

  (defun dadd () '(+ a b))

And suppose that at the top level you have

  (madd)
  (eval (dadd))

These do the same thing, namely evaluate (+ a b) but this is only
the same only because you are at the top level. Suppose the
evaluation is nested in a form:

  (let ((a 1) (b 2))
    (madd))

This is the same as

  (let ((a 1) (b 2))
    (+ a b))

which of course produces 3.

But

  (defvar a 0)
  (defvar b 0)

  (let ((a 1) (b 2))
    (eval (dadd)))

produces 0. The lexical environment is not available in the eval,
so it simply refers to the dynamic bindings of a and b, established
in the defvar forms.

So that is a huge difference, which makes it incorrect to say that
macro expansions are subject to eval.

Sometimes macros need to be broken down into functions, because they
are too complex to write as one big function. They also may refer to
variables or constants. Then the macro expansion time starts to
matter, because if a Lisp compiler expands the macro in its own bowels,
the functions are not available by default. Suppose you are compiling
the following:

  (defun cool-expander (...) ...)

  (defmacro cool-macro (args) ... (cool-expander ...))

The compiler might do something like this: it will read cool-expander,
compile it and deposit the results into the object file, but it won't
actually evaluate the defun to instantiate the function! That would just waste
space and time in the compiler. The compiler's job is to produce translate the
function definitions, not to carry them out, right? Most of the time,
it doesn't need the definitions, only the target program needs them.

And so then the macro refers to an undefined function; when an expansion is
attempted, it will fail. Because the macro does need the function at compile
time.

To make the compiler define the cool-expander function to itself, so that your
macro can call it during expansion, you have to surround the defun with this:

  (eval-when (:compile-toplevel :load-toplevel :execute)
    (defun cool-expander (...) ...))

  Not precisely correct, but pretty close.  A macro invocation is replaced by
  the return value from the macro (expansion) function, and this would, when
  evaluated or compiled, retain the lexical environment of the form.  If you
  have used a macro in a different setting, which is certainly possible, you
  would find that it would not necessarily be evaluated.

| Are the following two forms semantically equivalent?

  Yes, but this is a very special case that probably only confuses you.

| They return the same result, but is there some subtle difference somewhere?

  Well, yes, the call to `eval´. :)

| What about in the general case?

  In the general case, a macro is called when encountered in the recursive
  evaluation of each argument.  Recall that `eval´ is a recursive function
  that effectively does (funcall (car form) (mapcar #'eval (cdr form))) for a
  function call.  For this to work properly, the innermost `eval´ call must
  return a form that works in the environment of the next outer `eval´.  For
  this to work, the macro function stored in the function slot of a macro
  takes two parameters, the whole macro form and the environment in which it
  is to be understood. This is not entirely trivial to explain, since the
  environment has to be queried to find the macro expansion function.

  You can try this, instead

(let ((form '(madd)))
  (funcall (macro-function (car form)) form nil))

| And one more question.  When is a macro expanded?

  When the form is about to acquire its meaning in the compiler or interpreter
  as defined by the language.  This need not necessarily lead to evaluation
  right away, but sooner or later it normally will.  At issue is how late this
  happens and how much environmental baggage the evaluation requires.

  In general, a macro is called with all the information that the enclosing
  forms present to the expander.  This is much more than just `eval´ does.

  I think the best way to understand macros is to study an implementation of
  `eval´.  Using the traditional Scheme textbooks and exercies for this will
  work, if you remember that a macro call is replaced by its return value and
  then re-interpreted as if the return value was the original form and that
  this does not mean immediate evaluation, except insofar as the macro
  function has side-effects.  (E.g., it could perform some book-keeping.)

--
Erik Naggum, Oslo, Norway

And thanks to all of you.

--
Thomas.

  Not to belabor the obvious, but it occurred to me that you need write

(defmacro madd ()
  `(+ 5 10))

  to make it return the list (+ 5 10).  Note the backquote/grave, but you
  could have used a simple ' in this case.  If you want it to return '(+ 5
  10), you actually have write

(defmacro madd ()
  ''(+ 5 10))

--
Erik Naggum, Oslo, Norway