I don't know about claiming things to be "impossible" in general,
but Lisp does have an extremely powerful macro facility that is
part of the language definition.  Lisp macros are generally used to
extend the Lisp language as needed.

There is a link to "Tutorial on Good Lisp Programming Style"
by Peter Norvig and Kent Pitman

It has a number of macro examples.

(page 81 has the comment "Don't use a macro where a function would suffice.")

I also don't think it's possible to produce a Common Lisp macro and
say, "There, you see, _no_ other language can do _that_".

If you want to learn about CL macros you should try to get "On Lisp"
by Paul Graham.  Amazon.co.uk had copies last time I looked.  It is
packed with examples of macros which would be difficult or _nigh_
impossible to write any other way.

You can get the code from the book by ftp at

  ftp://ftp.das.harvard.edu/pub/onlisp

        (my-macro-name arguments)

LISP programs execute by the evaluation of forms, which you can think
of as trees. Every language feature is some kind of form: function
definitions, loops and so on.

Most programming languages parse constructs and turn them
into some kind of internal tree structure which is evaluated
(interpreter mode) or processed to produce a translation.

In LISP, you basically write these trees directly, thanks
to its minimal syntax. And these trees can be also treated
as data; so there is a blur in the systemic distinction between
code and data.

Macros are procedures which, at run time, construct trees
which are then evaluated.

When you call an ordinary LISP function, all of the arguments are
evaluated, and then the functions's parameters are bound to the resulting
values. When the function is done, it produces some value, or multiple
values.

Some built-in forms are special, in that they don't evaluate
their parmeters. For example, a form for declaring a variable
obviously cannot evaluate the variable name being declared.
Or a form which iterates over some items, an on each iteration
binds a name to each item in turn, also cannot evaluate that
name.

Unlike functions, LISP macros allow you to create your own forms that
behave just like special forms. So you can introduce new language
constructs. If you want a new kind of loop you can create a macro which
provides it, completely seamlessly.

Macros receive their parameters unevaluated, and compute a form,
usually by carrying out some computation and then substituting
parameters into a template. The constructed form is returned and
then evaluated. Because this is done at run time, there is considerable
power and flexibility in what macros can do. The same macro can compute
something different each time; slightly different or even radically
different. For example, suppose that your newly invented language
construct needs to be able to invent a symbol for internal use. When
invoked, your macro can use the LISP (gensym) function to generate a
unique new symbol, which you can stick into appropriate places in the
returned form.

One feature of macros that helps them seamlessly emulate built-in forms
is called destructuring. Rather than taking only flat parameter lists,
macros can take nested parameter lists, so that

        (mymacro (a (b c (d e)) f))

can be a valid macro call in which the arguments a, b, c, d, ...
are mapped to parameters. Thus macros can emulate language
features that have this type of rich, nested syntax. Within each
parameter nesting, you can have all the bells and whistles
of a full LISP parameter list: optional parameters, keyword
parameters and all that.

Here is a very simple macro example which shows some of the features:
destructuring, substituting into a template to produce a form,
using (gensym) to invent a needed symbol. The purpose of this macro
is to introduce a very simple language feature; a loop construct
which executes a sequence of forms N times, where N is a parameter.
We implement our new kind of loop, which we will call ``ntimes''
in terms of the Common LISP ``dotimes' loop:

    (defmacro ntimes (count . forms) (let ((counter (gensym)))
                                       `(dotimes
                                           (,counter ,count) ,@forms)))

The special dot syntax in the macro parameter list means that the
first argument to the macro will be known as ``count'', and ``forms''
will represent the entire list of remaining arguments.  Of course, all
the arguments are unevaluated, which is important because the forms are
program statements that we want to execute in the loop, and they need
to be incorporated verbatim into the loop product we are trying to build.

The first thing the macro does is create a lexical environment with
local variables via let, in which a local variable called counter is
bound to the value produced by calling (gensym). In other words, the
value of counter is a unique symbol (symbols are values in LISP). In
this let environment there is but one form that is evaluated; this will
be the return value of the macro, which will be subject to a second
evaluation. This is where we specify the template and substitute things
into it to build up the loop:

        `(dotimes
           (,counter ,count) ,@forms)))

The backtick means that the entire form is quoted: evaluation
is suppresed, except for things preceded by , and ,@.  These
introduce parameters which are evaluated and interpolated
into the form. The value of counter is a unique symbol, so
,counter causes that unique symbol to be put into the tree.
,count expands to the form that the user specified which
determines the loop count. That form is pinned into our tree.
(See, this is like Christmas!). Lastly, ,@forms means expand
the value of ``forms'' into a list which is spliced into place.
``forms'' evaluates to the unevaluated list of forms.

So now you can write something like

        (ntimes 3 (format t "Hello, ") (format t " world~%"))

to print a message three times. This expands into

        (dotimes (<UNIQUE-SYM> 3) (format t "Hello, ") (format t " world~%"))

(dotimes is itself probably a macro, so more expansion takes place,
but we aren't interested in that).

Now perhaps making a simpler loop based on an existing construct is not
exactly ``undoable'' in other languages, even using primitive C token
preprocessing. But note the differences. This is completely seamless,
and does things at run time, like compute a unique name for a variable.
The only way the user can know that ntimes is a macro is to peek
at the definition or to perform:

        (macro-expand '(ntimes 3 'blah))

If you had full control over the the macro syntax, you
should be able to make it callable as (for instance):

        [ arg1 | arg2 | arg3 ] mymacro

I also use the term "syntax" to mean "tree structure".

So does the ANSI CL spec.

We usually refer to what you call syntax as "READ syntax" or
"reader syntax", but mostly we just assume that any attempt to refer
to text syntax is mere metaphor for the underlying text.  Most code
is indeed typed in, but when one asks "What is the syntax of with-open-file?"
one is *not* generally asking where the whitespace goes, and to the extent
one is asking where the parens go, one is generally using that as a
convenient notational metaphor for asking where the list groupings are.

There IS a legitimate expression syntax issue here as well, which is
what I thought you might be referring to (but apparently not) and which
did come up in the InterLisp dialect.  The question of whether you can
write [expression] syntax like

   (x * y)

and have the symbol * get control.  That's also something you can't do
in Lisp, so it's technically true even at the expression level that
macros aren't totally in control of their syntax.  But even there, we
as language designers rationalize this by saying there is an overarching
desire to have a language whose syntax has certain commonalities, and macros
do have the ability to grab control of the expression syntax to the extent
that any part of the language, even builtins, do.  That is, no system-provided
operator can do that either.

 Kaz> If you had full control over the the macro syntax, you
 Kaz> should be able to make it callable as (for instance):
 Kaz>        [ arg1 | arg2 | arg3 ] mymacro
 Kaz> So clearly, you don't have full control over syntax.
 Kaz> For that, you have to write a full blown parser.

Coby

Cheeky?  Yes, but there's no smiley there because in all seriousness
there is a separate macroexpand time.  It occurs as a sub-time of either
compile time _or_ eval time.

In terms of a program's potential behavior, all programming languages
are provably equivalent.  It may require extensive effort, but you can
do OOP with inheritance in BASIC.  In that sense there's no "undoable"
thing in any language.

However, in terms of a programming language's ability to extend itself,
there are obvious differences.  For example, the BASIC I used when I
first learned some programming had a fixed set of math operations.
I couldn't add my own.  I could make a program *behave* as if it had
other operations, but calling a BASIC subroutine required more code than
just doing, say, SIN(X).

When I moved on to Forth, I was delighted that now I could write my own
functions, and they were then as convenient to use as the ones built
into the language.  From a theoretical standpoint, this didn't enable
anything previously "undoable".  For me, however, it did.

Previously, I had read a magazine article that described a BASIC program
to make mazes.  I read and reread the article, but somehow couldn't get
the algorithm into my head.  It was too complicated.

After learning some Forth, I was able to go back and understand the
algorithm, because now my mind had a more convenient language to express
it in.  I wrote my own Forth program to generate mazes and was very
proud of myself.

Lisp macros make Lisp different from other languages in a way analagous
to how Forth is different from BASIC.  They allow the user to extend the
language in ways previously undoable, giving convenient expression to
concepts that the language designers might not have anticipated.

I'll give you an example.  Suppose you're looping through an
alphabetized table of words and their definitions.  There may be more
than one definition per word, but you want these grouped by word.

BRL (my Scheme-based web system) has macros useful for such situations.
You use the sql-repeat macro or brl-repeat macro to iterate through the
results.  Within that, you can use (group-beginning? word) to test if
you're at the beginning of a group.

You see, functions can only tell you something about the value of a
variable.  A function would be useless here, because it would only be
passed the current value of word, not its previous value.  (Side note:
group-beginning? can take any expression as its argument, not just a
variable.)

In languages without macros, you could work around this problem by
defining another variable, e.g. priorword, that you set to the value of
word at the end of each iteration, then compare word and priorword.  For
simple problems, this is not a big deal.  However, as things get more
complex, the additional code obscures the core problem being solved.
Even more code is needed for the equivalent of group-ending?.

Not long ago, a coworker was struggling with creating a complex HTML
report from a database using ASP/VBscript.  He had been trying for days,
but it still wasn't working.  He showed me what he was doing, but urged
me not to sink too much time into it.

It was a good deal trickier than your usual HTML report.  It took me 3
or 4 tries to get a BRL solution working.  "Another victory for BRL," my
coworker declared as I gave him the code.

What really surprised me was that, this experienced programmer was not
able to expand the BRL program I gave him into functionally equivalent
VBscript.  Probably some little bug kept him tripped up, and the excess
code made it difficult to track down.  Given enough time, I'm sure he
would have been able to get it.  For practical purposes, i.e. time
constraints, the macro enabled something "undoable" in another language.

In some implementations (e.g. CMUCL), there isn't even an interpreter.
EVAL works by compiling the form and then calling the resulting function.
So, while a traditional interpreter will generally delay expanding a macro
until it's is encountered during execution (which allows them to access
dynamic bindings in the containing form), these compiler-only
implementations will expand all the macros in an expression before
executing anything.

You can see the difference with:

(eval-when (:compile-toplevel :eval)
  (defvar *foo* 1))

(defmacro bad-macro ()
  `(quote ,*foo*))

(eval '(let ((*foo* 2))
         (bad-macro)))

With a typical interpreter, this will return 2, but in a compiler-only
system it will return 1.

LISP macros allow you to code such interpreters without making a sharp
systemic division between LISP and the new language. Functions and
data pass between LISP and the newly constructed language freely and
seamlessly; for instance one does not give up LISP programming when
using the Common LISP Object System. The system is built up in LISP
and then mixes right in. Basic data types become members of classes,
objects can be put into lists, functions can now be generic, and so on.

The old-named keywords are eval, compile, load.

Actually, CMU CL does have 2 interpreters, one very simplistic one
that can only handle very trivial forms like function application,
progn, setq, etc., but doesn't depend on (part of) the compiler, and
another interpreter that works directly on the internal representation
of the first pass of the compiler (called IR1).  This can handle all legal
forms, but requires (the first pass) of the compiler to be present.
The simplistic interpreter hands off complex forms to the normal
interpreter.

The simplistic interpreter is important for runtime-only images which
leave out the compiler, but need a way to get the compiled code
running.

So while CMU CL doesn't do a full compile for interpreted code, it
does a full IR1 conversion for complex forms first, and this obviously
includes the macro-expansion phase, thereby making interpreter and
compiler equivalent in this regard.

MCL is an example of a system that AFAIK really is a compiler-only
system.

Regs, Pierre.

 & arg1 arg2 arg3 $ fn

or something like that where it reads forms up to a $ and then one form
after, constructing (fn arg1 arg2 arg3).  But you can't just have

 1 2 3 +

like in PostScript if you expect to mix it with Lisp, because what
would you do in the case of

 (- 1 2 3 +)

Would that mean

 (- (+ 1 2 3))

or

 (- 1 (+ 2 3))

or

 (- 1 2 (+ 3))

or

 (- 1 2 3 (+))

The Lisp Machine, for example, used a funny non-ascii character I'll
represent as $ here to do:

 #$ a*b-c*d $

Kent M Pitman <pit...@world.std.com> writes:

        #I( a*b-c*d / sin(x) )

and more.  THe original version used #\$ as delimiters (a
la' TeX), but that conflicted with a use of #\$ in MCL, so it was
changed to #I().

Cheers

(set-macro-character #\] (get-macro-character #\)))

(set-dispatch-macro-character #\# #\[
  (lambda (stream char1 char2)
    (declare (ignore char1 char2))
    (let* ((command-line (read-delimited-list #\] stream t))
           (last (car (last command-line))))
      (cond ((equal last '+) (reverse command-line))
            (t command-line)))))

[2]> #[1 2 3 +]
6
[3]> #[- 7 #[1 2 3 +]]
1
[4]> #[list #[- 7 #[1 2 3 +]]]
(1)
[5]> #[- 1 2 3 +]

*** - argument to + should be a number: (+ 3 2 1 -)
1. Break [6]> (quit)

Process shell finished