Macro questions
Neil Toronto
This is about Lisp-style (sexpr-replacement) macros in general, not
specific to any particular Lisp.
I'm sure I'm wrong on quite a few points, so if anyone has the time,
I'd like to know what those are.
1. A language doesn't need to be devoid of syntax in order to
implement macros, as long as there's a simple, unambiguous mapping
from surface syntax to the syntax tree.
2. Macros don't need a *functional* language to operate, they need an
*expression-based* language. Lisp macros don't break down when using
an imperative style.
C macros - the evil gits - currently work at a token level. If C were
completely expression-based and had a standardized syntax tree, a
compiler could be written that implemented Lisp-style macros. (You'd
probably have to compile them separately as a shared library, but it
should work.)
3. Macros seem to expand in the wrong order. Why isn't it postorder
(evaluation order)? It seems postorder expansion could solve quite a
few composition issues. As it is, macros expand preorder, which runs a
little counter to intuition.
Thanks for any and all replies.
Neil
Barry Margolin
<e5b81d1c-0d94-4372...@s37g2000prg.googlegroups.com>,
Neil Toronto <neil.t...@gmail.com> wrote:
> 3. Macros seem to expand in the wrong order. Why isn't it postorder
> (evaluation order)? It seems postorder expansion could solve quite a
> few composition issues. As it is, macros expand preorder, which runs a
> little counter to intuition.
Because the "parameters" to macros are not necessarily expressions.
This is what allows macros to implement new syntax, which is one of the
major benefits of macros.
For example, if macros were postorder, how would you implement
(with-open-file (if :direction :IN)
;; IF is short for Input File
(read if))
A postorder expander would try to expand (if :direction :out) as if it
were an ordinary IF expression.
--
Barry Margolin, bar...@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***
*** PLEASE don't copy me on replies, I'll read them in the group ***
Paul Donnelly
> 3. Macros seem to expand in the wrong order. Why isn't it postorder
> (evaluation order)? It seems postorder expansion could solve quite a
> few composition issues. As it is, macros expand preorder, which runs a
> little counter to intuition.
Consider a macro invocation like:
(dotimes (i 3) (whatever))
If they expanded in the usual order, (i 3) would be treated as an
attempt at a function call, which is not what is intended. The whole
point of a macro is to interpret the macro's arguments in a non-standard
way. It would be self defeating if a macro's arguments got messed with
before the macro got to see them.
Neil Toronto
wrote:
Wouldn't that only be a problem if there were a macro named "i"?
Similarly, it seems the issue Barry brings up is only a problem if
there is a macro named "if". (I suppose some Lisp implementation might
do that.) Or is there more information passed to a macro than just
lists of lists of atoms?
Thanks for the speedy and helpful replies, by the way.
Neil
Barry Margolin
<7085ee9c-86bd-44f8...@j20g2000hsi.googlegroups.com>,
Neil Toronto <neil.t...@gmail.com> wrote:
> On Feb 11, 10:29 pm, Paul Donnelly <paul-donne...@sbcglobal.net>
> wrote:
> > Neil Toronto <neil.toro...@gmail.com> writes:
> > > 3. Macros seem to expand in the wrong order. Why isn't it postorder
> > > (evaluation order)? It seems postorder expansion could solve quite a
> > > few composition issues. As it is, macros expand preorder, which runs a
> > > little counter to intuition.
> >
> > Consider a macro invocation like:
> >
> > (dotimes (i 3) (whatever))
> >
> > If they expanded in the usual order, (i 3) would be treated as an
> > attempt at a function call, which is not what is intended. The whole
> > point of a macro is to interpret the macro's arguments in a non-standard
> > way. It would be self defeating if a macro's arguments got messed with
> > before the macro got to see them.
>
> Wouldn't that only be a problem if there were a macro named "i"?
So you're suggesting that it expand macros if it can, but leave things
alone if they're not macro invocations? What about macro invocations in
deeply nested lists? The problem is that it's the outer macro that
defines which of these nested lists are the ones that will eventually be
evaluated as expressions. So how can the evaluator know in general
which ones to expand postorder?
>
> Similarly, it seems the issue Barry brings up is only a problem if
> there is a macro named "if". (I suppose some Lisp implementation might
Sorry, I forgot that IF is a special operator and COND is the macro
(this is the opposite of how things were in the original Lisp dialects I
learned, and there's no particular reason to favor one or the other as
the primitive). So how about:
(dotimes (cond 3) (whatever))
> do that.) Or is there more information passed to a macro than just
> lists of lists of atoms?
No, that's it.
Here's another example of why postorder would not be correct:
(defmacro my-quote (thing)
`(quote ,thing))
(defmacro test-macro (x)
`(list ,x ,x))
(my-quote (test-macro 3))
With postorder expansion, this will return (LIST 3 3) rather than
(TEST-MACRO 3).
D Herring
> On Feb 11, 10:29 pm, Paul Donnelly <paul-donne...@sbcglobal.net> wrote:
>> Neil Toronto <neil.toro...@gmail.com> writes:
>>> 3. Macros seem to expand in the wrong order. Why isn't it postorder
>>> (evaluation order)? It seems postorder expansion could solve quite a
>>> few composition issues.
>>
>> (dotimes (i 3) (whatever))
>>
>> If they expanded in the usual order, (i 3) would be treated as an
>> attempt at a function call, which is not what is intended. The whole
>> point of a macro is to interpret the macro's arguments in a non-standard
>> way. It would be self defeating if a macro's arguments got messed with
>> before the macro got to see them.
>
> Wouldn't that only be a problem if there were a macro named "i"?
Introducing more composition issues?
Composition only works for certain classes of problems; it frequently
breaks down where nonlinear behavior is desired...
One beauty of lisp macros is that the syntax tree isn't treated as
such until all the macros are done working. By default, the outer
scope sees everything raw (nice for printing code before its output).
Nothing stops you from letting a macro macroexpand its contents
before manipulating them, but the Lisp order gives each macro's author
that choice.
Flexibility trumps all.
- Daniel
Kent M Pitman
> On Feb 11, 10:29 pm, Paul Donnelly <paul-donne...@sbcglobal.net> wrote:
> > Consider a macro invocation like:
> >
> > (dotimes (i 3) (whatever))
> >
> > If they expanded in the usual order, (i 3) would be treated as an
> > attempt at a function call, which is not what is intended. The whole
> > point of a macro is to interpret the macro's arguments in a non-standard
> > way. It would be self defeating if a macro's arguments got messed with
> > before the macro got to see them.
>
> Wouldn't that only be a problem if there were a macro named "i"?
Consider the concept of quotation: You'd only be able to quote
structures that were not programs. And since a common reason to do
quotation is to talk about programs, that makes no sense at all.
The whole point is that the process of macro expansion is not intended to
be heuristic, so the very notion that you could be relying on the lucky
accident that nothing that was in the macro body was, coincidentally,
also evaluable places holes in the syntactic space that you want to have
available for reliable macro processing.
You can certainly design a macro system that works as you describe. It
just won't be anything I'd want to use.
Maciej Katafiasz
>> do that.) Or is there more information passed to a macro than just
>> lists of lists of atoms?
>
> No, that's it.
Strictly speaking, there's also lexical environment which you can opt to
receive. In ANSI CL you can't do all that much with them (CLtL2 gave you
a more expressive toolkit, and I have no idea about how Scheme does
things), but they're still crucial if you want to properly expand other
macros in your own body.
Cheers,
Maciej
Pascal Costanza
> More like potential conclusions, anyway, and I want to check them.
> This is about Lisp-style (sexpr-replacement) macros in general, not
> specific to any particular Lisp.
>
> I'm sure I'm wrong on quite a few points, so if anyone has the time,
> I'd like to know what those are.
>
> 1. A language doesn't need to be devoid of syntax in order to
> implement macros, as long as there's a simple, unambiguous mapping
> from surface syntax to the syntax tree.
The syntax doesn't even need to be simple, only unambiguous. See, for
example, "Explicit Programming" at
http://www.cs.ubc.ca/labs/spl/projects/explicit.html
Simple syntax makes macro programming more convenient, though (which is
a subjective assessment).
> 2. Macros don't need a *functional* language to operate, they need an
> *expression-based* language. Lisp macros don't break down when using
> an imperative style.
There is no guarantee how often a macro gets expanded. For example, an
IDE may have debugging tools that allow you to partially to fully expand
source code for inspection, and they allow you to do this as often as
you want. A macro itself may also partially to fully expand the
parameters it receives, to make its expansion depend on characteristics
of its "inner" code.
Therefore, the side effects you use in a macro definition should better
be restricted to local variables only, and shouldn't affect the outcome
of future macro expansions.
> C macros - the evil gits - currently work at a token level. If C were
> completely expression-based and had a standardized syntax tree, a
> compiler could be written that implemented Lisp-style macros. (You'd
> probably have to compile them separately as a shared library, but it
> should work.)
Yes.
> 3. Macros seem to expand in the wrong order. Why isn't it postorder
> (evaluation order)? It seems postorder expansion could solve quite a
> few composition issues. As it is, macros expand preorder, which runs a
> little counter to intuition.
You can have postorder, if you want to, by calling macroexpand in your
macro body yourself. You should watch out for the &environment parameter
in that case, and you probably need a more complete code walker.
However, the situations where this is useful are very rare. (Google for
"expansion-passing style", though, which is quite similar.)
Getting preorder if you already have postorder would be much harder.
Pascal
--
1st European Lisp Symposium (ELS'08)
http://prog.vub.ac.be/~pcostanza/els08/
My website: http://p-cos.net
Common Lisp Document Repository: http://cdr.eurolisp.org
Closer to MOP & ContextL: http://common-lisp.net/project/closer/
Andreas Thiele
I think the simple point that macros don't evaluate their arguments is the
cause of the expansion order. Because arguments are not evaluated you
are 'stuck' with the very first 'outer' macro. Not evaluating the arguments
is needed for the macro to do it job - transforming source. Am I wrong?
Andreas
Pascal J. Bourguignon
Not at all. Nobody can tell what the arguments to a macro are, until
the macro returns a form.
(defmacro m (a b c)
`(print "Hello"))
(m (m 1 2 3) (m 4 5 6) (m m m m))
--
__Pascal Bourguignon__
Stanisław Halik
> I think the simple point that macros don't evaluate their arguments is
> the cause of the expansion order. Because arguments are not evaluated
> you are 'stuck' with the very first 'outer' macro.
Except for readmacros, which come handy when generating CASE/COND
bodies.
--
Nawet świnka wejdzie na drzewo kiedy ją chwalą.
Neil Toronto
> > 3. Macros seem to expand in the wrong order. Why isn't it postorder
> > (evaluation order)? It seems postorder expansion could solve quite a
> > few composition issues. As it is, macros expand preorder, which runs a
> > little counter to intuition.
>
> You can have postorder, if you want to, by calling macroexpand in your
> macro body yourself. You should watch out for the &environment parameter
> in that case, and you probably need a more complete code walker.
> However, the situations where this is useful are very rare. (Google for
> "expansion-passing style", though, which is quite similar.)
I read the original paper, and this seems like a fantastic idea. Why
didn't it catch on?
> Getting preorder if you already have postorder would be much harder.
Point.
Thanks for your help, including the stuff I snipped. :)
Neil
Victor Kryukov
>> 3. Macros seem to expand in the wrong order. Why isn't it postorder
>> (evaluation order)? It seems postorder expansion could solve quite a
>> few composition issues. As it is, macros expand preorder, which runs a
>> little counter to intuition.
>
> You can have postorder, if you want to, by calling macroexpand in your
> macro body yourself. You should watch out for the &environment
> parameter in that case, and you probably need a more complete code
> walker. However, the situations where this is useful are very
> rare. (Google for "expansion-passing style", though, which is quite
> similar.)
>
Coincidentally, I've just wrote[1] a simple extension for CL-WHO which
allows with-html-output to expand some of the macros in its body
before applying main CL-WHO machinery.
E.g. you can right
(defpackage :test
(:use :cl :cl-who))
(in-package :test)
(def-syntax-macro html-list (&body body)
`(:ul
,@(loop
for elem in body
collect `(:li ,elem))))
(with-html-output-to-string (s)
(:html
(:head
(:title "Test"))
(:body
(:h1 "A glorious example")
(html-list
(:strong "First")
"Second"
"Third"))))
and html-list will be expanded _before_ with-html-output-to-string
will see it[2], thus producing
"<html><head><title>Test</title></head><body><h1>A glorious example</h1><ul><li><strong>First</strong></li><li>Second</li><li>Third</li></ul></body></html>"
Main functionality is achieved by the following code:
(defun macroexpand-tree (tree)
"Recursively expands all macros from *macro-to-expand* list in TREE."
(apply-to-tree
(lambda (subtree)
(macroexpand-tree (macroexpand-1 subtree)))
(lambda (subtree)
(and (consp subtree)
(member (first subtree) *macro-to-expand*
:test #'eq)))
tree))
where *macro-to-expand* is a list of symbol names of macros we have to
expand and (apply-to-tree fn test tree) applies FN to every subtree
(not only nodes) of TREE that satisifies TEST predicate.
Victor.
[1]
http://common-lisp.net/pipermail/cl-who-devel/2008-February/000127.html
[2] Or more precisely, the result will be the same as if html-list
was expanded before old, unmodified with-html-output has seen it, as I
obviously had to modify with-html-output and some auxilary functions.
Pascal Costanza
> On Feb 12, 3:46 am, Pascal Costanza <p...@p-cos.net> wrote:
>>> 3. Macros seem to expand in the wrong order. Why isn't it postorder
>>> (evaluation order)? It seems postorder expansion could solve quite a
>>> few composition issues. As it is, macros expand preorder, which runs a
>>> little counter to intuition.
>> You can have postorder, if you want to, by calling macroexpand in your
>> macro body yourself. You should watch out for the &environment parameter
>> in that case, and you probably need a more complete code walker.
>> However, the situations where this is useful are very rare. (Google for
>> "expansion-passing style", though, which is quite similar.)
>
> I read the original paper, and this seems like a fantastic idea. Why
> didn't it catch on?
As far as I know, Bigloo uses expansion-passing style macros at its core.
Furthermore, Common Lisp's macro system is actually quite close to
expansion-passing style due to the possibility to access the current
lexical environment via macroexpand, and to modify the lexical
environment for enclosed code by generating new local macros.
frank.adr...@gmail.com
> 1. A language doesn't need to be devoid of syntax in order to
> implement macros, as long as there's a simple, unambiguous mapping
> from surface syntax to the syntax tree.
It doesn't need to. However, the simpler the syntax, the easier it is
to write a macro. Think of macros as code generators. Depending on
the code fragment being generated, there are contextual constraints
that need to be satisfied. The less onerous the contextual
constraints, the more use you can make of the generated code and the
easier it is to write the macro.
For instance, lets say I was writing a macro for C. Depending on the
context, were I generating code for an if statement, I would either
need to use the statement syntax:
if (<expr>)
<stmt>;
else
<stmt>;
or the expression syntax:
(<expr>)?<expr>:<expr>;
For general use, I would need a mechanism that looked at the context
where the macro was invoked, and a decision process about which of
these to use. This means that the macro would need to be much more
involved than a simple lisp macro that would produce:
(if <expr> <expr> <expr>)
which could be used in either context. In short, you cannot hide the
difference between a statement and an expression in C and this filters
through to the macro system.
Also, you'll need different sub-tree builders for each kind of
statement (and although you can use pure textual substitution, you're
not gaining much power over standard C macros in that case). You'll
have to understand operator precedence (including the structure of the
containing tree). The same thing happens with other nits of syntax
all over the place.
I worked on a system that did code generation in Smalltalk. Although
Smalltalk has a very minimal syntax, even then one had to know whether
one was generating expressions or methods (for return purposes) and
had to understand the context for particular generated blocks.
The bottom line is that, yes, as long as you have a syntax tree, node
generation tools, and splicing tools, you can build a macro system.
However, the more heterogeneous your tree structure is, the harder it
will be to write the macros and to know where you can use them.
faa