Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

What's up with Scheme macros?

86 views
Skip to first unread message

Ken Tilton

unread,
Feb 12, 2008, 4:46:15 AM2/12/08
to
While trying to explain my irrational prejudice against Scheme macros I
got curious as to why I hated them, so I googled up this:

http://www.xs4all.nl/~hipster/lib/scheme/gauche/define-syntax-primer.txt

Boys and girls, that is almost eleven thousand words long!!!!!

I see the problem, they have this whole pattern-matching syntax-rules
language moving from source input to output. I have done (and am
supposed to be doing) enough Prolog to realize Scheme macro writers must
be way smarter than me. But... why? No, not why are they smarter. Why do
they have to be smarter? What was wrong with treating the source as an
ordinary tree of data and just whinging it about with Scheme itself to
produce the output? Please restrict yourself to one or more of these
answers:

1. The hygiene made us do it.

2. We actually thought it would be way cool and super powerful.

3. We were embarrassd by the small size of the standard and decided to
make it up in the macro language

4. Other _____________________

Jes curious,

kenny


--
http://smuglispweeny.blogspot.com/
http://www.theoryyalgebra.com/

"In the morning, hear the Way;
in the evening, die content!"
-- Confucius

Shiro Kawai

unread,
Feb 12, 2008, 5:14:13 AM2/12/08
to
I use both (CL and Scheme) at work but more inclined to Scheme.
And I do have a mixed feeling about the Scheme macros.

Ken Tilton <kennytil...@optonline.net> wrote:
> 1. The hygiene made us do it.

If I need to pick one from the first three, probably this is it.

Theoretically, hygiene has nothing to do with pattern matching
sublanguage.
The pattern matching part is for the convenience and efficiency, if
I understand correctly. To realize hygiene, every identifier has to
carry the syntactic context, which is added or removed in every macro
expansion.

It would be very inefficient to do that context handling
eagerly if the macro's input form is huge S-expr but the macro
expander
only shuffles its toplevel elements, so you want to do the context
handling lazily---that is, instead of applying the context to every
leaf
of the S-expr, you just feed a tuple of <S-expr, Context> to the macro
expander, which "peels" off the context on demand (e.g. <(a (b c)),
Context>
is transfomed to (<a, Context> <(b c), Context>) if the expander needs
to see the toplevel elements.) Because of this tupling, you cannot
use
ordinary car/cdr to decompose the input.

The reason I have a mixed feeling is that I do like hygiene (it is
cool
that an exported macro of my module foo expands into a form that
inserts
the reference to the foo's internal macros/procedures and it just
works.
I'd use package prefix in CL, but I feel it's like hack.) but carrying
pattern language is... say, too heavy. Lots of smart people have
tackled
this problem so there may not be the way to avoid that, but I hope
otherwise.

For the meantime, I use both legacy macros and hygienic marcos in my
Scheme code at work; legacy macros are handy if you have total control
over the source so that nobody will step on your toe or vice versa.

Pascal Costanza

unread,
Feb 12, 2008, 6:22:06 AM2/12/08
to
Ken Tilton wrote:
> While trying to explain my irrational prejudice against Scheme macros I
> got curious as to why I hated them, so I googled up this:
>
> http://www.xs4all.nl/~hipster/lib/scheme/gauche/define-syntax-primer.txt
>
> Boys and girls, that is almost eleven thousand words long!!!!!
>
> I see the problem, they have this whole pattern-matching syntax-rules
> language moving from source input to output. I have done (and am
> supposed to be doing) enough Prolog to realize Scheme macro writers must
> be way smarter than me. But... why? No, not why are they smarter. Why do
> they have to be smarter? What was wrong with treating the source as an
> ordinary tree of data and just whinging it about with Scheme itself to
> produce the output? Please restrict yourself to one or more of these
> answers:
>
> 1. The hygiene made us do it.

No, there are hygienic macro systems where source is a tree of data.
Actually, in most hygienic macro systems, source is still a tree of data.

> 2. We actually thought it would be way cool and super powerful.

Lisp-style macros (as in Common Lisp) provide backquote as a convenient
means to construct new s-expressions. One nice thing about backquote is
that you can make backquote expressions look very close the code you
have in your mind. (Compare `(progn ,@body) to (cons 'progn body), for
example.)

Common Lisp also provides destructuring macro lambda lists, where you
can already preprocess the arguments a macro receives by tearing them
apart into subparts. For example: (defmacro with ((&rest bindings) &body
body) ...) instead of (defmacro with (&rest code) ...).

Backquote works pretty well. However, destructuring macro lambda lists
are limited - note how to you can't see the constituents of the bindings
list in the with example. You have to 'manually' tear them apart. (Loop
is pretty good at such things, but you still have to do it.)

If you want to make destructuring more fine-grained, you also have to
make changes to the corresponding construct for reconstructing code
(like backquote). That's where the complexity comes in.

Syntax-rules and syntax-case work quite well in that regard (although I
personally prefer to use LOOP).

> 3. We were embarrassd by the small size of the standard and decided to
> make it up in the macro language

They could have kept the spec smaller by sticking to Lisp-style macros
and adding a rename construct. The other macro systems can be built on
top of that. I guess removing the weaknesses and restrictions that make
additional features appear necessary is not so important after all. :-P

To be more serious: Syntax-rules is a macro system that is very simple
to use for very simple macros. You sometimes want to define your own
macros to illustrate some points in academic papers. Syntax-rules macros
are more than good enough for such purposes, and the macro definitions
can be read by people who are not used to macro programming. That's a
good thing, if the macro is not the main part of what you want to
discuss. Macro definitions in such papers hardly need to be complex (and
can always be simplified for illustration purposes).

You know, there is more you can do with programming languages than
writing applications...


Pascal

--
1st European Lisp Symposium (ELS'08)
http://prog.vub.ac.be/~pcostanza/els08/

My website: http://p-cos.net
Common Lisp Document Repository: http://cdr.eurolisp.org
Closer to MOP & ContextL: http://common-lisp.net/project/closer/

Jens Axel Soegaard

unread,
Feb 12, 2008, 8:21:09 AM2/12/08
to
Ken Tilton wrote:
> While trying to explain my irrational prejudice against Scheme macros I
> got curious as to why I hated them, so I googled up this:
>
> http://www.xs4all.nl/~hipster/lib/scheme/gauche/define-syntax-primer.txt
>
> Boys and girls, that is almost eleven thousand words long!!!!!

And "On Lisp" is even longer!!!!!

"Primer" is obviously not to be taken literally.

> I see the problem, they have this whole pattern-matching syntax-rules
> language moving from source input to output. I have done (and am
> supposed to be doing) enough Prolog to realize Scheme macro writers must
> be way smarter than me. But... why? No, not why are they smarter. Why do
> they have to be smarter? What was wrong with treating the source as an
> ordinary tree of data and just whinging it about with Scheme itself to
> produce the output?

The use of pattern matching is a detail.

That's not why Schemers abandoned the old way. There are *two* goals of
modern Scheme macro systems. Hygiene (i.e. automatic renaming) as
another Paris Hilton got all the press, while the referentially
transparent property was left behind in the dust.

Referentially transparent macros?

If a macro transformer inserts a free reference to an identifier,
the reference refers to the binding that was visible where the
transformer was specified, regardless of any local bindings that may
surround the use of the macro.

In short the meaning of the expansion of a macro doesn't depend on where
it is used. In order to implement that you need to attach information
to variables, and thus symbols are replaced by "syntax-objects
representing symbols".


For a CL view on the matters see these posts in a very informative
thread on Scheme macros in comp.lang.lisp in 2004.

Anton van Straaten in comp.lang.lisp:
http://groups.google.com/group/comp.lang.lisp/msg/4cfdf326287b3d01

Pascal Constanza in comp.lang.lisp:
http://groups.google.com/group/comp.lang.lisp/msg/67d80b9d8da8c085

Anyhow, stop reading about syntax-rules (so last millenium) and treat
yourself with some light reading on syntax-case. Dybvig's

Writing Hygenic Macros in Scheme with Syntax-Case
ftp://ftp.cs.indiana.edu/pub/scheme-repository/doc/pubs/iucstr356.ps.gz

is a must read.

So is Dybvig's contribution to the book "Beautiful code":
http://www.cs.indiana.edu/~dyb/pubs/bc-syntax-case.pdf


--
Jens Axel Søgaard

Kjetil S. Matheussen

unread,
Feb 12, 2008, 8:32:26 AM2/12/08
to
On Tue, 12 Feb 2008, Jens Axel Soegaard wrote:

> Ken Tilton wrote:
> > While trying to explain my irrational prejudice against Scheme macros I
> > got curious as to why I hated them, so I googled up this:
> >
> > http://www.xs4all.nl/~hipster/lib/scheme/gauche/define-syntax-primer.txt
> >
> > Boys and girls, that is almost eleven thousand words long!!!!!
>
> And "On Lisp" is even longer!!!!!
>
> "Primer" is obviously not to be taken literally.
>
> > I see the problem, they have this whole pattern-matching syntax-rules
> > language moving from source input to output. I have done (and am
> > supposed to be doing) enough Prolog to realize Scheme macro writers must
> > be way smarter than me. But... why? No, not why are they smarter. Why do
> > they have to be smarter? What was wrong with treating the source as an
> > ordinary tree of data and just whinging it about with Scheme itself to
> > produce the output?
>
> The use of pattern matching is a detail.
>
> That's not why Schemers abandoned the old way.

I wouldn't say that. My impression is that most scheme implementations,
and all the large ones (anyone who doesn't?), support define-macro. I
would actually also be surprised if hygenic macros is more used by
schemeres than "the old way", although I don't have any data to back
that up with. Anyway, I would never use a scheme where you couldn't
write macros the old way.

andreu...@yahoo.com

unread,
Feb 12, 2008, 10:00:32 AM2/12/08
to
On Feb 12, 5:14 am, Shiro Kawai <shiro.ka...@gmail.com> wrote:

> It would be very inefficient to do that context handling
> eagerly if the macro's input form is huge S-expr but the macro
> expander
> only shuffles its toplevel elements, so you want to do the context
> handling lazily---that is, instead of applying the context to every
> leaf
> of the S-expr, you just feed a tuple of <S-expr, Context> to the macro
> expander, which "peels" off the context on demand (e.g. <(a (b c)),
> Context>
> is transfomed to (<a, Context> <(b c), Context>) if the expander needs
> to see the toplevel elements.) Because of this tupling, you cannot
> use
> ordinary car/cdr to decompose the input.

That is not accurate, but the misconception is understandable given
the language of R6RS.

For example, the explicit renaming system can be
implemented with linear complexity with ordinary s-expressions that
are
decomposable using car, cdr, etc. The same is true for syntax-case.
For example, my (Andre van Tonder's) syntax-case expander uses an
eager
algorithm based on a variant of explicit renaming and using ordinary
s-expressions decomposable using car/cdr. Thus, syntax-case can be
viewed as a convenience layer on top of explicit renaming.
Explicit renaming is really quite simple. See
http://groups.google.com/group/comp.lang.scheme/msg/a4817ec9ffcccf39

I was disappointed that the R6RS editors decided to describe the lazy
algorithm in the R6RS standard, when a simple declarative description
of
hygiene/referential transparency as in R5RS would have sufficed.
Instead they chose to describe, as part of the standard, a specific,
non-unique implementation strategy, in fact probably the most
complicated implementation strategy anyone has bothered to
come up with to date, virtually ensuring that hardly anyone will
understand the document. I don't follow it, and I'm an implementor!


Jens Axel Soegaard

unread,
Feb 12, 2008, 11:11:51 AM2/12/08
to
Kjetil S. Matheussen wrote:

>> That's not why Schemers abandoned the old way.
>
> I wouldn't say that. My impression is that most scheme implementations,
> and all the large ones (anyone who doesn't?), support define-macro. I
> would actually also be surprised if hygenic macros is more used by
> schemeres than "the old way", although I don't have any data to back
> that up with. Anyway, I would never use a scheme where you couldn't
> write macros the old way.

Nowadays there are (depending on point of view) two different
types of define-macro forms. One type is define-macro implemented
on top of define-syntax - this type is generally found in
implementations that have "full" syntax-case integration.

The other one is found in implementations where syntax-case
is treated as an "add on" builds on a "native" define-macro.

The two types are for simple macros equivalent, but for macros
that require phase 2 [*] computations hell is likely to break
loose.

There are two things I look for, when I evaluate macro systems:

1) How easy is it to give error messages in terms of the
*original* user syntax?

2) Can the module system handle the macro system?
[I.e. how (if at all) is the phase problem solved?]

Most "native" define-macro systems handle these points on an
ad hoc basis.


[*]
http://pre.plt-scheme.org/docs/html/reference/Evaluation_Model.html#(part~20module-phase)

andreu...@yahoo.com

unread,
Feb 12, 2008, 11:26:54 AM2/12/08
to
On Feb 12, 4:46 am, Ken Tilton <kennytil...@optonline.net> wrote:

> I see the problem, they have this whole pattern-matching syntax-rules
> language moving from source input to output. I have done (and am
> supposed to be doing) enough Prolog to realize Scheme macro writers must
> be way smarter than me. But... why? No, not why are they smarter. Why do
> they have to be smarter? What was wrong with treating the source as an
> ordinary tree of data and just whinging it about with Scheme itself to
> produce the output?

Nothing is wrong with your suggestion. Here is a simple macro
that will work in one of the R6RS reference implementations.

(define-syntax swap!
(lambda (form)
(let ((a (cadr form))
(b (caddr form)))
#`(let ((temp #,a))
(set! #,a #,b)
(set! #,b temp))))))

It is almost identical to the corresponding defmacro, except that
you use #` and #, instead of ` and ,. Also, no gensym is necessary.
You can think, as a fisrt approximation, of #` as automatically
inserting the necessary gensyms for you.

Andre

Raffael Cavallaro

unread,
Feb 12, 2008, 12:31:36 PM2/12/08
to
On 2008-02-12 11:26:54 -0500, andreu...@yahoo.com said:

> Nothing is wrong with your suggestion. Here is a simple macro
> that will work in one of the R6RS reference implementations.
>
> (define-syntax swap!
> (lambda (form)
> (let ((a (cadr form))
> (b (caddr form)))
> #`(let ((temp #,a))
> (set! #,a #,b)
> (set! #,b temp))))))
>
> It is almost identical to the corresponding defmacro, except that
> you use #` and #, instead of ` and ,. Also, no gensym is necessary.
> You can think, as a fisrt approximation, of #` as automatically
> inserting the necessary gensyms for you.

Not that I doubt the power of your system Andre, but could you show us
a simple example of intentional capture inside a #` form?

Maciej Katafiasz

unread,
Feb 12, 2008, 12:45:09 PM2/12/08
to
Den Tue, 12 Feb 2008 04:46:15 -0500 skrev Ken Tilton:

> While trying to explain my irrational prejudice against Scheme macros I
> got curious as to why I hated them, so I googled up this:
>
> http://www.xs4all.nl/~hipster/lib/scheme/gauche/define-syntax-
primer.txt
>
> Boys and girls, that is almost eleven thousand words long!!!!!

Woah there, back in my day, a three-article technical *series* was
absolutely, unquestionably, your editor is really good at using Delete
thank you very much limited to 6K words (including intros, conclusions
and recaps). So that's two complete series there.

> Please restrict yourself to one or more of these
> answers:
>

> 4. Other _____________________

They had to write articles with strict wordcount limits before and decide
that this time fuck it, *they* will decide when to stop.

Cheers,
Maciej

michele....@gmail.com

unread,
Feb 12, 2008, 1:56:50 PM2/12/08
to
I have been skeptical about Scheme macros for a long time,
thinking that syntax-rules is too simple and syntax-case too complex,
and I have been using define-macro instead. However, in the last
couple of months I have somewhat changed my mind. I figured out that
the biggest issue I had with syntax-case was notational. If you are
willing to dress it with a minor amount of sugar, syntax-case becomes
pretty easy to use and quite readable too. Here is the swap! example:

(define-syntax-case swap! ()
((swap! x y) (let ((t x)) (set! x y) (set! y t))))

define-syntax-case is just sugar around syntax-case:

;; tested on Ikarus Scheme
(define-syntax define-syntax-case
(syntax-rules ()
((_ name (literal ...)
((_ arg ...) templ) ...)
(define-syntax name
(lambda (x)
(syntax-case x (<expand> literal ...)
((ctx <expand> arg ...) #`'templ) ...
((ctx arg ...) #`templ) ...))))
((_ name (literal ...)
((_ arg ...) fender templ) ...)
(define-syntax name
(lambda (x)
(syntax-case x (<expand> literal ...)
((ctx <expand> arg ...) fender #`'templ) ...
((ctx arg ...) fender #`templ) ...))))
))

Notice that I also added an <expand> feature to see the expansion
of the macro:

(swap! <expand> x y) ;=> (let ((t x)) (set! x y) (set! y t))


Michele Simionato

andreu...@yahoo.com

unread,
Feb 12, 2008, 2:07:56 PM2/12/08
to
On Feb 12, 12:31 pm, Raffael Cavallaro <raffaelcavallaro@pas-d'espam-
s'il-vous-plait-mac.com> wrote:

> Not that I doubt the power of your system Andre, but could you show us
> a simple example of intentional capture inside a #` form?

(define-syntax if-it
(lambda (form)
(let ((it (datum->syntax (car form) 'it)))
#`(let ((,it ,(cadr form)))
(if #,it
#,(caddr form)
#,(cadddr form))))))

DATUM->SYNTAX overrides hygiene by creating the identifier IT as if
it
appeared in the source at the position of (CAR FORM), i.e., the IF-IT
at the usage site of the macro. You may think of this, very roughly,
as a standard way of making an identifier that is not affected by the
automatic gensym that would otherwise be applied by #`.

This is slightly longer than the defmacro way, but macros that
capture
are usually a minority. Most defmacros can be trivially converted
to Scheme hygienic macros (in this R6RS reference implementation)
by omitting gensyms and putting #s in front of the `s,'s, and ,s
and would therefore be shortened rather than lengthened.

Unfortunately this is not true for all implementations of R6RS Scheme
macros. Some unfortunately do not represent code as s-expressions
and rely on pattern matching to decompose their input.
While there are reasons for their design choice, I think it very
unfortunately obscures the fact that pattern matching is absolutely
orthogonal to - and unnecessary for - the hygiene mechanism. As a
result,
people like the OP get the impression that hygiene is more
complicated
than it really is, and cannot entirely be blamed for giving
up in disgust.

Andre

andreu...@yahoo.com

unread,
Feb 12, 2008, 2:37:54 PM2/12/08
to
On Feb 12, 2:07 pm, andreuri2...@yahoo.com wrote:

> (define-syntax if-it
> (lambda (form)
> (let ((it (datum->syntax (car form) 'it)))
> #`(let ((,it ,(cadr form)))
> (if #,it
> #,(caddr form)
> #,(cadddr form))))))


Sorry. I meant:

(define-syntax if-it
(lambda (form)
(let ((it (datum->syntax (car form) 'it)))

#`(let ((#,it #,(cadr form)))


(if #,it
#,(caddr form)
#,(cadddr form))))))

Andre

Abdulaziz Ghuloum

unread,
Feb 12, 2008, 3:24:15 PM2/12/08
to
michele....@gmail.com wrote:

> (define-syntax define-syntax-case
> (syntax-rules ()
> ((_ name (literal ...)
> ((_ arg ...) templ) ...)
> (define-syntax name
> (lambda (x)
> (syntax-case x (<expand> literal ...)
> ((ctx <expand> arg ...) #`'templ) ...
> ((ctx arg ...) #`templ) ...))))
> ((_ name (literal ...)
> ((_ arg ...) fender templ) ...)
> (define-syntax name
> (lambda (x)
> (syntax-case x (<expand> literal ...)
> ((ctx <expand> arg ...) fender #`'templ) ...
> ((ctx arg ...) fender #`templ) ...))))
> ))

I wonder how this would look using s-expression manipulation
(cars/cdrs). Andre, since you seem to be the expert on this,
do you want to give it a shot?

Eli Barzilay

unread,
Feb 12, 2008, 3:45:14 PM2/12/08
to
"Kjetil S. Matheussen" <k.s.mat...@notam02.no> writes:

> I wouldn't say that. My impression is that most scheme
> implementations, and all the large ones (anyone who doesn't?),
> support define-macro. I would actually also be surprised if hygenic
> macros is more used by schemeres than "the old way", although I

> don't have any data to back that up with. [...]

Out of curiosity, I inspected the PLT collects tree. Here's a
summary:

* about 400K lines of Scheme code

* 12 occurrences of `define-macro'; 5 are in the define-macro library
source, 3 in comments, 4 are in misc meta-code (keyword colors,
indentation, etc), that leaves 0 uses in real code.

* 1438 occurrences of `define-syntax' (and `...-syntaxes')

* 1360 of `syntax-case'

* 468 of `syntax-rules'

It's interesting to compare this to version 103 (which is before PLT
switched to `syntax-case' being native):

* About 150K lines

* 125 `define-macro'

* 7 `define-syntax', 1 `syntax-rules', 1 `syntax-case' none is
actually used

And here's an interesting result: if the switch to the new system
would mean changing all `define-macro's to `define-syntax', and
assuming the same "macro use density" per line of code, then you'd
expect to see about 300 syntax definitions. So the new macro system is
robust enough that it is used about 4.5 times more often than
define-macro.

--
((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay:
http://www.barzilay.org/ Maze is Life!

Eli Barzilay

unread,
Feb 12, 2008, 3:50:02 PM2/12/08
to
Ken Tilton <kenny...@optonline.net> writes:

> While trying to explain my irrational prejudice against Scheme
> macros I got curious as to why I hated them, so I googled up this:
>
> http://www.xs4all.nl/~hipster/lib/scheme/gauche/define-syntax-primer.txt
>
> Boys and girls, that is almost eleven thousand words long!!!!!

Note the name of the document -- it describes `syntax-rules' and how
to do with it things that seem impossible from a very restricted
syntax rewriting system. Using *just* that is on the same masochistic
level as using *just* r5rs for real programming.

Abdulaziz Ghuloum

unread,
Feb 12, 2008, 4:55:30 PM2/12/08
to
Eli Barzilay wrote:

> So the new macro system is robust enough that it is
> used about 4.5 times more often than define-macro.

Or, it may be that you guys are just macro noobs. You
can't handle the power of lisp macros, and you want
this thing called "hygiene" to protect you from the so
called "unintended capture" straw-man that you throw
around at every occasion.

The real programmer does not need no stinkin' hygiene.

andreu...@yahoo.com

unread,
Feb 12, 2008, 6:03:52 PM2/12/08
to
On Feb 12, 3:24 pm, Abdulaziz Ghuloum <aghul...@cee.ess.indiana.edu>
wrote:

I'll leave that to the Lispniks, who have more practice
with this kind of thing. It might even occur to some of
them that pattern matching, despite being orthogonal to and
unnecessary for hygiene, is a quite useful thing to have.

Shiro Kawai

unread,
Feb 12, 2008, 7:01:33 PM2/12/08
to
andreuri2...@yahoo.com wrote:
> That is not accurate, but the misconception is understandable given
> the language of R6RS.
>
> For example, the explicit renaming system can be
> implemented with linear complexity with ordinary s-expressions that
> are
> decomposable using car, cdr, etc. The same is true for syntax-case.
> For example, my (Andre van Tonder's) syntax-case expander uses an
> eager
> algorithm based on a variant of explicit renaming and using ordinary
> s-expressions decomposable using car/cdr. Thus, syntax-case can be
> viewed as a convenience layer on top of explicit renaming.
> Explicit renaming is really quite simple. Seehttp://groups.google.com/group/comp.lang.scheme/msg/a4817ec9ffcccf39

Aha! I've seen the explicit renaming expander but have never looked
at
it closely. Now I read the articles and found it made a lot of sense,
and it feels more Scheme-way to me. I'll give it a shot. Thanks!

michele....@gmail.com

unread,
Feb 12, 2008, 11:41:56 PM2/12/08
to
On Feb 13, 12:03 am, andreuri2...@yahoo.com wrote:

> I'll leave that to the Lispniks, who have more practice
> with this kind of thing. It might even occur to some of
> them that pattern matching, despite being orthogonal to and
> unnecessary for hygiene, is a quite useful thing to have.

Actually one of the reasons why I changed my mind about Scheme
macros is that I realized the benefits of pattern matching
when writing complex macros. BTW, I just noticed that I have posted
the wrong version of define-syntax-case, the correct one is

(define-syntax define-syntax-case
(syntax-rules ()
((_ name (literal ...)

((ctx arg ...) templ) ...)


(define-syntax name
(lambda (x)
(syntax-case x (<expand> literal ...)
((ctx <expand> arg ...) #`'templ) ...
((ctx arg ...) #`templ) ...))))
((_ name (literal ...)

((ctx arg ...) fender templ) ...)

Alex Shinn

unread,
Feb 13, 2008, 8:39:28 AM2/13/08
to
>>>>> "Ken" == Ken Tilton <kenny...@optonline.net> writes:

[ ... why syntax-rules? ... ]

Ken> 4. Other _____________________

I won't presume as to the R5RS editors original reasons, but
I suspect it has to do with the fact that SYNTAX-RULES was
and remains the simplest and most natural macro system for
describing all of the derived syntax in R5RS itself.

There are plenty of other cases where SYNTAX-RULES clearly
wins. Just consider the confusion that a newbie recently
ran into trying to use DEFINE-MACRO:

http://list.cs.brown.edu/pipermail/plt-scheme/2008-February/022777.html

... and how SYNTAX-RULES saves the day:

http://list.cs.brown.edu/pipermail/plt-scheme/2008-February/022782.html

--
Alex

Eli Barzilay

unread,
Feb 13, 2008, 9:25:22 AM2/13/08
to
Abdulaziz Ghuloum <aghu...@cee.ess.indiana.edu> writes:

> Eli Barzilay wrote:
>
>> So the new macro system is robust enough that it is
>> used about 4.5 times more often than define-macro.
>
> Or, it may be that you guys are just macro noobs.

No.


> You can't handle the power of lisp macros, and you want this thing
> called "hygiene" to protect you from the so called "unintended
> capture" straw-man that you throw around at every occasion.
>
> The real programmer does not need no stinkin' hygiene.

--

Pascal Costanza

unread,
Feb 13, 2008, 3:32:07 PM2/13/08
to
andreu...@yahoo.com wrote:
> On Feb 12, 12:31 pm, Raffael Cavallaro <raffaelcavallaro@pas-d'espam-
> s'il-vous-plait-mac.com> wrote:
>
>> Not that I doubt the power of your system Andre, but could you show us
>> a simple example of intentional capture inside a #` form?
>
> (define-syntax if-it
> (lambda (form)
> (let ((it (datum->syntax (car form) 'it)))
> #`(let ((,it ,(cadr form)))
> (if #,it
> #,(caddr form)
> #,(cadddr form))))))
>
> DATUM->SYNTAX overrides hygiene by creating the identifier IT as if
> it appeared in the source at the position of (CAR FORM), i.e., the
> IF-IT at the usage site of the macro. You may think of this, very
> roughly, as a standard way of making an identifier that is not
> affected by the automatic gensym that would otherwise be applied by
> #`.

This sounds like I have to say twice where the identifier is inserted.
That's weird. Why is that the case?

> This is slightly longer than the defmacro way, but macros that
> capture are usually a minority.

Says who?

> Some unfortunately do not represent code as s-expressions and rely on
> pattern matching to decompose their input. While there are reasons
> for their design choice, I think it very unfortunately obscures the
> fact that pattern matching is absolutely orthogonal to - and
> unnecessary for - the hygiene mechanism. As a result, people like the
> OP get the impression that hygiene is more complicated than it really
> is, and cannot entirely be blamed for giving up in disgust.

Yes, you're macro system looks more attractive than syntax-case to me. I
understand that syntax-rules can be neat when it's all you need, but for
more complex stuff, s-expressions are simply better (IMHO, and at all
that, of course).

Does your system also support the kinds of macros that are mentioned for
example in
http://www.lispworks.com/documentation/HyperSpec/Issues/iss066_w.htm ?

Kaz Kylheku

unread,
Feb 13, 2008, 3:47:01 PM2/13/08
to
On Feb 13, 5:39 am, Alex Shinn <alexsh...@gmail.com> wrote:
> There are plenty of other cases where SYNTAX-RULES clearly
> wins.  Just consider the confusion that a newbie recently
> ran into trying to use DEFINE-MACRO:
>
>  http://list.cs.brown.edu/pipermail/plt-scheme/2008-February/022777.html
>
> ... and how SYNTAX-RULES saves the day:
>
>  http://list.cs.brown.edu/pipermail/plt-scheme/2008-February/022782.html

I can hardly make heads or tails out of the nested defspel. I need to
see the nested backquotes to see what is the meta-level and what is
meta-meta, otherwise my eyes glaze over. I can't see what part of the
template is literal, and what part is spliced in, because nothing is
denoting that. And look, even though things are quoted, they are still
substituted. You have expressions like (eq? 'object 'obj), but this is
really like `(eq? (quote ,object) (quote ,obj)). It looks like these
are simple quoted literal symbols, but no, there is still evaluation
taking place. Confusing! Suppose you're a sloppy Frenchman and make a
typo and write (eq? 'objet 'obj). The 'objet is now literal, because
objet isn't a macro parameter. That's evil!

This basically brings us back to the stone age of C language macros:

#define STR(X) #X
#define SPEL(OBJECT, SUBJECT) foo(SUBJECT, STR(OBJET))

Look at this, here, too, there is no ugly backquote to confuse
newbies! C preprocessor saves the day! Of course, when you call
SPEL(X, Y) you end up with foo(X, "OBJET") instead of foo(X, "Y").
Oops!

William D Clinger

unread,
Feb 13, 2008, 4:52:55 PM2/13/08
to
When you get right down to it, the real reason we
added syntax-case to Scheme was #' envy.

Will

Raffael Cavallaro

unread,
Feb 13, 2008, 6:30:45 PM2/13/08
to
On 2008-02-12 14:37:54 -0500, andreu...@yahoo.com said:

> Sorry. I meant:
>
> (define-syntax if-it
> (lambda (form)
> (let ((it (datum->syntax (car form) 'it)))
> #`(let ((#,it #,(cadr form)))
> (if #,it
> #,(caddr form)
> #,(cadddr form))))))
>
> Andre

Thanks. I can get this to work using:

(ex:repl '((import (rnrs)) ...


but is there some way to get an *interactive* repl in one of the
compatible schemes? I've tried the existing
r6rs-expander-vantonder.plt, but it's kind annoying to use. For
example, if I enter your definition above:

R5.97RS> (define-syntax if-it


(lambda (form)
(let ((it (datum->syntax (car form) 'it)))
#`(let ((#,it #,(cadr form)))
(if #,it
#,(caddr form)
#,(cadddr form))))))

Syntax violation: invalid reference

Attempt to use binding of lambda in library () at invalid level 1.
Binding is only available at levels: 0

Form: lambda

Trace:

(lambda (form)
(let ((it (datum->syntax (car form) 'it)))

(quasisyntax
(let (((unsyntax it) (unsyntax (cadr form))))
(if (unsyntax it) (unsyntax (caddr form)) (unsyntax (cadddr form)))))))

syntax-violation: Integrate with host error handling here

More or less defeats the purpose of a repl if I have to define a
library for such simple things - unless I'm misunderstanding the error
altogether.

William D Clinger

unread,
Feb 13, 2008, 10:12:38 PM2/13/08
to
Raffael Cavallaro wrote:
> Thanks. I can get this to work using:
>
> (ex:repl '((import (rnrs)) ...
>
> but is there some way to get an *interactive* repl in one of the
> compatible schemes?

Larceny's ERR5RS mode uses Andre's interactive REPL as
its native REPL:

% larceny -err5rs
Larceny v0.961 "Fluoridation" (Jan 2 2008 04:27:56,
precise:SunOS5:split)
ERR5RS mode (no libraries have been imported)

> (import (rnrs))
Autoloading (rnrs)
Autoloading (rnrs enums)
Autoloading (rnrs lists)
Autoloading (rnrs syntax-case)
Autoloading (rnrs hashtables)
Autoloading (rnrs arithmetic bitwise)
Autoloading (rnrs programs)
Autoloading (rnrs files)
Autoloading (rnrs io ports)
Autoloading (larceny deprecated)
Autoloading (rnrs conditions)
Autoloading (rnrs exceptions)
Autoloading (rnrs records syntactic)
Autoloading (err5rs records procedural)
Autoloading (rnrs records procedural)
Autoloading (rnrs control)
Autoloading (rnrs sorting)
Autoloading (rnrs bytevectors)
Autoloading (rnrs unicode)

> (define-syntax if-it
(lambda (form)
(let ((it (datum->syntax (car form) 'it)))
#`(let ((#,it #,(cadr form)))
(if #,it
#,(caddr form)
#,(cadddr form))))))

>

Will

llama

unread,
Feb 14, 2008, 12:25:58 AM2/14/08
to
"William D Clinger" <cesu...@yahoo.com> wrote in message
news:5c7a0076-f785-4011...@e25g2000prg.googlegroups.com...


That's nice. Now show an usage example. Every Scheme I tried choked on on
the application of that macro.

Abdulaziz Ghuloum

unread,
Feb 14, 2008, 1:15:03 AM2/14/08
to
llama wrote:

> Now show an usage example. Every Scheme I tried choked on
> on the application of that macro.

Andre already told you that this macro that he's written uses
an extension that's not supported by any any implementation
except larceny (by virtue of using his system). Fortunate or
unfortunate, there *is* a portable way (as far as R6RS goes)
of writing this macro as shown below (it might run on other
systems that I have not tested). The portable version also
happens to be more robust--it gives a better error message if
the input is malformed instead of giving the ``error in
caddr'' sort of error that you'd get otherwise.

Pick the version that better suits your needs.

Aziz,,,

$ ikarus
Ikarus Scheme version 0.0.3+ (revision 1384, build 2008-02-13)
Copyright (c) 2006-2008 Abdulaziz Ghuloum

> (define-syntax if-it
(lambda (stx)
(syntax-case stx ()
((ctxt test conseq altern)
(let ((it (datum->syntax #'ctxt 'it)))
#`(let ((#,it test))
(if #,it conseq altern)))))))
> (if-it 17 it 18)
17


$ larceny -err5rs
Larceny v0.961 "Fluoridation" (Jan 2 2008 04:30:22, precise:Posix
Unix:unified)
larceny.heap, built on Wed Jan 2 04:41:29 EST 2008


ERR5RS mode (no libraries have been imported)

> (import (rnrs))
Autoloading ...

> (define-syntax if-it
(lambda (stx)
(syntax-case stx ()
((ctxt test conseq altern)
(let ((it (datum->syntax #'ctxt 'it)))
#`(let ((#,it test))
(if #,it conseq altern)))))))

> (if-it 17 it 18)
17


$ mzscheme
Welcome to MzScheme v3.99.0.10 [3m], Copyright (c) 2004-2008 PLT Scheme Inc.
> (define-syntax if-it
(lambda (stx)
(syntax-case stx ()
((ctxt test conseq altern)
(let ((it (datum->syntax #'ctxt 'it)))
#`(let ((#,it test))
(if #,it conseq altern)))))))
> (if-it 17 it 18)
17


$ petite
Petite Chez Scheme Version 7.4
Copyright (c) 1985-2007 Cadence Research Systems

> (define-syntax if-it
(lambda (stx)
(syntax-case stx ()
((ctxt test conseq altern)
(let ((it (datum->syntax #'ctxt 'it)))
#`(let ((#,it test))
(if #,it conseq altern)))))))
> (if-it 17 it 18)
17


William D Clinger

unread,
Feb 14, 2008, 1:24:30 AM2/14/08
to
llama wrote:
> That's nice. Now show an usage example. Every Scheme I tried choked on on
> the application of that macro.

It works in Larceny. See below.

Will

--------

% larceny -err5rs
Larceny v0.961 "Fluoridation" (Jan 19 2008 12:07:22,

> (if-it #t 'then 'else)
then

> (if-it #f 'then 'else)
else

> (if-it #t it 'else)
#t

> ;; Now for a more familiar use of intentional capture.
(define-syntax myloop
(lambda (form)
(let ((break (datum->syntax (car form) 'break)))
#`(call/cc
(lambda (k)
(let ((#,break k))
(do ()
(#f)
(begin #,@(cdr form)))))))))

> (let ((x '(3 2 1 0 -1 -2 -3 -4)))
(myloop (if (zero? (car x))
(break (cdr x))
(set! x (cdr x)))))
(-1 -2 -3 -4)

Raffael Cavallaro

unread,
Feb 14, 2008, 1:52:02 AM2/14/08
to
On 2008-02-13 22:12:38 -0500, William D Clinger <cesu...@yahoo.com> said:

> Larceny's ERR5RS mode uses Andre's interactive REPL as
> its native REPL:
>
> % larceny -err5rs
> Larceny v0.961 "Fluoridation" (Jan 2 2008 04:27:56,
> precise:SunOS5:split)
> ERR5RS mode (no libraries have been imported)
>
>> (import (rnrs))

Ah, I remember now reading something about this on the larceny mailing
list. Thanks - I've just written, compiled, loaded, imported and run my
first err5rs/r6rs library - works nicely.

bunn...@gmail.com

unread,
Feb 14, 2008, 4:46:06 AM2/14/08
to
On Feb 12, 9:45 pm, Eli Barzilay <e...@barzilay.org> wrote:

> "Kjetil S. Matheussen" <k.s.matheus...@notam02.no> writes:
>
> > I wouldn't say that. My impression is that most scheme
> > implementations, and all the large ones (anyone who doesn't?),
> > support define-macro. I would actually also be surprised if hygenic
> > macros is more used by schemeres than "the old way", although I
> > don't have any data to back that up with. [...]
>
> Out of curiosity, I inspected the PLT collects tree. Here's a
> summary:
>
> * about 400K lines of Scheme code
>
> * 12 occurrences of `define-macro'; 5 are in the define-macro library
> source, 3 in comments, 4 are in misc meta-code (keyword colors,
> indentation, etc), that leaves 0 uses in real code.
>
> * 1438 occurrences of `define-syntax' (and `...-syntaxes')
>
> * 1360 of `syntax-case'
>
> * 468 of `syntax-rules'
>

As a comparison, here a grep through the chicken code repository
(including tagged versions):

define-macro: 9235
define-syntax: 5504
syntax-rules: 5249


cheers,
felix

William D Clinger

unread,
Feb 14, 2008, 8:44:31 AM2/14/08
to
Tabulating data collected by Eli Barzilay and
Felix Winkelmann:

PLT Chicken Larceny
define-macro 12 9235 67
define-syntax 1438 5504 1806
syntax-rules 468 5249 1703
syntax-case 1360 ? 271

For Larceny, most of the macros are defined in
libraries and benchmarks written by people who
are not themselves developers of Larceny.

Will

Pascal Costanza

unread,
Feb 14, 2008, 9:05:06 AM2/14/08
to
Alex Shinn wrote:
>>>>>> "Ken" == Ken Tilton <kenny...@optonline.net> writes:
>
> [ ... why syntax-rules? ... ]
>
> Ken> 4. Other _____________________
>
> I won't presume as to the R5RS editors original reasons, but
> I suspect it has to do with the fact that SYNTAX-RULES was
> and remains the simplest and most natural macro system for
> describing all of the derived syntax in R5RS itself.

I don't think that syntax rules. ;)

llama

unread,
Feb 14, 2008, 10:51:56 AM2/14/08
to
"Abdulaziz Ghuloum" <aghu...@cee.ess.indiana.edu> wrote in message
news:fp0m97$iki$1...@aioe.org...

> llama wrote:
>
>> Now show an usage example. Every Scheme I tried choked on on the
>> application of that macro.
>
> Andre already told you that this macro that he's written uses
> an extension that's not supported by any any implementation
> except larceny (by virtue of using his system).

Thanks, I must have missed that part.

William D Clinger

unread,
Feb 14, 2008, 4:40:06 PM2/14/08
to
Felix Winkelmann wrote:
> As a comparison, here a grep through the chicken code repository
> (including tagged versions):

If a macro is present in many tagged versions, then it
gets counted once for every version. That bothered me,
so I decided to do the grep myself for all three systems.
I did a wget from

http://pre.plt-scheme.org/plt-clean-tree.tgz

and an svn export from

https://galinha.ucpel.tche.br/svn/chicken-eggs/
http://larceny.ccs.neu.edu/svn/trunk/larceny_src

From chicken-eggs, I then deleted all directories named
"tags" or "branches". This left:

all files Scheme files
-------------------
MB # KLOC
PLT 85 MB 24 2655 627
Chicken 185 MB 53 5008 1470
Larceny 47 MB 12 1193 344

I then grepped the remaining .sch, .scm, and .ss files
for "define-macro", "define-syntax", "syntax-rules",
and "syntax-case". Results:

PLT Chicken Larceny
define-macro 66 4234 67
define-syntax 1908 3583 1807
syntax-rules 651 3297 1704
syntax-case 1675 1110 272

Each of the three columns shows a different pattern, so
generalizations based on a single programmer's experience
or on the code written for any one implementation should
be regarded with skepticism.

Will

Abdulaziz Ghuloum

unread,
Feb 14, 2008, 7:30:33 PM2/14/08
to
Ken Tilton wrote:
> What was wrong with treating the source as an ordinary tree
> of data and just whinging it about with Scheme itself to
> produce the output?

There is nothing really wrong with that. However, permitting
alternative representation of the source opens up the opportunity
for the implementation to do more. An example of that is having
the implementation report contextual information about where
syntax errors occurred and to keep that information around so
that debuggers and profilers show information in the context of
the original source code, not the expanded code.

Here is a small example to illustrate the first case.

$ cat test.ss
#!/usr/bin/env scheme-script
(import (rnrs))

(define-syntax foo
(syntax-rules ()
[(_ a b c) (let ([a b]) c)]))

(define-syntax bar
(syntax-rules ()
[(_ a b c) (foo b a c)]))

(foo a 12 (bar b 17 (list a b)))


$ ikarus --r6rs-script test.ss
Unhandled exception:
Condition components:
1. &who: let
2. &message: "not an identifier"
3. &syntax:
form: (let ((17 b)) (list a b))
subform: 17
4. &trace: #<syntax (let ((17 b)) (list a b))>
5. &trace: #<syntax (foo 17 b (list a b))>
6. &trace: #<syntax (bar b 17 (list a b)) [byte 200 of test.ss]>


As you can see, it shows the original file name and position in
terms of the user's original source code. It also shows the
macro expansion steps that contributed to producing that error.
The outer call to foo is not shown because it expanded properly.
The inner call to bar expanded to a call to foo, which expanded
to a let, which caught the error. The transformation steps are
shown in the trace output.

Some may think that this advantage does not outweigh dropping
the raw s-expression form. But others value things differently.
These are just different points in the design space. Neither
is "wrong": they just cater to different needs.

Aziz,,,

Alex Shinn

unread,
Feb 14, 2008, 7:34:49 PM2/14/08
to
On Feb 15, 6:40 am, William D Clinger <cesur...@yahoo.com> wrote:
>
> PLT Chicken Larceny
> define-macro 66 4234 67
> define-syntax 1908 3583 1807
> syntax-rules 651 3297 1704
> syntax-case 1675 1110 272

sc-macro-transformer 16

Though in all fairness 7 of those were from the
syntactic-closures egg itself.

Chicken has no active uses of er-macro-transformer.

--
Alex

michele....@gmail.com

unread,
Feb 15, 2008, 12:20:08 AM2/15/08
to
On Feb 15, 1:30 am, Abdulaziz Ghuloum <aghul...@cee.ess.indiana.edu>
wrote:list a b)))

>
> $ ikarus --r6rs-script test.ss
> Unhandled exception:
> Condition components:
> 1. &who: let
> 2. &message: "not an identifier"
> 3. &syntax:
> form: (let ((17 b)) (list a b))
> subform: 17
> 4. &trace: #<syntax (let ((17 b)) (list a b))>
> 5. &trace: #<syntax (foo 17 b (list a b))>
> 6. &trace: #<syntax (bar b 17 (list a b)) [byte 200 of test.ss]>

Why I don't get the traceback? My output stops at point 3. Are you
referring to a development version of Ikarus, or there is a debug
flag I should set somewhere to see the traceback? I am using
Ikarus 0.0.3.

Michele Simionato

Kaz Kylheku

unread,
Feb 15, 2008, 12:36:46 AM2/15/08
to
On Feb 14, 4:30 pm, Abdulaziz Ghuloum <aghul...@cee.ess.indiana.edu>
wrote:

> $ ikarus --r6rs-script test.ss
> Unhandled exception:
>   Condition components:
>     1. &who: let
>     2. &message: "not an identifier"
>     3. &syntax:
>         form: (let ((17 b)) (list a b))
>         subform: 17
>     4. &trace: #<syntax (let ((17 b)) (list a b))>
>     5. &trace: #<syntax (foo 17 b (list a b))>
>     6. &trace: #<syntax (bar b 17 (list a b)) [byte 200 of test.ss]>
>
> As you can see, it shows the original file name and position in
> terms of the user's original source code.  

S-expressions read from a file (and all their constituent
expressions), can be associated this information in a global hash
table. Given some erroneous form, the macro can see if there is
associated information with that form.

In principle, the origin of every literal object (or at least that of
every boxed one) that was read from some source file can be
pinpointed.

> Some may think that this advantage does not outweigh dropping
> the raw s-expression form.

It doesn't seem to /technically/ require such dropping.

> But others value things differently.

E.g. eating the cake and having it too.

Abdulaziz Ghuloum

unread,
Feb 15, 2008, 12:37:29 AM2/15/08
to
michele....@gmail.com wrote:

> Why I don't get the traceback? My output stops at point 3. Are you
> referring to a development version of Ikarus, or there is a debug
> flag I should set somewhere to see the traceback? I am using
> Ikarus 0.0.3.

I just added that yesterday :-) So, you can get it from the repo:

$ bzr checkout --lightweight http://www.cs.indiana.edu/~aghuloum/ikarus.dev

Abdulaziz Ghuloum

unread,
Feb 15, 2008, 12:52:34 AM2/15/08
to
Kaz Kylheku wrote:

>> Some may think that this advantage does not outweigh dropping
>> the raw s-expression form.
>
> It doesn't seem to /technically/ require such dropping.

Neither did I say it was, nor am I going to attempt to
prove otherwise. Actually, there may already exist some
system that somehow manages to pull it off, but to my
limited knowledge, I have not seen it done. Moreover,
adding the trace to my expander took no more than 45
minutes of doing minor edits to one file. Additionally,
the trace information (which is associated with the syntax
objects themselves) is not interned anywhere, and works
across files and will just work even with separately
compiled libraries.

All I show here is the advantage that *I* got from having
special syntax objects over having to use s-expressions.
Mileages vary.

Aziz,,,

Raffael Cavallaro

unread,
Feb 15, 2008, 1:54:58 AM2/15/08
to
On 2008-02-14 16:40:06 -0500, William D Clinger <cesu...@yahoo.com> said:

> Results:
>
> PLT Chicken Larceny
> define-macro 66 4234 67
> define-syntax 1908 3583 1807
> syntax-rules 651 3297 1704
> syntax-case 1675 1110 272
>
> Each of the three columns shows a different pattern, so
> generalizations based on a single programmer's experience
> or on the code written for any one implementation should
> be regarded with skepticism.

I wonder what the column for Gambit would look like - syntax-case and
define-syntax are not loaded by default in Gambit so I would suspect
they don't figure heavily in Gambit's own source.

andreu...@yahoo.com

unread,
Feb 15, 2008, 10:18:39 AM2/15/08
to
On Feb 13, 3:47 pm, Kaz Kylheku <kkylh...@gmail.com> wrote:

> I can hardly make heads or tails out of the nested defspel. I need to
> see the nested backquotes to see what is the meta-level and what is
> meta-meta, otherwise my eyes glaze over.

This may be a question of familiarity. I tend to glaze
over when confronted by the nested quasiquotations that tend to
appear
in macro-generating macros. Here is a snippet from
one of my own old define-macros

`(define-macro (,name ,k ,@args)
`(,,k ,',supers ,',labels ,@,args)))))

and a little quiz:

(let ((x '(a b c))) ``(,,x ,@,x ,,@x ,@,@x)) => ??

Define-syntax templates are easier for me in these situations.

> I can't see what part of the
> template is literal, and what part is spliced in, because nothing is
> denoting that.

That is certainly a valid complaint. It could be mitigated somewhat
by
a notational convention like

(define-syntax swap
(syntax-rules ()
((_ x y) (LET ((TEMP x))
(SET! x y)
(SET! y TEMP)))))

or by text colorization in a sufficiently smart editor. But this is
not typically something that seems to trip up Scheme macro writers
too much.

Joel J. Adamson

unread,
Feb 15, 2008, 9:22:44 AM2/15/08
to
Raffael Cavallaro <raffaelcavallaro@pas-d'espam-s'il-vous-plait-mac.com>
writes:

We were simultaneously having a similar discussion on gambit-list: A
quick check of the examples turned up (obviously) no uses of syntax-case
style macros. This discussion started with my mistaken presumption that
I needed to use syntax-case macros and dealing with the problems created
by loading the syntax-case library packaged with Gambit. The response
from a few readers was that Gambit favors define-macro, and indeed
that's all that I find in the examples and I, personally find their
construction more intuitive than syntax-rules templates.

Joel

--
Joel J. Adamson
Biostatistician
Pediatric Psychopharmacology Research Unit
Massachusetts General Hospital
Boston, MA 02114
(617) 643-1432
(303) 880-3109

llama

unread,
Feb 15, 2008, 10:56:59 AM2/15/08
to
<andreu...@yahoo.com> wrote in message
news:c1f34d0e-a312-46e7...@u10g2000prn.googlegroups.com...

> and a little quiz:
>
> (let ((x '(a b c))) ``(,,x ,@,x ,,@x ,@,@x)) => ??
>

Are the last 2 cases (,,@x and ,@,@x) even valid?

andreu...@yahoo.com

unread,
Feb 15, 2008, 12:02:51 PM2/15/08
to
On Feb 15, 10:56 am, "llama" <l...@winamp.com> wrote:
> <andreuri2...@yahoo.com> wrote in message

>
> > (let ((x '(a b c))) ``(,,x ,@,x ,,@x ,@,@x)) => ??
>
> Are the last 2 cases (,,@x and ,@,@x) even valid?

They are valid in R6RS Scheme. I'm not sure about
Common Lisp.

llama

unread,
Feb 15, 2008, 12:41:11 PM2/15/08
to
<andreu...@yahoo.com> wrote in message
news:789c3a40-a41f-46e4...@s8g2000prg.googlegroups.com...


Sorry, I didnt mean syntactically valid, I meant valid in the sense of the
computation. So will it produce a result?

What is the behaviour around unquoting/unquotesplicing a spliced list?

I prefer nested quasiquoting with a bit of space between them :)

Kaz Kylheku

unread,
Feb 15, 2008, 12:45:29 PM2/15/08
to
On Feb 15, 7:18 am, andreuri2...@yahoo.com wrote:
> On Feb 13, 3:47 pm, Kaz Kylheku <kkylh...@gmail.com> wrote:
>
> > I can hardly make heads or tails out of the nested defspel. I need to
> > see the nested backquotes to see what is the meta-level and what is
> > meta-meta, otherwise my eyes glaze over.
>
> This may be a question of familiarity. I tend to glaze
> over when confronted by the nested quasiquotations that tend to
> appear in macro-generating macros.
>
> Here is a snippet from
> one of my own old define-macros
>
>        `(define-macro (,name ,k ,@args)
>          `(,,k ,',supers ,',labels ,@,args)))))

Even in the absence of enough context, I know what's going on here. I
don't have the bindings of name, k, supers and labels are, but I can
see exactly how they are being integrated into the structure. I can
tell that define-macro is a literal symbol in the template, regardless
of any binding.

> and a little quiz:
>
>   (let ((x '(a b c))) ``(,,x ,@,x ,,@x ,@,@x))   => ??

The result is the equivalent of this object (itself, not its value):

`(,(a b c) ,@(a b c) ,a ,b ,c ,@a ,@b ,@c)

The ,x's are straightforward; they are simply replaced by (a b c). If
we look at in in the Scheme way, the splicing ,@x's under under an
unquote or unquote-splicing operator effectively cause it to have
multiple arguments. Both operators apply the same evaluation each of
their arguments, and then splice in the resulting values, resulting in
a distributive law: (unquote a b c) means the same thing as (unquote
a) (unquote b) (unquote c), and by means of this reduction to single-
argument forms, we can turn it back into the comma notation ,a ,b ,c.

I simply remember the distributive law in terms of the shorthand
operators themselves. ,,@x means that x is spliced in, and the inner
comma distributes into the elements of the splice.

Nested backquote is indeed something you have to ``get'', and that can
take time.

andreu...@yahoo.com

unread,
Feb 15, 2008, 1:41:29 PM2/15/08
to
Let me point out that the simplicity of defmacro is
sometimes exaggerated when compared to define-syntax.
For example, the following let-macro

(define-syntax let
(syntax-rules ()
((_ ((x v) ...) b1 b2 ...)
((lambda (x ...) b1 b2 ...) v ...))))

is /not/ the same as the superficially simple

(define-macro (let bindings . body)
`((lambda ,(map car bindings) ,@body) ,@(map cdr bindings)))

but rather expresses the following:

(define-macro (let . rest)
(unless
(and (list? rest)
(>= (length rest) 2))
(syntax-error 'let
"Does not match (((x v) ...) b1 b2 ...)" rest))
(unless
(and (list? (car rest))
(andmap (lambda (binding)
(= (length binding) 2))
(car rest)))
(syntax-error 'let
"Does not match ((x v) ...)" (car rest)))
`((lambda ,(map car (car rest)) ,@(cdr rest))
,@(map cdr (car rest))))

Andre

Abdulaziz Ghuloum

unread,
Feb 15, 2008, 2:23:23 PM2/15/08
to
andreu...@yahoo.com wrote:

> (and (list? (car rest))
> (andmap (lambda (binding)

(and (list? binding)

Kaz Kylheku

unread,
Feb 15, 2008, 2:34:16 PM2/15/08
to
On Feb 15, 9:02 am, andreuri2...@yahoo.com wrote:
> On Feb 15, 10:56 am, "llama" <l...@winamp.com> wrote:
>
> > <andreuri2...@yahoo.com> wrote in message
>
> > >  (let ((x '(a b c))) ``(,,x ,@,x ,,@x ,@,@x))   => ??
>
> > Are the last 2 cases (,,@x and ,@,@x) even valid?
>
> They are valid in R6RS Scheme.

By golly, R5RS doesn't seem define this, only R5R6, which allows
multiple arguments to unquote and unquote-splicing, and contains
nested backquote examples with this type of syntax.

Could it be really be that Scheme had broken backquotes until
recently? I'm rubbing my eyes.

> I'm not sure about Common Lisp.

The distributed-law behavior pops out of the axioms of the semantic
description of backquote expansion in the standard.

Essentially, `(x1 x2 x3 ... . atom) is defined as being equivalent to
(append [x1] [x2] ... [xn]), where if [xi] looks like ,form it is
interpreted as (list form) and consequently ,,@form means
(list ,@form). I.e. the unquote is defined in terms of LIST, which
handles additional arguments similarly to how the Scheme unquote does,
as of R6RS. The splicing case is similar. If [xi] looks like ,@form
it is simply reduced to form. So ,@,@form becomes ,@form.

Thus if we expand the inner backquote of this, and leave the outer one
alone, then this:

``(,,a ,,@b ,@,@c)

becomes this:

`(append (list ,a) (list ,@b) ,@c)

Thus if C is bound to a list of forms, they all get spliced into this
append. And so in the next evaluation, each of these forms (except the
last one) must produce lists which are spliced together. Thus the ,@
distributes into the ,@c.

Pascal Costanza

unread,
Feb 15, 2008, 4:12:42 PM2/15/08
to

Too complicated.

(defmacro let ((&rest bindings) &body body)
(loop for (var val . rest) in bindings
collect var into vars
collect val into vals
do (assert (null rest) ()
"Too many values for ~S." var)
finally (return `((lambda ,vars ,@body) ,@vals))))

LOOP is a pretty versatile tool for macro programming.

Alan Crowe

unread,
Feb 15, 2008, 4:09:13 PM2/15/08
to
andreu...@yahoo.com writes:

> and a little quiz:
>
> (let ((x '(a b c))) ``(,,x ,@,x ,,@x ,@,@x)) => ??
>

I thought I nearly knew this,

`(,(a b c) ,@(a b c) ,a ,b ,c and err, not sure...

but CMUCL is giving me either


`(,(A B C) ,@(A B C) ,A ,@B) with 19a

or

`(,(A B C) (,@(A B C)) (,A ,@B)) with 19c

while

SBCL says `(,(A B C) ,@(A B C) ,A ,B ,C ,@A ,@B ,@C)

which is convincing

and CLISP says

`(,(A B C) ,@(A B C) (SYSTEM::UNQUOTE A B C) ,@'(SYSTEM::UNQUOTE A B C))

so I'm wondering if there is a bug in CMUCL.

Alan Crowe
Edinburgh
Scotland

Jeff Barnett

unread,
Feb 15, 2008, 4:24:44 PM2/15/08
to
I'm Lisp curious, if there are any declarations (or strings) in the body
portion of this macro, will the behavior of the let and the lambda be
guaranteed identical?

-- Jeff Barnett

Kaz Kylheku

unread,
Feb 15, 2008, 8:15:22 PM2/15/08
to
On Feb 15, 1:09 pm, Alan Crowe <a...@cawtech.freeserve.co.uk> wrote:

> andreuri2...@yahoo.com writes:
> > and a little quiz:
>
> > (let ((x '(a b c))) ``(,,x ,@,x ,,@x ,@,@x)) => ??
>
> and CLISP says
>
> `(,(A B C) ,@(A B C) (SYSTEM::UNQUOTE A B C) ,@'(SYSTEM::UNQUOTE A B C))

Wow, how old is your CLISP? That looks suspiciously like the old
backquote implementation that was replaced some five years ago.

Now I'm a bit out of date myself here with 2.38, but I get:

[1]> (let ((x '(a b c))) ``(,,x ,@,x ,,@x ,@,@x))
(CONS (A B C) (APPEND (A B C) (LIST* A B C (APPEND A B C))))

It's not as pretty an answer as SBCL's, but right.

andreu...@yahoo.com

unread,
Feb 15, 2008, 8:28:41 PM2/15/08
to
On Feb 15, 4:12 pm, Pascal Costanza <p...@p-cos.net> wrote:

> andreuri2...@yahoo.com wrote:
> > Let me point out that the simplicity of defmacro is
> > sometimes exaggerated when compared to define-syntax.
> > For example, the following let-macro
>
> > (define-syntax let
> > (syntax-rules ()
> > ((_ ((x v) ...) b1 b2 ...)
> > ((lambda (x ...) b1 b2 ...) v ...))))

> (defmacro let ((&rest bindings) &body body)


> (loop for (var val . rest) in bindings
> collect var into vars
> collect val into vals
> do (assert (null rest) ()
> "Too many values for ~S." var)
> finally (return `((lambda ,vars ,@body) ,@vals))))

This does not appear equivalent. What happened to the elegant
failure if a binding is not a list, or has length less than 2?
Also, what happened to the test that the body not be empty?

Andre


Kent M Pitman

unread,
Feb 15, 2008, 9:42:15 PM2/15/08
to
Kaz Kylheku <kkyl...@gmail.com> writes:

Just eyeballing it, I expected:

(let ((a '(a1 a2 a3))
(b '(b1 b2 b3))
(c '(c1 c2 c3)))
(flet ((a (&rest foo) `(a-fun ,foo))
(b (&rest foo) `(b-fun ,foo))
(c (&rest foo) `(c-fun ,foo)))
(macrolet ((show-it ()
(let ((x '(a b c))) ``(,,x ,@,x ,,@x ,@,@x))))
(show-it))))
`(,(a b c) ,@(a b c) ,a ,b ,c ,@a ,@b ,@c)

to return the same as:

(let ((a '(a1 a2 a3))
(b '(b1 b2 b3))
(c '(c1 c2 c3)))
(flet ((a (&rest foo) `(a-fun ,foo))
(b (&rest foo) `(b-fun ,foo))
(c (&rest foo) `(c-fun ,foo)))
(macrolet ((show-it ()
'(append (list (a b c))
(a b c)
(list a b c)
a b c)))
(show-it))))

... and it seems to do that.

No, I don't recommend using these idioms all the time. But that's not
how they come up. They happen gradually.

Complicated syntax isn't there because you're always supposed to use it.
It's there so that if you drift into it gradually you won't suddenly fall
off a cliff and have to suddenly be forced to use a different syntax.

You can make very long sentences in English (and other natural
languages), too, even though style guides will tell you only to use a
subset of what you theoretically can do.

Alan Crowe

unread,
Feb 16, 2008, 4:47:20 AM2/16/08
to
Kaz Kylheku <kkyl...@gmail.com> writes:

> On Feb 15, 1:09 pm, Alan Crowe <a...@cawtech.freeserve.co.uk> wrote:
> > andreuri2...@yahoo.com writes:
> > > and a little quiz:
> >
> > > (let ((x '(a b c))) ``(,,x ,@,x ,,@x ,@,@x)) => ??
> >
> > and CLISP says
> >
> > `(,(A B C) ,@(A B C) (SYSTEM::UNQUOTE A B C) ,@'(SYSTEM::UNQUOTE A B C))
>
> Wow, how old is your CLISP? That looks suspiciously like the old
> backquote implementation that was replaced some five years ago.
>

[1]> (lisp-implementation-version)
"2000-03-06 (March 2000)"

Yes, I've fallen behind a little. I think I mentioned
recently that I've not been well.

Alan Crowe
Edinburgh
Scotland

Pascal Costanza

unread,
Feb 16, 2008, 6:47:55 AM2/16/08
to
Jeff Barnett wrote:

> Pascal Costanza wrote:
>> (defmacro let ((&rest bindings) &body body)
>> (loop for (var val . rest) in bindings
>> collect var into vars
>> collect val into vals
>> do (assert (null rest) ()
>> "Too many values for ~S." var)
>> finally (return `((lambda ,vars ,@body) ,@vals))))
>>
>> LOOP is a pretty versatile tool for macro programming.
>>
>> Pascal
>>
> I'm Lisp curious, if there are any declarations (or strings) in the body
> portion of this macro, will the behavior of the let and the lambda be
> guaranteed identical?

Yes, including special declarations.

Pascal Costanza

unread,
Feb 16, 2008, 7:02:23 AM2/16/08
to
andreu...@yahoo.com wrote:
> On Feb 15, 4:12 pm, Pascal Costanza <p...@p-cos.net> wrote:
>> andreuri2...@yahoo.com wrote:
>>> Let me point out that the simplicity of defmacro is
>>> sometimes exaggerated when compared to define-syntax.
>>> For example, the following let-macro
>>> (define-syntax let
>>> (syntax-rules ()
>>> ((_ ((x v) ...) b1 b2 ...)
>>> ((lambda (x ...) b1 b2 ...) v ...))))
>
>> (defmacro let ((&rest bindings) &body body)
>> (loop for (var val . rest) in bindings
>> collect var into vars
>> collect val into vals
>> do (assert (null rest) ()
>> "Too many values for ~S." var)
>> finally (return `((lambda ,vars ,@body) ,@vals))))
>
> This does not appear equivalent.

Sure. This is Common Lisp, not Scheme. Things are expected to behave a
bit different over here. (That's one thing Schemers seem to find hard to
get: When Common Lispers say that defmacro is good, they mean defmacro,
not define-macro. There are some strong differences between the Common
Lisp version and the Scheme versions.)

> What happened to the elegant
> failure if a binding is not a list, or has length less than 2?
> Also, what happened to the test that the body not be empty?

- If a binding is not a list: That's covered by the destructuring
expression in the LOOP form.

(let (a b c) a)

*** - CAR: A is not a list

Actually, that's not what a Common Lisper would expect. A Common Lisper
would expect that the variables are accepted anyway and implicitly bound
to nil. This is easy to achieve. Define the following function:

(defun prepare-binding (binding)
(if (consp binding) binding (list binding)))

And loop over (mapcar #'prepare-binding bindings) instead of just bindings.


- If a binding has length less than 2: A Common Lisper would expect an
implicit binding to nil, and this is what happens here.

(let ((a)) a)
NIL

If you insist on being more strict, you can handle that case in
prepare-binding.


- If the body is empty: Empty lambda bodies implicitly return nil.
That's what Common Lispers would expect.

>((lambda ()))
NIL

Again, if you insist on being more strict, just put an (assert body) in
the beginning of the macro definition. You'll get a nice error message then.

The implicit rules are there to make programming with such constructs
more convenient (and convenience is what new language constructs are all
about). Syntax-rules is nice for simple macros, but as soon as you want
to add more bells and whistles, it becomes harder to do so. (How do you
localize the different cases how to treat the different variations of
what a binding could look like?)

Scheme has a tendency to disapprove of bells and whistles, but Common
Lisp encourages them. Whether that's a good idea or not is a subjective
assessment.

andreu...@yahoo.com

unread,
Feb 16, 2008, 1:35:46 PM2/16/08
to
On Feb 16, 7:02 am, Pascal Costanza <p...@p-cos.net> wrote:
> >> andreuri2...@yahoo.com wrote:
> >>> Let me point out that the simplicity of defmacro is
> >>> sometimes exaggerated when compared to define-syntax.
> >>>
> >>> (define-syntax let
> >>> (syntax-rules ()
> >>> ((_ ((x v) ...) b1 b2 ...)
> >>> ((lambda (x ...) b1 b2 ...) v ...))))
> >>>
> >> (defmacro let ((&rest bindings) &body body)
> >> (loop for (var val . rest) in bindings
> >> collect var into vars
> >> collect val into vals
> >> do (assert (null rest) ()
> >> "Too many values for ~S." var)
> >> finally (return `((lambda ,vars ,@body) ,@vals))))
>
> That's one thing Schemers seem to find hard to
> get: When Common Lispers say that defmacro is good, they mean defmacro,
> not define-macro.

Okay, but comparing the above syntax-rules to your defmacro,
I personally find the syntax-rules more concise and legible
(effectively only two lines of BNF-like notation). YMMV.

> > What happened to the elegant
> > failure if a binding is not a list, or has length less than 2?
>

> - If a binding is not a list: That's covered by the destructuring
> expression in the LOOP form.
>
> (let (a b c) a)
>
> *** - CAR: A is not a list

That's not quite what was meant by elegant failure, though...

> Actually, that's not what a Common Lisper would expect. A Common Lisper
> would expect that the variables are accepted anyway and implicitly bound
> to nil.
>

> - If a binding has length less than 2: A Common Lisper would expect an
> implicit binding to nil,

Good grief.

Here is an idiomatic way of expressing that, trivially, in Scheme
(R6RS).

(define-syntax let
(lambda (form)
(syntax-case form ()
((_ (b ...) e ...)
(with-syntax
((((x v) ...)
(map (lambda (b)
(syntax-case b ()
((x v) (identifier? x) #'(x v))
((x) (identifier? x) #'(x '()))
(x (identifier? x) #'(x '()))))
#'(b ...))))
#'((lambda (x ...) e ...) v ...))))))

I don't think this is so much about fundamental properties of
macro systems as it is about programming style.
Pattern matching is just as easy to use in Lisp defmacros
as it is in Scheme, yet it has never caught on.
Why Lispers prefer not to use it, and many Schemers do,
is probably more of a cultural question than any
fundamental property of defmacro.
One could just as well ask why Schemers themselves like to use
patterns for macros yet prefer not to use pattern matching for
function definitions as in ML.

Andre

Pascal Costanza

unread,
Feb 16, 2008, 2:07:21 PM2/16/08
to
andreu...@yahoo.com wrote:
> On Feb 16, 7:02 am, Pascal Costanza <p...@p-cos.net> wrote:
>>>> andreuri2...@yahoo.com wrote:
>>>>> Let me point out that the simplicity of defmacro is
>>>>> sometimes exaggerated when compared to define-syntax.
>>>>>
>>>>> (define-syntax let
>>>>> (syntax-rules ()
>>>>> ((_ ((x v) ...) b1 b2 ...)
>>>>> ((lambda (x ...) b1 b2 ...) v ...))))
>>>>>
>>>> (defmacro let ((&rest bindings) &body body)
>>>> (loop for (var val . rest) in bindings
>>>> collect var into vars
>>>> collect val into vals
>>>> do (assert (null rest) ()
>>>> "Too many values for ~S." var)
>>>> finally (return `((lambda ,vars ,@body) ,@vals))))
>> That's one thing Schemers seem to find hard to
>> get: When Common Lispers say that defmacro is good, they mean defmacro,
>> not define-macro.
>
> Okay, but comparing the above syntax-rules to your defmacro,
> I personally find the syntax-rules more concise and legible
> (effectively only two lines of BNF-like notation). YMMV.

Sure, for small and simple macros, syntax-rules is better. I'm not
convinced about more complex macros. (Schemers seem to have a tendency
towards a stronger functional programming style, so that may not be as
important then.)

>>> What happened to the elegant
>>> failure if a binding is not a list, or has length less than 2?
>> - If a binding is not a list: That's covered by the destructuring
>> expression in the LOOP form.
>>
>> (let (a b c) a)
>>
>> *** - CAR: A is not a list
>
> That's not quite what was meant by elegant failure, though...

What is inelegant about that?

>> Actually, that's not what a Common Lisper would expect. A Common Lisper
>> would expect that the variables are accepted anyway and implicitly bound
>> to nil.
>>
>> - If a binding has length less than 2: A Common Lisper would expect an
>> implicit binding to nil,
>
> Good grief.
>
> Here is an idiomatic way of expressing that, trivially, in Scheme
> (R6RS).
>
> (define-syntax let
> (lambda (form)
> (syntax-case form ()
> ((_ (b ...) e ...)
> (with-syntax
> ((((x v) ...)
> (map (lambda (b)
> (syntax-case b ()
> ((x v) (identifier? x) #'(x v))
> ((x) (identifier? x) #'(x '()))
> (x (identifier? x) #'(x '()))))
> #'(b ...))))
> #'((lambda (x ...) e ...) v ...))))))

See? Now the Scheme macro becomes more complicated as well...

> I don't think this is so much about fundamental properties of
> macro systems as it is about programming style.

Agreed.

> Pattern matching is just as easy to use in Lisp defmacros
> as it is in Scheme, yet it has never caught on.
> Why Lispers prefer not to use it, and many Schemers do,
> is probably more of a cultural question than any
> fundamental property of defmacro.

Right.

> One could just as well ask why Schemers themselves like to use
> patterns for macros yet prefer not to use pattern matching for
> function definitions as in ML.

As I said elsewhere in this thread (I think), syntax-rules is nice for
simple cases, which is exactly what you need for example in papers or
other situations where you want to illustrate something in neat and
brief ways. I guess, syntax-rules is probably also somewhat nicer to
read and understand by non-Lispers/Schemers than define-macro/defmacro,
which helps for exposition as well.

In practical settings (and when not being fixated on a functional
programming style), you need a 'real' macro system (whichever concrete
macro system that may be). The reason is that it's more important that
the macros are convenient to use, rather than that it's convenient to
write those macros.

John Thingstad

unread,
Feb 16, 2008, 2:09:23 PM2/16/08
to
På Sat, 16 Feb 2008 19:35:46 +0100, skrev <andreu...@yahoo.com>:

> Good grief.
>
> Here is an idiomatic way of expressing that, trivially, in Scheme
> (R6RS).
>
> (define-syntax let
> (lambda (form)
> (syntax-case form ()
> ((_ (b ...) e ...)
> (with-syntax
> ((((x v) ...)
> (map (lambda (b)
> (syntax-case b ()
> ((x v) (identifier? x) #'(x v))
> ((x) (identifier? x) #'(x '()))
> (x (identifier? x) #'(x '()))))
> #'(b ...))))
> #'((lambda (x ...) e ...) v ...))))))
>

We must have different ideas of trivial..

> One could just as well ask why Schemers themselves like to use
> patterns for macros yet prefer not to use pattern matching for
> function definitions as in ML.

I wonder about that one myself.

--------------
John Thingstad

andreu...@yahoo.com

unread,
Feb 16, 2008, 3:32:26 PM2/16/08
to
On Feb 16, 2:07 pm, Pascal Costanza <p...@p-cos.net> wrote:
> andreuri2...@yahoo.com wrote:

> >> *** - CAR: A is not a list
>
> > That's not quite what was meant by elegant failure, though...
>
> What is inelegant about that?

The message does not even indicate that the error was a syntax error.

> As I said elsewhere in this thread (I think), syntax-rules is nice for
> simple cases, which is exactly what you need for example in papers or
> other situations where you want to illustrate something in neat and
> brief ways. I guess, syntax-rules is probably also somewhat nicer to
> read and understand by non-Lispers/Schemers than define-macro/defmacro,
> which helps for exposition as well.
>
> In practical settings (and when not being fixated on a functional
> programming style), you need a 'real' macro system (whichever concrete
> macro system that may be). The reason is that it's more important that
> the macros are convenient to use, rather than that it's convenient to
> write those macros.

As has been said elsewhere, Scheme has, for years, had 'real' macro
systems that are not syntax-rules, making macros convenient to write
and
use. The Scheme example I wrote of the generalized binding type LET
was
certainly not written in syntax-rules. Comparing defmacro to syntax-
rules
is simply being out of date. Scheme macros are considered by Schemers
to
be more robust than Lisp defmacros, given the referential
transparency
guarantees of Scheme macros, although I know Lispers tend not to care
about this issue.

Andre

andreu...@yahoo.com

unread,
Feb 16, 2008, 3:47:03 PM2/16/08
to
On Feb 16, 2:09 pm, "John Thingstad" <jpth...@online.no> wrote:

> På Sat, 16 Feb 2008 19:35:46 +0100, skrev <andreuri2...@yahoo.com>:
>
> > Here is an idiomatic way of expressing that, trivially, in Scheme
> > (R6RS).
>
> > (define-syntax let
> > (lambda (form)
> > (syntax-case form ()
> > ((_ (b ...) e ...)
> > (with-syntax
> > ((((x v) ...)
> > (map (lambda (b)
> > (syntax-case b ()
> > ((x v) (identifier? x) #'(x v))
> > ((x) (identifier? x) #'(x '()))
> > (x (identifier? x) #'(x '()))))
> > #'(b ...))))
> > #'((lambda (x ...) e ...) v ...))))))
>
> We must have different ideas of trivial..

I can assure you that a Scheme programmer
familiar with syntax-case would find this utterly trivial.
I am not sure how this macro might be expressed idiomatically
in Lisp, but probably a Lisper would find the result more
transparent than the above, while I would probably find it much
less trivial. Again, that is cultural, and it would be silly
to argue that Scheme macros are, therefore, inferior.

> > One could just as well ask why Schemers themselves like to use
> > patterns for macros yet prefer not to use pattern matching for
> > function definitions as in ML.
>
> I wonder about that one myself.

Perhaps because Schemers tend to come from more academic backgrounds
where using a BNF-type notation for syntax comes naturally.
On the other hand, there is no built-in emphasis on algebraic data
types as in ML.

Andre

Jeff Barnett

unread,
Feb 16, 2008, 6:00:20 PM2/16/08
to
Pascal Costanza wrote:
> Jeff Barnett wrote:
>> Pascal Costanza wrote:
>>> (defmacro let ((&rest bindings) &body body)
>>> (loop for (var val . rest) in bindings
>>> collect var into vars
>>> collect val into vals
>>> do (assert (null rest) ()
>>> "Too many values for ~S." var)
>>> finally (return `((lambda ,vars ,@body) ,@vals))))
>>>
>>> LOOP is a pretty versatile tool for macro programming.
>>>
>>> Pascal
>>>
>> I'm Lisp curious, if there are any declarations (or strings) in the
>> body portion of this macro, will the behavior of the let and the
>> lambda be guaranteed identical?
>
> Yes, including special declarations.
>
>
> Pascal
>
The reason I asked was historical. When the common lisp standard was
being debated, there was a difference. Declarations within a let could
effect the calculation of binding presets; on the other hand,
declarations in a lambda could not effect the evaluation of its
arguments. Way back when, I argued that this was a semantic glitch. I'm
glad to hear the problem was finally resolved.

-- Jeff Barnett

Kent M Pitman

unread,
Feb 16, 2008, 7:43:35 PM2/16/08
to
[ comp.lang.lisp only; http://www.nhplace.com/kent/PFAQ/cross-posting.html ]

Jeff Barnett <jbb...@ca.rr.com> writes:

Hmm. My recollection is that Jeff is right. I'll have to look into it
more. I think the difference is related to the scope of the specials
onto the init values. In some cases, I think the special can go into
the init value. But maybe I'm misremembering.

I tried this in LWW 4.4.5 and LWW 5.0.1 and I was surprised by the answer.

(let ((a 'outer))
(declare (special a))
(let ((a 'inner))
;; this a is lexical
(let ((b a))
(declare (special a))
(list b (symbol-value 'a)))))
=> (INNER OUTER)

I would have expected the answer to be (OUTER OUTER), but maybe I'm wrong.
I don't have time investigate this right now, so I figured I'd put it up and
maybe someone could do the research while I was off doing other things. :)

But in any case, my point is that I think there are places where
(let (...) (declare ...) ...)
is not the same as
((lambda (...) (declare ...)) ...)
becuase the declare is not spanning the declarations in the lambda case,
and might need to be.

Disclaimer: No one should take any of my "I think"'s in this as some
sort of claim of truth. They're just my recollection and impression
of the moment. I reserve the right to change my mind.

John Thingstad

unread,
Feb 16, 2008, 7:52:59 PM2/16/08
to
På Sat, 16 Feb 2008 21:32:26 +0100, skrev <andreu...@yahoo.com>:

> is simply being out of date. Scheme macros are considered by Schemers
> to be more robust than Lisp defmacros, given the referential
> transparency guarantees of Scheme macros, although I know Lispers tend
> not to care
> about this issue.
>

We care. We just take care of it through protocol of writing.
gensym is a bit lightweight. With the help of a couple of macroes named
'with-unique-names' and 'rebinding' it is fairly easy to write
referentially transparent code. Rebinding for rebinding input variables so
they are evaluated before use. with-unique-values for introducing new
variables.

(defmacro lister (p q)
(with-unique-names (x y)
`(let ((,x (x-function))
(,y (y-function)))
(list ,p ,q ,x ,y))))

the form (lister i j) macroexpands to

(LET* ((#:X-88 (X-FUNCTION))
(#:Y-89 (Y-FUNCTION)))
(LIST i j #:X-88 #:Y-89))

(defmacro lister (x y)
(rebinding (x y)
'(list ,x ,y)))

the form (lister i j) macroexpands to

(LET* ((#:X-77 I)
(#:Y-78 J))
(LIST #:X-77 #:Y-78))

This seems to cover most cases.

--------------
John Thingstad

Pascal Costanza

unread,
Feb 16, 2008, 8:43:20 PM2/16/08
to
John Thingstad wrote:
> På Sat, 16 Feb 2008 21:32:26 +0100, skrev <andreu...@yahoo.com>:
>
>> is simply being out of date. Scheme macros are considered by Schemers
>> to be more robust than Lisp defmacros, given the referential
>> transparency guarantees of Scheme macros, although I know Lispers tend
>> not to care
>> about this issue.
>>
>
> We care. We just take care of it through protocol of writing.
> gensym is a bit lightweight. With the help of a couple of macroes named
> 'with-unique-names' and 'rebinding' it is fairly easy to write
> referentially transparent code. Rebinding for rebinding input variables
> so they are evaluated before use. with-unique-values for introducing new
> variables.
[...]

> This seems to cover most cases.

No, that's not enough. What you list here solves the problems of
unintended variable capture, and order and number of evaluations.
Referential transparency is something else, though.

Consider:

(let ((x 42))
(macrolet ((foo () 'x))
(let ((x 4711))
(foo))))

Now assume you're not allowed to change the names of the x variable
bindings. In a defmacro-style macro system, there is no way that you can
ensure that the code to which foo expands refers to the outer x by just
changing the macro definition. Whatever you may try, the outer x will be
shadowed by the inner x in the inner expansion of foo.

Hygienic macro systems solve this issue. The following evaluates to 42:

(let ((x 42))
(let-syntax ((foo (syntax-rules () ((foo) x))))
(let ((x 4711))
(foo))))

The problem can be relatively easily circumvented in Common Lisp by
using the package system and choosing names more carefully. After all,
the following actually does what you expect:

(let ((x 42))
(macrolet ((foo () 'x))
(let ((y 4711)) ; note: name changed
(foo))))

If Common Lisp had a module system instead of a package system, and if
Common Lisp were a Lisp-1 instead of a Lisp-2, referential transparency
would be a much more pressing issue. [1]

Nevertheless, hygienic macro systems are strictly more powerful in that
regard than defmacro-style macro systems.

Pascal

[1] That's the main reason why I personally prefer packages over modules
and Lisp-2 over Lisp-1, because I get much less nameclashes with that in
general (not only in macros).

Juho Snellman

unread,
Feb 16, 2008, 8:48:34 PM2/16/08
to
Kent M Pitman <pit...@nhplace.com> writes:
> I tried this in LWW 4.4.5 and LWW 5.0.1 and I was surprised by the answer.
>
> (let ((a 'outer))
> (declare (special a))
> (let ((a 'inner))
> ;; this a is lexical
> (let ((b a))
> (declare (special a))
> (list b (symbol-value 'a)))))
> => (INNER OUTER)
>
> I would have expected the answer to be (OUTER OUTER), but maybe I'm wrong.
> I don't have time investigate this right now, so I figured I'd put it up and
> maybe someone could do the research while I was off doing other things. :)

Lispworks is correct. The inner SPECIAL A declaration is a free
declaration, and per 3.3.4:

The scope of free declarations specifically does not include
initialization forms for bindings established by the form containing
the declarations.

--
Juho Snellman

Maciej Katafiasz

unread,
Feb 16, 2008, 9:13:43 PM2/16/08
to
Den Sat, 16 Feb 2008 19:43:35 -0500 skrev Kent M Pitman:

> I tried this in LWW 4.4.5 and LWW 5.0.1 and I was surprised by the
> answer.
>
> (let ((a 'outer))
> (declare (special a))
> (let ((a 'inner))
> ;; this a is lexical
> (let ((b a))
> (declare (special a))
> (list b (symbol-value 'a)))))
> => (INNER OUTER)
>
> I would have expected the answer to be (OUTER OUTER), but maybe I'm
> wrong. I don't have time investigate this right now, so I figured I'd
> put it up and maybe someone could do the research while I was off doing
> other things. :)

No actual research, but this is consistent with my expectations. I see
declarations as happening conceptually at the same time as bindings, in
which case b can't see the value of special a, as it doesn't exist at the
time b is bound. Harder to decide would be what to do with a LET*,
though, there I fail to have any consistent expectations and would have
to consult the spec :)

Cheers,
Maciej

Rob Warnock

unread,
Feb 16, 2008, 11:17:52 PM2/16/08
to
Pascal Costanza <p...@p-cos.net> wrote:
+---------------

| (let ((x 42))
| (macrolet ((foo () 'x))
| (let ((x 4711))
| (foo))))
|
| Now assume you're not allowed to change the names of the x variable
| bindings. In a defmacro-style macro system, there is no way that you can
| ensure that the code to which foo expands refers to the outer x by just
| changing the macro definition. Whatever you may try, the outer x will be
| shadowed by the inner x in the inner expansion of foo.
+---------------

Not to disagree with your point about macro scope & shadowing,
but this paticular example is not at all compelling, since you
could easily have gotten the desired result with no macros at all!

> (let ((x 42))
(flet ((foo () x))
(let ((x 4711))
(foo))))

42
> (let ((x 42))
(flet ((foo () x))
(let ((x 4711))
(flet ((bar () x))
(let ((x 937))
(list (foo) (bar) x))))))

(42 4711 937)
>

Look, Ma, no macros!

Sometimes I think that with all the bickering about macro style
we sometimes forget that Scheme & CL both share the amazing power
of lexical closures.

+---------------


| Hygienic macro systems solve this issue.

+---------------

But so do closures... and with less "new syntax".


-Rob

-----
Rob Warnock <rp...@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607

Kent M Pitman

unread,
Feb 17, 2008, 12:38:01 AM2/17/08
to
Maciej Katafiasz <math...@gmail.com> writes:

> Den Sat, 16 Feb 2008 19:43:35 -0500 skrev Kent M Pitman:
>
> > I tried this in LWW 4.4.5 and LWW 5.0.1 and I was surprised by the
> > answer.
> >
> > (let ((a 'outer))
> > (declare (special a))
> > (let ((a 'inner))
> > ;; this a is lexical
> > (let ((b a))
> > (declare (special a))
> > (list b (symbol-value 'a)))))
> > => (INNER OUTER)
> >
> > I would have expected the answer to be (OUTER OUTER), but maybe I'm
> > wrong. I don't have time investigate this right now, so I figured I'd
> > put it up and maybe someone could do the research while I was off doing
> > other things. :)
>
> No actual research, but this is consistent with my expectations.

I think it was not the rule for some time, perhaps in CLTL times.
Looks like we did fix it. Must have left a bad taste in my mouth.
The prevailing issue was Issue DECLARATION-SCOPE:NO-HOISTING.
http://www.lispworks.com/documentation/HyperSpec/Issues/iss092_w.htm

> I see declarations as happening conceptually at the same time as
> bindings,

Well, that's my point. Since there is no binding, it seems to me that
we language designers should have defaulted to applying to the whole
LET (since there is no binding to stop it) rather than the case of
interest is.

> in which case b can't see the value of special a, as it doesn't
> exist at the time b is bound.

Point of view, I suppose. I think of the value of special A as
"always existing" since it is always accessible via symbol-value, a
clue that all specials with the same name share the same value cell.
The variables never come into existence, only the ability to reference
the variables. That's different from lexical variables, which are new
every time you execute a binding.

So, to me, the only question is whether referencing that cell has been
turned back on. That's a legitimate question (absent a spec). The spec
is clear, I just don't 100% like the definition.

But I think historically the behavior went back and forth, and it
looks like the present behavior does make the LET/LAMBDA thing
consistent, so I was wrong about worrying that wasn't so.

(At least I think that's how it all is. It's late and I'm tired, so I
can still have made a mistake.)

Abdulaziz Ghuloum

unread,
Feb 17, 2008, 1:29:55 AM2/17/08
to
Rob Warnock wrote:

> Not to disagree with your point about macro scope & shadowing,
> but this paticular example is not at all compelling, since you
> could easily have gotten the desired result with no macros at all!

Of course. His example was the simplest example to illustrate the
point.

> (42 4711 937)
> >
>
> Look, Ma, no macros!

Right, you can get the same thing by simply saying '(42 7411 937).
You don't need procedures or macros or anything really for that.

> Sometimes I think that with all the bickering about macro style
> we sometimes forget that Scheme & CL both share the amazing power
> of lexical closures.

In scheme, there is lexical scope for procedures and macros. In CL,
there is lexical scope for procedures and dynamic scope for macros.
In elisp, it's dynamic scope for both (no closures).


Aziz,,,

PS. I'm ignoring dynamic variables, fluid-let, parameterize, etc.

Rob Warnock

unread,
Feb 17, 2008, 3:45:57 AM2/17/08
to
Abdulaziz Ghuloum <aghu...@cee.ess.indiana.edu> wrote:
+---------------

| Rob Warnock wrote:
| > Sometimes I think that with all the bickering about macro style
| > we sometimes forget that Scheme & CL both share the amazing power
| > of lexical closures.
|
| In scheme, there is lexical scope for procedures and macros. In CL,
| there is lexical scope for procedures and dynamic scope for macros.
| In elisp, it's dynamic scope for both (no closures).
+---------------

Hmmm... Digging into this a bit, I think that for CL, it's actually a
bit *worse*[1] than if it were really just "dynamic scope for macros":

http://alu.org/HyperSpec/Body/speope_fletcm_scm_macrolet.html
Special Operator FLET, LABELS, MACROLET
...
macrolet
The macro-expansion functions defined by macrolet are defined in the
lexical environment in which the macrolet form appears. Declarations
and macrolet and symbol-macrolet definitions affect the local macro
definitions in a macrolet, but the consequences are undefined if
the local macro definitions reference any local variable or function
bindings that are visible in that lexical environment.

And then they give an example that is very close to Pascal_C's, except
using the LAMBDA-bound variables of a DEFUN as the outer ones that
can't be referenced instead of Pascal's LET-bound outer variable.

To repeat what Pascal Costanza <p...@p-cos.net> wrote:
+---------------
| (let ((x 42))
| (macrolet ((foo () 'x))
| (let ((x 4711))
| (foo))))
|
| Now assume you're not allowed to change the names of the x variable
| bindings. In a defmacro-style macro system, there is no way that you can
| ensure that the code to which foo expands refers to the outer x by just
| changing the macro definition. Whatever you may try, the outer x will be
| shadowed by the inner x in the inner expansion of foo.
+---------------

After reading the above CLHS quote, I now think that's probably correct.
The MACROLET definition cannot itself capture the lexical value of the
outer X [because it probably doesn't exist yet when the macro expansion
function is closed over]. You would have to use some *other* binding
form to perform that capture at :EXECUTE time [such as the FLETs I
showed previously], but then the MACROLET definition could certainly
expand into a *reference* to that other binding form. And the definition
of the macro expansion of that reference can even be done *before* the
other binding form is encountered:

> (let ((x 42))
(macrolet ((foo () '(list y x)))
(let ((y x))
(let ((x 4711))
(foo)))))

(42 4711)
>

*Or* after:

> (let ((x 42))
(let ((y x))
(macrolet ((foo () '(list y x)))
(let ((x 4711))
(foo)))))

(42 4711)
>

But the point being that in either case the macro *definition* cannot
reference the :EXECUTE lexical environment, only its *expansion* can.

Oh, well...


-Rob

[1] The reason I say this is "worse" than "just dynamic scope" is that
the dynamic values of variables in the macro *definition* might be
those at compile time, which could be altogether different than the
dynamic values of those very same variables at runtime (:EXECUTE).
Pascal Bourguignon's article in the "passing values from compile-time
to load-time" thread <news:87lk5ke...@thalassa.informatimago.com>
hints at this when he said:

When a macro is executing, the situation is :EXECUTE.

but I don't think he made it clear enough there -- at least, not
for the purposes of *this* discussion -- that the macro is probably
executing at compile time, and even though "the situation is :EXECUTE"
during its execution the dynamic environment at that time is the
environment of the *compiler*, which is not (or not necessarily)
the dynamic environment in which the code being compiled will
itself :EXECUTE.

Abdulaziz Ghuloum

unread,
Feb 17, 2008, 4:58:19 AM2/17/08
to
Rob Warnock wrote:

> But the point being that in either case the macro *definition*
> cannot reference the :EXECUTE lexical environment, only its
> *expansion* can.

That's consistent with Scheme.

> (let ((x 5))
(let-syntax ((f (lambda (stx) x)))
(let ((x 6))
(f))))
=>
Unhandled exception
Condition components:
1. &who: x
2. &message: "identifier out of context"
3. &syntax:
form: x

But:

> (let ((x 5))
(let-syntax ((f (lambda (stx) #'x)))
(let ((x 6))
(f))))
=> 5

The syntax object obtained by #'x contains a lexical environment
which closes over all lexical identifiers visible at that point
(it contains a mapping from x to the outer x among other things).

In Pascal's example:

> (let ((x 5))
(macrolet ((f () 'x))
(let ((x 6))
(f))))
=> 6

The x in f is just a symbol. It has no lexical environment
attached to it. The expansion of (f) will use the environment
at the macro use site (e.g., where (f) occurred) and not the
macro definition site (where the 'x occurred).

Abdulaziz Ghuloum

unread,
Feb 17, 2008, 6:49:54 AM2/17/08
to
Abdulaziz Ghuloum wrote:

> The syntax object obtained by #'x contains a lexical environment
> which closes over all lexical identifiers visible at that point
> (it contains a mapping from x to the outer x among other things).

Just to drive the point home. Suppose Scheme had a macrolet[*],
then we can write:

> (let ((x 'outer-x))
(macrolet ((f () #'x))
(let ((x 'inner-x))
(list x (f)))))
=>
(inner-x outer-x)

I said that #'x contains an environment that contains all
lexically visible identifiers at that point. Scheme has the
procedure datum->syntax that attaches the environment of one
identifier to any s-expression, in essence, obtaining an
expression as if it had been written in the location where
the identifier occurred.

> (let ((x 'outer-x) (y 'outer-y))
(macrolet ((f (sym) (datum->syntax #'here sym)))
(let ((x 'inner-x) (y 'inner-y))
(list x y (f x) (f y)))))
=>
(inner-x inner-y outer-x outer-y)

The syntax object obtained from #'here has a lexical closure
which contains the bindings of the outer x and y among other
things. The macro call (f x) attaches the #'here environment
to the symbol x. The result is an identifier x as if it had
occurred in place of the #'here, which is the outer x.

I hope this clarifies things a bit.

Aziz,,,

[*] This is the Scheme macrolet that I'm using for these
examples:

(define-syntax macrolet
(syntax-rules ()
((_ ((names (args ...) exprs) ...) b b* ...)
(let-syntax ((names
(lambda (stx)
(syntax-case stx ()
((_ args ...)
(let ((args (syntax->datum #'args)) ...)
exprs)))))
...)
b b* ...))))

Jens Axel Soegaard

unread,
Feb 17, 2008, 7:03:47 AM2/17/08
to
Kaz Kylheku wrote:
> On Feb 14, 4:30 pm, Abdulaziz Ghuloum <aghul...@cee.ess.indiana.edu>

>> As you can see, it shows the original file name and position in
>> terms of the user's original source code.
>
> S-expressions read from a file (and all their constituent
> expressions), can be associated this information in a global hash
> table. Given some erroneous form, the macro can see if there is
> associated information with that form.

How does a hash table system handle S-expressions generated
by macros? Does it store every return value generated by
macro transformers along with the location of the macro
definition?

--
Jens Axel Søgaard

Pascal Costanza

unread,
Feb 17, 2008, 7:39:33 AM2/17/08
to
Kaz Kylheku wrote:
> On Feb 14, 4:30 pm, Abdulaziz Ghuloum <aghul...@cee.ess.indiana.edu>
> wrote:
>> $ ikarus --r6rs-script test.ss
>> Unhandled exception:
>> Condition components:
>> 1. &who: let
>> 2. &message: "not an identifier"
>> 3. &syntax:
>> form: (let ((17 b)) (list a b))
>> subform: 17
>> 4. &trace: #<syntax (let ((17 b)) (list a b))>
>> 5. &trace: #<syntax (foo 17 b (list a b))>
>> 6. &trace: #<syntax (bar b 17 (list a b)) [byte 200 of test.ss]>

>>
>> As you can see, it shows the original file name and position in
>> terms of the user's original source code.
>
> S-expressions read from a file (and all their constituent
> expressions), can be associated this information in a global hash
> table. Given some erroneous form, the macro can see if there is
> associated information with that form.
>
> In principle, the origin of every literal object (or at least that of
> every boxed one) that was read from some source file can be
> pinpointed.

No, not in general. S-expressions can share subforms, and then such
information is not unambiguous anymore.


Pascal

Pascal Costanza

unread,
Feb 17, 2008, 7:58:49 AM2/17/08
to
Abdulaziz Ghuloum wrote:
> In Pascal's example:
>
> > (let ((x 5))
> (macrolet ((f () 'x))
> (let ((x 6))
> (f))))
> => 6
>
> The x in f is just a symbol. It has no lexical environment
> attached to it. The expansion of (f) will use the environment
> at the macro use site (e.g., where (f) occurred) and not the
> macro definition site (where the 'x occurred).

To explain how this is solved in Common Lisp: There are essentially two
cases, either local or global macro definitions.

The example above is a local macro. It's not really an interesting case
in practice because of that, because all the code there belongs to you
anyway. So just rename one of the x's, and you're done.

For global macros, let's assume for a while we have lexically global
variables: [1]

(defglobal x 5)

(defmacro f () 'x)

(let ((x 6)) (f)) => 6

This code still has the problem that f is not referentially transparent.
However, you can use the package system to solve that problem. Again,
there are two cases: Either (a) you don't export the variable x, or (b)
you export it.

(a)

(defpackage my-library
(:export f)) ;; note: x not exported

(in-package my-library)

(defglobal x 5)

(defmacro f () 'x)


(defpackage my-client-code
(:use my-library))

(in-package my-client-code)

(let ((x 6)) (f)) => 5

Here, the problem doesn't occur anymore because my-library::x and
my-client-code::x are different symbols! (That's why a package system
makes writing such macros more reliable. In a module system, the two
symbols would be the same and you would still have the problem!)

(b)

(defpackage my-library
(:export f x)) ;; note: both f and x exported

(in-package my-library)

(defglobal x 5)

(defmacro f () 'x)


(defpackage my-client-code
(:use my-library))

(in-package my-client-code)

(let ((x 6)) (f)) => 6

Here, the problem seems to occur, BUT: Since x is exported, the ability
to rebind it becomes part of the interface (the "API") of my-library,
it's one of the operations you can "perform" on such names. So it's the
responsibility of my-library to _document_ what happens in such cases
and to point out that there are possible interactions with f (or
reorganize the code such that such interactions don't occur).

Another important point here is that as soon as you export symbols from
a package, it's a good idea to use more convincing names than f and x
(and in practice, this is where using
long-and-descriptive-names-for-your-identifiers pays off, as is
typically done in Common Lisp).


I think this analysis covers all important cases, and it shows that the
problem doesn't really exist in practice, or at least doesn't hurt that
much. Note that this comes from the fact that Common Lisp has a package
system, not a module system. The fact that Common Lisp is a Lisp-2 also
plays a role here (but I haven't covered that part in my discussion).

In a module system, all definitions are always 'local', in a sense.
That's why the problem is much more pressing in Scheme...

Pascal

[1] Google for defglobal to see how to implement that in Common Lisp.
Discussing global lexicals makes the point a bit clearer, but the
discussion also holds for other lexical entities (like functions).

Ron Garret

unread,
Feb 17, 2008, 12:36:57 PM2/17/08
to
In article <61qpc9F...@mid.individual.net>,
Pascal Costanza <p...@p-cos.net> wrote:

> Abdulaziz Ghuloum wrote:
> > In Pascal's example:
> >
> > > (let ((x 5))
> > (macrolet ((f () 'x))
> > (let ((x 6))
> > (f))))
> > => 6
> >
> > The x in f is just a symbol. It has no lexical environment
> > attached to it. The expansion of (f) will use the environment
> > at the macro use site (e.g., where (f) occurred) and not the
> > macro definition site (where the 'x occurred).
>
> To explain how this is solved in Common Lisp: There are essentially two
> cases, either local or global macro definitions.
>
> The example above is a local macro. It's not really an interesting case
> in practice because of that, because all the code there belongs to you
> anyway. So just rename one of the x's, and you're done.
>
> For global macros, let's assume for a while we have lexically global
> variables: [1]
>
> (defglobal x 5)
>
> (defmacro f () 'x)
>
> (let ((x 6)) (f)) => 6
>
> This code still has the problem that f is not referentially transparent.
> However, you can use the package system to solve that problem.

Or you can solve it this way:

(defmacro f () x)

Or, if you don't trust your implementation to do the Right Thing with
lexical scope in macro definitions (because the spec doesn't actually
require it to):

(defun my-macro-expander () x)

(defmacro f () (my-macro-expander))

rg

Pascal Costanza

unread,
Feb 17, 2008, 1:19:15 PM2/17/08
to

A definition like (defmacro f () 'x) ensures that the variable lookup to
which this macro expands is done at runtime (or better, at the execution
time of the code into which it expands). A definition like (defmacro f
() x) will perform the lookup of x at macro-expansion time. For
interpreted code, that does not make a big difference, but for compiled
code, it certainly does: Macro-expansion time and compile item is
practically the same for compiled code, and the respective lexical
bindings will generally simply not exist yet at compile time.

That the spec doesn't require macro functions to respect lexical scoping
is nonsense: ANSI Common Lisp is exceptionally clear in that regard.
Just look up the entries for defmacro and macrolet in the HyperSpec.


Pascal

Jens Axel Soegaard

unread,
Feb 17, 2008, 1:59:56 PM2/17/08
to
Pascal Costanza wrote:

> The problem can be relatively easily circumvented in Common Lisp by
> using the package system and choosing names more carefully. After all,
> the following actually does what you expect:

> If Common Lisp had a module system instead of a package system, and if

> Common Lisp were a Lisp-1 instead of a Lisp-2, referential transparency
> would be a much more pressing issue. [1]
>
> Nevertheless, hygienic macro systems are strictly more powerful in that
> regard than defmacro-style macro systems.


I wonder how long it takes the Arcers (?) to realize the problems.

--
Jens Axel Søgaard

Pascal Costanza

unread,
Feb 17, 2008, 2:01:59 PM2/17/08
to

Good question. What's the experience with define-macro in Scheme
implementations. Does the problem actually occur? A lot?


Pascal

Ron Garret

unread,
Feb 17, 2008, 2:11:18 PM2/17/08
to
In article <61rc54F...@mid.individual.net>,
Pascal Costanza <p...@p-cos.net> wrote:

Then it's not lexically scoped. I thought the whole point of the
present exercise was to get X to refer to the lexically apparent binding
at the point of macro definition.

> A definition like (defmacro f
> () x) will perform the lookup of x at macro-expansion time.

Yes. Isn't that exactly what you want in this case? (Maybe I've
dropped some context here.)

> For
> interpreted code, that does not make a big difference, but for compiled
> code, it certainly does: Macro-expansion time and compile item is
> practically the same for compiled code, and the respective lexical
> bindings will generally simply not exist yet at compile time.

Of course they will. Compile-time for the invoker of the macro *is*
run-time for the macroexpander function, so any lexical bindings in the
macro expander function must exist at that point (otherwise by
definition they are not lexical bindings).

> That the spec doesn't require macro functions to respect lexical scoping
> is nonsense: ANSI Common Lisp is exceptionally clear in that regard.
> Just look up the entries for defmacro and macrolet in the HyperSpec.

Yeah, it surprised me too. I'm going off something Rob Warnock posted
earlier in this thread:

http://alu.org/HyperSpec/Body/speope_fletcm_scm_macrolet.html
Special Operator FLET, LABELS, MACROLET
...
macrolet
The macro-expansion functions defined by macrolet are defined in the
lexical environment in which the macrolet form appears. Declarations
and macrolet and symbol-macrolet definitions affect the local macro
definitions in a macrolet, but the consequences are undefined if
the local macro definitions reference any local variable or function
bindings that are visible in that lexical environment.

Maybe I misinterpreted what this section of the spec is saying.

rg

Pascal Costanza

unread,
Feb 17, 2008, 2:38:39 PM2/17/08
to

Yes, but without evaluating it at macro-expansion time. That's the
point: We want to have the cake and eat it at the same time. Not either
or. And that's what hygienic macro systems allow you to do.

>> A definition like (defmacro f
>> () x) will perform the lookup of x at macro-expansion time.
>
> Yes. Isn't that exactly what you want in this case? (Maybe I've
> dropped some context here.)

No, that's not what we want.

I have the impression you misunderstand that section.

Consider this (tested under Allegro Common Lisp, because it macroexpands
during evaluation in interpreted mode):

CL-USER(1): (defun test ()
(let ((x 42))
(macrolet ((foo () (if (> x 20) 'y 'z)))
(let ((y 0) (z 1))
(foo)))))
TEST
CL-USER(2): (test)
0
CL-USER(3): (compile 'test)
; While compiling TEST:
Error: Attempt to take the value of the unbound variable `X'.
[condition type: UNBOUND-VARIABLE]

What happens here is that when (foo) is invoked, only then the macro
invocation is expanded. The macro function for foo is lexically scoped
(as required by ANSI Common Lisp!), so it can see the local variable x.

However, when you compile the code, macroexpansion has to be fully
performed at compile time (again as required by ANSI Common Lisp!). At
macro expansion time, though, the binding for x doesn't exist yet.
That's why it refuses to compile.

The fact that in interpreted mode this code works and in compiled mode
it doesn't is why the spec says that "the consequences are undefined."

Compare the situation with this code:

CL-USER(1): (defun test2 ()
(symbol-macrolet ((x 42))
(macrolet ((foo () (if (> x 20) 'y 'z)))
(let ((y 0) (z 1))
(foo)))))
TEST2
CL-USER(2): (test2)
0
CL-USER(3): (compile 'test2)
; While compiling TEST2:
Warning: Variable Z is never used.
TEST2
T
NIL
CL-USER(4): (test2)
0

Works fine, lexical scoping is respected perfectly, and all that... ;)

Ron Garret

unread,
Feb 17, 2008, 3:20:04 PM2/17/08
to
In article <61rgpvF...@mid.individual.net>,
Pascal Costanza <p...@p-cos.net> wrote:

> >>> (defun my-macro-expander () x)
> >>>
> >>> (defmacro f () (my-macro-expander))
> >> A definition like (defmacro f () 'x) ensures that the variable lookup to
> >> which this macro expands is done at runtime (or better, at the execution
> >> time of the code into which it expands).
> >
> > Then it's not lexically scoped. I thought the whole point of the
> > present exercise was to get X to refer to the lexically apparent binding
> > at the point of macro definition.
>
> Yes, but without evaluating it at macro-expansion time. That's the
> point: We want to have the cake and eat it at the same time. Not either
> or. And that's what hygienic macro systems allow you to do.

Ah. Let me make sure I understand. You want the following:

? (defglobal x 1)
1
? (defmacro m () x)
M
? (defun f () (m))
F
? (let ((x 2)) (f))
;Compiler warnings :
; Unused lexical variable X, in an anonymous lambda form.
1

But then you also want:

? (setf x 2)
2
? (f)
2

(It actually returns 1 as things stand.)

Is that right?

(Just FYI, the reason I'm interested in understanding this is because I
want to make macros do the Right Thing in my lexicon library.)


> I have the impression you misunderstand that section.

Could be. I was under the impression that:

(macrolet ((foo () body)) ...

was equivalent to

(flet ((foo-expander () body))
(macrolet ((foo () (foo-expander))

but I guess it isn't. I need to go back and re-read some of this.

Thanks for the example. That was very helpful.

rg

Ron Garret

unread,
Feb 17, 2008, 3:20:54 PM2/17/08
to
In article <47b88433$0$15898$edfa...@dtext01.news.tele.dk>,

They realize the problem (or at least Paul does). He just doesn't
believe that it's a problem.

rg

Abdulaziz Ghuloum

unread,
Feb 17, 2008, 3:37:30 PM2/17/08
to
Ron Garret wrote:

> They realize the problem (or at least Paul does). He just doesn't
> believe that it's a problem.

If describing unintended capture as "kind of freaky" does not
qualify as admitting the problem, I don't know what does.

http://arclanguage.org/item?id=2504

Pascal Costanza

unread,
Feb 17, 2008, 3:41:54 PM2/17/08
to

Yes.

> (Just FYI, the reason I'm interested in understanding this is because I
> want to make macros do the Right Thing in my lexicon library.)

Then you probably want hygienic macros. Look at Will Clinger's "Hygienic
Macros Through Explicit Renaming" paper for a simple version.

>> I have the impression you misunderstand that section.
>
> Could be. I was under the impression that:
>
> (macrolet ((foo () body)) ...
>
> was equivalent to
>
> (flet ((foo-expander () body))
> (macrolet ((foo () (foo-expander))
>
> but I guess it isn't.

Yes, it is. The second version has the same problem as the first one:
foo-expander is only available at runtime, but not at compile time.

> I need to go back and re-read some of this.
>
> Thanks for the example. That was very helpful.

I'm happy that it helped...

John Thingstad

unread,
Feb 17, 2008, 3:45:06 PM2/17/08
to
På Sun, 17 Feb 2008 02:43:20 +0100, skrev Pascal Costanza <p...@p-cos.net>:

>
> No, that's not enough. What you list here solves the problems of
> unintended variable capture, and order and number of evaluations.
> Referential transparency is something else, though.
>
> Consider:
>
> (let ((x 42))
> (macrolet ((foo () 'x))
> (let ((x 4711))
> (foo))))
>

Right you are.
You have given me something to thing about..
I was thinging of a solution like..

(let ((x 42))
(capture (x)


(macrolet ((foo () 'x))
(let ((x 4711))

(foo))))))

42

But the realized I needed special handeling for 'special' variables...
Guess I'll think some more..

--------------
John Thingstad

Ron Garret

unread,
Feb 17, 2008, 3:49:46 PM2/17/08
to
In article <61rkgiF...@mid.individual.net>,
Pascal Costanza <p...@p-cos.net> wrote:

> > (Just FYI, the reason I'm interested in understanding this is because I
> > want to make macros do the Right Thing in my lexicon library.)
>
> Then you probably want hygienic macros. Look at Will Clinger's "Hygienic
> Macros Through Explicit Renaming" paper for a simple version.

Maybe. I'm thinking there might be a sneakier solution though :-)

> >> I have the impression you misunderstand that section.
> >
> > Could be. I was under the impression that:
> >
> > (macrolet ((foo () body)) ...
> >
> > was equivalent to
> >
> > (flet ((foo-expander () body))
> > (macrolet ((foo () (foo-expander))
> >
> > but I guess it isn't.
>
> Yes, it is. The second version has the same problem as the first one:
> foo-expander is only available at runtime, but not at compile time.

Ah. Thanks, you just saved me a bunch of reading :-)

rg

Ron Garret

unread,
Feb 17, 2008, 3:52:12 PM2/17/08
to
In article <fpa5uj$jjk$1...@aioe.org>,
Abdulaziz Ghuloum <aghu...@cee.ess.indiana.edu> wrote:

Like I said, Paul is aware of the problem. He just thinks it can be
dealt with informally, through naming conventions or some such thing.

See http://www.paulgraham.com/arcchallenge.html

rg

John Thingstad

unread,
Feb 17, 2008, 4:16:31 PM2/17/08
to
På Sun, 17 Feb 2008 21:45:06 +0100, skrev John Thingstad
<jpt...@online.no>:


I should add I would use a code walker and the fact that the macrolet
hasn't expanded yet to substitute the variable with a unique one.

--------------
John Thingstad

Pascal Costanza

unread,
Feb 17, 2008, 4:31:47 PM2/17/08
to

Yes, that's what hygienic macro systems typically do.

John Thingstad

unread,
Feb 17, 2008, 6:20:33 PM2/17/08
to
På Sun, 17 Feb 2008 22:31:47 +0100, skrev Pascal Costanza <p...@p-cos.net>:

>
> Yes, that's what hygienic macro systems typically do.
>
> Pascal
>

Indeed but explicitly. I am talking about by protocol. But I still have to
consider special varaibles.

--------------
John Thingstad

Ken Tilton

unread,
Feb 17, 2008, 6:36:53 PM2/17/08
to

Abdulaziz Ghuloum wrote:
> Ron Garret wrote:
>
>> They realize the problem (or at least Paul does). He just doesn't
>> believe that it's a problem.
>
>
> If describing unintended capture as "kind of freaky" does not
> qualify as admitting the problem, I don't know what does.

Uh, that would be "not choosing it for Arc and then mentioning it (along
with (is nil '()) -> t) as a deliberate design choice".

In a couple of messages pg has laid down a constraint on whining about
Arc: only stuff that creates a problem /in practice/ is of interest.

Unhygienic macros, as freaky as they may be, never make a problem in
practice (except for foot-shooters who skimp on gensyms) and for some
stupid pet tricks variable capture is the only way to go.

btw, even after all these years Cells scares me, esp. when I sneak them
into a database such that it runs around updating itself. But I know
Cells works, so while I may joke about trying to get it past change
control (and doubt I could anywhere I am not the change control
committee), that joking should not be construed as a sober engineering
conviction. As if I am ever sober.

kt

--
http://smuglispweeny.blogspot.com/
http://www.theoryyalgebra.com/

"In the morning, hear the Way;
in the evening, die content!"
-- Confucius

Abdulaziz Ghuloum

unread,
Feb 17, 2008, 7:13:17 PM2/17/08
to
Ken Tilton wrote:

> In a couple of messages pg has laid down a constraint on whining about
> Arc: only stuff that creates a problem /in practice/ is of interest.

NOTHING creates a problem in practice. Dynamic scope: not a problem,
just look at all the elisp code out there. Nonhygienic macros: not a
problem, just gensym and go. Lack of module system: follow a naming
convention. No static type system: we never have type errors around
here. And so on and so on. People always find workarounds. (often
times, these are excuses, not solutions, since there is no problem
to begin with.)

Remember that Fortran has no problems in practice if one is willing
to dismiss every innovation in programming languages over the last
N years as a theoretical solution to a nonexistent problem.

Pascal Costanza

unread,
Feb 17, 2008, 8:17:02 PM2/17/08
to
Abdulaziz Ghuloum wrote:
> Ken Tilton wrote:
>
>> In a couple of messages pg has laid down a constraint on whining about
>> Arc: only stuff that creates a problem /in practice/ is of interest.
>
> NOTHING creates a problem in practice. Dynamic scope: not a problem,
> just look at all the elisp code out there. Nonhygienic macros: not a
> problem, just gensym and go. Lack of module system: follow a naming
> convention. No static type system: we never have type errors around
> here. And so on and so on. People always find workarounds. (often
> times, these are excuses, not solutions, since there is no problem
> to begin with.)

It's not about workarounds, it's about trade offs. Hygienic macro
systems screw up the idea that code is made up of conses, symbols and
atoms, and destroys the conceptual simplicity of that model. Lisp-style
macro systems are nice in that regard, because I can use _any_ library
that operates on conses on program representations as well, because
conses _are_ also program representations in Lisp. I don't need to think
of programs being something substantially different from my other data
structures that I use every day.

Some people find that conceptual simplicity more compelling than fixing
problems that don't really occur in practice. (Note we're talking about
Common Lisp here, in which problems with referential transparency indeed
hardly ever occur. That seems to be different in Scheme.)

You have similar trade offs for the other examples you mention as well.

> Remember that Fortran has no problems in practice if one is willing
> to dismiss every innovation in programming languages over the last
> N years as a theoretical solution to a nonexistent problem.

Fortran is not a programmable programming language. ;)

It is loading more messages.
0 new messages