Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

What's up with Scheme macros?

86 views
Skip to first unread message

Ken Tilton

unread,
Feb 12, 2008, 4:46:15 AM2/12/08
to
While trying to explain my irrational prejudice against Scheme macros I
got curious as to why I hated them, so I googled up this:

http://www.xs4all.nl/~hipster/lib/scheme/gauche/define-syntax-primer.txt

Boys and girls, that is almost eleven thousand words long!!!!!

I see the problem, they have this whole pattern-matching syntax-rules
language moving from source input to output. I have done (and am
supposed to be doing) enough Prolog to realize Scheme macro writers must
be way smarter than me. But... why? No, not why are they smarter. Why do
they have to be smarter? What was wrong with treating the source as an
ordinary tree of data and just whinging it about with Scheme itself to
produce the output? Please restrict yourself to one or more of these
answers:

1. The hygiene made us do it.

2. We actually thought it would be way cool and super powerful.

3. We were embarrassd by the small size of the standard and decided to
make it up in the macro language

4. Other _____________________

Jes curious,

kenny


--
http://smuglispweeny.blogspot.com/
http://www.theoryyalgebra.com/

"In the morning, hear the Way;
in the evening, die content!"
-- Confucius

Shiro Kawai

unread,
Feb 12, 2008, 5:14:13 AM2/12/08
to
I use both (CL and Scheme) at work but more inclined to Scheme.
And I do have a mixed feeling about the Scheme macros.

Ken Tilton <kennytil...@optonline.net> wrote:
> 1. The hygiene made us do it.

If I need to pick one from the first three, probably this is it.

Theoretically, hygiene has nothing to do with pattern matching
sublanguage.
The pattern matching part is for the convenience and efficiency, if
I understand correctly. To realize hygiene, every identifier has to
carry the syntactic context, which is added or removed in every macro
expansion.

It would be very inefficient to do that context handling
eagerly if the macro's input form is huge S-expr but the macro
expander
only shuffles its toplevel elements, so you want to do the context
handling lazily---that is, instead of applying the context to every
leaf
of the S-expr, you just feed a tuple of <S-expr, Context> to the macro
expander, which "peels" off the context on demand (e.g. <(a (b c)),
Context>
is transfomed to (<a, Context> <(b c), Context>) if the expander needs
to see the toplevel elements.) Because of this tupling, you cannot
use
ordinary car/cdr to decompose the input.

The reason I have a mixed feeling is that I do like hygiene (it is
cool
that an exported macro of my module foo expands into a form that
inserts
the reference to the foo's internal macros/procedures and it just
works.
I'd use package prefix in CL, but I feel it's like hack.) but carrying
pattern language is... say, too heavy. Lots of smart people have
tackled
this problem so there may not be the way to avoid that, but I hope
otherwise.

For the meantime, I use both legacy macros and hygienic marcos in my
Scheme code at work; legacy macros are handy if you have total control
over the source so that nobody will step on your toe or vice versa.

Pascal Costanza

unread,
Feb 12, 2008, 6:22:06 AM2/12/08
to
Ken Tilton wrote:
> While trying to explain my irrational prejudice against Scheme macros I
> got curious as to why I hated them, so I googled up this:
>
> http://www.xs4all.nl/~hipster/lib/scheme/gauche/define-syntax-primer.txt
>
> Boys and girls, that is almost eleven thousand words long!!!!!
>
> I see the problem, they have this whole pattern-matching syntax-rules
> language moving from source input to output. I have done (and am
> supposed to be doing) enough Prolog to realize Scheme macro writers must
> be way smarter than me. But... why? No, not why are they smarter. Why do
> they have to be smarter? What was wrong with treating the source as an
> ordinary tree of data and just whinging it about with Scheme itself to
> produce the output? Please restrict yourself to one or more of these
> answers:
>
> 1. The hygiene made us do it.

No, there are hygienic macro systems where source is a tree of data.
Actually, in most hygienic macro systems, source is still a tree of data.

> 2. We actually thought it would be way cool and super powerful.

Lisp-style macros (as in Common Lisp) provide backquote as a convenient
means to construct new s-expressions. One nice thing about backquote is
that you can make backquote expressions look very close the code you
have in your mind. (Compare `(progn ,@body) to (cons 'progn body), for
example.)

Common Lisp also provides destructuring macro lambda lists, where you
can already preprocess the arguments a macro receives by tearing them
apart into subparts. For example: (defmacro with ((&rest bindings) &body
body) ...) instead of (defmacro with (&rest code) ...).

Backquote works pretty well. However, destructuring macro lambda lists
are limited - note how to you can't see the constituents of the bindings
list in the with example. You have to 'manually' tear them apart. (Loop
is pretty good at such things, but you still have to do it.)

If you want to make destructuring more fine-grained, you also have to
make changes to the corresponding construct for reconstructing code
(like backquote). That's where the complexity comes in.

Syntax-rules and syntax-case work quite well in that regard (although I
personally prefer to use LOOP).

> 3. We were embarrassd by the small size of the standard and decided to
> make it up in the macro language

They could have kept the spec smaller by sticking to Lisp-style macros
and adding a rename construct. The other macro systems can be built on
top of that. I guess removing the weaknesses and restrictions that make
additional features appear necessary is not so important after all. :-P

To be more serious: Syntax-rules is a macro system that is very simple
to use for very simple macros. You sometimes want to define your own
macros to illustrate some points in academic papers. Syntax-rules macros
are more than good enough for such purposes, and the macro definitions
can be read by people who are not used to macro programming. That's a
good thing, if the macro is not the main part of what you want to
discuss. Macro definitions in such papers hardly need to be complex (and
can always be simplified for illustration purposes).

You know, there is more you can do with programming languages than
writing applications...


Pascal

--
1st European Lisp Symposium (ELS'08)
http://prog.vub.ac.be/~pcostanza/els08/

My website: http://p-cos.net
Common Lisp Document Repository: http://cdr.eurolisp.org
Closer to MOP & ContextL: http://common-lisp.net/project/closer/

Jens Axel Soegaard

unread,
Feb 12, 2008, 8:21:09 AM2/12/08
to
Ken Tilton wrote:
> While trying to explain my irrational prejudice against Scheme macros I
> got curious as to why I hated them, so I googled up this:
>
> http://www.xs4all.nl/~hipster/lib/scheme/gauche/define-syntax-primer.txt
>
> Boys and girls, that is almost eleven thousand words long!!!!!

And "On Lisp" is even longer!!!!!

"Primer" is obviously not to be taken literally.

> I see the problem, they have this whole pattern-matching syntax-rules
> language moving from source input to output. I have done (and am
> supposed to be doing) enough Prolog to realize Scheme macro writers must
> be way smarter than me. But... why? No, not why are they smarter. Why do
> they have to be smarter? What was wrong with treating the source as an
> ordinary tree of data and just whinging it about with Scheme itself to
> produce the output?

The use of pattern matching is a detail.

That's not why Schemers abandoned the old way. There are *two* goals of
modern Scheme macro systems. Hygiene (i.e. automatic renaming) as
another Paris Hilton got all the press, while the referentially
transparent property was left behind in the dust.

Referentially transparent macros?

If a macro transformer inserts a free reference to an identifier,
the reference refers to the binding that was visible where the
transformer was specified, regardless of any local bindings that may
surround the use of the macro.

In short the meaning of the expansion of a macro doesn't depend on where
it is used. In order to implement that you need to attach information
to variables, and thus symbols are replaced by "syntax-objects
representing symbols".


For a CL view on the matters see these posts in a very informative
thread on Scheme macros in comp.lang.lisp in 2004.

Anton van Straaten in comp.lang.lisp:
http://groups.google.com/group/comp.lang.lisp/msg/4cfdf326287b3d01

Pascal Constanza in comp.lang.lisp:
http://groups.google.com/group/comp.lang.lisp/msg/67d80b9d8da8c085

Anyhow, stop reading about syntax-rules (so last millenium) and treat
yourself with some light reading on syntax-case. Dybvig's

Writing Hygenic Macros in Scheme with Syntax-Case
ftp://ftp.cs.indiana.edu/pub/scheme-repository/doc/pubs/iucstr356.ps.gz

is a must read.

So is Dybvig's contribution to the book "Beautiful code":
http://www.cs.indiana.edu/~dyb/pubs/bc-syntax-case.pdf


--
Jens Axel Søgaard

Kjetil S. Matheussen

unread,
Feb 12, 2008, 8:32:26 AM2/12/08
to
On Tue, 12 Feb 2008, Jens Axel Soegaard wrote:

> Ken Tilton wrote:
> > While trying to explain my irrational prejudice against Scheme macros I
> > got curious as to why I hated them, so I googled up this:
> >
> > http://www.xs4all.nl/~hipster/lib/scheme/gauche/define-syntax-primer.txt
> >
> > Boys and girls, that is almost eleven thousand words long!!!!!
>
> And "On Lisp" is even longer!!!!!
>
> "Primer" is obviously not to be taken literally.
>
> > I see the problem, they have this whole pattern-matching syntax-rules
> > language moving from source input to output. I have done (and am
> > supposed to be doing) enough Prolog to realize Scheme macro writers must
> > be way smarter than me. But... why? No, not why are they smarter. Why do
> > they have to be smarter? What was wrong with treating the source as an
> > ordinary tree of data and just whinging it about with Scheme itself to
> > produce the output?
>
> The use of pattern matching is a detail.
>
> That's not why Schemers abandoned the old way.

I wouldn't say that. My impression is that most scheme implementations,
and all the large ones (anyone who doesn't?), support define-macro. I
would actually also be surprised if hygenic macros is more used by
schemeres than "the old way", although I don't have any data to back
that up with. Anyway, I would never use a scheme where you couldn't
write macros the old way.

andreu...@yahoo.com

unread,
Feb 12, 2008, 10:00:32 AM2/12/08
to
On Feb 12, 5:14 am, Shiro Kawai <shiro.ka...@gmail.com> wrote:

> It would be very inefficient to do that context handling
> eagerly if the macro's input form is huge S-expr but the macro
> expander
> only shuffles its toplevel elements, so you want to do the context
> handling lazily---that is, instead of applying the context to every
> leaf
> of the S-expr, you just feed a tuple of <S-expr, Context> to the macro
> expander, which "peels" off the context on demand (e.g. <(a (b c)),
> Context>
> is transfomed to (<a, Context> <(b c), Context>) if the expander needs
> to see the toplevel elements.) Because of this tupling, you cannot
> use
> ordinary car/cdr to decompose the input.

That is not accurate, but the misconception is understandable given
the language of R6RS.

For example, the explicit renaming system can be
implemented with linear complexity with ordinary s-expressions that
are
decomposable using car, cdr, etc. The same is true for syntax-case.
For example, my (Andre van Tonder's) syntax-case expander uses an
eager
algorithm based on a variant of explicit renaming and using ordinary
s-expressions decomposable using car/cdr. Thus, syntax-case can be
viewed as a convenience layer on top of explicit renaming.
Explicit renaming is really quite simple. See
http://groups.google.com/group/comp.lang.scheme/msg/a4817ec9ffcccf39

I was disappointed that the R6RS editors decided to describe the lazy
algorithm in the R6RS standard, when a simple declarative description
of
hygiene/referential transparency as in R5RS would have sufficed.
Instead they chose to describe, as part of the standard, a specific,
non-unique implementation strategy, in fact probably the most
complicated implementation strategy anyone has bothered to
come up with to date, virtually ensuring that hardly anyone will
understand the document. I don't follow it, and I'm an implementor!


Jens Axel Soegaard

unread,
Feb 12, 2008, 11:11:51 AM2/12/08
to
Kjetil S. Matheussen wrote:

>> That's not why Schemers abandoned the old way.
>
> I wouldn't say that. My impression is that most scheme implementations,
> and all the large ones (anyone who doesn't?), support define-macro. I
> would actually also be surprised if hygenic macros is more used by
> schemeres than "the old way", although I don't have any data to back
> that up with. Anyway, I would never use a scheme where you couldn't
> write macros the old way.

Nowadays there are (depending on point of view) two different
types of define-macro forms. One type is define-macro implemented
on top of define-syntax - this type is generally found in
implementations that have "full" syntax-case integration.

The other one is found in implementations where syntax-case
is treated as an "add on" builds on a "native" define-macro.

The two types are for simple macros equivalent, but for macros
that require phase 2 [*] computations hell is likely to break
loose.

There are two things I look for, when I evaluate macro systems:

1) How easy is it to give error messages in terms of the
*original* user syntax?

2) Can the module system handle the macro system?
[I.e. how (if at all) is the phase problem solved?]

Most "native" define-macro systems handle these points on an
ad hoc basis.


[*]
http://pre.plt-scheme.org/docs/html/reference/Evaluation_Model.html#(part~20module-phase)

andreu...@yahoo.com

unread,
Feb 12, 2008, 11:26:54 AM2/12/08
to
On Feb 12, 4:46 am, Ken Tilton <kennytil...@optonline.net> wrote:

> I see the problem, they have this whole pattern-matching syntax-rules
> language moving from source input to output. I have done (and am
> supposed to be doing) enough Prolog to realize Scheme macro writers must
> be way smarter than me. But... why? No, not why are they smarter. Why do
> they have to be smarter? What was wrong with treating the source as an
> ordinary tree of data and just whinging it about with Scheme itself to
> produce the output?

Nothing is wrong with your suggestion. Here is a simple macro
that will work in one of the R6RS reference implementations.

(define-syntax swap!
(lambda (form)
(let ((a (cadr form))
(b (caddr form)))
#`(let ((temp #,a))
(set! #,a #,b)
(set! #,b temp))))))

It is almost identical to the corresponding defmacro, except that
you use #` and #, instead of ` and ,. Also, no gensym is necessary.
You can think, as a fisrt approximation, of #` as automatically
inserting the necessary gensyms for you.

Andre

Raffael Cavallaro

unread,
Feb 12, 2008, 12:31:36 PM2/12/08
to
On 2008-02-12 11:26:54 -0500, andreu...@yahoo.com said:

> Nothing is wrong with your suggestion. Here is a simple macro
> that will work in one of the R6RS reference implementations.
>
> (define-syntax swap!
> (lambda (form)
> (let ((a (cadr form))
> (b (caddr form)))
> #`(let ((temp #,a))
> (set! #,a #,b)
> (set! #,b temp))))))
>
> It is almost identical to the corresponding defmacro, except that
> you use #` and #, instead of ` and ,. Also, no gensym is necessary.
> You can think, as a fisrt approximation, of #` as automatically
> inserting the necessary gensyms for you.

Not that I doubt the power of your system Andre, but could you show us
a simple example of intentional capture inside a #` form?

Maciej Katafiasz

unread,
Feb 12, 2008, 12:45:09 PM2/12/08
to
Den Tue, 12 Feb 2008 04:46:15 -0500 skrev Ken Tilton:

> While trying to explain my irrational prejudice against Scheme macros I
> got curious as to why I hated them, so I googled up this:
>
> http://www.xs4all.nl/~hipster/lib/scheme/gauche/define-syntax-
primer.txt
>
> Boys and girls, that is almost eleven thousand words long!!!!!

Woah there, back in my day, a three-article technical *series* was
absolutely, unquestionably, your editor is really good at using Delete
thank you very much limited to 6K words (including intros, conclusions
and recaps). So that's two complete series there.

> Please restrict yourself to one or more of these
> answers:
>

> 4. Other _____________________

They had to write articles with strict wordcount limits before and decide
that this time fuck it, *they* will decide when to stop.

Cheers,
Maciej

michele....@gmail.com

unread,
Feb 12, 2008, 1:56:50 PM2/12/08
to
I have been skeptical about Scheme macros for a long time,
thinking that syntax-rules is too simple and syntax-case too complex,
and I have been using define-macro instead. However, in the last
couple of months I have somewhat changed my mind. I figured out that
the biggest issue I had with syntax-case was notational. If you are
willing to dress it with a minor amount of sugar, syntax-case becomes
pretty easy to use and quite readable too. Here is the swap! example:

(define-syntax-case swap! ()
((swap! x y) (let ((t x)) (set! x y) (set! y t))))

define-syntax-case is just sugar around syntax-case:

;; tested on Ikarus Scheme
(define-syntax define-syntax-case
(syntax-rules ()
((_ name (literal ...)
((_ arg ...) templ) ...)
(define-syntax name
(lambda (x)
(syntax-case x (<expand> literal ...)
((ctx <expand> arg ...) #`'templ) ...
((ctx arg ...) #`templ) ...))))
((_ name (literal ...)
((_ arg ...) fender templ) ...)
(define-syntax name
(lambda (x)
(syntax-case x (<expand> literal ...)
((ctx <expand> arg ...) fender #`'templ) ...
((ctx arg ...) fender #`templ) ...))))
))

Notice that I also added an <expand> feature to see the expansion
of the macro:

(swap! <expand> x y) ;=> (let ((t x)) (set! x y) (set! y t))


Michele Simionato

andreu...@yahoo.com

unread,
Feb 12, 2008, 2:07:56 PM2/12/08
to
On Feb 12, 12:31 pm, Raffael Cavallaro <raffaelcavallaro@pas-d'espam-
s'il-vous-plait-mac.com> wrote:

> Not that I doubt the power of your system Andre, but could you show us
> a simple example of intentional capture inside a #` form?

(define-syntax if-it
(lambda (form)
(let ((it (datum->syntax (car form) 'it)))
#`(let ((,it ,(cadr form)))
(if #,it
#,(caddr form)
#,(cadddr form))))))

DATUM->SYNTAX overrides hygiene by creating the identifier IT as if
it
appeared in the source at the position of (CAR FORM), i.e., the IF-IT
at the usage site of the macro. You may think of this, very roughly,
as a standard way of making an identifier that is not affected by the
automatic gensym that would otherwise be applied by #`.

This is slightly longer than the defmacro way, but macros that
capture
are usually a minority. Most defmacros can be trivially converted
to Scheme hygienic macros (in this R6RS reference implementation)
by omitting gensyms and putting #s in front of the `s,'s, and ,s
and would therefore be shortened rather than lengthened.

Unfortunately this is not true for all implementations of R6RS Scheme
macros. Some unfortunately do not represent code as s-expressions
and rely on pattern matching to decompose their input.
While there are reasons for their design choice, I think it very
unfortunately obscures the fact that pattern matching is absolutely
orthogonal to - and unnecessary for - the hygiene mechanism. As a
result,
people like the OP get the impression that hygiene is more
complicated
than it really is, and cannot entirely be blamed for giving
up in disgust.

Andre

andreu...@yahoo.com

unread,
Feb 12, 2008, 2:37:54 PM2/12/08
to
On Feb 12, 2:07 pm, andreuri2...@yahoo.com wrote:

> (define-syntax if-it
> (lambda (form)
> (let ((it (datum->syntax (car form) 'it)))
> #`(let ((,it ,(cadr form)))
> (if #,it
> #,(caddr form)
> #,(cadddr form))))))


Sorry. I meant:

(define-syntax if-it
(lambda (form)
(let ((it (datum->syntax (car form) 'it)))

#`(let ((#,it #,(cadr form)))


(if #,it
#,(caddr form)
#,(cadddr form))))))

Andre

Abdulaziz Ghuloum

unread,
Feb 12, 2008, 3:24:15 PM2/12/08
to
michele....@gmail.com wrote:

> (define-syntax define-syntax-case
> (syntax-rules ()
> ((_ name (literal ...)
> ((_ arg ...) templ) ...)
> (define-syntax name
> (lambda (x)
> (syntax-case x (<expand> literal ...)
> ((ctx <expand> arg ...) #`'templ) ...
> ((ctx arg ...) #`templ) ...))))
> ((_ name (literal ...)
> ((_ arg ...) fender templ) ...)
> (define-syntax name
> (lambda (x)
> (syntax-case x (<expand> literal ...)
> ((ctx <expand> arg ...) fender #`'templ) ...
> ((ctx arg ...) fender #`templ) ...))))
> ))

I wonder how this would look using s-expression manipulation
(cars/cdrs). Andre, since you seem to be the expert on this,
do you want to give it a shot?

Eli Barzilay

unread,
Feb 12, 2008, 3:45:14 PM2/12/08
to
"Kjetil S. Matheussen" <k.s.mat...@notam02.no> writes:

> I wouldn't say that. My impression is that most scheme
> implementations, and all the large ones (anyone who doesn't?),
> support define-macro. I would actually also be surprised if hygenic
> macros is more used by schemeres than "the old way", although I

> don't have any data to back that up with. [...]

Out of curiosity, I inspected the PLT collects tree. Here's a
summary:

* about 400K lines of Scheme code

* 12 occurrences of `define-macro'; 5 are in the define-macro library
source, 3 in comments, 4 are in misc meta-code (keyword colors,
indentation, etc), that leaves 0 uses in real code.

* 1438 occurrences of `define-syntax' (and `...-syntaxes')

* 1360 of `syntax-case'

* 468 of `syntax-rules'

It's interesting to compare this to version 103 (which is before PLT
switched to `syntax-case' being native):

* About 150K lines

* 125 `define-macro'

* 7 `define-syntax', 1 `syntax-rules', 1 `syntax-case' none is
actually used

And here's an interesting result: if the switch to the new system
would mean changing all `define-macro's to `define-syntax', and
assuming the same "macro use density" per line of code, then you'd
expect to see about 300 syntax definitions. So the new macro system is
robust enough that it is used about 4.5 times more often than
define-macro.

--
((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay:
http://www.barzilay.org/ Maze is Life!

Eli Barzilay

unread,
Feb 12, 2008, 3:50:02 PM2/12/08
to
Ken Tilton <kenny...@optonline.net> writes:

> While trying to explain my irrational prejudice against Scheme
> macros I got curious as to why I hated them, so I googled up this:
>
> http://www.xs4all.nl/~hipster/lib/scheme/gauche/define-syntax-primer.txt
>
> Boys and girls, that is almost eleven thousand words long!!!!!

Note the name of the document -- it describes `syntax-rules' and how
to do with it things that seem impossible from a very restricted
syntax rewriting system. Using *just* that is on the same masochistic
level as using *just* r5rs for real programming.

Abdulaziz Ghuloum

unread,
Feb 12, 2008, 4:55:30 PM2/12/08
to
Eli Barzilay wrote:

> So the new macro system is robust enough that it is
> used about 4.5 times more often than define-macro.

Or, it may be that you guys are just macro noobs. You
can't handle the power of lisp macros, and you want
this thing called "hygiene" to protect you from the so
called "unintended capture" straw-man that you throw
around at every occasion.

The real programmer does not need no stinkin' hygiene.

andreu...@yahoo.com

unread,
Feb 12, 2008, 6:03:52 PM2/12/08
to
On Feb 12, 3:24 pm, Abdulaziz Ghuloum <aghul...@cee.ess.indiana.edu>
wrote:

I'll leave that to the Lispniks, who have more practice
with this kind of thing. It might even occur to some of
them that pattern matching, despite being orthogonal to and
unnecessary for hygiene, is a quite useful thing to have.

Shiro Kawai

unread,
Feb 12, 2008, 7:01:33 PM2/12/08
to
andreuri2...@yahoo.com wrote:
> That is not accurate, but the misconception is understandable given
> the language of R6RS.
>
> For example, the explicit renaming system can be
> implemented with linear complexity with ordinary s-expressions that
> are
> decomposable using car, cdr, etc. The same is true for syntax-case.
> For example, my (Andre van Tonder's) syntax-case expander uses an
> eager
> algorithm based on a variant of explicit renaming and using ordinary
> s-expressions decomposable using car/cdr. Thus, syntax-case can be
> viewed as a convenience layer on top of explicit renaming.
> Explicit renaming is really quite simple. Seehttp://groups.google.com/group/comp.lang.scheme/msg/a4817ec9ffcccf39

Aha! I've seen the explicit renaming expander but have never looked
at
it closely. Now I read the articles and found it made a lot of sense,
and it feels more Scheme-way to me. I'll give it a shot. Thanks!

michele....@gmail.com

unread,
Feb 12, 2008, 11:41:56 PM2/12/08
to
On Feb 13, 12:03 am, andreuri2...@yahoo.com wrote:

> I'll leave that to the Lispniks, who have more practice
> with this kind of thing. It might even occur to some of
> them that pattern matching, despite being orthogonal to and
> unnecessary for hygiene, is a quite useful thing to have.

Actually one of the reasons why I changed my mind about Scheme
macros is that I realized the benefits of pattern matching
when writing complex macros. BTW, I just noticed that I have posted
the wrong version of define-syntax-case, the correct one is

(define-syntax define-syntax-case
(syntax-rules ()
((_ name (literal ...)

((ctx arg ...) templ) ...)


(define-syntax name
(lambda (x)
(syntax-case x (<expand> literal ...)
((ctx <expand> arg ...) #`'templ) ...
((ctx arg ...) #`templ) ...))))
((_ name (literal ...)

((ctx arg ...) fender templ) ...)

Alex Shinn

unread,
Feb 13, 2008, 8:39:28 AM2/13/08
to
>>>>> "Ken" == Ken Tilton <kenny...@optonline.net> writes:

[ ... why syntax-rules? ... ]

Ken> 4. Other _____________________

I won't presume as to the R5RS editors original reasons, but
I suspect it has to do with the fact that SYNTAX-RULES was
and remains the simplest and most natural macro system for
describing all of the derived syntax in R5RS itself.

There are plenty of other cases where SYNTAX-RULES clearly
wins. Just consider the confusion that a newbie recently
ran into trying to use DEFINE-MACRO:

http://list.cs.brown.edu/pipermail/plt-scheme/2008-February/022777.html

... and how SYNTAX-RULES saves the day:

http://list.cs.brown.edu/pipermail/plt-scheme/2008-February/022782.html

--
Alex

Eli Barzilay

unread,
Feb 13, 2008, 9:25:22 AM2/13/08
to
Abdulaziz Ghuloum <aghu...@cee.ess.indiana.edu> writes:

> Eli Barzilay wrote:
>
>> So the new macro system is robust enough that it is
>> used about 4.5 times more often than define-macro.
>
> Or, it may be that you guys are just macro noobs.

No.


> You can't handle the power of lisp macros, and you want this thing
> called "hygiene" to protect you from the so called "unintended
> capture" straw-man that you throw around at every occasion.
>
> The real programmer does not need no stinkin' hygiene.

--

Pascal Costanza

unread,
Feb 13, 2008, 3:32:07 PM2/13/08
to
andreu...@yahoo.com wrote:
> On Feb 12, 12:31 pm, Raffael Cavallaro <raffaelcavallaro@pas-d'espam-
> s'il-vous-plait-mac.com> wrote:
>
>> Not that I doubt the power of your system Andre, but could you show us
>> a simple example of intentional capture inside a #` form?
>
> (define-syntax if-it
> (lambda (form)
> (let ((it (datum->syntax (car form) 'it)))
> #`(let ((,it ,(cadr form)))
> (if #,it
> #,(caddr form)
> #,(cadddr form))))))
>
> DATUM->SYNTAX overrides hygiene by creating the identifier IT as if
> it appeared in the source at the position of (CAR FORM), i.e., the
> IF-IT at the usage site of the macro. You may think of this, very
> roughly, as a standard way of making an identifier that is not
> affected by the automatic gensym that would otherwise be applied by
> #`.

This sounds like I have to say twice where the identifier is inserted.
That's weird. Why is that the case?

> This is slightly longer than the defmacro way, but macros that
> capture are usually a minority.

Says who?

> Some unfortunately do not represent code as s-expressions and rely on
> pattern matching to decompose their input. While there are reasons
> for their design choice, I think it very unfortunately obscures the
> fact that pattern matching is absolutely orthogonal to - and
> unnecessary for - the hygiene mechanism. As a result, people like the
> OP get the impression that hygiene is more complicated than it really
> is, and cannot entirely be blamed for giving up in disgust.

Yes, you're macro system looks more attractive than syntax-case to me. I
understand that syntax-rules can be neat when it's all you need, but for
more complex stuff, s-expressions are simply better (IMHO, and at all
that, of course).

Does your system also support the kinds of macros that are mentioned for
example in
http://www.lispworks.com/documentation/HyperSpec/Issues/iss066_w.htm ?

Kaz Kylheku

unread,
Feb 13, 2008, 3:47:01 PM2/13/08
to
On Feb 13, 5:39 am, Alex Shinn <alexsh...@gmail.com> wrote:
> There are plenty of other cases where SYNTAX-RULES clearly
> wins.  Just consider the confusion that a newbie recently
> ran into trying to use DEFINE-MACRO:
>
>  http://list.cs.brown.edu/pipermail/plt-scheme/2008-February/022777.html
>
> ... and how SYNTAX-RULES saves the day:
>
>  http://list.cs.brown.edu/pipermail/plt-scheme/2008-February/022782.html

I can hardly make heads or tails out of the nested defspel. I need to
see the nested backquotes to see what is the meta-level and what is
meta-meta, otherwise my eyes glaze over. I can't see what part of the
template is literal, and what part is spliced in, because nothing is
denoting that. And look, even though things are quoted, they are still
substituted. You have expressions like (eq? 'object 'obj), but this is
really like `(eq? (quote ,object) (quote ,obj)). It looks like these
are simple quoted literal symbols, but no, there is still evaluation
taking place. Confusing! Suppose you're a sloppy Frenchman and make a
typo and write (eq? 'objet 'obj). The 'objet is now literal, because
objet isn't a macro parameter. That's evil!

This basically brings us back to the stone age of C language macros:

#define STR(X) #X
#define SPEL(OBJECT, SUBJECT) foo(SUBJECT, STR(OBJET))

Look at this, here, too, there is no ugly backquote to confuse
newbies! C preprocessor saves the day! Of course, when you call
SPEL(X, Y) you end up with foo(X, "OBJET") instead of foo(X, "Y").
Oops!

William D Clinger

unread,
Feb 13, 2008, 4:52:55 PM2/13/08
to
When you get right down to it, the real reason we
added syntax-case to Scheme was #' envy.

Will

Raffael Cavallaro

unread,
Feb 13, 2008, 6:30:45 PM2/13/08
to
On 2008-02-12 14:37:54 -0500, andreu...@yahoo.com said:

> Sorry. I meant:
>
> (define-syntax if-it
> (lambda (form)
> (let ((it (datum->syntax (car form) 'it)))
> #`(let ((#,it #,(cadr form)))
> (if #,it
> #,(caddr form)
> #,(cadddr form))))))
>
> Andre

Thanks. I can get this to work using:

(ex:repl '((import (rnrs)) ...


but is there some way to get an *interactive* repl in one of the
compatible schemes? I've tried the existing
r6rs-expander-vantonder.plt, but it's kind annoying to use. For
example, if I enter your definition above:

R5.97RS> (define-syntax if-it


(lambda (form)
(let ((it (datum->syntax (car form) 'it)))
#`(let ((#,it #,(cadr form)))
(if #,it
#,(caddr form)
#,(cadddr form))))))

Syntax violation: invalid reference

Attempt to use binding of lambda in library () at invalid level 1.
Binding is only available at levels: 0

Form: lambda

Trace:

(lambda (form)
(let ((it (datum->syntax (car form) 'it)))

(quasisyntax
(let (((unsyntax it) (unsyntax (cadr form))))
(if (unsyntax it) (unsyntax (caddr form)) (unsyntax (cadddr form)))))))

syntax-violation: Integrate with host error handling here

More or less defeats the purpose of a repl if I have to define a
library for such simple things - unless I'm misunderstanding the error
altogether.

William D Clinger

unread,
Feb 13, 2008, 10:12:38 PM2/13/08
to
Raffael Cavallaro wrote:
> Thanks. I can get this to work using:
>
> (ex:repl '((import (rnrs)) ...
>
> but is there some way to get an *interactive* repl in one of the
> compatible schemes?

Larceny's ERR5RS mode uses Andre's interactive REPL as
its native REPL:

% larceny -err5rs
Larceny v0.961 "Fluoridation" (Jan 2 2008 04:27:56,
precise:SunOS5:split)
ERR5RS mode (no libraries have been imported)

> (import (rnrs))
Autoloading (rnrs)
Autoloading (rnrs enums)
Autoloading (rnrs lists)
Autoloading (rnrs syntax-case)
Autoloading (rnrs hashtables)
Autoloading (rnrs arithmetic bitwise)
Autoloading (rnrs programs)
Autoloading (rnrs files)
Autoloading (rnrs io ports)
Autoloading (larceny deprecated)
Autoloading (rnrs conditions)
Autoloading (rnrs exceptions)
Autoloading (rnrs records syntactic)
Autoloading (err5rs records procedural)
Autoloading (rnrs records procedural)
Autoloading (rnrs control)
Autoloading (rnrs sorting)
Autoloading (rnrs bytevectors)
Autoloading (rnrs unicode)

> (define-syntax if-it
(lambda (form)
(let ((it (datum->syntax (car form) 'it)))
#`(let ((#,it #,(cadr form)))
(if #,it
#,(caddr form)
#,(cadddr form))))))

>

Will

llama

unread,
Feb 14, 2008, 12:25:58 AM2/14/08
to
"William D Clinger" <cesu...@yahoo.com> wrote in message
news:5c7a0076-f785-4011...@e25g2000prg.googlegroups.com...


That's nice. Now show an usage example. Every Scheme I tried choked on on
the application of that macro.

Abdulaziz Ghuloum

unread,
Feb 14, 2008, 1:15:03 AM2/14/08
to
llama wrote:

> Now show an usage example. Every Scheme I tried choked on
> on the application of that macro.

Andre already told you that this macro that he's written uses
an extension that's not supported by any any implementation
except larceny (by virtue of using his system). Fortunate or
unfortunate, there *is* a portable way (as far as R6RS goes)
of writing this macro as shown below (it might run on other
systems that I have not tested). The portable version also
happens to be more robust--it gives a better error message if
the input is malformed instead of giving the ``error in
caddr'' sort of error that you'd get otherwise.

Pick the version that better suits your needs.

Aziz,,,

$ ikarus
Ikarus Scheme version 0.0.3+ (revision 1384, build 2008-02-13)
Copyright (c) 2006-2008 Abdulaziz Ghuloum

> (define-syntax if-it
(lambda (stx)
(syntax-case stx ()
((ctxt test conseq altern)
(let ((it (datum->syntax #'ctxt 'it)))
#`(let ((#,it test))
(if #,it conseq altern)))))))
> (if-it 17 it 18)
17


$ larceny -err5rs
Larceny v0.961 "Fluoridation" (Jan 2 2008 04:30:22, precise:Posix
Unix:unified)
larceny.heap, built on Wed Jan 2 04:41:29 EST 2008


ERR5RS mode (no libraries have been imported)

> (import (rnrs))
Autoloading ...

> (define-syntax if-it
(lambda (stx)
(syntax-case stx ()
((ctxt test conseq altern)
(let ((it (datum->syntax #'ctxt 'it)))
#`(let ((#,it test))
(if #,it conseq altern)))))))

> (if-it 17 it 18)
17


$ mzscheme
Welcome to MzScheme v3.99.0.10 [3m], Copyright (c) 2004-2008 PLT Scheme Inc.
> (define-syntax if-it
(lambda (stx)
(syntax-case stx ()
((ctxt test conseq altern)
(let ((it (datum->syntax #'ctxt 'it)))
#`(let ((#,it test))
(if #,it conseq altern)))))))
> (if-it 17 it 18)
17


$ petite
Petite Chez Scheme Version 7.4
Copyright (c) 1985-2007 Cadence Research Systems

> (define-syntax if-it
(lambda (stx)
(syntax-case stx ()
((ctxt test conseq altern)
(let ((it (datum->syntax #'ctxt 'it)))
#`(let ((#,it test))
(if #,it conseq altern)))))))
> (if-it 17 it 18)
17


William D Clinger

unread,
Feb 14, 2008, 1:24:30 AM2/14/08
to
llama wrote:
> That's nice. Now show an usage example. Every Scheme I tried choked on on
> the application of that macro.

It works in Larceny. See below.

Will

--------

% larceny -err5rs
Larceny v0.961 "Fluoridation" (Jan 19 2008 12:07:22,

> (if-it #t 'then 'else)
then

> (if-it #f 'then 'else)
else

> (if-it #t it 'else)
#t

> ;; Now for a more familiar use of intentional capture.
(define-syntax myloop
(lambda (form)
(let ((break (datum->syntax (car form) 'break)))
#`(call/cc
(lambda (k)
(let ((#,break k))
(do ()
(#f)
(begin #,@(cdr form)))))))))

> (let ((x '(3 2 1 0 -1 -2 -3 -4)))
(myloop (if (zero? (car x))
(break (cdr x))
(set! x (cdr x)))))
(-1 -2 -3 -4)

Raffael Cavallaro

unread,
Feb 14, 2008, 1:52:02 AM2/14/08
to
On 2008-02-13 22:12:38 -0500, William D Clinger <cesu...@yahoo.com> said:

> Larceny's ERR5RS mode uses Andre's interactive REPL as
> its native REPL:
>
> % larceny -err5rs
> Larceny v0.961 "Fluoridation" (Jan 2 2008 04:27:56,
> precise:SunOS5:split)
> ERR5RS mode (no libraries have been imported)
>
>> (import (rnrs))

Ah, I remember now reading something about this on the larceny mailing
list. Thanks - I've just written, compiled, loaded, imported and run my
first err5rs/r6rs library - works nicely.

bunn...@gmail.com

unread,
Feb 14, 2008, 4:46:06 AM2/14/08
to
On Feb 12, 9:45 pm, Eli Barzilay <e...@barzilay.org> wrote:

> "Kjetil S. Matheussen" <k.s.matheus...@notam02.no> writes:
>
> > I wouldn't say that. My impression is that most scheme
> > implementations, and all the large ones (anyone who doesn't?),
> > support define-macro. I would actually also be surprised if hygenic
> > macros is more used by schemeres than "the old way", although I
> > don't have any data to back that up with. [...]
>
> Out of curiosity, I inspected the PLT collects tree. Here's a
> summary:
>
> * about 400K lines of Scheme code
>
> * 12 occurrences of `define-macro'; 5 are in the define-macro library
> source, 3 in comments, 4 are in misc meta-code (keyword colors,
> indentation, etc), that leaves 0 uses in real code.
>
> * 1438 occurrences of `define-syntax' (and `...-syntaxes')
>
> * 1360 of `syntax-case'
>
> * 468 of `syntax-rules'
>

As a comparison, here a grep through the chicken code repository
(including tagged versions):

define-macro: 9235
define-syntax: 5504
syntax-rules: 5249


cheers,
felix

William D Clinger

unread,
Feb 14, 2008, 8:44:31 AM2/14/08
to
Tabulating data collected by Eli Barzilay and
Felix Winkelmann:

PLT Chicken Larceny
define-macro 12 9235 67
define-syntax 1438 5504 1806
syntax-rules 468 5249 1703
syntax-case 1360 ? 271

For Larceny, most of the macros are defined in
libraries and benchmarks written by people who
are not themselves developers of Larceny.

Will

Pascal Costanza

unread,
Feb 14, 2008, 9:05:06 AM2/14/08
to
Alex Shinn wrote:
>>>>>> "Ken" == Ken Tilton <kenny...@optonline.net> writes:
>
> [ ... why syntax-rules? ... ]
>
> Ken> 4. Other _____________________
>
> I won't presume as to the R5RS editors original reasons, but
> I suspect it has to do with the fact that SYNTAX-RULES was
> and remains the simplest and most natural macro system for
> describing all of the derived syntax in R5RS itself.

I don't think that syntax rules. ;)

llama

unread,
Feb 14, 2008, 10:51:56 AM2/14/08
to
"Abdulaziz Ghuloum" <aghu...@cee.ess.indiana.edu> wrote in message
news:fp0m97$iki$1...@aioe.org...

> llama wrote:
>
>> Now show an usage example. Every Scheme I tried choked on on the
>> application of that macro.
>
> Andre already told you that this macro that he's written uses
> an extension that's not supported by any any implementation
> except larceny (by virtue of using his system).

Thanks, I must have missed that part.

William D Clinger

unread,
Feb 14, 2008, 4:40:06 PM2/14/08
to
Felix Winkelmann wrote:
> As a comparison, here a grep through the chicken code repository
> (including tagged versions):

If a macro is present in many tagged versions, then it
gets counted once for every version. That bothered me,
so I decided to do the grep myself for all three systems.
I did a wget from

http://pre.plt-scheme.org/plt-clean-tree.tgz

and an svn export from

https://galinha.ucpel.tche.br/svn/chicken-eggs/
http://larceny.ccs.neu.edu/svn/trunk/larceny_src

From chicken-eggs, I then deleted all directories named
"tags" or "branches". This left:

all files Scheme files
-------------------
MB # KLOC
PLT 85 MB 24 2655 627
Chicken 185 MB 53 5008 1470
Larceny 47 MB 12 1193 344

I then grepped the remaining .sch, .scm, and .ss files
for "define-macro", "define-syntax", "syntax-rules",
and "syntax-case". Results:

PLT Chicken Larceny
define-macro 66 4234 67
define-syntax 1908 3583 1807
syntax-rules 651 3297 1704
syntax-case 1675 1110 272

Each of the three columns shows a different pattern, so
generalizations based on a single programmer's experience
or on the code written for any one implementation should
be regarded with skepticism.

Will

Abdulaziz Ghuloum

unread,
Feb 14, 2008, 7:30:33 PM2/14/08
to
Ken Tilton wrote:
> What was wrong with treating the source as an ordinary tree
> of data and just whinging it about with Scheme itself to
> produce the output?

There is nothing really wrong with that. However, permitting
alternative representation of the source opens up the opportunity
for the implementation to do more. An example of that is having
the implementation report contextual information about where
syntax errors occurred and to keep that information around so
that debuggers and profilers show information in the context of
the original source code, not the expanded code.

Here is a small example to illustrate the first case.

$ cat test.ss
#!/usr/bin/env scheme-script
(import (rnrs))

(define-syntax foo
(syntax-rules ()
[(_ a b c) (let ([a b]) c)]))

(define-syntax bar
(syntax-rules ()
[(_ a b c) (foo b a c)]))

(foo a 12 (bar b 17 (list a b)))


$ ikarus --r6rs-script test.ss
Unhandled exception:
Condition components:
1. &who: let
2. &message: "not an identifier"
3. &syntax:
form: (let ((17 b)) (list a b))
subform: 17
4. &trace: #<syntax (let ((17 b)) (list a b))>
5. &trace: #<syntax (foo 17 b (list a b))>
6. &trace: #<syntax (bar b 17 (list a b)) [byte 200 of test.ss]>


As you can see, it shows the original file name and position in
terms of the user's original source code. It also shows the
macro expansion steps that contributed to producing that error.
The outer call to foo is not shown because it expanded properly.
The inner call to bar expanded to a call to foo, which expanded
to a let, which caught the error. The transformation steps are
shown in the trace output.

Some may think that this advantage does not outweigh dropping
the raw s-expression form. But others value things differently.
These are just different points in the design space. Neither
is "wrong": they just cater to different needs.

Aziz,,,

Alex Shinn

unread,
Feb 14, 2008, 7:34:49 PM2/14/08
to
On Feb 15, 6:40 am, William D Clinger <cesur...@yahoo.com> wrote:
>
> PLT Chicken Larceny
> define-macro 66 4234 67
> define-syntax 1908 3583 1807
> syntax-rules 651 3297 1704
> syntax-case 1675 1110 272

sc-macro-transformer 16

Though in all fairness 7 of those were from the
syntactic-closures egg itself.

Chicken has no active uses of er-macro-transformer.

--
Alex

michele....@gmail.com

unread,
Feb 15, 2008, 12:20:08 AM2/15/08
to
On Feb 15, 1:30 am, Abdulaziz Ghuloum <aghul...@cee.ess.indiana.edu>
wrote:list a b)))

>
> $ ikarus --r6rs-script test.ss
> Unhandled exception:
> Condition components:
> 1. &who: let
> 2. &message: "not an identifier"
> 3. &syntax:
> form: (let ((17 b)) (list a b))
> subform: 17
> 4. &trace: #<syntax (let ((17 b)) (list a b))>
> 5. &trace: #<syntax (foo 17 b (list a b))>
> 6. &trace: #<syntax (bar b 17 (list a b)) [byte 200 of test.ss]>

Why I don't get the traceback? My output stops at point 3. Are you
referring to a development version of Ikarus, or there is a debug
flag I should set somewhere to see the traceback? I am using
Ikarus 0.0.3.

Michele Simionato

Kaz Kylheku

unread,
Feb 15, 2008, 12:36:46 AM2/15/08
to
On Feb 14, 4:30 pm, Abdulaziz Ghuloum <aghul...@cee.ess.indiana.edu>
wrote:

> $ ikarus --r6rs-script test.ss
> Unhandled exception:
>   Condition components:
>     1. &who: let
>     2. &message: "not an identifier"
>     3. &syntax:
>         form: (let ((17 b)) (list a b))
>         subform: 17
>     4. &trace: #<syntax (let ((17 b)) (list a b))>
>     5. &trace: #<syntax (foo 17 b (list a b))>
>     6. &trace: #<syntax (bar b 17 (list a b)) [byte 200 of test.ss]>
>
> As you can see, it shows the original file name and position in
> terms of the user's original source code.  

S-expressions read from a file (and all their constituent
expressions), can be associated this information in a global hash
table. Given some erroneous form, the macro can see if there is
associated information with that form.

In principle, the origin of every literal object (or at least that of
every boxed one) that was read from some source file can be
pinpointed.

> Some may think that this advantage does not outweigh dropping
> the raw s-expression form.

It doesn't seem to /technically/ require such dropping.

> But others value things differently.

E.g. eating the cake and having it too.

Abdulaziz Ghuloum

unread,
Feb 15, 2008, 12:37:29 AM2/15/08
to
michele....@gmail.com wrote:

> Why I don't get the traceback? My output stops at point 3. Are you
> referring to a development version of Ikarus, or there is a debug
> flag I should set somewhere to see the traceback? I am using
> Ikarus 0.0.3.

I just added that yesterday :-) So, you can get it from the repo:

$ bzr checkout --lightweight http://www.cs.indiana.edu/~aghuloum/ikarus.dev

Abdulaziz Ghuloum

unread,
Feb 15, 2008, 12:52:34 AM2/15/08
to
Kaz Kylheku wrote:

>> Some may think that this advantage does not outweigh dropping
>> the raw s-expression form.
>
> It doesn't seem to /technically/ require such dropping.

Neither did I say it was, nor am I going to attempt to
prove otherwise. Actually, there may already exist some
system that somehow manages to pull it off, but to my
limited knowledge, I have not seen it done. Moreover,
adding the trace to my expander took no more than 45
minutes of doing minor edits to one file. Additionally,
the trace information (which is associated with the syntax
objects themselves) is not interned anywhere, and works
across files and will just work even with separately
compiled libraries.

All I show here is the advantage that *I* got from having
special syntax objects over having to use s-expressions.
Mileages vary.

Aziz,,,

Raffael Cavallaro

unread,
Feb 15, 2008, 1:54:58 AM2/15/08
to
On 2008-02-14 16:40:06 -0500, William D Clinger <cesu...@yahoo.com> said:

> Results:
>
> PLT Chicken Larceny
> define-macro 66 4234 67
> define-syntax 1908 3583 1807
> syntax-rules 651 3297 1704
> syntax-case 1675 1110 272
>
> Each of the three columns shows a different pattern, so
> generalizations based on a single programmer's experience
> or on the code written for any one implementation should
> be regarded with skepticism.

I wonder what the column for Gambit would look like - syntax-case and
define-syntax are not loaded by default in Gambit so I would suspect
they don't figure heavily in Gambit's own source.

andreu...@yahoo.com

unread,
Feb 15, 2008, 10:18:39 AM2/15/08
to
On Feb 13, 3:47 pm, Kaz Kylheku <kkylh...@gmail.com> wrote:

> I can hardly make heads or tails out of the nested defspel. I need to
> see the nested backquotes to see what is the meta-level and what is
> meta-meta, otherwise my eyes glaze over.

This may be a question of familiarity. I tend to glaze
over when confronted by the nested quasiquotations that tend to
appear
in macro-generating macros. Here is a snippet from
one of my own old define-macros

`(define-macro (,name ,k ,@args)
`(,,k ,',supers ,',labels ,@,args)))))

and a little quiz:

(let ((x '(a b c))) ``(,,x ,@,x ,,@x ,@,@x)) => ??

Define-syntax templates are easier for me in these situations.

> I can't see what part of the
> template is literal, and what part is spliced in, because nothing is
> denoting that.

That is certainly a valid complaint. It could be mitigated somewhat
by
a notational convention like

(define-syntax swap
(syntax-rules ()
((_ x y) (LET ((TEMP x))
(SET! x y)
(SET! y TEMP)))))

or by text colorization in a sufficiently smart editor. But this is
not typically something that seems to trip up Scheme macro writers
too much.

Joel J. Adamson

unread,
Feb 15, 2008, 9:22:44 AM2/15/08