Syntactic Abstraction under the New Model
Aaron W. Hsu
systems should exist in the base or core Scheme standard and that
SYNTAX-RULES should be relegated to a separate library. I wholeheartedly
disagree with this sentiment. There are significant flaws to the DEF-MACRO
system, which was the reason that high-level, hygienic macro systems were
created in the first place. If any macro system should be in the core
Scheme, it should be one that permits many transformations without
breaking Hygiene. The ability to break hygiene should be considered an
unnecessary feature for the minimalist Schemer. Reliability and
predictability in terms of evaluation set Scheme apart from other Lisps.
This reliability should be maintained at the syntactic abstraction level.
I also hold that the SYNTAX-CASE macro system is a better macro system on
which to standardize for macro systems that can break hygiene. It does not
require the use of DEF-MACRO, and instead works with DEFINE-SYNTAX, it has
only a few basic extensions necessary for its use, provides a high-level
interface for destructing syntax in the same form as SYNTAX-RULES, and
most importantly in my eyes: it preserves hygiene by default. This last
behavior is really important. Other macro systems that do not do this
force the programmer to the defensive, rather than letting the programmer
choose selectively, at his leisure, when and where to break hygiene. It
should be trivial in a macro system that allows unhygienic macros to
create hygienic macros with the same ease as SYNTAX-RULES, it should not
be easier to create unhygienic ones than hygienic ones.
Aaron W. Hsu
--
Of all tyrannies, a tyranny sincerely exercised for the good of its
victims may be the most oppressive. -- C. S. Lewis
Nick Keighley
I understood the more sophisticated macro system (syntax-case?) could
be used to implement the less sophisticated systems. Couldn't syntax-
case
and def-macro both be in the core language. As a beginner the syntax-
*
forms just baffle me.
namekuseijin
system is to be there, it should be easy enough for someone wanting to
write a Core Scheme interpreter for educational purposes. I think
def-macro fits the bill better than syntax-case.
--
a game sig: http://tinyurl.com/d3rxz9
Aaron W. Hsu
<nick_keigh...@hotmail.com> wrote:
> I understood the more sophisticated macro system (syntax-case?) could
> be used to implement the less sophisticated systems. Couldn't syntax-
> case
> and def-macro both be in the core language. As a beginner the syntax-
> *
> forms just baffle me.
The problem with the defmacro form is that it is inherently unsafe, as I
understand it. At least the forms I have seen are based on passing in
s-expressions and return s-expressions. You lose syntactic information
that is necessary to preserving hygiene. You can achieve the same thing
using higher-lever macro systems based on DEFINE-SYNTAX that still make it
possible to break hygiene, so there is no reason to have an arguably
broken DEFMACRO construct in the core language. I don't believe DEFMACRO
is necessary for any of the major unhygienic macro systems to be used,
such as SC, ER, or SYNTAX-CASE.
If you are confused by the SYNTAX-RULES forms, that is understandable, but
as a beginner, I'm not sure it is wise to be working in the unhygienic
world until you understand macros. On the other hand, it isn't hard to get
DEFMACRO like utility using SYNTAX-CASE (though it is absolutely *wrong*,
IMO, to do so).
(define-syntax my-macro
(lambda (x)
(let ([sexpr (syntax->datum x)])
(datum->syntax x
<do your sexpr work here and return a datum>))))
I have seen these transformations done sometimes when converting old
macros to new systems, but this is bad bad bad! You lose all syntactic
information, and that is a very bad thing.
Aaron W. Hsu
wrote:
> Isn't Core Scheme supposed to be geared towards education? If any macro
> system is to be there, it should be easy enough for someone wanting to
> write a Core Scheme interpreter for educational purposes. I think
> def-macro fits the bill better than syntax-case.
I would never use DEFMACRO in education. Encouraging its use when it is so
vastly inferior to the other systems out there makes no sense from an
educational standpoint. You don't teach something that is bad because it
is easy. Instead, you find ways of making the right approach more
approachable. SYNTAX-RULES is not very hard to grasp, especially if
taught, as opposed to just thrown at the student.
namekuseijin
> On Wed, 26 Aug 2009 13:20:56 -0400, namekuseijin
> <nameku...@gmail.com> wrote:
>
>> Isn't Core Scheme supposed to be geared towards education? If any
>> macro system is to be there, it should be easy enough for someone
>> wanting to write a Core Scheme interpreter for educational purposes.
>> I think def-macro fits the bill better than syntax-case.
>
> I would never use DEFMACRO in education. Encouraging its use when it is
> so vastly inferior to the other systems out there makes no sense from an
> educational standpoint. You don't teach something that is bad because it
> is easy. Instead, you find ways of making the right approach more
> approachable. SYNTAX-RULES is not very hard to grasp, especially if
> taught, as opposed to just thrown at the student.
I agree. So, no macro systems whatsoever in Core Scheme.
namekuseijin
Notice I wasn't talking about ease of using syntax-case, but of
implementing it for educational purposes.
Russell McManus
"Aaron W. Hsu" <arc...@sacrideo.us> writes:
> Reliability and predictability in terms of evaluation set Scheme apart
> from other Lisps.
Hardly. Unspecified order of evaluation of function arguments leaps to
mind. Common Lisp specfies left to right. Which one is more
predictable?
-russ
Peter Keller
Well, if in the common lisp case, I know that a side effect
while computing argument 1 will affect an object very far away
lexically/semantically before argument 2 needs it in some arcane control
path, what I will have discovered is that the code was crappy. You shouldn't
write unmaintainable code like that. :)
I'd *much* rather have unpredictable argument evaluation and know that to
be the case. It makes it easier to lift arguments out of a function call
and into a let-form for better readability.
Now whether a compiler can do enough abstract interpretation to tell you that
this dataflow assumption exists is a different story.
-pete
Grant Rettke
> Some people have tried to suggest that DEFMACRO or other "low-level" macro
> systems should exist in the base or core Scheme standard and that
> SYNTAX-RULES should be relegated to a separate library.
With what rationale?
I want lexical scoping in those macros. Scheme's hygienic-macros will
go down in history as a major contribution (at least if R7RS takes off
and grows 100K+ users ;)).
Pascal Costanza
> Russell McManus <russell...@yahoo.com> wrote:
>> "Aaron W. Hsu" <arc...@sacrideo.us> writes:
>>
>>> Reliability and predictability in terms of evaluation set Scheme apart
>>> from other Lisps.
>> Hardly. Unspecified order of evaluation of function arguments leaps to
>> mind. Common Lisp specfies left to right. Which one is more
>> predictable?
>
> Well, if in the common lisp case, I know that a side effect
> while computing argument 1 will affect an object very far away
> lexically/semantically before argument 2 needs it in some arcane control
> path, what I will have discovered is that the code was crappy. You shouldn't
> write unmaintainable code like that. :)
(let ((x 0))
(foo (incf x) (incf x)))
a) The variable affected is not 'very far away.'
b) The control path is not 'arcane.'
> I'd *much* rather have unpredictable argument evaluation and know that to
> be the case. It makes it easier to lift arguments out of a function call
> and into a let-form for better readability.
(let* ((x 0)
(y (incf x))
(z (incf x)))
(foo y z))
c) Lifting out the arguments wasn't any harder. (Maybe you mean
something else here?)
Pascal
--
My website: http://p-cos.net
Common Lisp Document Repository: http://cdr.eurolisp.org
Closer to MOP & ContextL: http://common-lisp.net/project/closer/
Pascal Costanza
> On Aug 25, 11:08 pm, "Aaron W. Hsu" <arcf...@sacrideo.us> wrote:
>> Some people have tried to suggest that DEFMACRO or other "low-level" macro
>> systems should exist in the base or core Scheme standard and that
>> SYNTAX-RULES should be relegated to a separate library.
>
> With what rationale?
You can do more with defmacro than with syntax-rules, and I personally,
subjectively, find defmacro much easier to understand than syntax-case.
> I want lexical scoping in those macros. Scheme's hygienic-macros will
> go down in history as a major contribution (at least if R7RS takes off
> and grows 100K+ users ;)).
True.
Aaron W. Hsu
wrote:
> So, no macro systems whatsoever in Core Scheme.
I guess now would be a good time to remark that I don't think saying Core
Scheme is for Education is a good metric. This might also be a good time
to say that I think macros are so essential to what make Scheme neat and
cool, that SYNTAX-RULES must be in the small standard, if only for
compatibility with the previous standards, but other than that, I'd also
suggest that Education spans a much broader scope than entry-level Comp.
Sci. where Macros might be out of the question.
Sam TH
> Grant Rettke wrote:
> > On Aug 25, 11:08 pm, "Aaron W. Hsu" <arcf...@sacrideo.us> wrote:
> >> Some people have tried to suggest that DEFMACRO or other "low-level" macro
> >> systems should exist in the base or core Scheme standard and that
> >> SYNTAX-RULES should be relegated to a separate library.
>
> > With what rationale?
>
> You can do more with defmacro than with syntax-rules,
This is false. The macros expressible with `defmacro' are
incomparable with the macros expressible with `syntax-rules'.
sam th
Aaron W. Hsu
Maybe I should clarify. I consider both of these models to be predictable
in the sense that it is possible to right proper code within their
semantic constraints that will behave correctly all the time or at least,
will not change in behavior. That is, I have nothing against
underspecification, or removing restrictions that shouldn't be there. On
the other hand, DEFMACRO is another beast entirely, because you simply
have *no* way of guaranteeing the behavior of your code. With unspecified
order of evaluations, we can program without relying on that order, and
we'll do fine. With DEFMACRO, it is not practically feasible to do so
without assuming that certain bindings are what we assume they are.
There's no way of ensuring that the expanded code really will do what we
expect it to do.
Aaron W. Hsu
> On Aug 25, 11:08 pm, "Aaron W. Hsu" <arcf...@sacrideo.us> wrote:
>> Some people have tried to suggest that DEFMACRO or other "low-level"
>> macro
>> systems should exist in the base or core Scheme standard and that
>> SYNTAX-RULES should be relegated to a separate library.
> With what rationale?
That is allows macro creation in terms of familiar basic primitives like
CAR, CDR, and the like. Hardly convincing, IMO.
Aaron W. Hsu
> You can do more with defmacro than with syntax-rules, and I personally,
> subjectively, find defmacro much easier to understand than syntax-case.
There are other unhygienic macro systems that don't rely on DEFMACRO that
allow you to do just as much. They also allow you to actually right code
that will work, as opposed to code that has inherent bugs. DEFMACRO is
easier to understand, but that doesn't make it better. Code with bugs in
it can often seem easier to understand too, but you want to take the time
to make sure the code is correct.
Pascal Costanza
Right. Let me rephrase: You can do more of the things that I am
interested in with defmacro than with syntax-rules. ;)
Pascal Costanza
> On Wed, 26 Aug 2009 17:10:15 -0400, Pascal Costanza <p...@p-cos.net> wrote:
>
>> You can do more with defmacro than with syntax-rules, and I
>> personally, subjectively, find defmacro much easier to understand than
>> syntax-case.
>
> There are other unhygienic macro systems that don't rely on DEFMACRO
> that allow you to do just as much. They also allow you to actually right
> code that will work, as opposed to code that has inherent bugs. DEFMACRO
> is easier to understand, but that doesn't make it better. Code with bugs
> in it can often seem easier to understand too, but you want to take the
> time to make sure the code is correct.
You seem to imply that defmacro automatically implies more bugs, but
that's totally not true in my and a lot of other people's experience (in
a Common Lisp setting!).
Your argument is similar to what proponents of static type systems
claim, that dynamically typed languages inherently lead to more bugs.
But that's also incorrect, for pretty much the same reasons, and
essentially boils down to lack of experience.
Yes, Scheme is probably better off with a more elaborate macro system
that guarantees hygiene by default.
namekuseijin
> Yes, Scheme is probably better off with a more elaborate macro system
> that guarantees hygiene by default.
always important in influenza times... :)
Ray
> Isn't Core Scheme supposed to be geared towards education? If any macro
> system is to be there, it should be easy enough for someone wanting to
> write a Core Scheme interpreter for educational purposes.
If we want to include enough tools to write a core scheme interpreter
in core scheme, you need a way to make dynamic environments. Scheme's
user code is lexically scoped by default, but the interpreter must
maintain a dynamic environment for things like the current-output-port,
current-input-port, etc.
There's a terrible, ugly hack for it involving winding continuations,
but get serious; dynamic environments shouldn't involve doing hard
work to save and restore dynamic variables every time control gets
transferred.
Bear
Aaron W. Hsu
> Let me rephrase: You can do more of the things that I am interested in
> with defmacro than with syntax-rules.
Why don't you use one of the many other macros that allow you to break
hygiene, but still preserve hygiene where it matters?
Aaron W. Hsu
> Your argument is similar to what proponents of static type systems
> claim, that dynamically typed languages inherently lead to more bugs.
> But that's also incorrect, for pretty much the same reasons, and
> essentially boils down to lack of experience.
Correct me if I am wrong, but my understanding of the way DEFMACRO works
(and let's remember that we are dealing with a single namespace here, I'm
willing to admit that the multiple namespaces alleviates some of these
problems, even if it doesn't entirely eliminate them) comes partly from
the following example [1]:
Say we have an OR2 macro that expands like:
(OR2 e1 e2) => (let ([t e1]) (if t t e2))
We then have the obvious problem of capturing, but in DEFMACRO, this can
be somewhat intercepted by using generated names and carefully working
with the macro (notice that this is harder and more work than having
hygiene handled automatically -- it places the burden of correctness
unnecessarily on the macro writer). However, there is also the issue of
"Referential Transparency" identified in the same paper referenced above
[1]. In this situation, we have thr problem of this sort:
(let ([if #f]) (or2 if "okay"))
which ought to evaluate to "okay," but that incorrectly results in an
error if the macro system does not enforce referential transparency. That
is, the free variables of the macro expansion ought to be scoped at where
the definition of the macro occurs, and not in the end result. To my
knowledge, DEFMACRO does not preserve such referential transparency. I
would be pleasantly surprised if it did, and if it does, I am woefully
misinformed, and should be presently corrected. Other macro systems give
you the same power as DEFMACRO, but do the right thing by default: do not
capture by default, and do not break referential transparency by default.
Aaron W. Hsu
[1] R. Kent Dybvig, Robert Hieb, and Carl Bruggeman. Syntactic abstraction
in Scheme. Lisp and Symbolic Computation, 5(4):295-326, 1993.
<http://www.cs.indiana.edu/~dyb/pubs/LaSC-5-4-pp295-326.pdf>
Pascal Costanza
> On Thu, 27 Aug 2009 02:39:15 -0400, Pascal Costanza <p...@p-cos.net> wrote:
>
>> Let me rephrase: You can do more of the things that I am interested in
>> with defmacro than with syntax-rules.
>
> Why don't you use one of the many other macros that allow you to break
> hygiene, but still preserve hygiene where it matters?
...because this adds complexity without much reward.
It is important to keep in mind that I am arguing from a Common Lisp
point of view. That language has some properties and features that allow
you to effectively deal with hygiene issues in ad-hoc ways. Admittedly,
they don't really solve it, but they are good enough for practical
purposes, and don't pose any serious problems in my experience.
You can actually implement a hygiene-compatible macro system on top of
Common Lisp's unhygienic macros, as I described here:
http://p-cos.net/documents/hygiene.pdf - for 'everyday macro hacking',
you essentially emulate this manually (and it's by far not as hard as it
may seem).
To put it in other words: You can either ensure desirable properties of
your programs by pushing them into language constructs that
automatically enforce them, or you can use idioms/patterns/styles to
achieve them manually. In a Common Lisp setting, the idiomatic approach
for ensuring macro hygiene works best for me.
Pascal Costanza
Common Lisp's DEFMACRO doesn't solve this by itself, but the combination
of Common Lisp's rules for disallowing redefinitions of predefined
symbols, Common Lisp's package system, and the fact that Common Lisp has
multiple namespaces pretty much makes this a non-issue.
Scheme has a single namespace, and typical Scheme implementations have a
different idea about how to do modules, so there hygienic macro systems
are a saner default.
Pascal Costanza
> Aaron W. Hsu wrote:
>> On Thu, 27 Aug 2009 02:39:15 -0400, Pascal Costanza <p...@p-cos.net> wrote:
>>
>>> Let me rephrase: You can do more of the things that I am interested
>>> in with defmacro than with syntax-rules.
>>
>> Why don't you use one of the many other macros that allow you to break
>> hygiene, but still preserve hygiene where it matters?
>
> ...because this adds complexity without much reward.
...to illustrate this further, I find the macros in
http://www.cs.indiana.edu/~dfried/ooo.pdf horribly complex, and Dan
Friedman himself told me that it took a couple of experts to actually
get them right.
I reimplemented them using defmacro in Common Lisp, which was still a
relatively complex undertaking, but I found it comparatively
straightforward. (I hope I have time to publish that code with some
explanations at some stage in one way or the other...)
Nicolas Neuss
> Common Lisp's DEFMACRO doesn't solve this by itself, but the combination of
> Common Lisp's rules for disallowing redefinitions of predefined symbols,
> Common Lisp's package system, and the fact that Common Lisp has multiple
> namespaces pretty much makes this a non-issue.
>
> Scheme has a single namespace, and typical Scheme implementations have a
> different idea about how to do modules, so there hygienic macro systems are
> a saner default.
Nevertheless, I would be interested in havin fully working SYNTAX-RULES
style macros also in Common Lisp. At least, I appreciated those very much
when teaching Lisp to students (it is one of the main reasons that I have
chosen Scheme for that course).
Nicolas
Pascal J. Bourguignon
I don't think there'd be any problem in implementing SYNTAX-RULES in
Common Lisp.
--
__Pascal Bourguignon__
Pascal Costanza
Using Scheme as a teaching language is a very good idea anyway. It
prepares very well for going into different kinds of directions afterwards.
If you want only the pattern matching of syntax-rules in Common Lisp,
that should be fairly easy. Integrating hygiene by default is more
complicated.
You could use the implementation of the 'alias' system (which is
essentially a portable reimplementation of renaming) described in
http://p-cos.net/documents/hygiene.pdf as a starting point to implement
syntax-rules on top. It's unlikely that I am going to do that, but the
code is available with an open source license, so somebody else could
certainly do this.
Aaron W. Hsu
> It is important to keep in mind that I am arguing from a Common Lisp
> point of view. That language has some properties and features that allow
> you to effectively deal with hygiene issues in ad-hoc ways. Admittedly,
> they don't really solve it, but they are good enough for practical
> purposes, and don't pose any serious problems in my experience.
Of course, I have agreed that the Common Lisp use of DEFMACRO is more
justified by the lesser hit rate on these sort of problems in practice.
Since, however, we are discussing Scheme macros and the use of DEFMACRO in
a Scheme context, I think a different environment leads to a stronger
argument against DEFMACRO in practice.
> To put it in other words: You can either ensure desirable properties of
> your programs by pushing them into language constructs that
> automatically enforce them, or you can use idioms/patterns/styles to
> achieve them manually. In a Common Lisp setting, the idiomatic approach
> for ensuring macro hygiene works best for me.
Within Common Lisp, that's understandable, but not within Scheme.
Aaron W. Hsu
> Scheme has a single namespace, and typical Scheme implementations have a
> different idea about how to do modules, so there hygienic macro systems
> are a saner default.
I think we are agreed here. And I believe this is a sane default for the
standard, as opposed to introducing DEFMACRO.
Aaron W. Hsu
Ray
>> Pascal Costanza <p...@p-cos.net> writes:
>>> Scheme has a single namespace, and typical Scheme implementations have a
>>> different idea about how to do modules, so there hygienic macro systems
>>> are a saner default.
> Using Scheme as a teaching language is a very good idea anyway. It
> prepares very well for going into different kinds of directions
> afterwards.
I think the right way to teach the topic is to first introduce
defmacro, then show what happens when there is an inadvertent
variable capture, then explain the idea of hygiene and introduce
syntax-rules.
I remember syntax-rules seeming pretty random and overcomplicated
back when I first learnt it; I hadn't the benefit of understanding
why (and when) hygiene was important first.
Bear
Brian Adkins
> Pascal Costanza wrote:
> [...]
> ...to illustrate this further, I find the macros in
> http://www.cs.indiana.edu/~dfried/ooo.pdf horribly complex, and Dan
> Friedman himself told me that it took a couple of experts to actually
> get them right.
>
> I reimplemented them using defmacro in Common Lisp, which was still a
> relatively complex undertaking, but I found it comparatively
> straightforward. (I hope I have time to publish that code with some
> explanations at some stage in one way or the other...)
That would be great.
--
Brian Adkins
http://lojic.com/