History of &environment
Pascal Costanza
This is a historical question: If I understand correctly, CLtL1 was the
first specification of a Lisp dialect that mentions an &environment
parameter for defmacro. However, I cannot find any material that
describes the historical background for introducing such a parameter. I
can only guess that this is related to the switch of a dynamically
scoped Lisp to a lexically scoped one.
Does anyone know who suggested to add this, why it was suggested, and/or
whether there was prior practice that justified its introduction? Was a
similar feature already available in previous Lisp dialects? Under what
name / concept?
Are there even papers or articles that describe the relationships?
Thanks a lot in advance,
Pascal
--
My website: http://p-cos.net
Common Lisp Document Repository: http://cdr.eurolisp.org
Closer to MOP & ContextL: http://common-lisp.net/project/closer/
Rainer Joswig
Pascal Costanza <p...@p-cos.net> wrote:
> Hi,
>
> This is a historical question: If I understand correctly, CLtL1 was the
> first specification of a Lisp dialect that mentions an &environment
> parameter for defmacro. However, I cannot find any material that
> describes the historical background for introducing such a parameter. I
> can only guess that this is related to the switch of a dynamically
> scoped Lisp to a lexically scoped one.
I would guess that it has been added during early work on Common Lisp.
Possibly because with Common Lisp one tried to say something about
the role of the compiler and macro definitions that are kept
only during compilation (or similar situations).
Lisp Machine Lisp in 1981 did not mention them, the manual
of 1984 with Common Lisp influence does.
Just a guess...
>
> Does anyone know who suggested to add this, why it was suggested, and/or
> whether there was prior practice that justified its introduction? Was a
> similar feature already available in previous Lisp dialects? Under what
> name / concept?
>
> Are there even papers or articles that describe the relationships?
>
> Thanks a lot in advance,
> Pascal
Pascal Costanza
> In article <5b5ffcF...@mid.individual.net>,
> Pascal Costanza <p...@p-cos.net> wrote:
>
>> Hi,
>>
>> This is a historical question: If I understand correctly, CLtL1 was the
>> first specification of a Lisp dialect that mentions an &environment
>> parameter for defmacro. However, I cannot find any material that
>> describes the historical background for introducing such a parameter. I
>> can only guess that this is related to the switch of a dynamically
>> scoped Lisp to a lexically scoped one.
>
> I would guess that it has been added during early work on Common Lisp.
> Possibly because with Common Lisp one tried to say something about
> the role of the compiler and macro definitions that are kept
> only during compilation (or similar situations).
>
> Lisp Machine Lisp in 1981 did not mention them, the manual
> of 1984 with Common Lisp influence does.
>
> Just a guess...
OK, thanks. I have digged a little deeper starting from that
observation, and I have found the following:
- The Lisp Machine Manual of 1984 has this to say about &environment:
"&environment causes the variable that follows it to be bound to the
/local macros environment/ of the macro call being expanded. This is
useful if the code for expanding this macro needs to invoke macroexpand
on subforms of the macro call. Then, to achieve correct interaction with
macrolet, this local macros environment should be passed to macroexpand
as its second argument."
That's roughly the same as what CLtL1. (I haven't been aware of the fact
that the &environment parameter in CLtL1 actually refers to macros only.)
- Another interesting tidbit is the following statement by David Moon in
the Common Lisp email history about apparently a draft version of CLtL1:
"On page 93 it says that MACROEXPAND ignores local macros established by
MACROLET. This is clearly incorrect; MACROEXPAND has to get called with
an appropriate lexical context available to it in the same way that EVAL
does. They are both parts of the interpreter. I don't have anything to
propose about this now; I just want to point out that there is an
issue." (email of August 29, 1982)
This strongly suggests that the "local macros environment" had been
introduced because of MACROLET, to ensure correct interaction.
I think it's very likely that using macroexpand locally in a macro was
already standard practice (the chinuals seem to suggest this).
So the question actually boils down to when macrolet / local macro
definition had been introduced. Was there any Lisp dialect before Common
Lisp that had local macros?
Pascal Costanza
> This strongly suggests that the "local macros environment" had been
> introduced because of MACROLET, to ensure correct interaction.
>
> So the question actually boils down to when macrolet / local macro
> definition had been introduced. Was there any Lisp dialect before Common
> Lisp that had local macros?
OK, I think the question is resolved. From
http://www.dreamsongs.com/Separation.html :
"In dialects preceding Common Lisp, such as MacLisp, [...] FLET, LABELS,
MACROLET did not exist."
I am surprised to hear that even flet and labels didn't exist before
Common Lisp, but apparently, that's how it is.
Rainer Joswig
Pascal Costanza <p...@p-cos.net> wrote:
> Pascal Costanza wrote:
>
> > This strongly suggests that the "local macros environment" had been
> > introduced because of MACROLET, to ensure correct interaction.
> >
> > So the question actually boils down to when macrolet / local macro
> > definition had been introduced. Was there any Lisp dialect before Common
> > Lisp that had local macros?
>
> OK, I think the question is resolved. From
> http://www.dreamsongs.com/Separation.html :
>
> "In dialects preceding Common Lisp, such as MacLisp, [...] FLET, LABELS,
> MACROLET did not exist."
>
> I am surprised to hear that even flet and labels didn't exist before
> Common Lisp, but apparently, that's how it is.
Lisp 1.5 had 'LABEL'.
>
>
> Pascal
Pascal Costanza
Yes, but that's substantially different. LABEL introduces a name for a
lambda expression with which it can refer to itself. It doesn't create a
scope for some embedded code. So (label foo (lambda (...) ...)) is
equivalent to (labels ((foo (...) ...)) #'foo).
BTW, I have double-checked, and labels actually comes from Scheme. In
R1RS and R2RS it is introduced (and the text refers to it being taken
from the PLASMA language, probably a Lisp dialect as well). In R3RS,
labels has been renamed to letrec.
Kent M Pitman
> - Another interesting tidbit is the following statement by David Moon
> in the Common Lisp email history about apparently a draft version of
> CLtL1:
>
> "On page 93 it says that MACROEXPAND ignores local macros established
> by MACROLET. This is clearly incorrect; MACROEXPAND has to get called
> with an appropriate lexical context available to it in the same way
> that EVAL does. They are both parts of the interpreter. I don't have
> anything to propose about this now; I just want to point out that
> there is an issue." (email of August 29, 1982)
>
> This strongly suggests that the "local macros environment" had been
> introduced because of MACROLET, to ensure correct interaction.
I'm pretty sure this is right.
What I recall is that, at around that time, closures had not previously
been something you could return upward in Lisp. Scheme had them, but
it was not something Lisp did. Certainly not MACLISP. And maybe Zetalisp
didn't have them then either, I'm not sure...
I _think_ what drove things was the fact that if you just did
(macrolet ((foo ...))
(macroexpand-1 '(foo ...)))
you could have dataflow from macrolet to macroexpand-1 dynamically.
But once you could do:
(macrolet ((foo ...)) ;#1
(funcall (macrolet ((foo ...)) ;#2
#'(lambda ()
(macrolet ((bar ()
`',(macroexpand-1 '(foo ..))))
...)))))
you had to worry that if you didn't do a macroexpansion prepass and instead
delayed the expansion of bar until after you started to execute the funcall,
you wouldn't have access to the foo any more if it had been stored only
dynamically.
So the problem may not have been MACROLET per se, but MACROEXPAND-1 and
MACROEXPAND with one arg were being stretched to their limit once you had
upward closures, since they had to do one of:
* pass the environment explicitly
* reliably have access to it in dynamic environment
* ignore the environment reliably
or else no one would know what was up. Moon's comment speaks to why the third
option was untenable. The example I gave above (if I wrote it right)
tells you why the second one was a problem. I think that left the first.
A lot of the business about returning things upward was very new and there
were myriad bugs and strangenesses that were discovered. It took a long
time to get the kinks out of returns and gos to tags that escaped their
creation context, for example. As well as side-effects to variables in
shared environments. It was all messy and lots of people used kludges in
their implementations that didn't stand up or hadn't been tested for a long
time in many of the obscure cases. Macros were just one of the things that
got shaken up in this.
You should also remember that at the time of CLTL, lexical scoping at all
was very new to the Lisp community and regarded as quite tentative to all
but a few. Many people just couldn't convince themselves that it was
really reliably going to work. They were very used to quasi-dynamic
scoping of Maclisp and it seemed so experimental. Of course, Scheme had
been around a couple years showing it was an ok thing, and more importantly
Algol had been around for many years showing it worked, which is where scheme
got many of its ideas. But even so, it's always hard to put one's head into
the notion of how the sense of unfamiliarity can make things move slowly
and painfully, but this was one of those things. What seems obvious now is
only obvious because we finally dove in and embraced it and got used to it.
I also have the vaguest memory that the problem was revisited later when we
went to do the "environment access" stuff (which we later gave up on and
removed, but which is snapshotted in approximate form in CLTL2). I think
the &environment objects were holding macro bindings fine, but not all
implementations passed ALL compiler information there. So things like
type declarations were not, in all implementations, riding that object.
And when we started saying you could get that data from there, some
implementations had to scurry to revamp how they passed that info.
Often it was a matter of creating a fiction in the language spec and then
allowing the implementations to catch up.
Or such is my memory. I don't mean any of this as gospel truth. But
more as things that might be usefully researched. I do have records
of some of this stuff in boxes of drafts and vote documents in my
basement. I wish I had a place to store that for better archival
purposes, but the process of getting a proper place for that is going
slowly. But either way, I don't have it at my fingertips.
> I think it's very likely that using macroexpand locally in a macro was
> already standard practice (the chinuals seem to suggest this).
It could be, but check macroexpand and macroexpand-1.
In MACLISP, macroexpand and macroexpand-1 didn't take an environment argument.
MACLISP had no MACROLET, though, so didn't need it.
> So the question actually boils down to when macrolet / local macro
> definition had been introduced. Was there any Lisp dialect before
> Common Lisp that had local macros?
I can't recall any Lisp dialect that had MACROLET before the Lispm.
That doesn't mean it didn't happen, though.
Pascal Costanza
> Pascal Costanza <p...@p-cos.net> writes:
>
>> - Another interesting tidbit is the following statement by David Moon
>> in the Common Lisp email history about apparently a draft version of
>> CLtL1:
>>
>> "On page 93 it says that MACROEXPAND ignores local macros established
>> by MACROLET. This is clearly incorrect; MACROEXPAND has to get called
>> with an appropriate lexical context available to it in the same way
>> that EVAL does. They are both parts of the interpreter. I don't have
>> anything to propose about this now; I just want to point out that
>> there is an issue." (email of August 29, 1982)
>>
>> This strongly suggests that the "local macros environment" had been
>> introduced because of MACROLET, to ensure correct interaction.
>
> I'm pretty sure this is right.
[... a lot of interesting stuff ...]
Thanks a lot for the valuable information!