Remembering information during compilation
Spiros Bousbouras
memory (&rest body) and doing (memory form) should expand to
(SYMBOL-MACROLET ((CELL XXX))
form)
where XXX should be replaced by the number of surrounding memory
invocations. So for example
(memory
(format t "CELL has value ~a~%" cell)
(memory (format t "CELL has value ~a~%" cell))
(format t "CELL has value ~a~%" cell))
should expand to
(SYMBOL-MACROLET ((CELL 1))
(FORMAT T "CELL has value ~a~%" CELL)
(SYMBOL-MACROLET ((CELL 2))
(FORMAT T "CELL has value ~a~%" CELL))
(FORMAT T "CELL has value ~a~%" CELL))
and produce
CELL has value 1
CELL has value 2
CELL has value 1
Note how in the second invocation of memory, CELL is assigned the form
2 because it is inside 2 nested invocations of memory. No matter how
deeply nested (within reason) the invocations of memory are CELL should
get the correct value. Below are my efforts but they have not met with
success.
(let ((a 0))
(defmacro memory (&rest body)
(prog2
(incf a)
`(symbol-macrolet ((cell ,a))
,@body)
(decf a))))
With the test I gave above it produces
CELL has value 1
CELL has value 1
CELL has value 1
So the second invocation of memory in the test does not increase by 1
the value of CELL.
The second attempt wouldn't be completely satisfactory even if it
worked because it exposes to the outside world an internal part of the
implementation , the variable B. But it doesn't work.
(defparameter b 0)
(defmacro memory2 (&rest body)
(prog2
(defparameter b (+ b 1))
`(symbol-macrolet ((cell ,b))
,@body)
(defparameter b (- b 1))))
Once again doing
(memory2
(format t "CELL has value ~a~%" cell)
(memory2 (format t "CELL has value ~a~%" cell))
(format t "CELL has value ~a~%" cell))
produces
CELL has value 1
CELL has value 1
CELL has value 1
I tried some variations of the above but no luck. So first why do my
attempts above not work ? I assume that by the time the code generated
from the macro expansion gets compiled the statement (decf a) has
already been executed. Am I right ?
It would be convenient if Lisp had some kind of minimal-compile
function so that you could do (minimal-compile form) and it would
return what results after minimal compilation of the form. With
such a thing one could define memory as
(let ((a 0))
(defmacro memory (&rest body)
(prog2
(incf a)
(minimal-compile
`(symbol-macrolet ((cell ,a))
,@body))
(decf a))))
Does anyone know an implementation with such a facility ? Any other
solutions to the problem ?
--
Shakespeare also predicted structured programming:
"Go to, go to, thou are a foolish fellow. Let me be clear of thee."
Twelfth Night, Act IV, Scene 1.
from http://www.iidb.org/vbb/showthread.php?p=3352214#post3352214
Pascal J. Bourguignon
> I'm trying to construct a macro called memory.
(defparameter *cell* 0)
(define-symbol-macro cell *cell*)
(defmacro memory (&rest body)
`(let ((*cell* (1+ *cell*)))
,@body)))
C/USER[100]> (memory
(format t "CELL has value ~a~%" cell)
(memory (format t "CELL has value ~a~%" cell))
(format t "CELL has value ~a~%" cell))
CELL has value 1
CELL has value 2
CELL has value 1
NIL
--
__Pascal Bourguignon__
Kaz Kylheku
> It would be convenient if Lisp had some kind of minimal-compile
Indeed, it would help a certain troll in another thread to have a fraction of a
valid point.
Kaz Kylheku
> memory (&rest body) and doing (memory form) should expand to
>
> (SYMBOL-MACROLET ((CELL XXX))
> form)
>
> where XXX should be replaced by the number of surrounding memory
> invocations. So for example
>
> (memory
> (format t "CELL has value ~a~%" cell)
> (memory (format t "CELL has value ~a~%" cell))
> (format t "CELL has value ~a~%" cell))
>
> should expand to
>
> (SYMBOL-MACROLET ((CELL 1))
> (FORMAT T "CELL has value ~a~%" CELL)
> (SYMBOL-MACROLET ((CELL 2))
> (FORMAT T "CELL has value ~a~%" CELL))
> (FORMAT T "CELL has value ~a~%" CELL))
>
> and produce
> CELL has value 1
> CELL has value 2
> CELL has value 1
Why does cell have to be a symbol macro? How about simply generating:
(let ((cell (1+ cell)) ...)
Take advantage of the LET scoping rule which allows you to shadow an outer
variable, but use its avlue to initialize the shadowing one.
You just need to bootstrap this somehow for the outermost invocation. Here is
where a symbol macro comes in: a global one made with define-symbol-macro,
to avoid introducing cell as a dynamic variable.
Pascal Costanza
> I'm trying to construct a macro called memory. The syntax is
> memory (&rest body) and doing (memory form) should expand to
>
> (SYMBOL-MACROLET ((CELL XXX))
> form)
>
> where XXX should be replaced by the number of surrounding memory
> invocations. So for example
>
> (memory
> (format t "CELL has value ~a~%" cell)
> (memory (format t "CELL has value ~a~%" cell))
> (format t "CELL has value ~a~%" cell))
>
> should expand to
>
> (SYMBOL-MACROLET ((CELL 1))
> (FORMAT T "CELL has value ~a~%" CELL)
> (SYMBOL-MACROLET ((CELL 2))
> (FORMAT T "CELL has value ~a~%" CELL))
> (FORMAT T "CELL has value ~a~%" CELL))
>
> and produce
> CELL has value 1
> CELL has value 2
> CELL has value 1
What do you want the following to evaluate to?
(setq f (memory
(lambda ()
(memory (print cell)))))
(funcall f)
If you want this to evaluate to 1, the other Pascal's solution is fine.
IF you want this to evaluate to 2, here is another solution:
(define-symbol-macro cell 0)
(defmacro memory (&body body &environment env)
(let ((new-cell-value (1+ (macroexpand 'cell env))))
`(symbol-macrolet ((cell ,new-cell-value))
,@body)))
One important thing to keep in mind: Don't use side effects in macro
definitions. In both your solutions, you tried to achieve the result by
assigning to a local or global variable as part of the macroexpansion
process. However, you cannot predict whether or when, and how often, a
particular macro will be invoked. For example, development environments
may offer users to expand macros on demand, compilers may expand macros
several times for analysis, higher-order macros may expand inner macro
for analysis as well, etc., etc. So make sure that your macros are
always side-effect-free (unless you really know what you're doing).
Pascal
--
My website: http://p-cos.net
Common Lisp Document Repository: http://cdr.eurolisp.org
Closer to MOP & ContextL: http://common-lisp.net/project/closer/
Vassil Nikolov
On Sat, 07 Nov 2009 18:18:19 GMT, Spiros Bousbouras <spi...@gmail.com> said:
> I'm trying to construct a macro called memory. The syntax is
> memory (&rest body) and doing (memory form) should expand to
> (SYMBOL-MACROLET ((CELL XXX))
> form)
> where XXX should be replaced by the number of surrounding memory
> invocations.
Surrounding how? Lexically or dynamically?
---Vassil.
--
"Even when the muse is posting on Usenet, Alexander Sergeevich?"
Spiros Bousbouras
On 7 Nov, 18:41, p...@informatimago.com (Pascal J. Bourguignon) wrote:
> Spiros Bousbouras <spi...@gmail.com> writes:
> > I'm trying to construct a macro called memory.
>
> (defparameter *cell* 0)
> (define-symbol-macro cell *cell*)
> (defmacro memory (&rest body)
> `(let ((*cell* (1+ *cell*)))
> ,@body)))
This is a good approximation to what I asked for. The following is
even better:
(defparameter b 0)
(defmacro memory (&rest body)
`(let ((b (1+ b)))
(symbol-macrolet ((cell b))
,@body)))
It makes cell only available within body not everywhere. A difference
from my vision is that in the code above cell expands to a variable
instead of a numerical constant but that's fairly unimportant. I'm a
bit more annoyed by the fact that your solution also exposes an
implementation detail to the world. I would prefer a 100% clean
solution like what we would have if memory were a function when we
would use closures.
--
Who's your mama ?
Spiros Bousbouras
On 7 Nov, 18:53, Kaz Kylheku <kkylh...@gmail.com> wrote:
>
> Why does cell have to be a symbol macro? How about simply generating:
>
> (let ((cell (1+ cell)) ...)
>
> Take advantage of the LET scoping rule which allows you to shadow an outer
> variable, but use its avlue to initialize the shadowing one.
It doesn't have to be a symbol macro. The point of the exercise is to
investigate methods by which different invocations of the same macro
can share information. Many of the technical details are unimportant.
Having said that , it would be nice to have a way for this sharing of
information to happen through objects only the macro knows about.
> You just need to bootstrap this somehow for the outermost invocation. Here is
> where a symbol macro comes in: a global one made with define-symbol-macro,
> to avoid introducing cell as a dynamic variable.
It would be good if this bootstrapping could be done through the
macro
definition itself.
--
To have absolutely if l' this large Mister is loved!
Translation on 2-11-2009 by
http://uk.babelfish.yahoo.com/translate_txt of
"A posséder absolument si l'on aime ce grand monsieur!
Spiros Bousbouras
> On Sat, 07 Nov 2009 18:18:19 GMT, Spiros Bousbouras <spi...@gmail.com> said:
>
> > I'm trying to construct a macro called memory. The syntax is
> > memory (&rest body) and doing (memory form) should expand to
> > (SYMBOL-MACROLET ((CELL XXX))
> > form)
> > where XXX should be replaced by the number of surrounding memory
> > invocations.
>
> Surrounding how? Lexically or dynamically?
Lexically. See the examples above. I'm not even sure what a
dynamic surrounding would be.
Madhu
Wrote on Sat, 7 Nov 2009 18:44:28 +0000 (UTC):
Assuming you are baiting me here by calling me a troll, I think you are
rather sore for pointing out you had no clue what minimal compilation
was and its implications to symbol macro-expansion
--
Madhu
Barry Margolin
<52b2c0f6-8c87-498c...@o10g2000yqa.googlegroups.com>,
Spiros Bousbouras <spi...@gmail.com> wrote:
(defun fun1 ()
(memory
(print cell)
(fun2)
(print cell)))
(defun fun2
(memory
(print cell)))
When PRINT in FUN2 is called, there's one lexical surrounding
invocation, two dynamic ones.
--
Barry Margolin, bar...@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***
*** PLEASE don't copy me on replies, I'll read them in the group ***
Vassil Nikolov
On Sat, 7 Nov 2009 14:23:36 -0800 (PST), Spiros Bousbouras <spi...@gmail.com> said:
> On 7 Nov, 19:55, Vassil Nikolov <vniko...@pobox.com> wrote:
>> On Sat, 07 Nov 2009 18:18:19 GMT, Spiros Bousbouras <spi...@gmail.com> said:
>>
>> > I'm trying to construct a macro called memory. The syntax is
>> > memory (&rest body) and doing (memory form) should expand to
>> > (SYMBOL-MACROLET ((CELL XXX))
>> > form)
>> > where XXX should be replaced by the number of surrounding memory
>> > invocations.
>>
>> Surrounding how? Lexically or dynamically?
> Lexically.
Then a solution cannot be based on a special variable (at least not
a na�ve solution; and even if a complicated solution could, I don't
think it would be worth the candles).
Spiros Bousbouras
>
> What do you want the following to evaluate to?
No preference , the question was for educational purposes.
>
> (setq f (memory
> (lambda ()
> (memory (print cell)))))
>
> (funcall f)
>
> If you want this to evaluate to 1, the other Pascal's solution is fine.
>
> IF you want this to evaluate to 2, here is another solution:
>
> (define-symbol-macro cell 0)
>
> (defmacro memory (&body body &environment env)
> (let ((new-cell-value (1+ (macroexpand 'cell env))))
> `(symbol-macrolet ((cell ,new-cell-value))
> ,@body)))
This is interesting. I did some experiments trying to understand why
it behaves the way it does. Let's also define
(defmacro memory2 (&body body)
(let ((new-cell-value (1+ (macroexpand 'cell))))
`(symbol-macrolet ((cell ,new-cell-value))
,@body)))
(let ((f1
(memory
(lambda ()
(memory (format t "Calling f1 gives ~a~%" cell)))))
(f2
(memory2
(lambda ()
(memory2 (format t "Calling f2 gives ~a~%" cell))))))
(funcall f1)
(funcall f2)
'| |)
I get
Calling f1 gives 2
Calling f2 gives 1
Let me see if I understand the reason for the difference between
memory and memory2. The first call to the two macros expands to the
same thing and places what follows in the scope of (after expansions)
(symbol-macrolet ((cell 1))
The difference lies in the behaviour of the line
(let ((new-cell-value (1+ (macroexpand 'cell env))))
vs
(let ((new-cell-value (1+ (macroexpand 'cell))))
in the second call to the macros. There
(macroexpand 'cell env) evaluates to 1 whereas
(macroexpand 'cell) evaluates to 0.
The reason is the following excerpt in the macroexpand page in
the HS:
In addition to macro definitions in the global environment,
any local macro definitions established within env by
macrolet or symbol-macrolet are considered. If only form is
supplied as an argument, then the environment is effectively
null, and only global macro definitions as established by
defmacro are considered.
I assume the last line should actually read "defmacro or
define-symbol-macro are considered".
Is my understanding correct ?
--
<Khassaki> HI EVERYBODY!!!!!!!!!!
<Judge-Mental> try pressing the the Caps Lock key
<Khassaki> O THANKS!!! ITS SO MUCH EASIER TO WRITE NOW!!!!!!!
<Judge-Mental> fuck me
http://www.bash.org/?835030
Spiros Bousbouras
> definitions. In both your solutions, you tried to achieve the result by
> assigning to a local or global variable as part of the macroexpansion
> process. However, you cannot predict whether or when, and how often, a
> particular macro will be invoked. For example, development environments
> may offer users to expand macros on demand, compilers may expand macros
> several times for analysis, higher-order macros may expand inner macro
> for analysis as well, etc., etc. So make sure that your macros are
> always side-effect-free (unless you really know what you're doing).
So you're saying that setq (or setf) should not appear in the code of
the macroexpansion function ? Why , what could go wrong ?
Pascal J. Bourguignon
(defvar *c* 0)
(defmacro m ()
(incf *c*)
`',*c*)
(defun f ()
(list (m) (m)))
(f) --> (1 2) ; or
--> (42 87) ; or anything else.
(f) --> (1 2) ; or
--> (3 4) ; or
--> (36 99) ; or anything else.
For example, here is what you get with clisp:
C/USER[5]> (f) ; interpreted
(1 2)
C/USER[6]> (f) ; minimal compilation is cached.
(1 2)
C/USER[7]> (compile 'f) ; execute minimal compilation once again
F ;
NIL ;
NIL
C/USER[8]> (f) ; hence the new result
(3 4)
C/USER[9]> (f)
(3 4)
--
__Pascal Bourguignon__
http://www.informatimago.com
Pascal Costanza
Yes, this is correct. The Common Lisp environment objects effectively
give you a first-class representation of (compile-time) lexical
environments.
Pascal Costanza
As I said, a macro may be expanded more than once. For example, in the
LispWorks IDE (and others), I can right-click on an s-expression and
macroexpand it, to see what the result of macroexpansion will be. This
will trigger any side effects the macro definition may produce.
Here is another example: Assume you have a 'with-locked macro that
supposedly takes a lock on an object and then executes some code. Since
taking locks is expensive, it may want to avoid doing so. So it does
something like this:
(defmacro with-lock (object &body body &environment env)
(if (contains object (walk body env))
`(call-with-locked-object ,object (lambda () ,@body))
`(progn ,@body)))
(defun contains (object expanded-body)
... some form of membership test ...)
(defun walk (code env)
... something that repeatedly calls
(macroexpand code env)
to expand all macros in code
...)
> Who's your mama ?
I don't know why that would be relevant in this thread.
Spiros Bousbouras
wrote:
> Spiros Bousbouras <spi...@gmail.com> writes:
> > On 7 Nov, 19:04, Pascal Costanza <p...@p-cos.net> wrote:
>
> >> One important thing to keep in mind: Don't use side effects in macro
> >> definitions. In both your solutions, you tried to achieve the result by
> >> assigning to a local or global variable as part of the macroexpansion
> >> process. However, you cannot predict whether or when, and how often, a
> >> particular macro will be invoked. For example, development environments
> >> may offer users to expand macros on demand, compilers may expand macros
> >> several times for analysis, higher-order macros may expand inner macro
> >> for analysis as well, etc., etc. So make sure that your macros are
> >> always side-effect-free (unless you really know what you're doing).
>
> > So you're saying that setq (or setf) should not appear in the code of
> > the macroexpansion function ? Why , what could go wrong ?
>
> (defvar *c* 0)
>
> (defmacro m ()
> (incf *c*)
> `',*c*)
If that's what Pascal meant I'm not surprised. But he referred to the
code in my first post, for example
(let ((a 0))
(defmacro memory (&rest body)
(prog2
(incf a)
`(symbol-macrolet ((cell ,a))
,@body)
(decf a))))
His phrasing above and the example gave me the impression that he was
saying that all assignments are dangerous even if the effects of the
assignment do not persist after the macroexpansion function has
finished executing. If that's what he meant then I do find it
surprising. For example I don't see what could go wrong with my code
above regardless of how many times the macro gets expanded.
In any case, it seems that if one wants to have macros share
information (apart from trivial cases perhaps) then this cannot be
achieved behind the scenes so to speak. What I mean behind the scenes
is the way you would do it with functions:
(let ((a))
(defun foo ...
(defun bar ...)
Any changes to a one function makes the other knows about but the
rest of the code is ignorant about a. With macros however one has to
put any changes to the shared information in the code the macro
produces thereby leaking what ought to be internal implementation. I
consider this a blemish in Lisp.
--
Capitalization is the difference between "I had to help my uncle Jack
off a horse.." and "I had to help my uncle jack off a horse.."
http://www.bash.org/?367896
Pascal Costanza
I didn't read your code carefully enough, so I indeed didn't notice that
the side effect didn't persist. However, this code could still create
problems, for example when a user tries to macroexpand some particular
form while some other code is compiled in a background thread, and two
invocations of 'memory are thus expanded at the same time. You would
either need to synchronize the accesses to the 'a variable, for example
by using locks or STM (har :P ), or you should just use local bindings
for 'a.
> In any case, it seems that if one wants to have macros share
> information (apart from trivial cases perhaps) then this cannot be
> achieved behind the scenes so to speak. What I mean behind the scenes
> is the way you would do it with functions:
>
> (let ((a))
> (defun foo ...
> (defun bar ...)
>
> Any changes to a one function makes the other knows about but the
> rest of the code is ignorant about a. With macros however one has to
> put any changes to the shared information in the code the macro
> produces thereby leaking what ought to be internal implementation. I
> consider this a blemish in Lisp.
You can hide the shared information in a package.
Thomas A. Russ
> If that's what Pascal meant I'm not surprised. But he referred to the
> code in my first post, for example
>
> (let ((a 0))
> (defmacro memory (&rest body)
> (prog2
> (incf a)
> `(symbol-macrolet ((cell ,a))
> ,@body)
> (decf a))))
>
> His phrasing above and the example gave me the impression that he was
> saying that all assignments are dangerous even if the effects of the
> assignment do not persist after the macroexpansion function has
> finished executing. If that's what he meant then I do find it
> surprising. For example I don't see what could go wrong with my code
> above regardless of how many times the macro gets expanded.
Well, I'm not convinced that this has to work the way you may wish it
to. In particular, the expansion of macros inside BODY doesn't have to
occur in the dynamic scope of the expansion of MEMORY.
So, for example, it seems to me it is perfectly permissible, and in fact
the most likely implementation, that when macros are being expanded the
following happens:
The macro MEMORY is encountered by the code walker or compiler. It is
expanded, and given the BODY contents. The BODY contents are not
evaluated, but merely returned inside the SYMBOL-MACROLET form. The
macroexpansion function for MEMORY returns the new form.
This new form is now walked by the code walker, so if there is any
macros in the newly returned form, they will also be properly expanded.
But since the macro-expansion function has already returned, the value
of variable A has been restored. So the next expansion doesn't see any
dynamic change.
In fact, I'm pretty sure it has to work this way, since otherwise macros
that expand into code headed by other macros would not work properly.
There will not be any invocation of macro-expansion functions during the
invocation of a macro-expansion function.
You could probably observe this by tracing the macro-expansion function
that corresponds to your macro. I would expect to see it return before
any other macro expansion occurs.
--
Thomas A. Russ, USC/Information Sciences Institute
Tobias C. Rittweiler
> (let ((a 0))
> (defmacro memory (&rest body)
> (prog2
> (incf a)
> `(symbol-macrolet ((cell ,a))
> ,@body)
> (decf a))))
(Mind Thoms Russ' response!)
> In any case, it seems that if one wants to have macros share
> information (apart from trivial cases perhaps) then this cannot be
> achieved behind the scenes so to speak. What I mean behind the scenes
> is the way you would do it with functions:
>
> (let ((a))
> (defun foo ...
> (defun bar ...)
>
> Any changes to a one function makes the other knows about but the
> rest of the code is ignorant about a. With macros however one has to
> put any changes to the shared information in the code the macro
> produces thereby leaking what ought to be internal implementation.
Where is it leaking? Because you can introspectively look at a macro's
expansion? Do you consider reflection to be "blemish" in general?
> I consider this a blemish in Lisp.
Impenetrable encapsulation is only longed for by the timid. :-)
-T.
Spiros Bousbouras
> Spiros Bousbouras wrote:
>
> > (let ((a 0))
> > (defmacro memory (&rest body)
> > (prog2
> > (incf a)
> > `(symbol-macrolet ((cell ,a))
> > ,@body)
> > (decf a))))
>
> > His phrasing above and the example gave me the impression that he was
> > saying that all assignments are dangerous even if the effects of the
> > assignment do not persist after the macroexpansion function has
> > finished executing. If that's what he meant then I do find it
> > surprising. For example I don't see what could go wrong with my code
> > above regardless of how many times the macro gets expanded.
>
> I didn't read your code carefully enough, so I indeed didn't notice that
> the side effect didn't persist. However, this code could still create
> problems, for example when a user tries to macroexpand some particular
> form while some other code is compiled in a background thread, and two
> invocations of 'memory are thus expanded at the same time.
Is there any established way for how such parallel
expansion/compilation is supposed to work ? Intuitively it seems to
me that in the scenario you're describing the implementation ought to
create one copy of a for the compilation and another for any
expansions which are not part of the compilation. If it doesn't do
that what makes you think that even your code will work correctly ?
There may be clashes with the use of new-cell-value.
> > In any case, it seems that if one wants to have macros share
> > information (apart from trivial cases perhaps) then this cannot be
> > achieved behind the scenes so to speak. What I mean behind the scenes
> > is the way you would do it with functions:
>
> > (let ((a))
> > (defun foo ...
> > (defun bar ...)
>
> > Any changes to a one function makes the other knows about but the
> > rest of the code is ignorant about a. With macros however one has to
> > put any changes to the shared information in the code the macro
> > produces thereby leaking what ought to be internal implementation. I
> > consider this a blemish in Lisp.
>
> You can hide the shared information in a package.
You can but still I find it kludgy. You already have the datum in a
variable i.e. in memory but in order to share it with a different
invocation of the macro you have to "print" it and then read it back
in again. Very ugly. Apart from aesthetics there's another issue :
"Practical Common Lisp" says in footnote 4 of chapter 4 : "Not all
Lisp objects can be written out in a way that can be read back in".
What if it is such an object you want to share ?
--
Never attribute to conspiracy what can adequately be explained by
shared attitudes.
Pillsy
[...]
> In any case, it seems that if one wants to have macros share
> information (apart from trivial cases perhaps) then this cannot be
> achieved behind the scenes so to speak. What I mean behind the scenes
> is the way you would do it with functions:
> (let ((a))
> (defun foo ...
> (defun bar ...)
> Any changes to a one function makes the other knows about but the
> rest of the code is ignorant about a.
This is a trick I've used once or twice in the past, and I'm just
submitting it as an idea in part because I'd like to see if there are
problems with it that I haven't considered up until now. It's hardly
something I do every day, and I know this corner of macrology is
generally riddled with pitfalls.
The idea is that you have easy access to uninterned symbols, and those
symbols not only have their value cells, but also function cells and
property lists to play with. You could do something like this:
(let ((closed (closed "BACKING-")))
(defmacro foo (name &rest args)
(setf (get closed name) (stuff-from-foo-args args))
(generate-foo-expansion name args closed))
(defmacro bar (name &rest args)
(setf (get closed name) (stuff-from-bar-args args))
(generate-bar-expansion name args closed)))
Is there something horribly wrong with this approach?
Cheers,
Pillsy
Spiros Bousbouras
> Spiros Bousbouras <spi...@gmail.com> writes:
> > If that's what Pascal meant I'm not surprised. But he referred to the
> > code in my first post, for example
>
> > (let ((a 0))
> > (defmacro memory (&rest body)
> > (prog2
> > (incf a)
> > `(symbol-macrolet ((cell ,a))
> > ,@body)
> > (decf a))))
>
> > His phrasing above and the example gave me the impression that he was
> > saying that all assignments are dangerous even if the effects of the
> > assignment do not persist after the macroexpansion function has
> > finished executing. If that's what he meant then I do find it
> > surprising. For example I don't see what could go wrong with my code
> > above regardless of how many times the macro gets expanded.
>
> Well, I'm not convinced that this has to work the way you may wish it
> to.
I would be surprised if you or anyone else were convinced since I
already said in my opening post that I tested it and it doesn't work.
> The macro MEMORY is encountered by the code walker or compiler. It is
> expanded, and given the BODY contents. The BODY contents are not
> evaluated, but merely returned inside the SYMBOL-MACROLET form. The
> macroexpansion function for MEMORY returns the new form.
>
> This new form is now walked by the code walker, so if there is any
> macros in the newly returned form, they will also be properly expanded.
> But since the macro-expansion function has already returned, the value
> of variable A has been restored. So the next expansion doesn't see any
> dynamic change.
Or as I said in my OP : "I assume that by the time the code generated
from the macro expansion gets compiled the statement (decf a) has
already been executed."
--
The worst part of living on the West Coast is having to remember to
say "no mayonnaise" when ordering a burger.
tinyurl.com/mayo-quote
Pillsy
[...]
> (let ((closed (closed "BACKING-")))
> (defmacro foo (name &rest args)
> (setf (get closed name) (stuff-from-foo-args args))
> (generate-foo-expansion name args closed))
> (defmacro bar (name &rest args)
> (setf (get closed name) (stuff-from-bar-args args))
> (generate-bar-expansion name args closed)))
> Is there something horribly wrong with this approach?
Well, one thing that's potentially horribly wrong with the approach is
its dependence on compile-time side-effects. This problem seems pretty
avoidable if all you want to do is establish a private chanell that
can be accessed by your expansions; that may not be enough for your
purposes. But if it is, there's always
(let ((closed (gensym "CLOSED-")))
(defmacro foo (name &rest stuff)
`(progn
(setf (get ',closed ',name) ,(mangle-foo-args args))
,(foo-expand name args closed)))
(defmacro bar (name &rest stuff)
`(progn
(setf (get ',closed ',name) ,(mangle-bar-args args))
,(foo-expand name args closed))))
Cheers,
Pillsy
Spiros Bousbouras
wrote:
> Spiros Bousbouras <spi...@gmail.com> writes:
> > (let ((a 0))
> > (defmacro memory (&rest body)
> > (prog2
> > (incf a)
> > `(symbol-macrolet ((cell ,a))
> > ,@body)
> > (decf a))))
>
> (Mind Thoms Russ' response!)
>
> > In any case, it seems that if one wants to have macros share
> > information (apart from trivial cases perhaps) then this cannot be
> > achieved behind the scenes so to speak. What I mean behind the scenes
> > is the way you would do it with functions:
>
> > (let ((a))
> > (defun foo ...
> > (defun bar ...)
>
> > Any changes to a one function makes the other knows about but the
> > rest of the code is ignorant about a. With macros however one has to
> > put any changes to the shared information in the code the macro
> > produces thereby leaking what ought to be internal implementation.
>
> Where is it leaking?
In my code above nothing is leaking but my code doesn't do what I
want. In the solutions posted by the two Pascals a symbol-macro cell
is introduced in the global environment. That's what's leaking. In
Pascal B.'s solution also the dynamic variable *cell* is leaking.
> Because you can introspectively look at a macro's
> expansion?
The ability to expand macros is a good thing. As I said in my opening
post , if Lisp had a minimal-compile function (which in a sense would
be a stronger version of macroexpand) , I could do what I want in a
clean and more elegant manner. Actually I find it surprising that it
doesn't. minimal-compile would be more or less something which does
more than macroexpand but less than eval.
> Do you consider reflection to be "blemish" in general?
Not at all. The problem here is lack of the right kind of
"reflection".
> > I consider this a blemish in Lisp.
>
> Impenetrable encapsulation is only longed for by the timid. :-)
Timid or not , if one wants impenetrable encapsulation the language
should let them achieve it unless it conflicts with something else.
And the issues here are elegance , practicality and even speed of
compilation.
Tamas K Papp
> On 11 Nov, 08:31, "Tobias C. Rittweiler" <t...@freebits.de.invalid>
> wrote:
>> Where is it leaking?
>
> In my code above nothing is leaking but my code doesn't do what I want.
> In the solutions posted by the two Pascals a symbol-macro cell is
> introduced in the global environment. That's what's leaking. In Pascal
> B.'s solution also the dynamic variable *cell* is leaking.
As Pascal said, you can put *cell* in another package. End of story.
>> Because you can introspectively look at a macro's expansion?
>
> The ability to expand macros is a good thing. As I said in my opening
> post , if Lisp had a minimal-compile function (which in a sense would be
> a stronger version of macroexpand) , I could do what I want in a clean
> and more elegant manner. Actually I find it surprising that it doesn't.
> minimal-compile would be more or less something which does more than
> macroexpand but less than eval.
>
>> Do you consider reflection to be "blemish" in general?
>
> Not at all. The problem here is lack of the right kind of "reflection".
>
>> > I consider this a blemish in Lisp.
>>
>> Impenetrable encapsulation is only longed for by the timid. :-)
>
> Timid or not , if one wants impenetrable encapsulation the language
> should let them achieve it unless it conflicts with something else. And
> the issues here are elegance , practicality and even speed of
> compilation.
I don't see how you would gain elegance, practicality or even speed
in this case.
But fortunately, your problem might have a solution soon. Stroustrup
is rumored to be working on a new language with impenetrable
encapsulation. The problem with C++ was that even though it has
private slots, the programmer would _know_ that they are there and be
tempted to access them using devious tricks. They might even reside
on the same hard disk as some other code! The new language,
tentatively called Objective StraitJacket, will offer the following
method to deal with that:
The code is written to a hard disk, which put in sealed triple metal
container (made of gold, silver, and iron). A river is temporarily
diverted at a random location to bury the container. The workers, who
would still know the location, are then shot by archers. The archers
are eaten by wild boars. The boars are then read the operator
precedence rules of C++, until they die of sheer exasperation.
Until then, you just have to hide symbols in packages.
Cheers,
Tamas
Tobias C. Rittweiler
> On 11 Nov, 08:31, "Tobias C. Rittweiler" wrote:
>
> > Spiros Bousbouras <spi...@gmail.com> writes:
> >
> > > In any case, it seems that if one wants to have macros share
> > > information (apart from trivial cases perhaps) then this cannot be
> > > achieved behind the scenes so to speak. What I mean behind the scenes
> > > is the way you would do it with functions:
> >
> > > (let ((a))
> > > (defun foo ...
> > > (defun bar ...)
> >
> > > Any changes to a one function makes the other knows about but the
> > > rest of the code is ignorant about a. With macros however one has to
> > > put any changes to the shared information in the code the macro
> > > produces thereby leaking what ought to be internal implementation.
> >
> > Where is it leaking?
>
> In my code above nothing is leaking but my code doesn't do what I
> want. In the solutions posted by the two Pascals a symbol-macro cell
> is introduced in the global environment. That's what's leaking. In
> Pascal B.'s solution also the dynamic variable *cell* is leaking.
Pascal Costanza's example contained a global symbol macro, though you
don't need it.
Are internally used functions to implement other functions also an
example of leaking? Or variables? They all introduce stuff into the
global environment. What's the difference?
> > Because you can introspectively look at a macro's expansion?
>
> The ability to expand macros is a good thing. As I said in my opening
> post , if Lisp had a minimal-compile function (which in a sense would
> be a stronger version of macroexpand) , I could do what I want in a
> clean and more elegant manner. Actually I find it surprising that it
> doesn't. minimal-compile would be more or less something which does
> more than macroexpand but less than eval.
It's commonly called MACROEXPAND-ALL. Look into the different SWANK
backends how it could be implemented on many implementations.
> > Do you consider reflection to be "blemish" in general?
>
> Not at all. The problem here is lack of the right kind of
> "reflection".
MACROEXPAND-ALL can be useful, yeah.
> > > I consider this a blemish in Lisp.
> >
> > Impenetrable encapsulation is only longed for by the timid. :-)
>
> Timid or not , if one wants impenetrable encapsulation the language
> should let them achieve it unless it conflicts with something else.
> And the issues here are elegance , practicality and even speed of
> compilation.
How does MACROEXPAND-ALL help the speed of compilation? Using it, the
implementation has to walk the syntax tree twice.
Do you know that it can in fact lead to an _impairment_ of runtime speed
because implementation which would previously implement some macros as
special forms cannot do so?
-T.
Pascal Costanza
Huh? new-cell-value is lexically bound, so it's guaranteed to have its
own binding per invocation.
>>> In any case, it seems that if one wants to have macros share
>>> information (apart from trivial cases perhaps) then this cannot be
>>> achieved behind the scenes so to speak. What I mean behind the scenes
>>> is the way you would do it with functions:
>>> (let ((a))
>>> (defun foo ...
>>> (defun bar ...)
>>> Any changes to a one function makes the other knows about but the
>>> rest of the code is ignorant about a. With macros however one has to
>>> put any changes to the shared information in the code the macro
>>> produces thereby leaking what ought to be internal implementation. I
>>> consider this a blemish in Lisp.
>> You can hide the shared information in a package.
>
> You can but still I find it kludgy. You already have the datum in a
> variable i.e. in memory but in order to share it with a different
> invocation of the macro you have to "print" it and then read it back
> in again. Very ugly. Apart from aesthetics there's another issue :
> "Practical Common Lisp" says in footnote 4 of chapter 4 : "Not all
> Lisp objects can be written out in a way that can be read back in".
> What if it is such an object you want to share ?
This all has absolutely nothing to do with printing and reading.
Spiros Bousbouras
wrote:
> Spiros Bousbouras <spi...@gmail.com> writes:
>
> > In my code above nothing is leaking but my code doesn't do what I
> > want. In the solutions posted by the two Pascals a symbol-macro cell
> > is introduced in the global environment. That's what's leaking. In
> > Pascal B.'s solution also the dynamic variable *cell* is leaking.
>
> Pascal Costanza's example contained a global symbol macro, though you
> don't need it.
How can you do the same thing without a global symbol macro ?
> Are internally used functions to implement other functions also an
> example of leaking? Or variables? They all introduce stuff into the
> global environment. What's the difference?
What do you mean "internally" ? Show me some code and I'll tell you.
> > The ability to expand macros is a good thing. As I said in my opening
> > post , if Lisp had a minimal-compile function (which in a sense would
> > be a stronger version of macroexpand) , I could do what I want in a
> > clean and more elegant manner. Actually I find it surprising that it
> > doesn't. minimal-compile would be more or less something which does
> > more than macroexpand but less than eval.
>
> It's commonly called MACROEXPAND-ALL.
Brilliant. I will return to this, probably in a different thread.
> > And the issues here are elegance , practicality and even speed of
> > compilation.
>
> How does MACROEXPAND-ALL help the speed of compilation?
I'm not sure that it will but I thought that if it makes the
implementation of certain macros simpler or the expanded code smaller
then this might help the speed of compilation.
> Using it, the
> implementation has to walk the syntax tree twice.
Yes but the second time it won't have to expand any macros. So on one
hand it might make the macroexpansion function or the expanded code
of some macros smaller but on the other it will have to parse some
stuff twice. I was thinking that in some cases the first may outweigh
the second.
> Do you know that it can in fact lead to an _impairment_ of runtime speed
> because implementation which would previously implement some macros as
> special forms cannot do so?
No I didn't know it. Can you give me an example of something an
implementation can do without MACROEXPAND-ALL that it couldn't do if
MACROEXPAND-ALL existed ?
--
But, to wring a new twist on an old phrase, power corrupts. And
Mississippi Burning is corrupted by the reckless exhilaration it
betrays in wielding its own pumped-up, self-inflated cinematic power.
http://cinepad.com/reviews/mississippi.htm
Spiros Bousbouras
> Spiros Bousbouras wrote:
> > On 11 Nov, 00:40, Pascal Costanza <p...@p-cos.net> wrote:
> >> Spiros Bousbouras wrote:
>
> >>> (let ((a 0))
> >>> (defmacro memory (&rest body)
> >>> (prog2
> >>> (incf a)
> >>> `(symbol-macrolet ((cell ,a))
> >>> ,@body)
> >>> (decf a))))
[...]
> > Is there any established way for how such parallel
> > expansion/compilation is supposed to work ? Intuitively it seems to
> > me that in the scenario you're describing the implementation ought to
> > create one copy of a for the compilation and another for any
> > expansions which are not part of the compilation. If it doesn't do
> > that what makes you think that even your code will work correctly ?
> > There may be clashes with the use of new-cell-value.
>
> Huh? new-cell-value is lexically bound, so it's guaranteed to have its
> own binding per invocation.
Guaranteed by what ? This situation is not covered by the standard ,
is it ? In any case in my code the variable a is also lexically
bound.
Pascal Costanza
Lexical bindings? Of course they are covered by the standard.
> In any case in my code the variable a is also lexically bound.
But you're performing side effects on this lexically bound variable,
which may lead to race conditions when multiple threads expand this one
macro. This is then a situation of concurrent write accesses to shared
memory, which usually need to be synchronized. In a 'pure binding'
(purely functional) situation, you don't have these problems.
Yes, _multithreading_ is not covered by the standard, but there are
numerous multithreaded Common Lisp implementations, so if you want your
code to be portable, you better take that into account.
Tobias C. Rittweiler
> On 17 Nov, 18:01, "Tobias C. Rittweiler" wrote:
>
> > Spiros Bousbouras <spi...@gmail.com> writes:
> >
> > > In my code above nothing is leaking but my code doesn't do what I
> > > want. In the solutions posted by the two Pascals a symbol-macro cell
> > > is introduced in the global environment. That's what's leaking. In
> > > Pascal B.'s solution also the dynamic variable *cell* is leaking.
> >
> > Pascal Costanza's example contained a global symbol macro, though you
> > don't need it.
>
> How can you do the same thing without a global symbol macro ?
(defmacro memory (&body body &environment env)
(multiple-value-bind (old-cell-value expandedp)
(macroexpand 'cell env)
`(symbol-macrolet ((cell ,(1+ (if expandedp old-cell-value 0))))
,@body)))
> > Are internally used functions to implement other functions also an
> > example of leaking? Or variables? They all introduce stuff into the
> > global environment. What's the difference?
>
> What do you mean "internally" ? Show me some code and I'll tell you.
(defpackage :foof (:use :cl) (:export #:frobf))
(in-package :foof)
(defun internal-fn (x) (* x x)
(defun frobf (x y) (+ (internal-fn x) y))
A function (INTERNAL-FN) is introduced in the global environment solely
to implement the behaviour of FROBF. In your definition quoted above,
it's leaking something.
> > Do you know that it can in fact lead to an _impairment_ of runtime speed
> > because implementation which would previously implement some macros as
> > special forms cannot do so?
>
> No I didn't know it. Can you give me an example of something an
> implementation can do without MACROEXPAND-ALL that it couldn't do if
> MACROEXPAND-ALL existed ?
Say your implementation implements DOLIST as a special form[*] for
reasons of efficiency. In case of
(defun foo (list &aux (sum 0)) (dolist (i list sum) (incf sum i)))
the implementation can process DOLIST as a special form.
Contrarily on:
(defmacro walk-at-compile-time (form &environment env)
(macroexpand-all form env))
(defun bar (list &aux (sum 0))
(walk-at-compile-time (dolist (i list sum) (incf sum i))))
the implementation has to expand the DOLIST form into something probably
involving TAGBODY. We assumed that the implementation as special form
would be more efficient -- so this shows how code-walking can impose
runtime penalty.
-T.
[*] This is not just an academic point. For example, ABCL's interpreter
implements DOLIST etc. as special form because it can then execute the
loop directly in Java instead of having to interpret a Lisp loop.
Spiros Bousbouras
> On Nov 10, 4:36 pm, Spiros Bousbouras <spi...@gmail.com> wrote:
> [...]
>
> > In any case, it seems that if one wants to have macros share
> > information (apart from trivial cases perhaps) then this cannot be
> > achieved behind the scenes so to speak. What I mean behind the scenes
> > is the way you would do it with functions:
> > (let ((a))
> > (defun foo ...
> > (defun bar ...)
> > Any changes to a one function makes the other knows about but the
> > rest of the code is ignorant about a.
>
> This is a trick I've used once or twice in the past, and I'm just
> submitting it as an idea in part because I'd like to see if there are
> problems with it that I haven't considered up until now. It's hardly
> something I do every day, and I know this corner of macrology is
> generally riddled with pitfalls.
>
> The idea is that you have easy access to uninterned symbols, and those
> symbols not only have their value cells, but also function cells and
> property lists to play with. You could do something like this:
>
> (let ((closed (closed "BACKING-")))
Is the second "closed" meant to be "gensym" ?
> (defmacro foo (name &rest args)
> (setf (get closed name) (stuff-from-foo-args args))
> (generate-foo-expansion name args closed))
> (defmacro bar (name &rest args)
> (setf (get closed name) (stuff-from-bar-args args))
> (generate-bar-expansion name args closed)))
>
> Is there something horribly wrong with this approach?
I'm afraid I don't understand what the various ingredients like
closed , name , stuff-from-foo-args are supposed to do. Could you
post some specific example which uses your technique ?
Nevertheless your code gave me the idea for the following approach:
(let ((a (gensym "memory-var-")))
(defmacro memory (&body body)
`(progn
(if (boundp ',a)
(incf ,a)
(defvar ,a 1))
(symbol-macrolet ((cell ,a))
(prog1
,@body
(decf ,a))))))
It solves the problem of "reserving" for itself a name for a global
symbol. And yes , I know it uses side effects but I'm not convinced
this is something worrying about.
Spiros Bousbouras
> And yes , I know it uses side effects but I'm not convinced
> this is something worrying about.
something worth worrying about.
Spiros Bousbouras
wrote:
> Spiros Bousbouras <spi...@gmail.com> writes:
> > On 17 Nov, 18:01, "Tobias C. Rittweiler" wrote:
>
> > > Spiros Bousbouras <spi...@gmail.com> writes:
>
> > > > In my code above nothing is leaking but my code doesn't do what I
> > > > want. In the solutions posted by the two Pascals a symbol-macro cell
> > > > is introduced in the global environment. That's what's leaking. In
> > > > Pascal B.'s solution also the dynamic variable *cell* is leaking.
>
> > > Pascal Costanza's example contained a global symbol macro, though you
> > > don't need it.
>
> > How can you do the same thing without a global symbol macro ?
>
> (defmacro memory (&body body &environment env)
> (multiple-value-bind (old-cell-value expandedp)
> (macroexpand 'cell env)
> `(symbol-macrolet ((cell ,(1+ (if expandedp old-cell-value 0))))
> ,@body)))
Very nice.
> > > Are internally used functions to implement other functions also an
> > > example of leaking? Or variables? They all introduce stuff into the
> > > global environment. What's the difference?
>
> > What do you mean "internally" ? Show me some code and I'll tell you.
>
> (defpackage :foof (:use :cl) (:export #:frobf))
> (in-package :foof)
> (defun internal-fn (x) (* x x)
> (defun frobf (x y) (+ (internal-fn x) y))
>
> A function (INTERNAL-FN) is introduced in the global environment solely
> to implement the behaviour of FROBF. In your definition quoted above,
> it's leaking something.
What definition , I didn't give any definition. But I will give now a
precise description of what's bothering me. My concern is whether the
implementation of memory prevents the user from using certain names
for his own code. Obviously, if the user wants to use memory then he
cannnot use the name 'memory for something else and , based on the
specification of memory , he cannot use the name 'cell *inside* the
scope of a memory call to denote something different than what the
specification of memory specifies. But ideally he should be able to
use 'cell in whatever way he likes as long as it's not within the
scope of a memory call. It may be poor style to do so but I prefer to
let the user decide on his own what he considers good style. So my
general rule is:
The ideal implementation of a function, macro, library etc.
should not place any further restrictions on the user beyond
what logically follows from what the function, macro,
library etc. is supposed to achieve, taking into account the
restrictions posed by the language.
So for example it does not logically follow from the specification of
memory that the user should not be able to use 'cell in any way he
likes outside the scope of a memory call so any implementation of
memory which places such a restriction on the user is not ideal in my
book. *That's* what really bothers me about Pascal C.'s code , not
that he places some symbol in the global environment.
Your implementation of frobf , although it creates the global symbol
internal-fn , does not violate the rule above because you place
internal-fn in a separate package and the user is not supposed to
access unexported symbols of a package as a general rule. Even so I
would find it more elegant if frobf were written as
(defun frobf (x y)
(flet ((internal-fn (x) (* x x)))
(+ (internal-fn x) y)))
I suspect that if frobf was intended for actual use you would write
it this way yourself (or not use internal-fn at all). If in order to
implement frobf you needed a large number of auxiliary functions then
yes you would probably put them in a separate package but for just a
few it's not worth the hassle. Returning to memory , it similarly
feels like too much work to create a separate package just to
encapsulate one name , 'cell , so ideally I'm looking for a solution
which satisfies the rule above without creating a separate package.
With these thoughts in mind here's an improvement to your code:
(let ((a (gensym "memory-var-")))
(defmacro memory (&body body &environment env)
(multiple-value-bind (old-cell-value expandedp new-cell-value)
(macroexpand a env)
(setq new-cell-value (1+ (if expandedp old-cell-value 0)))
`(symbol-macrolet ((,a ,new-cell-value)
(cell ,new-cell-value))
,@body))))
Not only it works correctly but even if we do for example
(define-symbol-macro cell '(Thierry Henry \, villain or victim ?))
the macro still works correctly. Of course even my code won't work
correctly if a has been declared as special before memory is defined
but I'm willing to accept this as a reasonable restriction of the
language :·D
--
I confess I am somewhat dazed. It happened too fast. Years of
fantasies had not prepared me for the suddenness of his orgasm.
Somehow, I thought one as experienced in the Force as Obi-Wan would
have lasted longer.
Somewhere on the internets
Spiros Bousbouras
> Spiros Bousbouras <spi...@gmail.com> writes:
> > On 17 Nov, 18:01, "Tobias C. Rittweiler" wrote:
>
> > > Do you know that it can in fact lead to an _impairment_ of runtime speed
> > > because implementation which would previously implement some macros as
> > > special forms cannot do so?
>
> > No I didn't know it. Can you give me an example of something an
> > implementation can do without MACROEXPAND-ALL that it couldn't do if
> > MACROEXPAND-ALL existed ?
>
> Say your implementation implements DOLIST as a special form[*] for
> reasons of efficiency. In case of
>
> (defun foo (list &aux (sum 0)) (dolist (i list sum) (incf sum i)))
>
> the implementation can process DOLIST as a special form.
>
> Contrarily on:
>
> (defmacro walk-at-compile-time (form &environment env)
> (macroexpand-all form env))
>
> (defun bar (list &aux (sum 0))
> (walk-at-compile-time (dolist (i list sum) (incf sum i))))
>
> the implementation has to expand the DOLIST form into something probably
> involving TAGBODY. We assumed that the implementation as special form
> would be more efficient -- so this shows how code-walking can impose
> runtime penalty.
I don't see why macroexpand-all cannot just return the dolist form
itself if dolist is implemented as a special form. It would be
perfectly adequate for the purpose I stated in my OP. As a debugging
tool it might even be preferable if macroexpand-all did not continue
the expansion beyond objects defined in the Lisp standard. So if I
have something like
(my-macro1 arg1 (my-macro2 args))
and want to examine that it does the right thing it would be more
useful if
(macroexpand-all '(my-macro1 arg1 (my-macro2 args)))
only expanded my macros but not things like do , dolist etc.
And while we're at it , in an implementation where dolist is a
special form what are the return values of
(macroexpand '(dolist ...)) ?
--
Always remember that it is impossible to speak in such a way that
you cannot be misunderstood: there will always be some who
misunderstand you.
Sir Karl Popper
Pascal J. Bourguignon
> I don't see why macroexpand-all cannot just return the dolist form
> itself if dolist is implemented as a special form.
Because section "3.1.2.1.2.2 Macro Forms" says so.
DOLIST is specified to be a macro and as mentionned by section
"3.1.2.1.2.2 Macro Forms":
An implementation is free to implement a Common Lisp special
operator as a macro. An implementation is free to implement any
macro operator as a special operator, but only if an equivalent
definition of the macro is also provided.
it can be implemented as a special operator, however, in that case the
implementation must provide a macro definition expanding (eventually)
only to standard special operators, for the benefit of code walkers.
--
__Pascal Bourguignon__
Spiros Bousbouras
wrote:
But the standard does not mention macroexpand-all so it can behave
any way it likes. Furthermore it's not unreasonable that it would not
expand something which the implementation offers as a special form
even if the standard says is a macro. Or macroexpand-all could be
designed to take an additional argument by which the programmer can
switch between "expand everything the standard says is a macro" and
"expand everything that this implementation actually (only)
implements as a macro".
But more to the point earlier in the thread Tobias said:
How does MACROEXPAND-ALL help the speed of compilation? Using
it, the implementation has to walk the syntax tree twice.
Do you know that it can in fact lead to an _impairment_ of
runtime speed because implementation which would previously
implement some macros as special forms cannot do so?
Even now as I reread it it seems to be saying that just by making
available macroexpand-all an implementation is prevented from
implementing some macros as special forms. But I think what he meant
was that if a piece of code uses macroexpand-all then when compiling
this code an implementation can no longer use the special form
version of a macro (if it has one) and has to use instead the actual
macro expansion. So yes , I see how macroexpand-all can cause an
impairment of *runtime* speed even when it is only called during
compilation. But this could be solved by offering a switch as I
mention above.
Pascal J. Bourguignon
> On 21 Nov, 10:13, p...@informatimago.com (Pascal J. Bourguignon)
> wrote:
>> Spiros Bousbouras <spi...@gmail.com> writes:
>> > I don't see why macroexpand-all cannot just return the dolist form
>> > itself if dolist is implemented as a special form.
>>
>> Because section "3.1.2.1.2.2 Macro Forms" says so.
>>
>> DOLIST is specified to be a macro and as mentionned by section
>> "3.1.2.1.2.2 Macro Forms":
>>
>> An implementation is free to implement a Common Lisp special
>> operator as a macro. An implementation is free to implement any
>> macro operator as a special operator, but only if an equivalent
>> definition of the macro is also provided.
>>
>> it can be implemented as a special operator, however, in that case the
>> implementation must provide a macro definition expanding (eventually)
>> only to standard special operators, for the benefit of code walkers.
>
> But the standard does not mention macroexpand-all so it can behave
> any way it likes.
Right, macroexpand-all is not a Common Lisp function. My bad, sorry.
What I said applies only to CL:MACROEXPAND-1 and CL:MACROEXPAND.
> Furthermore it's not unreasonable that it would not
> expand something which the implementation offers as a special form
> even if the standard says is a macro. Or macroexpand-all could be
> designed to take an additional argument by which the programmer can
> switch between "expand everything the standard says is a macro" and
> "expand everything that this implementation actually (only)
> implements as a macro".
>
> But more to the point earlier in the thread Tobias said:
>
> How does MACROEXPAND-ALL help the speed of compilation? Using
> it, the implementation has to walk the syntax tree twice.
>
> Do you know that it can in fact lead to an _impairment_ of
> runtime speed because implementation which would previously
> implement some macros as special forms cannot do so?
>
> Even now as I reread it it seems to be saying that just by making
> available macroexpand-all an implementation is prevented from
> implementing some macros as special forms. But I think what he meant
> was that if a piece of code uses macroexpand-all then when compiling
> this code an implementation can no longer use the special form
> version of a macro (if it has one) and has to use instead the actual
> macro expansion. So yes , I see how macroexpand-all can cause an
> impairment of *runtime* speed even when it is only called during
> compilation. But this could be solved by offering a switch as I
> mention above.
Clearly, the expectation of na�ve users such as me about a
macroexpand-all function would be that it expands to the same special
operators as macroexpand or macroexpand-1.
I'd say that expanding to the special operators of the implementation
is a specific need, and would need a more specific function, such as
EXPAND-ALL-TO-IMPLEMENTATION-SPECIAL-OPERATORS.
--
__Pascal Bourguignon__
Spiros Bousbouras
wrote:
>
> Clearly, the expectation of naïve users such as me about a
> macroexpand-all function would be that it expands to the same special
> operators as macroexpand or macroexpand-1.
Whereas a more sophisticated user would read the documentation
instead of trying to figure out from just the name what it
does. I wonder what the expectations of a naive user would be with
regards to car or labels.
> I'd say that expanding to the special operators of the implementation
> is a specific need, and would need a more specific function, such as
> EXPAND-ALL-TO-IMPLEMENTATION-SPECIAL-OPERATORS.
And there is another "specific need" : macroexpand-all to things
defined in the standard but not below. So user macros would be
expanded but as soon as we reach standard macros then these would not
be expanded regardless of whether they are implemented as special
forms or not. This would be useful for debugging.
Spiros Bousbouras
> Spiros Bousbouras wrote:
> > On 17 Nov, 20:15, Pascal Costanza <p...@p-cos.net> wrote:
> >> Spiros Bousbouras wrote:
>
> >>> Is there any established way for how such parallel
> >>> expansion/compilation is supposed to work ? Intuitively it seems to
> >>> me that in the scenario you're describing the implementation ought to
> >>> create one copy of a for the compilation and another for any
> >>> expansions which are not part of the compilation. If it doesn't do
> >>> that what makes you think that even your code will work correctly ?
> >>> There may be clashes with the use of new-cell-value.
> >> Huh? new-cell-value is lexically bound, so it's guaranteed to have its
> >> own binding per invocation.
>
> > Guaranteed by what ? This situation is not covered by the standard ,
> > is it ?
>
> Lexical bindings? Of course they are covered by the standard.
I was of course referring to the interaction between multithreading
and lexical bindings. *That* is not covered by the standard.
> > In any case in my code the variable a is also lexically bound.
>
> But you're performing side effects on this lexically bound variable,
> which may lead to race conditions when multiple threads expand this one
> macro. This is then a situation of concurrent write accesses to shared
> memory, which usually need to be synchronized. In a 'pure binding'
> (purely functional) situation, you don't have these problems.
>
> Yes, _multithreading_ is not covered by the standard,
Which brings us back to what I ask above: Is there any established
way for how such parallel expansion/compilation is supposed to work ?
To put it otherwise you're saying that my code violates some rules
but yours doesn't but you're not saying what these rules are , how
universal they are among implementations and where one could read
about those rules. And it's not clear what you mean by "side
effects". Are you saying that things like setf are completely
forbidden in the code which does the expansion ?
> but there are
> numerous multithreaded Common Lisp implementations, so if you want your
> code to be portable, you better take that into account.
No , if I want my code to be portable I only need to take into
account the standard and possible bugs/non-conformances in the
implementations. Portability *and* thread safety on the other hand ,
is a much more severe requirement.
Having said that it still seems likely to me that my code will work
everywhere yours will and that any multithreading Lisp implementation
worth its salt will create separate copies of the variable a for each
thread or synchronise access automatically.
Duane Rettig
> On 21 Nov, 10:13, p...@informatimago.com (Pascal J. Bourguignon)
> wrote:
>
>
>
>
>
> > Spiros Bousbouras <spi...@gmail.com> writes:
> > > I don't see why macroexpand-all cannot just return the dolist form
> > > itself if dolist is implemented as a special form.
>
> > Because section "3.1.2.1.2.2 Macro Forms" says so.
>
> > DOLIST is specified to be a macro and as mentionned by section
> > "3.1.2.1.2.2 Macro Forms":
>
> > An implementation is free to implement a Common Lisp special
> > operator as a macro. An implementation is free to implement any
> > macro operator as a special operator, but only if an equivalent
> > definition of the macro is also provided.
>
> > it can be implemented as a special operator, however, in that case the
> > implementation must provide a macro definition expanding (eventually)
> > only to standard special operators, for the benefit of code walkers.
>
> But the standard does not mention macroexpand-all so it can behave
> any way it likes. Furthermore it's not unreasonable that it would not
> expand something which the implementation offers as a special form
> even if the standard says is a macro.
You'd have to figure out how to implement this in your macroexpand-all
function. Ultimately, this function will be implemented by calling
either macroexpand or macroexpand-1 at various times. The issue of
wether to macroexpand a special-form is a hard one, and one that
should be examined carefully at a lower level than the level you to
are discussing here - i.e. you are discussing "what" should happen,
when we need to deal with the "why" first wrt special operators.
Special operators are really black-boxes, but the concepts need to be
well-understood before it can be decided what to do with them.
> Or macroexpand-all could be
> designed to take an additional argument by which the programmer can
> switch between "expand everything the standard says is a macro" and
> "expand everything that this implementation actually (only)
> implements as a macro".
>
> But more to the point earlier in the thread Tobias said:
>
> How does MACROEXPAND-ALL help the speed of compilation? Using
> it, the implementation has to walk the syntax tree twice.
>
> Do you know that it can in fact lead to an _impairment_ of
> runtime speed because implementation which would previously
> implement some macros as special forms cannot do so?
>
> Even now as I reread it it seems to be saying that just by making
> available macroexpand-all an implementation is prevented from
> implementing some macros as special forms. But I think what he meant
> was that if a piece of code uses macroexpand-all then when compiling
> this code an implementation can no longer use the special form
> version of a macro (if it has one) and has to use instead the actual
> macro expansion. So yes , I see how macroexpand-all can cause an
> impairment of *runtime* speed even when it is only called during
> compilation. But this could be solved by offering a switch as I
> mention above.
Yes, this comes close to an understanding of why special operators are
identified as such, and why the spec allows implementations to make
special operators out of macros.
The issue is this:
A function is a first-class object that can be identified as the
operator for a function call form. As an implementor, it is easy for
me to get my compiler back-end to grab a call to (foo 10) and make it
something else, performing the functionality of foo without actually
calling foo. This allows for a faster implementation of foo's
functionality.
However, a macro is not a stopping point for macroexpand, and so the
compiler back-end never sees the macro form - it only sees an
implementation of the macro in terms of lower-level (including
recursive) definitions - either function call forms or special forms.
But if a macroexpansion process were to always expand past special
forms, then your lowest level implementation would always and only be
function forms. This is not only inefficient, it is ludicrous. It is
why the spec doesn't require the standard special operators to also
have macro definitions (though Allegro CL does provide macro
definitions for all special operators).
So let's take a specific CASE (pun intended): Consider the following
definition:
(defun foo (x)
(declare (optimize speed (safety 0)))
(case x
((1 3 5 7 9) 'hi)
((2 4 6 8 27) 'bye)
(t 'nope)))
Note that a macroexpand of the case form normally turns into a cond:
CL-USER(1): (pprint (macroexpand '(case x
((1 3 5 7 9) 'hi)
((2 4 6 8 27) 'bye)
(t 'nope))))
(LET ()
(COND ((OR (EQL '1 X) (EQL '3 X) (EQL '5 X) (EQL '7 X) (EQL '9 X))
'HI)
((OR (EQL '2 X) (EQL '4 X) (EQL '6 X) (EQL '8 X) (EQL '27 X))
'BYE)
(T 'NOPE)))
CL-USER(2):
However, if we compile it (this one happens to be on x86-64) we get
something completely different than a cond statement:
CL-USER(2): (compile
(defun foo (x)
(declare (optimize speed (safety 0)))
(case x
((1 3 5 7 9) 'hi)
((2 4 6 8 27) 'bye)
(t 'nope))))
FOO
NIL
NIL
CL-USER(3): (disassemble 'foo)
;; disassembly of #<Function FOO>
;; formals: X
;; constant vector:
0: NOPE
1: HI
2: BYE
;; code start: #x1000994588:
0: 48 c1 cf 03 ror rdi,$3
4: 48 83 ff 1b cmp rdi,$27
8: 0f 87 86 00 00 jnbe 148
00
14: 48 8d 15 0b 00 leaq rdx,[rip+11] ; 32
00 00
21: 48 63 04 ba movslq rax,[rdx+rdi*4+0]
25: 48 03 c2 addq rax,rdx
28: ff e0 jmp *eax
30: 66 90 .align 4
32: 74 00 00 00 .long $116 ; 0: 148
36: 7f 00 00 00 .long $127 ; 1: 159
40: 86 00 00 00 .long $134 ; 2: 166
44: 7f 00 00 00 .long $127 ; 3: 159
48: 86 00 00 00 .long $134 ; 4: 166
52: 7f 00 00 00 .long $127 ; 5: 159
56: 86 00 00 00 .long $134 ; 6: 166
60: 7f 00 00 00 .long $127 ; 7: 159
64: 86 00 00 00 .long $134 ; 8: 166
68: 7f 00 00 00 .long $127 ; 9: 159
72: 74 00 00 00 .long $116 ; 10: 148
76: 74 00 00 00 .long $116 ; 11: 148
80: 74 00 00 00 .long $116 ; 12: 148
84: 74 00 00 00 .long $116 ; 13: 148
88: 74 00 00 00 .long $116 ; 14: 148
92: 74 00 00 00 .long $116 ; 15: 148
96: 74 00 00 00 .long $116 ; 16: 148
100: 74 00 00 00 .long $116 ; 17: 148
104: 74 00 00 00 .long $116 ; 18: 148
108: 74 00 00 00 .long $116 ; 19: 148
112: 74 00 00 00 .long $116 ; 20: 148
116: 74 00 00 00 .long $116 ; 21: 148
120: 74 00 00 00 .long $116 ; 22: 148
124: 74 00 00 00 .long $116 ; 23: 148
128: 74 00 00 00 .long $116 ; 24: 148
132: 74 00 00 00 .long $116 ; 25: 148
136: 74 00 00 00 .long $116 ; 26: 148
140: 86 00 00 00 .long $134 ; 27: 166
144: 00 00 00 00 .long $0
148: 49 8b 7e 36 movq rdi,[r14+54] ; NOPE
152: f8 clc
153: 4c 8b 74 24 10 movq r14,[rsp+16]
158: c3 ret
159: 49 8b 7e 3e movq rdi,[r14+62] ; HI
163: f8 clc
164: eb f3 jmp 153
166: 49 8b 7e 46 movq rdi,[r14+70] ; BYE
170: f8 clc
171: eb ec jmp 153
173: 90 nop
CL-USER(4):
The first question is: if case is a macro, and it expands into a cond
statement, how does the compiler get the macroexpansion to stop when
it sees a special-operator? And isn't this precisely the kind of
thing you're trying to do?
The other question (and it is not rhetorical) is: if you are able to
get macroexpansion to stop at a special operator, what will you do
with the form you get back? There might be a number of things you can
do with it, but since you have now a form that is no longer defined in
terms of lower-level CL operators, but instead in terms of a black box
that only the compiler knows how to implement, what will you do with
the form?
Duane
Pascal Costanza
>> Yes, _multithreading_ is not covered by the standard,
>
> Which brings us back to what I ask above: Is there any established
> way for how such parallel expansion/compilation is supposed to work ?
> To put it otherwise you're saying that my code violates some rules
> but yours doesn't but you're not saying what these rules are , how
> universal they are among implementations and where one could read
> about those rules. And it's not clear what you mean by "side
> effects". Are you saying that things like setf are completely
> forbidden in the code which does the expansion ?
No, I'm saying that side effects are known to be potential sources for
race conditions, while binding constructs are known to avoid race
conditions.
>> but there are
>> numerous multithreaded Common Lisp implementations, so if you want your
>> code to be portable, you better take that into account.
>
> No , if I want my code to be portable I only need to take into
> account the standard and possible bugs/non-conformances in the
> implementations. Portability *and* thread safety on the other hand ,
> is a much more severe requirement.
>
> Having said that it still seems likely to me that my code will work
> everywhere yours will and that any multithreading Lisp implementation
> worth its salt will create separate copies of the variable a for each
> thread or synchronise access automatically.
One of the two options is a bet, the other isn't. I know which of the
two I prefer. Good luck with your code, though.
Spiros Bousbouras
Duane Rettig <du...@franz.com> wrote:
> On Nov 21, 3:24 am, Spiros Bousbouras <spi...@gmail.com> wrote:
>
> > But the standard does not mention macroexpand-all so it can behave
> > any way it likes. Furthermore it's not unreasonable that it would not
> > expand something which the implementation offers as a special form
> > even if the standard says is a macro.
>
> You'd have to figure out how to implement this in your macroexpand-all
> function.
I'm not writing a macroexpand-all function. The possibility occurred to
me and I considered starting a thread on the subject (and I may still
do) but I imagine it cannot be done portably. I was simply saying that
it would be convenient if the standard included a macroexpand-all
function so that every conforming implementation would have it.
> Ultimately, this function will be implemented by calling
> either macroexpand or macroexpand-1 at various times.
If one wants to do it portably then perhaps it's unavoidable that
macroexpand or macroexpand-1 would have to be used. But if the
implementation does it then it doesn't have such a restriction.
> The issue of
> wether to macroexpand a special-form is a hard one, and one that
> should be examined carefully at a lower level than the level you to
> are discussing here - i.e. you are discussing "what" should happen,
> when we need to deal with the "why" first wrt special operators.
In programming one usually starts with what one wants to achieve. I
have explained during the course of the thread why I would want
macroexpand-all but I suspect you haven't read the whole thread.
In my opening post I gave a specific piece of code but I used the name
minimal-compile instead of macroexpand-all because at the time I hadn't
come across the name macroexpand-all. I was informed in this thread
that macroexpand-all is an established (albeit non-standard) name for
the functionality I want and I also came across the information that
SBCL offers macroexpand-all so here's a working piece of code:
; Working but SBCL specific because of sb-cltl2:macroexpand-all and
; (require "sb-cltl2")
(require "sb-cltl2")
(let ((a 0))
(defmacro memory (&rest body)
(prog2
(incf a)
(sb-cltl2:macroexpand-all
`(symbol-macrolet ((cell ,a))
,@body))
(decf a))))
If macroexpand-all expands recursively user defined macros and the
arguments given to macros defined by the implementation but does not
expand the macros provided by the implementation then the above will
still work. So for example if I do
(macroexpand-all
'(memory ...
(when test-form form)))
and I get
(<expansion of memory>
(when <expansion of test-form> <expansion of form>))
instead of
(<expansion of memory>
(<expansion of when> <expansion of test-form> <expansion of form>))
it is still adequate for my purposes. That's why I didn't have to
concern myself with the issue of which standard macros the
implementation implements as special forms.
[...]
> However, a macro is not a stopping point for macroexpand, and so the
> compiler back-end never sees the macro form - it only sees an
> implementation of the macro in terms of lower-level (including
> recursive) definitions - either function call forms or special forms.
> But if a macroexpansion process were to always expand past special
> forms, then your lowest level implementation would always and only be
> function forms. This is not only inefficient, it is ludicrous.
Beyond ludicrous isn't it actually unworkable ? Function forms
evaluate all their arguments but some times you don't want all the
arguments evaluated.
> It is
> why the spec doesn't require the standard special operators to also
> have macro definitions (though Allegro CL does provide macro
> definitions for all special operators).
You mean that if I do
(some-allegro-specific-package:macroexpand-all
'(when test-form form))
I will get something which only has function calls ? I'd love to know
how you manage this.
[...]
> The first question is: if case is a macro, and it expands into a cond
> statement, how does the compiler get the macroexpansion to stop when
> it sees a special-operator? And isn't this precisely the kind of
> thing you're trying to do?
As I've said I'm not actually writing a macroexpand-all function but it
seems to me that this particular issue can be easily solved portably:
(defun macroexpand-all (form)
(if (and (consp form)
(special-operator-p (first form)))
; It's a special operator so it doesn't need to be expanded
...))
> The other question (and it is not rhetorical) is: if you are able to
> get macroexpansion to stop at a special operator, what will you do
> with the form you get back? There might be a number of things you can
> do with it, but since you have now a form that is no longer defined in
> terms of lower-level CL operators, but instead in terms of a black box
> that only the compiler knows how to implement, what will you do with
> the form?
I can think of at least 2 possibilities:
1) Pass the form to the compiler for further compilation. See the example
above.
2) Inspect the form visually for debugging purposes. So let's say for
example that my code defines macros mac1 and mac2 and that the
expansion of mac2 is influenced by whether it is called within the
scope of a mac1 call. In order to verify that the interaction between
mac1 and mac2 works correctly I may want to do something like
(macroexpand-all
'(mac1 (defun name (args) (mac2))))
and verify that I get the kind of expansion that I want. For this it
does not help me in any way if defun itself gets expanded because I
only want to verify that mac1 and mac2 expand to the right thing. On
the contrary the result will probably be clearer if defun does not get
expanded than if it gets expanded to some implementation specific
symbols which would be unfamiliar to me.
So , come to think of it , a macroexpand-all intended as a debugging
tool would be more useful if it accepted a key argument called :test.
This would be a function which accepts 1 argument which should be a
symbol and returns true or false. macroexpand-all would only try to
expand symbols for which the test function returns true (or all the
symbols if no test function was given). So for the example above you
would do
(macroexpand-all
'(mac1 (defun name (args) (mac2)))
:test #'(lambda (item)
(find item '(mac1 mac2))))
--
THE COURT: I forbid her from singing during the trial. I will not
permit singing in this Courtroom.
http://www.law.umkc.edu/faculty/projects/ftrials/Chicago7/Collins.html
Spiros Bousbouras
Pascal Costanza <p...@p-cos.net> posted:
> Spiros Bousbouras wrote:
>
> > Which brings us back to what I ask above: Is there any established
> > way for how such parallel expansion/compilation is supposed to work ?
> > To put it otherwise you're saying that my code violates some rules
> > but yours doesn't but you're not saying what these rules are , how
> > universal they are among implementations and where one could read
> > about those rules. And it's not clear what you mean by "side
> > effects". Are you saying that things like setf are completely
> > forbidden in the code which does the expansion ?
>
> No, I'm saying that side effects are known to be potential sources for
> race conditions, while binding constructs are known to avoid race
> conditions.
But since you're not explaining what "side effects" means in the context
of Lisp you're not really imparting any information.
> > Having said that it still seems likely to me that my code will work
> > everywhere yours will and that any multithreading Lisp implementation
> > worth its salt will create separate copies of the variable a for each
> > thread or synchronise access automatically.
>
> One of the two options is a bet, the other isn't.
Indeed. By using the full power of Lisp for writing macros (or anything
else) I make sure that my programming will be as quick and enjoyable as
possible. On the other hand if I were to adopt your restrictions I
would be reducing speed and enjoyment for uncertain benefit.
> I know which of the
> two I prefer.
Yeah , me too.
> Good luck with your code, though.
Thank you.
Spiros Bousbouras
Spiros Bousbouras <spi...@gmail.com> wrote:
>
> ; Working but SBCL specific because of sb-cltl2:macroexpand-all and
> ; (require "sb-cltl2")
> (require "sb-cltl2")
> (let ((a 0))
> (defmacro memory (&rest body)
> (prog2
> (incf a)
> (sb-cltl2:macroexpand-all
> `(symbol-macrolet ((cell ,a))
> ,@body))
> (decf a))))
>
> If macroexpand-all expands recursively user defined macros and the
> arguments given to macros defined by the implementation but does not
> expand the macros provided by the implementation then the above will
> still work. So for example if I do
>
> (macroexpand-all
> '(memory ...
> (when test-form form)))
>
> and I get
>
> (<expansion of memory>
> (when <expansion of test-form> <expansion of form>))
Perhaps this would work in the case of when but generally it would give
the wrong results since the macro (in place of when) may process the
arguments in a completely different manner than macroexpanding the
arguments would.
> instead of
>
> (<expansion of memory>
> (<expansion of when> <expansion of test-form> <expansion of form>))
>
>
> it is still adequate for my purposes.
[...]
> So , come to think of it , a macroexpand-all intended as a debugging
> tool would be more useful if it accepted a key argument called :test.
> This would be a function which accepts 1 argument which should be a
> symbol and returns true or false. macroexpand-all would only try to
> expand symbols for which the test function returns true (or all the
> symbols if no test function was given).
As much as this would be useful, in view of my comment above, it may
not be possible.