Strange SYMBOL-MACROLET behaviour

[Antsan]

I'm working on SBCL 1.0.57

I've got the following macro for a lazy evaluation library:
(defmacro with-lazy~ (args expr)
`(symbol-macrolet (,@(multiple-value-bind (req opt rest key)
(split-args args)
(declare (ignore rest))
(append (mapcar (lambda (arg)
`(,arg (force~ ,arg)))
(append req opt key))
(when rest
(list `(,rest (mapcar #'force~
,rest)))))))
,expr))
together with the following macros
(defmacro defun~ (name args expr)
`(defun ,name ,args
(with-lazy~ ,args
,expr)))

(defmacro ~defun (name args expr)
`(defun ,name ,args
(~with-lazy ,args
,expr)))
and the definition
(defun~ funcall~ (fn &rest args)
(apply #'funcall
fn args))

When I try to compile the file, I get an error telling me that the control stack is exhausted, probably due to some infinite recursion or yaddayaddayadda.

Now, when I macroexpand-1 that last form I get what I expected:
(defun funcall~ (fn &rest args)
(with-lazy~ (fn &rest args) (apply #'funcall fn args)))
and this expands to
The body of this DEFUN expands into:
(symbol-macrolet ((fn (force~ fn)) (args (mapcar #'force~ args)))
(apply #'funcall fn args))
Now, macroexpanding SYMBOL-MACROLET forms is not doable with any function I know, so I just go by what I know about SYMBOL-MACROLET and expect this to directly macroexpand into
(apply #'funcall (force~ fn) (mapcar #'force~ args))
If that where the case, everything would work out fine, but it doesn't, so that's not what that SYMBOL-MACROLET expands into.
I could imagine that for some reason this would expand into
(symbol-macrolet ((fn (force~ fn)) (args (mapcar #'force~ args)))
(apply #'funcall (force~ fn) (mapcar #'force~ args)))
instead and it would explain the infinite recursion, but that can't be the case because in Let Over Lambda there are a ton of macros exactly like WITH-LAZY~ without any of those problems.
It is also not a problem of the symbol macro `(,arg (mapcar #'force~ ,arg)) as the whole thing doesn't work even if I remove that part from the macro (which would kind of defeat the whole purpose of having the macro in the first place).

Any ideas? Did I make some kind of stupidly obvious mistake?

[Pascal J. Bourguignon]

In any case, ~with-lazy and force~ are not defined.


> […]

> Any ideas? Did I make some kind of stupidly obvious mistake?

Yes. You're expanding a symbol macro to an expression that contains
itself, so it is an infinite expansion.


(ql:quickload :com.informatimago.common-lisp)
(use-package :com.informatimago.common-lisp.lisp-sexp.source-form)

(defun split-args (lambda-list)
(flet ((names (ps) (mapcar (function parameter-name) ps)))
(let ((ll (parse-lambda-list lambda-list)))
(values (names (lambda-list-mandatory-parameters ll))
(names (lambda-list-optional-parameters ll))
(and (lambda-list-rest-parameter-p ll)
(parameter-name (lambda-list-rest-parameter ll)))
(names (lambda-list-keyword-parameters ll))))))

;; (split-args '(a &optional b &rest r &key ((:k kv))))
;; (split-args '())

(defun force~ (x) x)

(defmacro with-lazy~ (args expr)

(multiple-value-bind (req opt rest key) (split-args args)

(let* ((parameter-names (append req opt key (when rest (list rest))))
(parameter-temps (mapcar (lambda (x) (gensym (string x)))
parameter-names)))
`(let ,(mapcar (function list) parameter-temps parameter-names)
(symbol-macrolet (,@(append (mapcar (lambda (arg temp)
`(,arg (force~ ,temp)))
(append req opt key)
(append parameter-temps))
(when rest
(list `(,rest (mapcar #'force~ ,(car (last parameter-temps))))))) )
,expr)))))

(defmacro defun~ (name args expr)
`(defun ,name ,args
(with-lazy~ ,args
,expr)))

(defmacro ~defun (name args expr)
`(defun ,name ,args

(with-lazy~ ,args
,expr)))

(defun~ funcall~ (fn &rest args)
(apply #'funcall
fn args))

;; before:
(pprint (macroexpand '(with-lazy~ (a &optional b &rest r &key ((:k kv)))
(list a b r k))))
(symbol-macrolet ((a (force~ a))
(b (force~ b))
(kv (force~ kv))
(r (mapcar #'force~ r)))
(list a b r k))
t

;; after:
(pprint (macroexpand '(with-lazy~ (a &optional b &rest r &key ((:k kv)))
(list a b r k))))

(let ((#1=#:a68726 a) (#2=#:b68727 b) (#3=#:kv68728 kv) (#4=#:r68729 r))
(symbol-macrolet
((a (force~ #1#)) (b (force~ #2#)) (kv (force~ #3#)) (r (mapcar #'force~ #4#)))
(list a b r k)))
t


--
__Pascal Bourguignon__
http://www.informatimago.com/

[Antsan]

~WITH-LAZY and FORCE~ are not of importance, the same goes for ~DEFUN, no idea
why I posted that here.
FORCE~ is not involved in the macro expansion and works as expected at runtime.

> Yes. You're expanding a symbol macro to an expression that contains
> itself, so it is an infinite expansion.

Huh, why would it do that? Is there any case where that behavior was useful
instead of annoying? I mean, I cannot imagine where you could have any kind of
non-infinite recursion with symbol macros, why make them recursive in the first
place?
I'm not seriously complaining, I'm just curious.

[Antsan]

Thank you. It works this way.

[Pascal J. Bourguignon]

Antsan <thomas.b...@gmail.com> writes:

> ~WITH-LAZY and FORCE~ are not of importance, the same goes for ~DEFUN, no idea
> why I posted that here.

They are not of no importance, if you want US to debug YOUR code!

> FORCE~ is not involved in the macro expansion and works as expected at runtime.
>
>
>> Yes. You're expanding a symbol macro to an expression that contains
>> itself, so it is an infinite expansion.
>
> Huh, why would it do that? Is there any case where that behavior was useful
> instead of annoying? I mean, I cannot imagine where you could have any kind of
> non-infinite recursion with symbol macros, why make them recursive in the first
> place?
> I'm not seriously complaining, I'm just curious.

Well, I gave you the solution.

[Antsan]

Hey, no reason to get angry, I already *knew* that FORCE~ wasn't the problem and
thus omitted it - as I said, it isn't involved in the macro expansion and the
problem couldn't be anywhere but in the macro expansion itself as the bug
happened at compile time. The only way for infinite recursion to happen at compile
time is recursive macro expansion, so why would FORCE~ play any role at all?
Your supplement for FORCE~ worked as fine as the real version.
And I even wrote this reason for why FORCE~ is of no importance into my original
post, so I don't see why you're getting all up in my face as if I had insulted
your mother or something.
~WITH-LAZY is not important because it is only used in ~DEFUN which wasn't even
involved in the bug (although it suffered from the same bug, as both rely on
WITH-LAZY~, but still, there is just no need to know them to understand the
bug).

And I am still curious whether there is some kind of rationale behind the
behavior of SYMBOL-MACROLET.
I am not sure why you're acting so hostile towards me.

I applied your solution and it works like a charm. Thank you for that and I am
sorry I didn't write that before.

[jathd]

Antsan <thomas.b...@gmail.com> writes:

> Huh, why would it do that? Is there any case where that behavior was useful
> instead of annoying? I mean, I cannot imagine where you could have any kind of
> non-infinite recursion with symbol macros, why make them recursive in the first
> place?
> I'm not seriously complaining, I'm just curious.

No one made them specifically recursive. The behaviour you observe comes
from the way macroexpansion works in general: when processing a form for
evaluation (or compilation, or macroexpansion, or...), if it is a macro
invocation, replace it with the corresponding expansion and start the
processing from the start with the new form.

So you would have to actively do something special for symbol macros, as
opposed to regular macros, for them not to be "recursive".

--
jathd ; (format nil "~a...@gmail.com" "jathdr")

[Pascal Costanza]

A good solution is to make the symbol macro expand into a regular macro.

(macrolet ((regular-macro (...) ...))
(symbol-macrolet ((sym (regular-macro)))
...))


Pascal

--
My website: http://p-cos.net
Common Lisp Document Repository: http://cdr.eurolisp.org
Closer to MOP & ContextL: http://common-lisp.net/project/closer/
The views expressed are my own, and not those of my employer.

[Barry Margolin]

In article <m2bo3al...@domain.tld>, jathd <inv...@domain.tld>
wrote:

I think he's suggesting that the CL creators should have done something
special when designing symbol macros. Similar to the way shell aliases
work: if the expansion begins with the alias, it doesn't get
re-expanded, which allows things like:

alias rm='rm -i'

An alternative would be to provide a VARIABLE-VALUE special operator, so
the symbol macro can expand into that (this differs from SYMBOL-VALUE in
that it would work with lexical variables).

--
Barry Margolin, bar...@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***

[Pascal J. Bourguignon]

Pascal Costanza <p...@p-cos.net> writes:

> On 30/09/2013 06:53, jathd wrote:
>> Antsan <thomas.b...@gmail.com> writes:
>>
>>> Huh, why would it do that? Is there any case where that behavior was useful
>>> instead of annoying? I mean, I cannot imagine where you could have any kind of
>>> non-infinite recursion with symbol macros, why make them recursive in the first
>>> place?
>>> I'm not seriously complaining, I'm just curious.
>>
>> No one made them specifically recursive. The behaviour you observe comes
>> from the way macroexpansion works in general: when processing a form for
>> evaluation (or compilation, or macroexpansion, or...), if it is a macro
>> invocation, replace it with the corresponding expansion and start the
>> processing from the start with the new form.
>>
>> So you would have to actively do something special for symbol macros, as
>> opposed to regular macros, for them not to be "recursive".
>
> A good solution is to make the symbol macro expand into a regular macro.
>
> (macrolet ((regular-macro (...) ...))
> (symbol-macrolet ((sym (regular-macro)))
> ...))

That would still fail if the regular macro expansion included in a
macroexpandable place the symbol naming the symbol macrolet.

[Pascal Costanza]

Not necessarily. If you can encode a condition in the lexical
environment that stops the process, then this should be fine (although
weird, of course...)

[Antsan]

Makes sense.
I thought macro expansion worked by repeatedly doing macro expansion on the
source until a fixpoint is reached. This way SYMBOL-MACROLET would be
automatically non-recursive if it was removed by macro expansion.

Something like:
(symbol-macrolet (a (foo a))
a)
macroexpand-1> (foo a)
macroexpand-1> (foo a) ; fixpoint

No special case would be needed there, although I guess that this algorithm for
macro expansion would be slower.