Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

Problem with the lambda form

36 views
Skip to first unread message

Abdullah Abdul Khadir

unread,
Jul 22, 2009, 1:15:36 AM7/22/09
to
Hi,
I tried the following lambda form in common lisp and the ideal
behaviour I expected is as given below
((lambda (f) (f '(b c)))
'(lambda (x) (cons 'a x)))
==> ((lambda (x) (cons 'a x)) '(b c))
==> (cons 'a '(b c))
==> (a b c)

However, I am getting an error. Could someone please help?

--
Regards,

Abdullah Abdul Khadir
------------------------------------------------------------------
Home page : www.cmi.ac.in/~abdullah
My blog : www.linuxandlisp.blogspot.com

Nils M Holm

unread,
Jul 22, 2009, 1:28:40 AM7/22/09
to
Abdullah Abdul Khadir <abdull...@gmail.com> wrote:
> ((lambda (f) (f '(b c)))
> '(lambda (x) (cons 'a x)))
> ==> ((lambda (x) (cons 'a x)) '(b c))
> ==> (cons 'a '(b c))
> ==> (a b c)
>
> However, I am getting an error. Could someone please help?

(lambda (x) (cons 'a x)) is a function, but
'(lambda (x) (cons 'a x)) is a list.

--
Nils M Holm <n m h @ t 3 x . o r g> -- http://t3x.org/nmh/

Abdullah Abdul Khadir

unread,
Jul 22, 2009, 1:50:09 AM7/22/09
to
On Jul 22, 10:28 am, Nils M Holm <news2...@t3x.org> wrote:

So what is the right way of doing it ? What is the right syntax to get
the
lambda to take a function as argument?

milanj

unread,
Jul 22, 2009, 2:25:09 AM7/22/09
to
On Jul 22, 7:50 am, Abdullah Abdul Khadir <abdullah....@gmail.com>

((lambda (f) (funcall f 1 2)) #'(lambda (a b) (+ a b)))

Pascal J. Bourguignon

unread,
Jul 22, 2009, 4:32:06 AM7/22/09
to
milanj <mil...@gmail.com> writes:

Or just:

((lambda (f) (funcall f 1 2)) (lambda (a b) (+ a b)))

Since in Common Lisp, LAMBDA is a macro that expands to an expression
evaluating to a function.

--
__Pascal Bourguignon__

Abdullah Abdul Khadir

unread,
Jul 28, 2009, 9:21:26 PM7/28/09
to
On Jul 22, 1:32 pm, p...@informatimago.com (Pascal J. Bourguignon)
wrote:

> milanj <mil...@gmail.com> writes:
> > On Jul 22, 7:50 am, Abdullah Abdul Khadir <abdullah....@gmail.com>
> > wrote:
> >> On Jul 22, 10:28 am, Nils M Holm <news2...@t3x.org> wrote:
>
> >> > Abdullah Abdul Khadir <abdullah....@gmail.com> wrote:
>
> >> > >      ((lambda(f) (f '(b c)))
> >> > >          '(lambda(x) (cons 'a x)))
> >> > >     ==> ((lambda(x) (cons 'a x)) '(b c))

> >> > >     ==> (cons 'a '(b c))
> >> > >     ==> (a b c)
>
> >> > >       However, I am getting an error. Could someone please help?
>
> >> > (lambda(x) (cons 'a x)) is a function, but
> >> > '(lambda(x) (cons 'a x)) is a list.

>
> >> > --
> >> > Nils M Holm <n m h @ t 3 x . o r g> --http://t3x.org/nmh/
>
> >> So what is the right way of doing it ? What is the right syntax to get
> >> the
> >>lambdato take a function as argument?
>
> > ((lambda(f) (funcall f 1 2)) #'(lambda(a b) (+ a b)))

>
> Or just:
>
>   ((lambda(f) (funcall f 1 2)) (lambda(a b) (+ a b)))

@milanj Thank You
>
> Since in Common Lisp,LAMBDAis a macro that expands to an expression


> evaluating to a function.
>
> --
> __Pascal Bourguignon__

@pascal Thank You too.

Could someone please direct to me a good manual that includes
a good description of lambda and passing lambda forms as arguments
to functions.

w_a_x_man

unread,
Jul 29, 2009, 3:10:26 AM7/29/09
to
On Jul 22, 3:32 am, p...@informatimago.com (Pascal J. Bourguignon)
wrote:

proc{|f| f[1,2]}[ proc{|a,b| a+b} ]

Rainer Joswig

unread,
Jul 29, 2009, 3:13:07 AM7/29/09
to
On 29 Jul., 09:10, weirdo wrote:
>
> proc bla bla
* proc{|f| f[1,2]}[ proc{|a,b| a+b} ]

debugger invoked on a UNBOUND-VARIABLE: The variable |PROC{f| is
unbound.

André Thieme

unread,
Jul 31, 2009, 5:42:04 PM7/31/09
to
w_a_x_man schrieb:

> proc{|f| f[1,2]}[ proc{|a,b| a+b} ]

Uuuh, complicated and full of parens, bah. Why do you work with a lesser
language when you can have Lisp?
(#(% 1 2) #(+ %1 %2))


Andr�
--
Lisp is not dead. It�s just the URL that has changed:
http://clojure.org/

ACL

unread,
Aug 1, 2009, 2:25:17 AM8/1/09
to

Guy (gal?) you have to post an explanation of what it is.

While we're posting completely random code...

((lambda (x)
(list x (list (quote quote) x)))
(quote
(lambda (x)
(list x (list (quote quote) x)))))

Then type (eval *) (eval *) (eval *) in your repl and see what
happens.

http://www.nyx.net/~gthompso/quine.htm

You should write a ruby one and send it to the quine page.
Quines are the best.

Someone should also write a macro that takes ruby code and expands to
common lisp (Like with cl-python). Would be cool. Would also be funny
because then ruby guy would be like *linenoise* and I'd be like, woah,
it runs... That looks nothing like lisp!

vippstar

unread,
Aug 2, 2009, 8:35:39 AM8/2/09
to
On Aug 1, 9:25 am, ACL <anonymous.c.lis...@gmail.com> wrote:
<snip>

> Someone should also write a macro that takes ruby code and expands to
> common lisp (Like with cl-python). Would be cool. Would also be funny
> because then ruby guy would be like *linenoise* and I'd be like, woah,
> it runs... That looks nothing like lisp!

With CL macros you can have any embedded language expanding to lisp
code, but the embedded language can't have a language feature that CL
does not, that's the only limitation. If ruby doesn't have any
features over CL, then he's linenoise already (and I don't care if it
does or not).

Oxide Scrubber

unread,
Aug 2, 2009, 3:10:31 PM8/2/09
to
vippstar wrote:
> On Aug 1, 9:25 am, ACL <anonymous.c.lis...@gmail.com> wrote:
> <snip>
>> Someone should also write a macro that takes ruby code and expands to
>> common lisp (Like with cl-python). Would be cool. Would also be funny
>> because then ruby guy would be like *linenoise* and I'd be like, woah,
>> it runs... That looks nothing like lisp!
>
> With CL macros you can have any embedded language expanding to lisp
> code, but the embedded language can't have a language feature that CL
> does not, that's the only limitation.

Balderdash. Any language feature that can be implemented by a Turing
machine can be included, using a suitable macro to implement it. With
the right set of macros you could probably give your embedded language
first-class continuations, for instance. The macros will essentially
compile it to CL the way GCC compiles C to assembly.

Willem Broekema

unread,
Aug 2, 2009, 5:22:46 PM8/2/09
to
On Aug 2, 9:10 pm, Oxide Scrubber <jharri...@hatlop.de> wrote:

> vippstar wrote:
> > With CL macros you can have any embedded language expanding to lisp
> > code, but the embedded language can't have a language feature that CL
> > does not, that's the only limitation.
>
> Balderdash. Any language feature that can be implemented by a Turing
> machine can be included, using a suitable macro to implement it.

And don't forget reader macros: they gives you _complete_ control over
syntax. Python and Ruby source code can be supported directly, with
the right readtable. CLPython defines a readtable that dispatches
every input character to the Python parser, and Python source files
are then compiled using basically:

(let ((*readtable* *python-parsing-readtable*))
(compile-file "source.py"))


Even more interesting is a mixed-syntax Python/Lisp readtable. It's in
development, here is a preview:

clpython(213): (.parser::enter-mixed-lisp-python-syntax)
; The mixed Lisp/Python syntax mode is now enabled; Lispy *readtable*
is now set.
clpython(214): range(100)[98:2:-2]
#(98 96 94 92 90 88 86 84 82 80 ...)
clpython(215): import os
#<module `os'
:src #P"/System/Library/Frameworks/Python.framework/Versions/2.5/
lib/python2.5/os.py"
:binary #P"/Users/willem/.fasl/allegro-8.1m-macosx-x86/system/
library/frameworks/python.framework/versions/2.5/lib/python2.5/
os.fasl"
:src-time 3400632679 :bin-time 3457618461 @ #x121964e2>
clpython(216): os.urandom(1)
"M"
clpython(217): (funcall ~os.urandom 1)
"¹"
clpython(218): (format nil "OS is a ~A, and its source file is ~A"
(type-of ~os) (module-src-pathname ~os))
"OS is a module, and its source file is /System/Library/Frameworks/
Python.framework/Versions/2.5/lib/python2.5/os.py"


Compiling and running a mixed-syntax file, using the normal compile-
file and load functions:

- - - foo.lispy - - -
(in-package :cl-user)
(clpython.parser::enter-mixed-lisp-python-syntax)

def pyfact(x):
if x <= 1:
return 1
else:
return x * lispfact(x-1)

(defun lispfact (x)
(if (<= x 1)
1
(* x (clpython:py-call ~pyfact (1- x)))))

(trace lispfact)
print pyfact(10)
- - -

clpython(203): :cf
;;; Compiling file foo.lispy
; The mixed Lisp/Python syntax mode is now enabled; Lispy *readtable*
is now set.
;;; Writing fasl file foo.fasl
;;; Fasl write complete
clpython(204): :ld
; Fast loading foo.fasl
0[2]: (lispfact 9)
1[2]: (lispfact 7)
2[2]: (lispfact 5)
3[2]: (lispfact 3)
4[2]: (lispfact 1)
4[2]: returned 1
3[2]: returned 6
2[2]: returned 120
1[2]: returned 5040
0[2]: returned 362880
3628800


> With the right set of macros you could probably give your embedded language
> first-class continuations, for instance.

Good guess... CLPython has a Python CPS convertor, to support
generator expressions.

> The macros will essentially compile it to CL the way GCC compiles C to assembly.

Yeah, Common Lisp makes an excellent implementation language for
Python, and probably Ruby too. It's almost cheating.

- Willem

ACL

unread,
Aug 3, 2009, 2:28:40 AM8/3/09
to

Yeah, that was kind of my point.
Although I disagree with this point:

> but the embedded language can't have a language feature that CL
> does not

You could drop a level and use FFI and macro wrappers to implement
whatever... (Although most things are implementable in CL, if not
always efficiently...)

Nicolas Neuss

unread,
Aug 3, 2009, 6:14:50 AM8/3/09
to
Willem Broekema <meta...@gmail.com> writes:

> [...] Even more interesting is a mixed-syntax Python/Lisp readtable. It's


> in development, here is a preview:
>
> clpython(213): (.parser::enter-mixed-lisp-python-syntax)
> ; The mixed Lisp/Python syntax mode is now enabled; Lispy *readtable*
> is now set.

> [...]


> - - - foo.lispy - - -
> (in-package :cl-user)
> (clpython.parser::enter-mixed-lisp-python-syntax)

> [...]


> Yeah, Common Lisp makes an excellent implementation language for
> Python, and probably Ruby too. It's almost cheating.

Very nice. Now only SLIME would have to be enhanced by suitable local
switching to Python mode:-)

Nicolas

Willem Broekema

unread,
Aug 3, 2009, 6:31:45 AM8/3/09
to
On Aug 3, 12:14 pm, Nicolas Neuss <lastn...@math.uni-karlsruhe.de>
wrote:

> Very nice.  Now only SLIME would have to be enhanced by suitable local
> switching to Python mode:-)

Good integration with SLIME is _another_ ongoing goal. :) A while ago
I found SLIME already had hooks to overrule the default REPL input
handling, which appeared to give enough control to make things work.
At that time SLIME did not keep track of (named) readtables though.

- Willem

Kaz Kylheku

unread,
Aug 4, 2009, 4:48:34 PM8/4/09
to

If it was possible, we Common Lispers would have tripped over each other to
provide such a continuation library long ago.

Lisp macros cannot do anything which cannot be done by a /local/ code rewrite.

If you can't write a local piece of code that will simulate a feature (because
such a piece of code doesn't exist), then you can't make macro to write such a
piece of code, either.

Substitution macros can't do things that affect the semantics of the
surrounding language, like changing the meaning of what it means to call a
function or bind a variable---unless they apply at the top level (the macro
surrounds an entire compilation unit which it gets to transform arbitrarily,
and that compilation unit interoperates seamlessly only with other such
compilation units under the control of the same macro).

Suppose that you are at a point in some function which was reached by a chain
of activations which span multiple translation units. How do you capture all of
those frames in a continuation, to which you can return even if those frames
terminate? What piece of Lisp code do you write to make this continuation?

Show me that piece of code, and then we give it a name, identify the variant
parts, and turn them into macro arguments that get substituted into a template,
etc.

Oxide Scrubber

unread,
Aug 4, 2009, 5:31:29 PM8/4/09
to
Kaz Kylheku wrote:
> On 2009-08-02, Oxide Scrubber <jhar...@hatlop.de> wrote:
>> vippstar wrote:
>>> On Aug 1, 9:25 am, ACL <anonymous.c.lis...@gmail.com> wrote:
>>> <snip>
>>>> Someone should also write a macro that takes ruby code and expands to
>>>> common lisp (Like with cl-python). Would be cool. Would also be funny
>>>> because then ruby guy would be like *linenoise* and I'd be like, woah,
>>>> it runs... That looks nothing like lisp!
>>> With CL macros you can have any embedded language expanding to lisp
>>> code, but the embedded language can't have a language feature that CL
>>> does not, that's the only limitation.
>> Balderdash. Any language feature that can be implemented by a Turing
>> machine can be included, using a suitable macro to implement it. With
>> the right set of macros you could probably give your embedded language
>> first-class continuations, for instance.
>
> If it was possible, we Common Lispers would have tripped over each other to
> provide such a continuation library long ago. [rest of argument deleted]

The same argument "proves" that CLOS multimethods are impossible to
implement as a library, yet CLOS began as exactly that -- a library. It
follows that your argument is invalid in some way.

Of course, functions with continuations would have to be def'd specially
(just as multimethods are -- defmethod instead of defun). Say, with a
macro named defun-c that used macrolet to define a
continuation-returning macro within the body. The body would probably
end up turned into several lambdas.

I don't know how to do the details of the implementation, but I do
suspect that it is possible.

Ron Garret

unread,
Aug 4, 2009, 8:54:17 PM8/4/09
to
In article <20090814...@gmail.com>,
Kaz Kylheku <kkyl...@gmail.com> wrote:

> On 2009-08-02, Oxide Scrubber <jhar...@hatlop.de> wrote:
> > vippstar wrote:
> >> On Aug 1, 9:25 am, ACL <anonymous.c.lis...@gmail.com> wrote:
> >> <snip>
> >>> Someone should also write a macro that takes ruby code and expands to
> >>> common lisp (Like with cl-python). Would be cool. Would also be funny
> >>> because then ruby guy would be like *linenoise* and I'd be like, woah,
> >>> it runs... That looks nothing like lisp!
> >>
> >> With CL macros you can have any embedded language expanding to lisp
> >> code, but the embedded language can't have a language feature that CL
> >> does not, that's the only limitation.
> >
> > Balderdash. Any language feature that can be implemented by a Turing
> > machine can be included, using a suitable macro to implement it. With
> > the right set of macros you could probably give your embedded language
> > first-class continuations, for instance.
>
> If it was possible, we Common Lispers would have tripped over each other to
> provide such a continuation library long ago.
>
> Lisp macros cannot do anything which cannot be done by a /local/ code
> rewrite.

That's not true. Macros can have arbitrary side-effects.

rg

Paul Donnelly

unread,
Aug 4, 2009, 10:38:44 PM8/4/09
to
Ron Garret <rNOS...@flownet.com> writes:

So can rewritten local code, for that matter. ;)

Kaz Kylheku

unread,
Aug 4, 2009, 11:27:29 PM8/4/09
to

Yep; arbitrary side effects. For all values of ``arbitrary'' other than ``what
you need in order to implement continuations''.

gugamilare

unread,
Aug 5, 2009, 12:52:18 AM8/5/09
to
On 5 ago, 00:27, Kaz Kylheku <kkylh...@gmail.com> wrote:
> Yep; arbitrary side effects. For all values of ``arbitrary'' other than ``what
> you need in order to implement continuations''.

http://common-lisp.net/project/cl-cont/

The only problem encountered is portability, not possibility:

"Because compilers often expand standard macros into non-standard CL
code, supporting all special operators is not sufficient to support
all of Common Lisp."

[...]

"These limitations will be addressed as the need arises."

Pascal Costanza

unread,
Aug 5, 2009, 1:26:49 AM8/5/09
to

There are language extensions which cannot be implemented by way of
macros, because they require that a program is _systematically_ modified
throughout, including the runtime libraries (for example).

Pascal

--
My website: http://p-cos.net
Common Lisp Document Repository: http://cdr.eurolisp.org
Closer to MOP & ContextL: http://common-lisp.net/project/closer/

Pascal Costanza

unread,
Aug 5, 2009, 1:28:16 AM8/5/09
to

...but that's not very useful, since there is no guarantee whether and
how often a macro is expanded by an interpreter or compiler, so in
practice, macros don't (or shouldn't) use side effects.

Oxide Scrubber

unread,
Aug 5, 2009, 2:13:21 AM8/5/09
to

What about idempotent side effects? The original defmethod presumably
did this, adding or replacing a method dispatch so the same defmethod
side effect occurring multiple times resulting in a dispatch table with
the same semantics as if it had occurred once, or indeed any different
number of times.

Oxide Scrubber

unread,
Aug 5, 2009, 2:17:03 AM8/5/09
to
Pascal Costanza wrote:
> Oxide Scrubber wrote:
>> vippstar wrote:
>>> On Aug 1, 9:25 am, ACL <anonymous.c.lis...@gmail.com> wrote:
>>> <snip>
>>>> Someone should also write a macro that takes ruby code and expands to
>>>> common lisp (Like with cl-python). Would be cool. Would also be funny
>>>> because then ruby guy would be like *linenoise* and I'd be like, woah,
>>>> it runs... That looks nothing like lisp!
>>>
>>> With CL macros you can have any embedded language expanding to lisp
>>> code, but the embedded language can't have a language feature that CL
>>> does not, that's the only limitation.
>>
>> Balderdash. Any language feature that can be implemented by a Turing
>> machine can be included, using a suitable macro to implement it. With
>> the right set of macros you could probably give your embedded language
>> first-class continuations, for instance. The macros will essentially
>> compile it to CL the way GCC compiles C to assembly.
>
> There are language extensions which cannot be implemented by way of
> macros, because they require that a program is _systematically_ modified
> throughout, including the runtime libraries (for example).

Things like adding a new kind of foreign function interface, maybe.

Or maybe not. Suppose you wanted to add Pascal FFI to a CL that came
with C FFI only. Your macro would parse declarations from the target
Pascal source code, or some other data, and generate C code for a
bridge, invoke gcc, set a flag (in case the same Pascal bindings got hit
again, there's no point generating the same bridge DLL/.so twice), and
expand to an appropriate C call. :)

(Actually, you'd want it generating the bridge binary only at
development time; you'd ship that with your program binaries or even
source, avoiding an unwanted dependence on gcc.)

Oxide Scrubber

unread,
Aug 5, 2009, 2:19:05 AM8/5/09
to

Oh, and as Paul Donnelly pointed out, the expansion can have side
effects, and has full access to the run-time environment, so perhaps can
act with greater knowledge. And there is no ill-definedness to when or
how often those side effects occur: whenever the code is executed at
runtime, and whenever it wants to (given suitable (if ...) or (cond ...)
or whatever around these side effects).

Pascal Costanza

unread,
Aug 5, 2009, 2:20:42 AM8/5/09
to

What do you mean by "the original defmethod"?

You only need a dispatch table at runtime, so what does this have to do
with side effects performed by the macro?

Pascal Costanza

unread,
Aug 5, 2009, 2:21:08 AM8/5/09
to
Oxide Scrubber wrote:
> Pascal Costanza wrote:
>> Oxide Scrubber wrote:
>>> vippstar wrote:
>>>> On Aug 1, 9:25 am, ACL <anonymous.c.lis...@gmail.com> wrote:
>>>> <snip>
>>>>> Someone should also write a macro that takes ruby code and expands to
>>>>> common lisp (Like with cl-python). Would be cool. Would also be funny
>>>>> because then ruby guy would be like *linenoise* and I'd be like, woah,
>>>>> it runs... That looks nothing like lisp!
>>>>
>>>> With CL macros you can have any embedded language expanding to lisp
>>>> code, but the embedded language can't have a language feature that CL
>>>> does not, that's the only limitation.
>>>
>>> Balderdash. Any language feature that can be implemented by a Turing
>>> machine can be included, using a suitable macro to implement it. With
>>> the right set of macros you could probably give your embedded
>>> language first-class continuations, for instance. The macros will
>>> essentially compile it to CL the way GCC compiles C to assembly.
>>
>> There are language extensions which cannot be implemented by way of
>> macros, because they require that a program is _systematically_
>> modified throughout, including the runtime libraries (for example).
>
> Things like adding a new kind of foreign function interface, maybe.

No. Things like call/cc, or concurrency, etc.

Pascal Costanza

unread,
Aug 5, 2009, 2:24:13 AM8/5/09
to

Oxide Scrubber

unread,
Aug 5, 2009, 2:27:18 AM8/5/09
to
Pascal Costanza wrote:
> Oxide Scrubber wrote:
>> Oxide Scrubber wrote:
>>> Pascal Costanza wrote:
>>>> Ron Garret wrote:
>>>>> That's not true. Macros can have arbitrary side-effects.
>>>>
>>>> ...but that's not very useful, since there is no guarantee whether
>>>> and how often a macro is expanded by an interpreter or compiler, so
>>>> in practice, macros don't (or shouldn't) use side effects.
>>>
>>> What about idempotent side effects? The original defmethod presumably
>>> did this, adding or replacing a method dispatch so the same defmethod
>>> side effect occurring multiple times resulting in a dispatch table
>>> with the same semantics as if it had occurred once, or indeed any
>>> different number of times.
>>
>> Oh, and as Paul Donnelly pointed out, the expansion can have side
>> effects, and has full access to the run-time environment, so perhaps
>> can act with greater knowledge. And there is no ill-definedness to
>> when or how often those side effects occur: whenever the code is
>> executed at runtime, and whenever it wants to (given suitable (if ...)
>> or (cond ...) or whatever around these side effects).
>
> No, really not.

Yes, really.

Oxide Scrubber

unread,
Aug 5, 2009, 2:28:06 AM8/5/09
to

When CLOS was first implemented as a library in a CLOS-free early
version of CL.

> You only need a dispatch table at runtime, so what does this have to do
> with side effects performed by the macro?

Actually, they are probably (idempotent) side effects performed by the
macro's expansion; see the other recent posts.

Oxide Scrubber

unread,
Aug 5, 2009, 2:35:30 AM8/5/09
to

Yes.

You have some kind of obsession, don't you? I posted three posts to this
argument and I found three replies by you as soon as I simply reloaded
everything to mark my own posts read. You'd gotten to each one and
mindlessly attacked it within five minutes of my posting it. That speaks
to a seriously compulsive mindset, combined with some very compulsive
clicking of "refresh" over and over again waiting for something new to
pounce on, instead of saying your piece once, marking your own posts
read, signing off, and finding something constructive to do for the rest
of the day.

> Things like call/cc, or concurrency, etc.

Just as an example:

Concurrency could be added, in a low-level form, using C
foreign-function interface and pthreads (or whatever).

Higher level stuff could be layered on.

The low-level stuff you'd need, and could get through FFI, would be:
create, join, otherwise modify threads; and at least one of atomic
integer, semaphore, mutex lock/unlock, with at least one operation that
established a memory barrier on multicore machines.

I assume many CL implementations already have concurrency. I believe one
is JVM-hosted; if it has Java FFI it has fairly easy access to
concurrency. If a macro can condition on which CL implementation it's
running in (as even C's lousy macros can) it is possible to make a
concurrency library for CL that translates to the vendor-specific
concurrency functions and is itself uniform in its API.

Now quit arguing with me. This debate is over. Any plausible
continuation consists of us either repeating ourselves, or (worse)
stooping to slinging insults at each other.

Pascal Costanza

unread,
Aug 5, 2009, 3:56:08 AM8/5/09
to

Yes, macros can expand into code that performs side effects in a
predictable way. No, you still cannot implement "any language feature"
because of that.

Pascal Costanza

unread,
Aug 5, 2009, 3:59:25 AM8/5/09
to

I don't think that was ever the case. A CLOS implementation requires a
notion of funcallable objects, which didn't exist in the original Common
Lisp specification.

You can implement CLOS as a plain library, but that has drawbacks if you
don't have funcallable objects. (The other option would be to add weak
pointers / weak hashtables, which also cannot be added by way of macrology.)

Pascal Costanza

unread,
Aug 5, 2009, 4:04:20 AM8/5/09
to

I will ignore this rant because it doesn't add anything to the discussion.

>> Things like call/cc, or concurrency, etc.
>
> Just as an example:
>
> Concurrency could be added, in a low-level form, using C
> foreign-function interface and pthreads (or whatever).

That's not adding a new language construct, that is reusing the
capabilities of another language (and it's actually not that simple either).

Oxide Scrubber

unread,
Aug 5, 2009, 10:21:42 AM8/5/09
to
Pascal Costanza wrote:
> Oxide Scrubber wrote:
>> Pascal Costanza wrote:
>>> Oxide Scrubber wrote:
>>>> Oh, and as Paul Donnelly pointed out, the expansion can have side
>>>> effects, and has full access to the run-time environment, so perhaps
>>>> can act with greater knowledge. And there is no ill-definedness to
>>>> when or how often those side effects occur: whenever the code is
>>>> executed at runtime, and whenever it wants to (given suitable (if
>>>> ...) or (cond ...) or whatever around these side effects).
>>>
>>> No, really not.
>>
>> Yes, really.
>
> Yes, macros can expand into code that performs side effects in a
> predictable way.

Well, that's that then.

Oxide Scrubber

unread,
Aug 5, 2009, 10:23:05 AM8/5/09
to
Pascal Costanza wrote:
> Oxide Scrubber wrote:
>> Pascal Costanza wrote:
>>> Oxide Scrubber wrote:
>>>> What about idempotent side effects? The original defmethod
>>>> presumably did this, adding or replacing a method dispatch so the
>>>> same defmethod side effect occurring multiple times resulting in a
>>>> dispatch table with the same semantics as if it had occurred once,
>>>> or indeed any different number of times.
>>>
>>> What do you mean by "the original defmethod"?
>>
>> When CLOS was first implemented as a library in a CLOS-free early
>> version of CL.
>
> I don't think that was ever the case.

But it was.

> A CLOS implementation requires a notion of funcallable objects, which
> didn't exist in the original Common Lisp specification.

Yet somehow they got by.

> You can implement CLOS as a plain library

See? Told you.

Oxide Scrubber

unread,
Aug 5, 2009, 10:24:31 AM8/5/09
to
Pascal Costanza wrote:
> Oxide Scrubber wrote:
>> Pascal Costanza wrote:
>>> Oxide Scrubber wrote:
>>>> Pascal Costanza wrote:
>>>>> There are language extensions which cannot be implemented by way of
>>>>> macros, because they require that a program is _systematically_
>>>>> modified throughout, including the runtime libraries (for example).
>>>>
>>>> Things like adding a new kind of foreign function interface, maybe.
>>>
>>> No.
>>
>> Yes.
>>
>> You have some kind of obsession, don't you? I posted three posts to
>> this argument and I found three replies by you as soon as I simply
>> reloaded everything to mark my own posts read. You'd gotten to each
>> one and mindlessly attacked it within five minutes of my posting it.
>> That speaks to a seriously compulsive mindset, combined with some very
>> compulsive clicking of "refresh" over and over again waiting for
>> something new to pounce on, instead of saying your piece once, marking
>> your own posts read, signing off, and finding something constructive
>> to do for the rest of the day.

This time you managed to wait a whole hour and a half before you
couldn't hold it in any longer and had to jizz all over the newsgroup.

Maybe try for three hours next time?

>>> Things like call/cc, or concurrency, etc.
>>
>> Just as an example:
>>
>> Concurrency could be added, in a low-level form, using C
>> foreign-function interface and pthreads (or whatever).
>
> That's not

Sure that is.

Pascal Costanza

unread,
Aug 5, 2009, 11:18:14 AM8/5/09
to

You're a funny guy.

Ron Garret

unread,
Aug 5, 2009, 12:07:14 PM8/5/09
to
In article <7dsjjgF...@mid.individual.net>,
Pascal Costanza <p...@p-cos.net> wrote:

Of course there is a guarantee: a macro is guaranteed to be expanded at
least once before the code is executed. Granted, it's a PITA to use
this guarantee to produce reliable, portable, useful behavior, but it's
not impossible. And if you stick with one implementation and don't care
about portability then it can get much easier.

rg

Ron Garret

unread,
Aug 5, 2009, 12:14:40 PM8/5/09
to
In article <7dss8oF...@mid.individual.net>,
Pascal Costanza <p...@p-cos.net> wrote:

Even if that were true (and it's not) that's still not the same as the
original claim that "Lisp macros cannot do anything which cannot be done

by a /local/ code rewrite."

Lisp macros are Turing-complete, and they have access to the full Lisp
environment. If there was something they couldn't do that would be
quite the remarkable result.

rg

Ron Garret

unread,
Aug 5, 2009, 12:16:32 PM8/5/09
to
In article <87tz0n3...@plap.localdomain>,
Paul Donnelly <paul-d...@sbcglobal.net> wrote:

No, it can't. That's what "local" means.

rg

Kaz Kylheku

unread,
Aug 5, 2009, 12:24:55 PM8/5/09
to

DEFMETHOD merely generates code which does the above; it doesn't have that
effect itself. The output of DEFMETHOD may be compiled into a file such that
side effects you're talking about take place when that file is loaded.

Ron Garret

unread,
Aug 5, 2009, 12:27:35 PM8/5/09
to
In article <200908150...@gmail.com>,
Kaz Kylheku <kkyl...@gmail.com> wrote:

The difficulty of implementing call/cc in CL is not any limitation on
what macros can do, it's deciding on the semantics of call/cc in the
face of CL's other built-in control constructs. But in principle,
adding call/cc to CL is straightforward: just intercept all the defining
forms and compile then using your own compiler that makes the call stack
explicitly accessible. Actually doing this is of course far from an
elementary exercise, but it's not only possible, it's actually well
understood (except of course by people who don't understand these
things).

It is true that you can't add call/cc to CL simply by defining a CALL/CC
macro. Maybe that's what you meant?

rg

Ron Garret

unread,
Aug 5, 2009, 12:33:55 PM8/5/09
to
In article <7dsjgpF...@mid.individual.net>,
Pascal Costanza <p...@p-cos.net> wrote:

> Oxide Scrubber wrote:
> > vippstar wrote:
> >> On Aug 1, 9:25 am, ACL <anonymous.c.lis...@gmail.com> wrote:
> >> <snip>
> >>> Someone should also write a macro that takes ruby code and expands to
> >>> common lisp (Like with cl-python). Would be cool. Would also be funny
> >>> because then ruby guy would be like *linenoise* and I'd be like, woah,
> >>> it runs... That looks nothing like lisp!
> >>
> >> With CL macros you can have any embedded language expanding to lisp
> >> code, but the embedded language can't have a language feature that CL
> >> does not, that's the only limitation.
> >
> > Balderdash. Any language feature that can be implemented by a Turing
> > machine can be included, using a suitable macro to implement it. With
> > the right set of macros you could probably give your embedded language
> > first-class continuations, for instance. The macros will essentially
> > compile it to CL the way GCC compiles C to assembly.
>
> There are language extensions which cannot be implemented by way of
> macros, because they require that a program is _systematically_ modified
> throughout, including the runtime libraries (for example).

Yeah? So? Why is that a problem? Recompiling the runtime libraries
may be inconvenient, but it's certainly possible.

rg

Kaz Kylheku

unread,
Aug 5, 2009, 12:43:25 PM8/5/09
to
On 2009-08-05, Oxide Scrubber <jhar...@hatlop.de> wrote:
> Pascal Costanza wrote:
>> Oxide Scrubber wrote:
>>> Pascal Costanza wrote:
>>>> Oxide Scrubber wrote:
>>>>> What about idempotent side effects? The original defmethod
>>>>> presumably did this, adding or replacing a method dispatch so the
>>>>> same defmethod side effect occurring multiple times resulting in a
>>>>> dispatch table with the same semantics as if it had occurred once,
>>>>> or indeed any different number of times.
>>>>
>>>> What do you mean by "the original defmethod"?
>>>
>>> When CLOS was first implemented as a library in a CLOS-free early
>>> version of CL.
>>
>> I don't think that was ever the case.
>
> But it was.
>
>> A CLOS implementation requires a notion of funcallable objects, which
>> didn't exist in the original Common Lisp specification.
>
> Yet somehow they got by.

In CLOS, the type of a generic function isn't FUNCTION. See for yourself:
make a generic function with DEFGENERIC and then apply type-of:

(defgeneric foo (x))

(type-of #'foo) -> ???

How do you make this work with only macros in a Common Lisp in which
there is only a function type?

>> You can implement CLOS as a plain library
>
> See? Told you.

You can implement the ancient predecessor to CLOS as a library, but how about a
conforming implementation of modern CLOS?

If we lift the conformance requirement, then you can implement anything and
call it CLOS.

Just like if we loosen our definition of words like ``continuation'' and
``embedded language'' we can implement some half-baked hack which is nothing
like real continuations in Scheme.

Kaz Kylheku

unread,
Aug 5, 2009, 12:53:25 PM8/5/09
to
On 2009-08-05, Oxide Scrubber <jhar...@hatlop.de> wrote:
> Concurrency could be added, in a low-level form, using C
> foreign-function interface and pthreads (or whatever).

Adding concurrency is more than just reaching the entry points for
pthread_create and others.

> Higher level stuff could be layered on.

Higher level stuff?

What about lower level stuff such as, oh, a thread-safe evaluator, thread-aware
garbage collector, thread-safe code generated by the compiler, all library
functions being thread-safe (including ones that have global effecst like DEFUN
and LOAD). Thread-local bindings for dynamic variables. Etc.

Show me the code that a Common Lisp macro can write to make the underlying
garbage collector thread aware.

> The low-level stuff you'd need, and could get through FFI, would be:
> create, join, otherwise modify threads; and at least one of atomic
> integer, semaphore, mutex lock/unlock, with at least one operation that
> established a memory barrier on multicore machines.

FFI to reach atomic operations is a bad idea. Firstly, there is no
standardized library for these operations in C. They are usually
macros that expand to inline assembly, useable from a C compiler.
You'd similarly need these primitives in your Lisp compiler somehow.

You're assuming that you know all the low level stuff you'd need.

> I assume many CL implementations already have concurrency. I believe one
> is JVM-hosted

Irrelevant. The concurrency comes from the JVM, not from Lisp macros
written over a Lisp implementation that itself has no concurency.

Kaz Kylheku

unread,
Aug 5, 2009, 1:06:19 PM8/5/09
to

But no matter what semantics you decide on, there is also the difficulty
of deciding what piece of code will replace call/cc to make that semantics
happen. No reasonable semantics that you can assign to call/cc can
be implemented as a chunk of code that simply replaces it.

> But in principle,
> adding call/cc to CL is straightforward: just intercept all the defining
> forms and compile then using your own compiler that makes the call stack
> explicitly accessible.

Of course if we replace COMPILE, EVAL, COMPILE-FILE and LOAD, we can
do anything. Problem is, we also may have to replace most, if not all, of the
run-time library too. Code compiled by the compiler wants to call standard
functions and operators, and some of them, left as they are, are going to be
broken with regard to continuations. Those existing functions were,
after all, not processed by your custom defining forms! The garbage collector
may have to be tweaked. And so it goes.

Sure provide your own compiler, evaluator, loader and run-time library,
and you can accomplish anything.

But you're no longer making an /embedded/ macro-based language.

The claim being disputed is that continuations can exist simply as
another macro syntax embedded in normal Lisp code.

> Actually doing this is of course far from an
> elementary exercise, but it's not only possible, it's actually well
> understood (except of course by people who don't understand these
> things).

Like Oxide whatshisface, for one.

Ron Garret

unread,
Aug 5, 2009, 1:40:01 PM8/5/09
to
In article <200908151...@gmail.com>,
Kaz Kylheku <kkyl...@gmail.com> wrote:

That's true. But that's different than "Lisp macros cannot do anything

which cannot be done by a /local/ code rewrite."

> > But in principle,

> > adding call/cc to CL is straightforward: just intercept all the defining
> > forms and compile then using your own compiler that makes the call stack
> > explicitly accessible.
>
> Of course if we replace COMPILE, EVAL, COMPILE-FILE and LOAD, we can
> do anything.

Exactly.

> Problem is, we also may have to replace most, if not all, of
> the run-time library too.

Not replace. Just recompile. Very different proposition.

> Sure provide your own compiler, evaluator, loader and run-time library,
> and you can accomplish anything.

Yes, but one of the beauties of CL is that providing your own compiler
is not nearly the challenge it would be in other languages because the
target language can be Common Lisp (sans call/cc of course). Not a
trivial exercise to be sure, but certainly doable.

>
> But you're no longer making an /embedded/ macro-based language.

So?

>
> The claim being disputed is that continuations can exist simply as
> another macro syntax embedded in normal Lisp code.

That depends a great deal on exactly how one defines one's terms. IMHO,
adding call/cc would be an ambitious project, but well within the scope
of what CL purports to be useful for, at least if you buy into John
Foderaro's "programmable programming language" idea. (Whether the
payoff would be worth the effort is, of course, an entirely different
question.)

rg

Ron Garret

unread,
Aug 5, 2009, 1:49:35 PM8/5/09
to
In article <200908151...@gmail.com>,
Kaz Kylheku <kkyl...@gmail.com> wrote:

Are you kidding? That's not even hard.


(shadow '(defgeneric type-of))

(defvar *generic-functions* nil)

(defmacro defgeneric (name &rest args)
`(progn
(defun ,name ,(... args ...))
(push (symbol-function ',name) *generic-functions*)
',name))

(defun type-of (thing)
(if (member thing *generic-functions*)
'generic-function
(cl:type-of thing)))

This approach of using CLOS as an example of something that can't be
layered on top of CL is a sure loser because that is in fact how CLOS
was originally implemented. (Surely you knew that?)

rg

Pascal Costanza

unread,
Aug 5, 2009, 1:59:06 PM8/5/09
to

Sure, they are Turing-complete and thus can compute anything they want
at macroexpansion-time. But that's not what is usually meant when
somebody says that "a macro implements something." What you're usually
interested in is the output of the macro expansion, and no matter how
arbitrary the code is that you expand into, it is restricted by the very
fact that the resulting code is only inserted locally, and cannot touch
any other parts of a program.

There are, however, language extensions that can only be implemented by
inserting code systematically in various places of a program. The only
way to do that, if the language doesn't provide the language construct
already, is to insert such code systematically in those places by hand.
That's when you start to call something an idiom, pattern, or style.

For example, the only way to add first-class continuations to a language
without support for continuations is to systematically provide those
continuations by hand, for example by writing your code in
continuation-passing style, or simulating continuations by using
trampolines, etc.

Macros help you very little in such examples,except maybe for easing the
use of idioms/patterns/style a little bit, but you would still have to
take care of the various places you have to modify yourself.

Pascal Costanza

unread,
Aug 5, 2009, 2:02:14 PM8/5/09
to

Here is an example where a macro invocation is not expanded:

(defmacro foo (...) ...)
(defmacro bar (...) ...)

(defmacro baz (form1 form2)
(if (some-condition)
form1
form2))

Example:

(baz (foo ...) (bar ...))

=> Either the invocation of foo or bar will not be expanded.


FWIW, here is an even simpler example:

(if (some-condition)
(foo ...)
(bar ...))

If a compiler can do some complex constant propagation, it may be able
to figure that the invocation of some-condition is actually a constant,
so can determine that one of the two branches is dead code and avoid
expanding it.

Pascal Costanza

unread,
Aug 5, 2009, 2:03:05 PM8/5/09
to

Maybe the runtime libraries, or some third-party libraries you need for
your extensions, is not available as source code, but only in
precompiled form.

Ron Garret

unread,
Aug 5, 2009, 4:00:51 PM8/5/09
to
In article <7dtvqpF...@mid.individual.net>,
Pascal Costanza <p...@p-cos.net> wrote:

Maybe. Why focus on worst-case scenarios? Current reality is that a
vast array of useful libraries is available in source form, and the few
commercial vendors that are left tend to be quite attentive to their
customers' needs (since they have so few of them).

rg

Ron Garret

unread,
Aug 5, 2009, 4:04:42 PM8/5/09
to
In article <7dtvp6F...@mid.individual.net>,
Pascal Costanza <p...@p-cos.net> wrote:

[snip]

Non-sequitur. I didn't say that a macro is guaranteed to be expanded.
I said that a macro is guaranteed to be expanded AT LEAST ONCE BEFORE IT
IS EXECUTED. Obviously if it is never executed it may never be
expanded. But if it is never executed I can't imagine any realistic
circumstances under which one would care.

rg

Ron Garret

unread,
Aug 5, 2009, 4:32:24 PM8/5/09
to
On Aug 5, 10:59 am, Pascal Costanza <p...@p-cos.net> wrote:
> Ron Garret wrote:
> > In article <7dss8oF2dcu3...@mid.individual.net>,

What a colossal failure of imagination.

Everything you say is true. There is a very common (one might even
say overwhelmingly popular) coding style that uses macros in exactly
the way you describe and has all the drawbacks, constraints, and
restrictions that you say. But nowhere is it written that one must
confine themselves to this style. There are infinite possibilities
beyond that if you are willing to think outside the box even just a
little bit. For example, consider the following code:

(((λ (f) ((λ (g) (g g)) (λ (h) (λ (x) ((f (h h)) x)))))
(λ (f) (λ (n) (if (zerop n) 1 (* n (f (1- n)))))))
15)

There is a very strong argument to be made that it is "impossible" to
munge CL so that this code does anything other than generate an
error. And yet, here is a transcript of an actual CL implementation
executing this code and not generating an error:

? (((λ (f) ((λ (g) (g g)) (λ (h) (λ (x) ((f (h h)) x)))))
(λ (f) (λ (n) (if (zerop n) 1 (* n (f (1- n)))))))
15)
1307674368000
?

And just for the record:

? ((λ (f) ((λ (g) (g g)) (λ (h) (λ (x) ((f (h h)) x)))))
(λ (f) (λ (n) (if (zerop n) 1 (* n (f (1- n)))))))
#<COMPILED-LEXICAL-CLOSURE #x3000420A0FAF>
? (funcall * 15)
1307674368000

And no, I did not have to rewrite the compiler from scratch.

(BTW, I'm posting this through Google Groups because for some reason
the MT-Newswatcher that I usually use is for some reason not handling
UTF-8 encoding properly. If you see funny characters above, those are
supposed to be lower-case Greek lambdas.)

rg

Vassil Nikolov

unread,
Aug 6, 2009, 1:09:23 AM8/6/09
to

On Wed, 5 Aug 2009 13:32:24 -0700 (PDT), Ron Garret <ron.g...@gmail.com> said:
> ...

> There is a very strong argument to be made that it is "impossible" to
> munge CL so that this code does anything other than generate an
> error. And yet, here is a transcript of an actual CL implementation
> executing this code and not generating an error:

> ? (((λ (f) ((λ (g) (g g)) (λ (h) (λ (x) ((f (h h)) x)))))
> (λ (f) (λ (n) (if (zerop n) 1 (* n (f (1- n)))))))
> 15)
> 1307674368000

What does

(eval (LIST (LIST (LIST 'λ (LIST 'F) (LIST (LIST 'λ
(LIST 'G) (LIST 'G 'G)) (LIST 'λ (LIST 'H)
(LIST 'λ (LIST 'X) (LIST (LIST 'F (LIST 'H 'H)) 'X)))))
(LIST 'λ (LIST 'F) (LIST 'λ (LIST 'N)
(LIST 'IF (LIST 'ZEROP 'N) '1
(LIST '* 'N (LIST 'F (LIST '1- 'N))))))) '15))

return?

---Vassil.


--
"Even when the muse is posting on Usenet, Alexander Sergeevich?"

Pascal J. Bourguignon

unread,
Aug 6, 2009, 2:52:44 AM8/6/09
to
Vassil Nikolov <vnik...@pobox.com> writes:

Nothing, λ is not defined.
Nothing, since in CL, the operator can only be a symbol, or a lambda form.

Nonetheless, it is possible to munge CL so that it works.
For the former expression, by writting a REPL that interprets that new language,
for the eval, by shadowing it and redefining it to interpret that new language.

--
__Pascal Bourguignon__

Tobias C. Rittweiler

unread,
Aug 6, 2009, 2:55:39 AM8/6/09
to
Ron Garret <...> writes:

> Everything you say is true. There is a very common (one might even
> say overwhelmingly popular) coding style that uses macros in exactly
> the way you describe and has all the drawbacks, constraints, and
> restrictions that you say. But nowhere is it written that one must
> confine themselves to this style. There are infinite possibilities
> beyond that if you are willing to think outside the box even just a
> little bit. For example, consider the following code:
>
> (((λ (f) ((λ (g) (g g)) (λ (h) (λ (x) ((f (h h)) x)))))
> (λ (f) (λ (n) (if (zerop n) 1 (* n (f (1- n)))))))
> 15)
>
> There is a very strong argument to be made that it is "impossible" to
> munge CL so that this code does anything other than generate an
> error. And yet, here is a transcript of an actual CL implementation
> executing this code and not generating an error:

Extending your implementation to support lambda macros just proves
Pascal Costanza's point.

-T.

Pascal Costanza

unread,
Aug 6, 2009, 4:52:21 AM8/6/09
to

The original statement, quoted above, was: "Any language feature that

can be implemented by a Turing machine can be included, using a suitable

macro to implement it." That's a very general statement, and general
statements include "worst-case" scenarios.

But even if you had all the source code available, you would still have
problems with composability, if two different language extensions want
to modify the same source code in incompatible ways. A general statement
should cover such a scenario as well.

You have to qualify that statement, and the qualification that you can
perform only local transformations is a generally accepted restriction
of macros, according to the literature.

Pascal Costanza

unread,
Aug 6, 2009, 4:55:45 AM8/6/09
to

That wasn't clear. (What you actually want to say is that a macro is
expanded at least once before the code _it expands into_ is executed.)

Yes, you have that guarantee, but it's a very weak guarantee.

Pascal Costanza

unread,
Aug 6, 2009, 5:00:16 AM8/6/09
to

Is it necessary to use insults?

> Everything you say is true. There is a very common (one might even
> say overwhelmingly popular) coding style that uses macros in exactly
> the way you describe and has all the drawbacks, constraints, and
> restrictions that you say. But nowhere is it written that one must
> confine themselves to this style. There are infinite possibilities
> beyond that if you are willing to think outside the box even just a
> little bit. For example, consider the following code:
>

> (((οΏ½ (f) ((οΏ½ (g) (g g)) (οΏ½ (h) (οΏ½ (x) ((f (h h)) x)))))
> (οΏ½ (f) (οΏ½ (n) (if (zerop n) 1 (* n (f (1- n)))))))


> 15)
>
> There is a very strong argument to be made that it is "impossible" to
> munge CL so that this code does anything other than generate an
> error. And yet, here is a transcript of an actual CL implementation
> executing this code and not generating an error:
>

> ? (((οΏ½ (f) ((οΏ½ (g) (g g)) (οΏ½ (h) (οΏ½ (x) ((f (h h)) x)))))
> (οΏ½ (f) (οΏ½ (n) (if (zerop n) 1 (* n (f (1- n)))))))


> 15)
> 1307674368000
> ?
>
> And just for the record:
>

> ? ((οΏ½ (f) ((οΏ½ (g) (g g)) (οΏ½ (h) (οΏ½ (x) ((f (h h)) x)))))
> (οΏ½ (f) (οΏ½ (n) (if (zerop n) 1 (* n (f (1- n)))))))


> #<COMPILED-LEXICAL-CLOSURE #x3000420A0FAF>
> ? (funcall * 15)
> 1307674368000
>
> And no, I did not have to rewrite the compiler from scratch.

...but you actually changed the internals of two implementations (IIRC),
so this is an example where the claim that you can implement any
language feature using just macros does not hold.

On top of that, you actually didn't even implement a new language
feature, you just provided alternative syntactic sugar for a feature
that's already part of Common Lisp.

Oxide Scrubber

unread,
Aug 6, 2009, 12:06:53 PM8/6/09
to
Pascal Costanza wrote:
> Ron Garret wrote:
>> Maybe. Why focus on worst-case scenarios?
>
> The original statement, quoted above, was: "Any language feature that
> can be implemented by a Turing machine can be included, using a suitable
> macro to implement it." That's a very general statement, and general
> statements include "worst-case" scenarios.
>
> But even if you had all the source code available, you would still have
> problems with composability, if two different language extensions want
> to modify the same source code in incompatible ways.

Extensions in general can be incompatible. At least we have a package
system that can help with that sort of thing, unlike some languages. :)

Oxide Scrubber

unread,
Aug 6, 2009, 12:16:54 PM8/6/09
to
Kaz Kylheku wrote:
> On 2009-08-05, Oxide Scrubber <jhar...@hatlop.de> wrote:
>> Concurrency could be added, in a low-level form, using C
>> foreign-function interface and pthreads (or whatever).
>
> Adding concurrency is more than just reaching the entry points for
> pthread_create and others.

Yes, I know.

>> Higher level stuff could be layered on.
>
> Higher level stuff?
>
> What about lower level stuff such as, oh, a thread-safe evaluator, thread-aware
> garbage collector, thread-safe code generated by the compiler, all library
> functions being thread-safe (including ones that have global effecst like DEFUN
> and LOAD). Thread-local bindings for dynamic variables. Etc.

All possible. The bare minimum needed would be for the thread creation
primitive you added to create some thread-local structures, and for
there to be a way to get a globally-unique identifier for the current
thread. (The latter would require punting to pthreads.) With threads
only launched after all loading and defining was done, and appropriate
use of whatever synchronization facilities whenever they shared data,
that gives you a basic level of concurrency. The only hairy bit is the
GC: it probably has to block all threads until it's done. Common Lisp
does allow for program styles that don't exercise the GC though, by
creating enough conses and whatever else before launching any threads
and then recycling them afterward. If GC is only ever triggered by
dynamic allocation, globally shadowing the special forms that allocate
things to make them suspend all other threads until done will work,
though reduce the benefits of concurrency.

It is a weakness of CL that whereas there's a macro system to provide
hooks into the compiler there's nothing analogous to provide hooks into
the GC. Bypassing it entirely and implementing your own is also
possible, though.

>> The low-level stuff you'd need, and could get through FFI, would be:
>> create, join, otherwise modify threads; and at least one of atomic
>> integer, semaphore, mutex lock/unlock, with at least one operation that
>> established a memory barrier on multicore machines.
>
> FFI to reach atomic operations is a bad idea. Firstly, there is no
> standardized library for these operations in C. They are usually
> macros that expand to inline assembly, useable from a C compiler.

The atomic operations would be implemented in C, e.g. compare and set an
atomic boolean value, and FFI would be used to call these, not to
directly call the primitives.

> You're assuming

I am not.

>> I assume many CL implementations already have concurrency. I believe one
>> is JVM-hosted
>
> Irrelevant.

No, not irrelevant.

I don't understand why people keep attacking me. I've done nothing to
provoke this constant, nagging harassment. Let me be.

Oxide Scrubber

unread,
Aug 6, 2009, 12:18:31 PM8/6/09
to

I thought I'd made it clear that I was considering that macros could
both have side effects themselves, and expand into code that has side
effects, and that both of those capabilities put the kibosh on the
erroneous claim made by Kaz Kylheku.

Oxide Scrubber

unread,
Aug 6, 2009, 12:20:07 PM8/6/09
to
Pascal Costanza wrote:
> Ron Garret wrote:
>> ? ((οΏ½ (f) ((οΏ½ (g) (g g)) (οΏ½ (h) (οΏ½ (x) ((f (h h)) x)))))
>> (οΏ½ (f) (οΏ½ (n) (if (zerop n) 1 (* n (f (1- n)))))))
>> #<COMPILED-LEXICAL-CLOSURE #x3000420A0FAF>
>> ? (funcall * 15)
>> 1307674368000
>>
>> And no, I did not have to rewrite the compiler from scratch.
>
> ...but you actually changed the internals of two implementations (IIRC),
> so this is an example where the claim that you can implement any
> language feature using just macros does not hold.

Shadowing eval and possibly other functions used by the repl suffices
for a case like this, does it not?

Oxide Scrubber

unread,
Aug 6, 2009, 12:20:57 PM8/6/09
to

Not really, given that such extending can often be done just by
shadowing a few built-in functions.

Oxide Scrubber

unread,
Aug 6, 2009, 12:22:38 PM8/6/09
to
Ron Garret wrote:
> In article <87tz0n3...@plap.localdomain>,

> Paul Donnelly <paul-d...@sbcglobal.net> wrote:
>> Ron Garret <rNOS...@flownet.com> writes:
>>> In article <20090814...@gmail.com>,
>>> Kaz Kylheku <kkyl...@gmail.com> wrote:
>>>> Lisp macros cannot do anything which cannot be done by a /local/ code
>>>> rewrite.
>>> That's not true. Macros can have arbitrary side-effects.
>> So can rewritten local code, for that matter. ;)
>
> No, it can't.

Sure it can.

(defmacro foo (x y)
`(setq my-global ,x ...

Pascal Costanza

unread,
Aug 6, 2009, 12:37:26 PM8/6/09
to

No, because third-party code doesn't see your changed definitions.

Kaz Kylheku

unread,
Aug 6, 2009, 1:05:30 PM8/6/09
to
On 2009-08-06, Oxide Scrubber <jhar...@hatlop.de> wrote:
> Pascal Costanza wrote:
>> Ron Garret wrote:
>>> ? ((λ (f) ((λ (g) (g g)) (λ (h) (λ (x) ((f (h h)) x)))))
>>> (λ (f) (λ (n) (if (zerop n) 1 (* n (f (1- n)))))))

>>> #<COMPILED-LEXICAL-CLOSURE #x3000420A0FAF>
>>> ? (funcall * 15)
>>> 1307674368000
>>>
>>> And no, I did not have to rewrite the compiler from scratch.
>>
>> ...but you actually changed the internals of two implementations (IIRC),
>> so this is an example where the claim that you can implement any
>> language feature using just macros does not hold.
>
> Shadowing eval and possibly other functions used by the repl suffices
> for a case like this, does it not?

If you have to shadow eval to make something work, that something is not
an embedded language that is implemented strictly by macros.

You can't portably redefine eval in Common Lisp (this is undefined behavior).
You /can/ define a function which is named by some symbol whose name is "EVAL".
But this is not CL:EVAL. The problem is getting existing clients of CL:EVAL to
call MY:EVAL instead. To switch some function from calling CL:EVAL into calling
MY:EVAL, we need to process the textual source code of that function, because
symbol names are resolved at read-time.

Kaz Kylheku

unread,
Aug 6, 2009, 1:38:58 PM8/6/09
to
On 2009-08-06, Oxide Scrubber <jhar...@hatlop.de> wrote:
> Kaz Kylheku wrote:
>> On 2009-08-05, Oxide Scrubber <jhar...@hatlop.de> wrote:
>>> Concurrency could be added, in a low-level form, using C
>>> foreign-function interface and pthreads (or whatever).
>>
>> Adding concurrency is more than just reaching the entry points for
>> pthread_create and others.
>
> Yes, I know.
>
>>> Higher level stuff could be layered on.
>>
>> Higher level stuff?
>>
>> What about lower level stuff such as, oh, a thread-safe evaluator, thread-aware
>> garbage collector, thread-safe code generated by the compiler, all library
>> functions being thread-safe (including ones that have global effecst like DEFUN
>> and LOAD). Thread-local bindings for dynamic variables. Etc.
>
> All possible.

The claim isn't that the above is impossible, but that it amounts to frobbing
the implementation, and thus that it can't just be achieved by a program
running in that implementation. You and Garett have both lost sight of the
debated point, but I will hold you to it.

If you have a program which depends on the above features being present in
FooLisp, and I have a stock installation of FooLisp, the program won't run in
my stock installation. I have to download the source code for FooLisp, apply
your patches to it, rebuild it, and install the patched version. Only then will
your program run.

This is so risky to existing FooLisp programs, that I will install this patched
FooLisp in a separate location, just for running your program, and keep the
stock FooLisp for everything else, until the day when your patches are
accepted upstream into FooLisp, and it achieves stability with those changes.

Lisp programs that locally extend the language with macros do not require a new
version of FooLisp; they run on the stock: the unmodified binary installation
of FooLisp. My claim is that you can't add continuations to Lisp with just
a program of this type; you need the source code for the implementation, and
you have to patch it and rebuild it; moreover, everyone who runs programs that
need continuations will have to do the same.

Oxide Scrubber

unread,
Aug 6, 2009, 2:07:33 PM8/6/09
to
Kaz Kylheku wrote:
> On 2009-08-06, Oxide Scrubber <jhar...@hatlop.de> wrote:
>> Kaz Kylheku wrote:
>>> On 2009-08-05, Oxide Scrubber <jhar...@hatlop.de> wrote:
>>>> Concurrency could be added, in a low-level form, using C
>>>> foreign-function interface and pthreads (or whatever).
>>> Adding concurrency is more than just reaching the entry points for
>>> pthread_create and others.
>> Yes, I know.
>>
>>>> Higher level stuff could be layered on.
>>> Higher level stuff?
>>>
>>> What about lower level stuff such as, oh, a thread-safe evaluator, thread-aware
>>> garbage collector, thread-safe code generated by the compiler, all library
>>> functions being thread-safe (including ones that have global effecst like DEFUN
>>> and LOAD). Thread-local bindings for dynamic variables. Etc.
>> All possible.
>
> The claim isn't that the above is impossible, but that it amounts to frobbing
> the implementation, and thus that it can't just be achieved by a program
> running in that implementation.

It can be achieved. I don't know why you're creating the artificial,
novel constraint of "running in that implementation", whatever exactly
it means (but apparently it forbids the program customizing the
implementation it's running on, and therefore much of what's powerful in
Lisp).

My guess would be that you're doing so just for the sake of perpetuating
an argument.

> You and Garett have both lost sight of the debated point

No, we have not.

> If you have a program which depends on the above features being present in
> FooLisp, and I have a stock installation of FooLisp, the program won't run in
> my stock installation. I have to download the source code for FooLisp, apply
> your patches to it, rebuild it, and install the patched version. Only then will
> your program run.

That isn't what we've been discussing. We've been discussing cases where
the program itself can do this (e.g. with some of its early lines
altering the way FooLisp works) and maybe needs a DLL or two.

Most Lisp systems bootstrap, by the way: some Lisp code runs on startup
to define common functions and macros, and even to set up some aspects
of the language that may "seem" fundamental. (And that early bit of code
can't use that stuff until it sets it up.) At least one that I know of
does this with destructuring, for example.

> Lisp programs that locally extend the language with macros do not require a new
> version of FooLisp; they run on the stock: the unmodified binary installation
> of FooLisp.

So do Lisp programs that shadow or redefine library functions or whatever.

Oxide Scrubber

unread,
Aug 6, 2009, 2:12:55 PM8/6/09
to
Kaz Kylheku wrote:
> On 2009-08-06, Oxide Scrubber <jhar...@hatlop.de> wrote:
>> Pascal Costanza wrote:
>>> Ron Garret wrote:
>>>> ? ((λ (f) ((λ (g) (g g)) (λ (h) (λ (x) ((f (h h)) x)))))
>>>> (λ (f) (λ (n) (if (zerop n) 1 (* n (f (1- n)))))))
>>>> #<COMPILED-LEXICAL-CLOSURE #x3000420A0FAF>
>>>> ? (funcall * 15)
>>>> 1307674368000
>>>>
>>>> And no, I did not have to rewrite the compiler from scratch.
>>> ...but you actually changed the internals of two implementations (IIRC),
>>> so this is an example where the claim that you can implement any
>>> language feature using just macros does not hold.
>> Shadowing eval and possibly other functions used by the repl suffices
>> for a case like this, does it not?
>
> If you have to shadow eval to make something work, that something is not
> an embedded language that is implemented strictly by macros.

If you want an embedded language that is implemented strictly by macros,
you can have one, but it may need its own explicit stack, its own
explicit GC, or similarly.

If you want a language extension, don't require it to be "implemented
strictly by macros", and do want to not have to alter the compiler or
other development tools or the runtime's binaries, then shadowing eval
is perfectly legitimate.

> You can't portably redefine eval in Common Lisp (this is undefined behavior).

Portability can be an issue at times.

> You /can/ define a function which is named by some symbol whose name is "EVAL".
> But this is not CL:EVAL. The problem is getting existing clients of CL:EVAL to
> call MY:EVAL instead. To switch some function from calling CL:EVAL into calling
> MY:EVAL, we need to process the textual source code of that function, because
> symbol names are resolved at read-time.

And we're right back to something macros can do.

Another way to regain some portability may be to find out what
redefining eval actually does on various CL implementations, and write
per-platform code appropriately, awkward though that may be.

Oxide Scrubber

unread,
Aug 6, 2009, 2:25:55 PM8/6/09
to
Pascal Costanza wrote:
> Oxide Scrubber wrote:
>> Pascal Costanza wrote:
>>> Ron Garret wrote:
>>>> ? ((οΏ½ (f) ((οΏ½ (g) (g g)) (οΏ½ (h) (οΏ½ (x) ((f (h h)) x)))))
>>>> (οΏ½ (f) (οΏ½ (n) (if (zerop n) 1 (* n (f (1- n)))))))
>>>> #<COMPILED-LEXICAL-CLOSURE #x3000420A0FAF>
>>>> ? (funcall * 15)
>>>> 1307674368000
>>>>
>>>> And no, I did not have to rewrite the compiler from scratch.
>>>
>>> ...but you actually changed the internals of two implementations
>>> (IIRC), so this is an example where the claim that you can implement
>>> any language feature using just macros does not hold.
>>
>> Shadowing eval and possibly other functions used by the repl suffices
>> for a case like this, does it not?
>
> No

Yes.

> because third-party code doesn't see your changed definitions.

Firstly, third-party code may not need to, depending on what the changes
are and on what the third-party code does.

Second, third-party code will see changed definitions if the changes are
made in the definitions' own namespaces and are made before the
third-party code is encountered.

Kaz Kylheku

unread,
Aug 6, 2009, 5:12:34 PM8/6/09
to
On 2009-08-06, Oxide Scrubber <jhar...@hatlop.de> wrote:
> Kaz Kylheku wrote:
>> On 2009-08-06, Oxide Scrubber <jhar...@hatlop.de> wrote:
>>> Kaz Kylheku wrote:
>>>> On 2009-08-05, Oxide Scrubber <jhar...@hatlop.de> wrote:
>>>>> Concurrency could be added, in a low-level form, using C
>>>>> foreign-function interface and pthreads (or whatever).
>>>> Adding concurrency is more than just reaching the entry points for
>>>> pthread_create and others.
>>> Yes, I know.
>>>
>>>>> Higher level stuff could be layered on.
>>>> Higher level stuff?
>>>>
>>>> What about lower level stuff such as, oh, a thread-safe evaluator, thread-aware
>>>> garbage collector, thread-safe code generated by the compiler, all library
>>>> functions being thread-safe (including ones that have global effecst like DEFUN
>>>> and LOAD). Thread-local bindings for dynamic variables. Etc.
>>> All possible.
>>
>> The claim isn't that the above is impossible, but that it amounts to frobbing
>> the implementation, and thus that it can't just be achieved by a program
>> running in that implementation.
>
> It can be achieved.

The burden of proof is on those who claim such a thing.

> I don't know why you're creating the artificial,
> novel constraint of "running in that implementation", whatever exactly
> it means

We all know what it means, save for a few trolls. It means we run some
/usr/bin/lisp, or whatever, which is a binary full of compiled code that we
don't touch. We don't get to replace any of it with our own code, nor a custom
recompile of the original source.

Lisp macros work without doing anything like this.

> (but apparently it forbids the program customizing the
> implementation it's running on, and therefore much of what's powerful in
> Lisp).

There is no such thing in Lisp. (As in defined by the Lisp language).

Doing this may be possible in some implementations, but it is not a feature of
the language or its macro system.

> That isn't what we've been discussing. We've been discussing cases where
> the program itself can do this (e.g. with some of its early lines
> altering the way FooLisp works) and maybe needs a DLL or two.

Please give an example, along with a citation (chapter and verse) which
supports how the example is well-defined ANSI Common Lisp.

> Most Lisp systems bootstrap, by the way: some Lisp code runs on startup
> to define common functions and macros, and even to set up some aspects
> of the language that may "seem" fundamental. (And that early bit of code
> can't use that stuff until it sets it up.) At least one that I know of
> does this with destructuring, for example.

No portable Common Lisp application can require patches to be applied to the
implementation, and any of those boostrap stages to be repeated.

Such a requirement is not only nonportable, but inconvenient and impractical.
It's a big thing to require, far beyond what Lisp macros do.

>> Lisp programs that locally extend the language with macros do not require a new
>> version of FooLisp; they run on the stock: the unmodified binary installation
>> of FooLisp.
>
> So do Lisp programs that shadow or redefine library functions or whatever.

Those that redefine library functions have undefined behavior according to the
language specification. (That doesn't mean they are wrong, only nonportable).
Lisp macros are a well-defined language feature which can be used to do things
in a portable way.

``Shadow library functions'' is misleading, because what's going on is that a
symbol is created in some custom package which merely has the same name as some
other symbol in the Common Lisp package.

(E.g. Ron Garret's recent example of a custom TYPE-OF which recognizes a
generic function as having a different type from a regular function).

But it's actually a completely different symobol, not EQ to that one.

Lisp symbols are based on implementation identity, not name identity.

Name identity is a read-time hack; if you exploit name identity for shadowing a
function, that requires the read time to be invoked, which requires raw source
code.

Kaz Kylheku

unread,
Aug 6, 2009, 5:32:03 PM8/6/09
to
On 2009-08-06, Oxide Scrubber <jhar...@hatlop.de> wrote:
> Pascal Costanza wrote:
>> Oxide Scrubber wrote:
>>> Pascal Costanza wrote:
>>>> Ron Garret wrote:
>>> Shadowing eval and possibly other functions used by the repl suffices
>>> for a case like this, does it not?
>>
>> No
>
> Yes.
>
>> because third-party code doesn't see your changed definitions.
>
> Firstly, third-party code may not need to, depending on what the changes
> are and on what the third-party code does.

The assumption is that the third-party code does need to. We have our own
custom EVAL. The third party code has some logic that performs evaluation of
code. We would like to pass code to that third-party code. Problem is, the
third party code calls CL:EVAL, so the code can't be written in our custom
language that requires MY:EVAL. At least not directly. We can do something like
this:

(let ((our-code ... whatever))
(third-party `(my:eval ',our-code)))

instead of:

(third-party our-code)

I.e. we need a shim there which breaks the abstraction.

In this case we are lucky, because we are talking about eval. Since eval is a
code evaluator, this gives us an obvious opportunity to hook in our shim that
diverts to a custom eval.

This trick won't generalize to arbitrary functions.

> Second, third-party code will see changed definitions if the changes are
> made in the definitions' own namespaces and are made before the
> third-party code is encountered.

You must assume that the third-party code comes to you compiled. Thus, it
is not processed by the Lisp reader at all. So you don't get to re-route
its EVAL reference to MY:EVAL.

Alessio Stalla

unread,
Aug 6, 2009, 6:43:17 PM8/6/09
to
On Aug 6, 11:12 pm, Kaz Kylheku <kkylh...@gmail.com> wrote:

> On 2009-08-06, Oxide Scrubber <jharri...@hatlop.de> wrote:
>
>
>
> > Kaz Kylheku wrote:
> >> On 2009-08-06, Oxide Scrubber <jharri...@hatlop.de> wrote:
> >>> Kaz Kylheku wrote:

If you know FooLisp's internals, you might be able to change how
FooLisp itself works without modifying FooLisp's sources. Lisp is
dynamic enough to allow that. Of course this entirely depends on how
FooLisp is implemented, it is completely non-portable (possibly not
even across different versions of FooLisp), and probably not
composable too (i.e. two programs changing bits of the Lisp
implementations might not run correctly in the same Lisp image). I'd
try to avoid such a solution in the general case, but I can imagine it
to be useful in some particular situations.

> > (but apparently it forbids the program customizing the
> > implementation it's running on, and therefore much of what's powerful in
> > Lisp).
>
> There is no such thing in Lisp. (As in defined by the Lisp language).
>
> Doing this may be possible in some implementations, but it is not a feature of
> the language or its macro system.

Well, a lot of things in CL implementations are neither a feature of
CL nor of its macro system. This is indeed the benefit of having
multiple implementations.

> > That isn't what we've been discussing. We've been discussing cases where
> > the program itself can do this (e.g. with some of its early lines
> > altering the way FooLisp works) and maybe needs a DLL or two.
>
> Please give an example, along with a citation (chapter and verse) which
> supports how the example is well-defined ANSI Common Lisp.

ANSI Common Lisp, probably not; FooLisp version X, maybe yes. Lisp
does not end with the CL standard.

> > Most Lisp systems bootstrap, by the way: some Lisp code runs on startup
> > to define common functions and macros, and even to set up some aspects
> > of the language that may "seem" fundamental. (And that early bit of code
> > can't use that stuff until it sets it up.) At least one that I know of
> > does this with destructuring, for example.
>
> No portable Common Lisp application can require patches to be applied to the
> implementation, and any of those boostrap stages to be repeated.
>
> Such a requirement is not only nonportable, but inconvenient and impractical.
> It's a big thing to require, far beyond what Lisp macros do.

Sure. But maybe requiring such a thing will make your software product
practically doable, or more maintainable etc. in the end, it's a
tradeoff. Portability is not a goal in itself, nor X being written in
100% "Pure Common Lisp" magically ensures X is better than if it was
implemented in FooLisp version K.Z patched by me.

> >> Lisp programs that locally extend the language with macros do not require a new
> >> version of FooLisp; they run on the stock: the unmodified binary installation
> >> of FooLisp.
>
> > So do Lisp programs that shadow or redefine library functions or whatever.
>
> Those that redefine library functions have undefined behavior according to the
> language specification. (That doesn't mean they are wrong, only nonportable).
> Lisp macros are a well-defined language feature which can be used to do things
> in a portable way.
>
> ``Shadow library functions'' is misleading, because what's going on is that a
> symbol is created in some custom package which merely has the same name as some
> other symbol in the Common Lisp package.  
>
> (E.g. Ron Garret's recent example of a custom TYPE-OF which recognizes a
> generic function as having a different type from a regular function).
>
> But it's actually a completely different symobol, not EQ to that one.
>
> Lisp symbols are based on implementation identity, not name identity.
>
> Name identity is a read-time hack; if you exploit name identity for shadowing a
> function, that requires the read time to be invoked, which requires raw source
> code.

Here I agree with you: shadowing only works when all parties agree to
use the shadowed version, or when one cheats and uses read-time tricks
to sneak his shadowed version inside third-party code.

In the end, Lisp is heavily extensible; many extension points are
specified by the CL standard (macros, various kind of hooks) or are de-
facto standards (MOP). Others are implementation-specific - extensible
sequences, hooks in the compiler, in the GC, in the evaluator, ... I
see no conceptual difference between standard and non-standard
extension points. The first will work in more cases, that's all.

Alessio

Ron Garret

unread,
Aug 6, 2009, 6:48:44 PM8/6/09
to
In article <7dvkd0F...@mid.individual.net>,
Pascal Costanza <p...@p-cos.net> wrote:

> > What a colossal failure of imagination.
>
> Is it necessary to use insults?
>

If the shoe fits. See below...

> ...but you actually changed the internals of two implementations (IIRC),

The depends on how you define "changing the internals". I took a stock
CL, and simply loaded a small amount of code (<100 lines) into it to
produce that result. Obviously, that code is not ANSI CL. But I did
not have to rebuild the Lisp to do what I did.

> so this is an example where the claim that you can implement any
> language feature using just macros does not hold.

I never claimed that you could implement any language feature using just
macros. All I said was that the claim that macros could only do local
code rewrites was false.

> On top of that, you actually didn't even implement a new language
> feature, you just provided alternative syntactic sugar for a feature
> that's already part of Common Lisp.

I chose that example not because it's an earth-shattering new feature,
but because it's something that many people would say (and have said) is
not possible to do. Those people are clearly wrong. The only point I
want to make is that very few things are actually impossible in Lisp,
and more often than not when someone says that something is impossible
it is because they are bringing to bear some tacit but unwarranted
assumption, hence "a failure of imagination."

rg

Ron Garret

unread,
Aug 6, 2009, 6:57:35 PM8/6/09
to
In article <h5f656$lvm$1...@aioe.org>,
Oxide Scrubber <jhar...@hatlop.de> wrote:

>
> > You and Garett have both lost sight of the debated point
>
> No, we have not.
>

Speak for yourself. I came into this in the middle so I'm not sure I
ever knew what the original point *was*. I only stepped in to correct a
false claim: it is not the case that macros can only do local code
transformations. It is also not the case that it is impossible to add
continuations to CL using macros. It is possible. However, it is not
easy. And there are constraints that one can find oneself having to
operate under (like having to use precompiled libraries for which one
does not have source code available) that would in fact make it
impossible. And that's really all there is to say about it.

rg

Ron Garret

unread,
Aug 6, 2009, 7:00:42 PM8/6/09
to
In article <7dvk4hF...@mid.individual.net>,
Pascal Costanza <p...@p-cos.net> wrote:

Right.

> Yes, you have that guarantee, but it's a very weak guarantee.

Indeed it is. But it suffices to refute your point.

rg

Ron Garret

unread,
Aug 6, 2009, 7:10:46 PM8/6/09
to
In article <7dvju6F...@mid.individual.net>,
Pascal Costanza <p...@p-cos.net> wrote:

Well, hurm, OK. I guess it depends on whether we're talking about
theory or engineering. If we're talking about theory then I can even
deal with precompiled libraries for which I don't have source code. All
I have to do is write an emulator for the machine that the compiled code
runs in, and now I have all the control I need to do whatever I need to
do.

> But even if you had all the source code available, you would still have
> problems with composability, if two different language extensions want
> to modify the same source code in incompatible ways. A general statement
> should cover such a scenario as well.

Why? This is exactly the sort of tacit assumption that I've been taking
you to task for. Composabilty was *not* part of the original problem
statement, even as you just now re-quoted it. It's obvious that general
composabilty of arbitrary language features is not possible because the
features themselves could conflict (to say nothing of their
implementations).

> You have to qualify that statement, and the qualification that you can
> perform only local transformations is a generally accepted restriction
> of macros, according to the literature.

I don't dispute that. All I'm saying is that "It's not possible" is not
the same claim as "it's not possible if one assumes constraint C". The
former is false, while the latter may be true depending on what C is.

rg

Ron Garret

unread,
Aug 6, 2009, 7:17:58 PM8/6/09
to
In article <h5evt9$dm1$4...@aioe.org>,
Oxide Scrubber <jhar...@hatlop.de> wrote:

But not in this case. What I've done above is more than just define a
unicode synonym for lambda. The above code is required by ANSI CL to
generate an error. And yet it does not. And yet it is running in an
unmodified (at least initially) ANSI-compliant CL implementation. And I
don't shadow any symbols.

rg

Ron Garret

unread,
Aug 6, 2009, 7:25:56 PM8/6/09
to
In article <snz4osl...@luna.vassil.nikolov.name>,
Vassil Nikolov <vnik...@pobox.com> wrote:

> On Wed, 5 Aug 2009 13:32:24 -0700 (PDT), Ron Garret <ron.g...@gmail.com>
> said:
> > ...

> > There is a very strong argument to be made that it is "impossible" to
> > munge CL so that this code does anything other than generate an
> > error. And yet, here is a transcript of an actual CL implementation
> > executing this code and not generating an error:
>

> > ? (((λ (f) ((λ (g) (g g)) (λ (h) (λ (x) ((f (h h)) x)))))


> > (λ (f) (λ (n) (if (zerop n) 1 (* n (f (1- n)))))))
> > 15)

> > 1307674368000
>
> What does
>
> (eval (LIST (LIST (LIST 'λ (LIST 'F) (LIST (LIST 'λ
> (LIST 'G) (LIST 'G 'G)) (LIST 'λ (LIST 'H)
> (LIST 'λ (LIST 'X) (LIST (LIST 'F (LIST 'H 'H)) 'X)))))
> (LIST 'λ (LIST 'F) (LIST 'λ (LIST 'N)
> (LIST 'IF (LIST 'ZEROP 'N) '1
> (LIST '* 'N (LIST 'F (LIST '1- 'N))))))) '15))
>
> return?
>

> ---Vassil.

I'm on the road until next Tuesday, and my travel laptop doesn't have
Lisp installed. But I'm sure it would produce the same result.

rg

Ron Garret

unread,
Aug 6, 2009, 7:28:27 PM8/6/09
to
In article <87iqh17...@galatea.local>,
p...@informatimago.com (Pascal J. Bourguignon) wrote:

> Vassil Nikolov <vnik...@pobox.com> writes:
>
> > On Wed, 5 Aug 2009 13:32:24 -0700 (PDT), Ron Garret <ron.g...@gmail.com>
> > said:
> >> ...
> >> There is a very strong argument to be made that it is "impossible" to
> >> munge CL so that this code does anything other than generate an
> >> error. And yet, here is a transcript of an actual CL implementation
> >> executing this code and not generating an error:
> >

> >> ? (((位 (f) ((位 (g) (g g)) (位 (h) (位 (x) ((f (h h)) x)))))
> >> (位 (f) (位 (n) (if (zerop n) 1 (* n (f (1- n)))))))
> >> 15)
> >> 1307674368000
> >
> > What does
> >
> > (eval (LIST (LIST (LIST '位 (LIST 'F) (LIST (LIST '位
> > (LIST 'G) (LIST 'G 'G)) (LIST '位 (LIST 'H)
> > (LIST '位 (LIST 'X) (LIST (LIST 'F (LIST 'H 'H)) 'X)))))
> > (LIST '位 (LIST 'F) (LIST '位 (LIST 'N)


> > (LIST 'IF (LIST 'ZEROP 'N) '1
> > (LIST '* 'N (LIST 'F (LIST '1- 'N))))))) '15))
> >
> > return?
>

> Nothing, 位 is not defined.

Well, obviously I defined it.

> Nothing, since in CL, the operator can only be a symbol, or a lambda form.

Another failure of imagination.

> Nonetheless, it is possible to munge CL so that it works.

Now you're talking.

> For the former expression, by writting a REPL that interprets that new
> language,

That's one way to do it, but that's not how I did it.

> for the eval, by shadowing it and redefining it to interpret that new
> language.

No symbols were shadowed to produce that result.

rg

Pascal J. Bourguignon

unread,
Aug 6, 2009, 7:45:09 PM8/6/09
to
Ron Garret <rNOS...@flownet.com> writes:

Well I was also thinking about defining a reader macro for #\( but I
wonder how you would safely distinguish data from code.

--
__Pascal Bourguignon__

Oxide Scrubber

unread,
Aug 7, 2009, 12:07:31 AM8/7/09
to
Kaz Kylheku wrote:
> On 2009-08-06, Oxide Scrubber <jhar...@hatlop.de> wrote:
>> Pascal Costanza wrote:
>>> Oxide Scrubber wrote:
>>>> Pascal Costanza wrote:
>>>>> Ron Garret wrote:
>>>> Shadowing eval and possibly other functions used by the repl suffices
>>>> for a case like this, does it not?
>>> No
>> Yes.
>>
>>> because third-party code doesn't see your changed definitions.
>> Firstly, third-party code may not need to, depending on what the changes
>> are and on what the third-party code does.
>
> The assumption is that the third-party code does need to.

Your assumption is.

> We can do something like this:
>
> (let ((our-code ... whatever))
> (third-party `(my:eval ',our-code)))

That's one way to do it.

>> Second, third-party code will see changed definitions if the changes are
>> made in the definitions' own namespaces and are made before the
>> third-party code is encountered.
>
> You must assume

Why must I assume anything? You're the one who's fond of assuming things.

> that the third-party code comes to you compiled. Thus, it
> is not processed by the Lisp reader at all. So you don't get to re-route
> its EVAL reference to MY:EVAL.

Changing what the symbol eval points to in the global environment
doesn't depend on the Lisp reader.

Oxide Scrubber

unread,
Aug 7, 2009, 12:08:38 AM8/7/09
to

Which helps to prove my point.

f

Oxide Scrubber

unread,
Aug 7, 2009, 12:12:09 AM8/7/09
to
Kaz Kylheku wrote:
> On 2009-08-06, Oxide Scrubber <jhar...@hatlop.de> wrote:
>> Kaz Kylheku wrote:
>>> On 2009-08-06, Oxide Scrubber <jhar...@hatlop.de> wrote:
>>>> Kaz Kylheku wrote:
>>>>> On 2009-08-05, Oxide Scrubber <jhar...@hatlop.de> wrote:
>>>>>> Concurrency could be added, in a low-level form, using C
>>>>>> foreign-function interface and pthreads (or whatever).
>>>>> Adding concurrency is more than just reaching the entry points for
>>>>> pthread_create and others.
>>>> Yes, I know.
>>>>
>>>>>> Higher level stuff could be layered on.
>>>>> Higher level stuff?
>>>>>
>>>>> What about lower level stuff such as, oh, a thread-safe evaluator, thread-aware
>>>>> garbage collector, thread-safe code generated by the compiler, all library
>>>>> functions being thread-safe (including ones that have global effecst like DEFUN
>>>>> and LOAD). Thread-local bindings for dynamic variables. Etc.
>>>> All possible.
>>> The claim isn't that the above is impossible, but that it amounts to frobbing
>>> the implementation, and thus that it can't just be achieved by a program
>>> running in that implementation.
>> It can be achieved.
>
> The burden of proof is on those who claim such a thing.

And I've met the burden of proof.

>> I don't know why you're creating the artificial,
>> novel constraint of "running in that implementation", whatever exactly

>> it means (but apparently it forbids the program customizing the

>> implementation it's running on, and therefore much of what's powerful in
>> Lisp).
>
> There is no such thing in Lisp.

Nonsense.

>> That isn't what we've been discussing. We've been discussing cases where
>> the program itself can do this (e.g. with some of its early lines
>> altering the way FooLisp works) and maybe needs a DLL or two.
>

>> Most Lisp systems bootstrap, by the way: some Lisp code runs on startup
>> to define common functions and macros, and even to set up some aspects
>> of the language that may "seem" fundamental. (And that early bit of code
>> can't use that stuff until it sets it up.) At least one that I know of
>> does this with destructuring, for example.
>
> No portable Common Lisp application can require patches to be applied to the
> implementation, and any of those boostrap stages to be repeated.

Unless it has conditional code for many of the commoner implementations
(Allegro, SBCL, ABCL...)

Portability, BTW, being another constraint you invented that was not
part of the original problem as stated.

>>> Lisp programs that locally extend the language with macros do not require a new
>>> version of FooLisp; they run on the stock: the unmodified binary installation
>>> of FooLisp.
>> So do Lisp programs that shadow or redefine library functions or whatever.
>
> Those that redefine library functions have undefined behavior according to the
> language specification. (That doesn't mean they are wrong, only nonportable).

Nonportable is not a showstopper; see above.

> ``Shadow library functions'' is misleading

No. Nothing that I have written is misleading.

> Name identity is a read-time hack; if you exploit name identity for shadowing a
> function, that requires the read time to be invoked, which requires raw source
> code.

Oh wow -- a programming technique requires raw source code! How
horrible! That makes it impossible! Er, waitaminnit...

Oxide Scrubber

unread,
Aug 7, 2009, 12:13:08 AM8/7/09
to
Ron Garret wrote:
> It is also not the case that it is impossible to add
> continuations to CL using macros.

Ah. That is the original bone of contention, conceded in my favor. Thank
you.

Ron Garret

unread,
Aug 7, 2009, 12:24:33 AM8/7/09
to
In article <h5g9kk$q99$2...@aioe.org>,
Oxide Scrubber <jhar...@hatlop.de> wrote:

I think you need to keep better track of who you're debating with. You
and I have been on the same side of this discussion since I joined it.
If you were seeking a "concession" from someone it wasn't me.

rg

gugamilare

unread,
Aug 7, 2009, 12:55:46 AM8/7/09
to
On 6 ago, 20:45, p...@informatimago.com (Pascal J. Bourguignon) wrote:
> Ron Garret <rNOSPA...@flownet.com> writes:
> > In article <87iqh177sz....@galatea.local>,

> >  p...@informatimago.com (Pascal J. Bourguignon) wrote:
>
> >> For the former expression, by writting a REPL that interprets that new
> >> language,
>
> > That's one way to do it, but that's not how I did it.
>
> >> for the eval, by shadowing it and redefining it to interpret that new
> >> language.
>
> > No symbols were shadowed to produce that result.
>
> Well I was also thinking about defining a reader macro for #\( but I
> wonder how you would safely distinguish data from code.

Did someone think about the reader macro of λ?

cl-user> (set-macro-character #\λ #'(lambda (&rest args) (declare
(ignore args)) 'lambda))
t
cl-user> (λ (x) (+ x 3))
#<FUNCTION (lambda (x)) {100445AC49}>
cl-user> (funcall * 3)
6

Vassil Nikolov

unread,
Aug 7, 2009, 1:34:04 AM8/7/09
to

On Thu, 06 Aug 2009 15:48:44 -0700, Ron Garret <rNOS...@flownet.com> said:

> In article <7dvkd0F...@mid.individual.net>,
> Pascal Costanza <p...@p-cos.net> wrote:
> ...
>> ...but you actually changed the internals of two implementations (IIRC),

> ... depends on how you define "changing the internals". I took a stock

> CL, and simply loaded a small amount of code (<100 lines) into it to
> produce that result. Obviously, that code is not ANSI CL. But I did
> not have to rebuild the Lisp to do what I did.

That does sound to me like "changing the internals" by any
reasonable definition of the latter. By the way, it is somewhat
interesting to know whether any part of the implementation had to be
recompiled, in the sense in which doing the above would
involve---just in order to accomplish the change (i.e. not in order
to run any programs that make use of the change)---compiling any
code outside those fewer than a hundred lines mentioned above.

And were these fewer than a hundred lines just DEFMACRO forms?

---Vassil.


--
"Even when the muse is posting on Usenet, Alexander Sergeevich?"

Kaz Kylheku

unread,
Aug 7, 2009, 2:09:04 AM8/7/09
to
On 2009-08-07, Oxide Scrubber <jhar...@hatlop.de> wrote:
> Why must I assume anything? You're the one who's fond of assuming things.

These tiresome personal attacks don't prove that full-blown continuations can
be implemented purely as a sublanguage based on ANSI-conforming Common Lisp
macros.

Welcome back. I guess.

Oxide Scrubber

unread,
Aug 7, 2009, 3:38:54 AM8/7/09
to

What personal attacks? I merely stated that you were fond of assuming
things, which is a simple observation.

As for proving that macros can implement continuations, Ron Garrett and
I have both supplied compelling evidence that it's true.

> Welcome back. I guess.

I don't know what you mean by this; we've been arguing for a few days
now during which I have never "left" for more than a few hours.

Oxide Scrubber

unread,
Aug 7, 2009, 3:40:04 AM8/7/09
to
Ron Garret wrote:
> In article <h5g9kk$q99$2...@aioe.org>,
> Oxide Scrubber <jhar...@hatlop.de> wrote:
>
>> Ron Garret wrote:
>>> It is also not the case that it is impossible to add
>>> continuations to CL using macros.
>> Ah. That is the original bone of contention, conceded in my favor. Thank
>> you.
>
> I think you need to keep better track of who you're debating with.

I am doing that just fine, thank you.

> You and I have been on the same side of this discussion since I joined it.
> If you were seeking a "concession" from someone it wasn't me.

The point was for Kaz's benefit, and Pascal's: that I am not alone in
making my claim.

Pascal Costanza

unread,
Aug 7, 2009, 3:50:49 AM8/7/09
to

You're making a lot of restrictions now, which refutes the much more
general claim you initially made.


Pascal

--
My website: http://p-cos.net
Common Lisp Document Repository: http://cdr.eurolisp.org
Closer to MOP & ContextL: http://common-lisp.net/project/closer/

Pascal Costanza

unread,
Aug 7, 2009, 3:55:41 AM8/7/09
to
Ron Garret wrote:
> In article <7dvkd0F...@mid.individual.net>,
> Pascal Costanza <p...@p-cos.net> wrote:
>
>>> What a colossal failure of imagination.
>> Is it necessary to use insults?
>>
>
> If the shoe fits. See below...
>
>> ...but you actually changed the internals of two implementations (IIRC),
>
> The depends on how you define "changing the internals". I took a stock
> CL, and simply loaded a small amount of code (<100 lines) into it to
> produce that result. Obviously, that code is not ANSI CL. But I did
> not have to rebuild the Lisp to do what I did.

It doesn't matter how you changed the internals. But you did.

>
>> so this is an example where the claim that you can implement any
>> language feature using just macros does not hold.
>
> I never claimed that you could implement any language feature using just
> macros.

Well, that's the claim I was originally concerned with.

> All I said was that the claim that macros could only do local
> code rewrites was false.

You're extension requires a change to the internals of a Common Lisp
implementation. (It's secondary how you achieved these changes.)

This doesn't say much about what you can and cannot do with _just_ macros.

>> On top of that, you actually didn't even implement a new language
>> feature, you just provided alternative syntactic sugar for a feature
>> that's already part of Common Lisp.
>
> I chose that example not because it's an earth-shattering new feature,
> but because it's something that many people would say (and have said) is
> not possible to do. Those people are clearly wrong. The only point I
> want to make is that very few things are actually impossible in Lisp,
> and more often than not when someone says that something is impossible
> it is because they are bringing to bear some tacit but unwarranted
> assumption, hence "a failure of imagination."

You're stating a very trivial thing here, namely that if you can change
implementation internals, you can do pretty much anything.

Pascal Costanza

unread,
Aug 7, 2009, 3:59:48 AM8/7/09
to

This is not a very local change now, is it? ;)

>
>> But even if you had all the source code available, you would still have
>> problems with composability, if two different language extensions want
>> to modify the same source code in incompatible ways. A general statement
>> should cover such a scenario as well.
>
> Why? This is exactly the sort of tacit assumption that I've been taking
> you to task for. Composabilty was *not* part of the original problem
> statement, even as you just now re-quoted it. It's obvious that general
> composabilty of arbitrary language features is not possible because the
> features themselves could conflict (to say nothing of their
> implementations).

The original quote made a claim about _any_ language feature, not "only
language features that you want to use only locally in your own modules."

Pascal J. Bourguignon

unread,
Aug 7, 2009, 6:10:40 AM8/7/09
to
gugamilare <gugam...@gmail.com> writes:

That is not enough.

( (lambda (x) x) 1) is allowed by CL because (lambda (x) x) is a function name.

But

( ( (lambda (y) (lambda (x) (+ y x))) 2) 1) ; [2]

is not allowed by CL because ( (lambda (y) (lambda (x) (+ y x))) 2)
is not a function name. It's a form that when evaluated would return
a _function_.


Note that (lambda (x) x) is also a form that would return a function,
because CL:LAMBDA is a macro that expands to (function (lambda
... ...)) but the point is that CL does not evaluate the operator
position in applications. To be able to evaluate [2] as intended, it
would have to do like Scheme, to evaluate the operator.

Perhaps the reader macro for #\( will check that it gets a λ first to
choose between code and data. But so far I can only imagine it as a
dirty hack. That is, the question is what do you get if you evaluate:

(cl:quote (λ (x) x))

or:

(car (cl:quote (λ (x) x)))

?


--
__Pascal Bourguignon__

It is loading more messages.
0 new messages