Why should I care about Lisp and Scheme?
bobu...@yahoo.com
(http://www.catb.org/~esr/faqs/hacker-howto.html)
(http://www.paulgraham.com/icad.html)
(http://www.paulgraham.com/rootsoflisp.html)
(http://groups.google.com/group/comp.lang.scheme/browse_frm/thread/3ae4ac3836bbde34/8add0f479c70ad73#8add0f479c70ad73)
(http://groups.google.com/group/comp.lang.scheme/browse_frm/thread/1c62fe94aee4dcab/7b657644f6d6638e#7b657644f6d6638e)
Foreword to the book "Essentials of Programming Languages".
Special thanks to Brian Harvey, Pascal Bourguignon, Anton van Straaten
and others at comp.lang.scheme, whose advice and statements I've used
heavily.
I see this document as a first draft that needs to be worked on. Hence
any comments are appreciated.
It's an imaginary conversation between a newbie and a hacker.
---
Newbie: Why should I care about Lisp and Scheme?
Hacker: If your happy with what you've got so far you don't need to
care about Lisp or Scheme. But if you want to get more perspective
about what programming is all about, then Lisp or Scheme could be worth
trying. They are worth learning for the profound enlightenment
experience you will have when you finally get it. That experience will
make you a better programmer for the rest of your days, even if you
never actually use Lisp (or Scheme) itself a lot.
Newbie: Lisp is one of the oldest languages around. How come it
survived? What's the story behind Lisp?
Hacker: Lisp differs from other programming languages, because it was
not written as a language. It came as a byproduct of McCarthy trying to
axiomatize computation. He invented a notation representing Lisp
functions as Lisp data, and such a notation was devised as a pure
abstract concept. One of the functions McCarthy devised was eval and
when Steve Russell, one of McCarthy's grad students, looked at the
definition of eval and realized that if he translated it into machine
language, the result would be a Lisp interpreter. Here is what McCarthy
said about it later in an interview:
"Steve Russell said, look, why don't I program this eval..., and I said
to him, ho, ho, you're confusing theory with practice, this eval is
intended for reading, not for computing. But he went ahead and did it.
That is, he compiled the eval in my paper into [IBM] 704 machine code,
fixing bugs, and then advertised this as a Lisp interpreter, which it
certainly was. So at that point Lisp had essentially the form that it
has today.... "
So the reason why Lisp is different from many other programming
languages is that it is math, and math doesn't become old so fast as
technology does.
Newbie: What would you say is special about Lisp, in practice?
Hacker: One thing is that symbolic data (words, sentences, etc.) are
easy to manipulate. Another thing is that procedures are first class
objects. A third thing is that variables are not typed.
Newbie: Can you illustrate with a simple example that symbolic data is
easy to manipulate.
Hacker: Here's a simple example
> (car '(define (f x) (* x x)))
define
Now you know that the expression defines something ad so on.
Newbie: What do you mean by procedures being first class objects?
Hacker: I mean that functions can be used in programs without
restriction, just as if they were simple data. For example a function
can:
* be constructed at runtime.
* be stored in variable.
* be passed as a parameter to a procedure.
* be returned as the result of a procedure call.
Newbie: Can you give a simple example?
Hacker: Here's a simple procedure that takes a function as a parameter.
> (define (add x y) (+ x y))
> (define (sub x y) (- x y))
> (define (func g) (g 8 4))
> (func add)
12
> (func sub)
4
As you can see the parameter of the procedure func is a procedure
called g. If we pass the procedure add to func we get 12, and if we
pass the procedure sub to func we get 4.
Newbie: You also mentioned that in Lisp variables are not typed. What
does that mean?
Hacker: In Lisp data (and procedures) are stored in the heap and
variables hold pointers pointing at the data. In this way variables do
not need to have any special type.
Newbie: What's so cool with that?
Hacker: Since the variables are not typed, there are no restrictions on
which values may be associated with names. Therefore the same procedure
can be used with values of various types. Here's one example
(define (third seq)
(element seq 2))
(define (element seq index)
(cond ((vector? seq) (vector-ref seq index))
((list? seq) (list-ref seq index))
((string? seq) (string-ref seq index))
(else (errorstring "First argument must be a sequence"))))
; Test
(define L '(1 2 3 4 5))
(third L)
; ==> 3
(define V '#(1 2 3 4 5))
(third V)
; ==>
(define S '"12345")
(third S)
; ==> #\3
As you can see the function third accepts any kind of sequence. You can
feed it with a list, a vector or a string. It does it's job without
complaining.
Newbie: What you're saying reminds me of Python, which is also
dynamically typed. The other two features of Lisp that you mentioned,
the symbolic data and procedures being first class, also apply to
Python I think? Python doesn't have the simple structure that Lisp has,
but it has lists, dictionaries and tuples as primitive elements so in
principle those two features apply to Python as well.
Hacker: Python has no easy data=code. The point of symbols and lists in
Lisp is that they are used both to represent data and code, and you can
convert between data and code at run-time. You can't beat the ease of
Lisp to treat code as data. Even though some programming language has
some part that Lisp has, it's the whole philosophy behind Lisp, which
is more than the sum of the parts.
Newbie: The reason I started to look at Scheme was this concept of not
distinguishing between the data and code. I mean both of them are zeros
and ones inside a computer so why should you make difference between
them. That many programming languages do so, feels like blocking a
world of opportunities.
Hacker: Interpreters and compilers are the prime examples of programs
that utilizes that code is data. If you don't understand interpreters,
you can still write programs; you can even be a competent programmer.
But you can't be a master.
Newbie: Why is understanding interpreters so important.
Hacker: One reason is that understanding interpreters allows you to
create special-purpose "little languages" that are custom made for your
problem. Another reason is that in order to understand computers and
programming, you need to understand the link between the code you write
and what happens inside the computer when it runs your code. In other
words you have to understand the link between syntax and semantics. An
the interpreter is that link. Even programmers who haven't studied
interpreters develop an intuitive understanding of the link between
syntax and semantics. They do this in two main ways: by reasoning about
programs, i.e. interpreting them in their head; and by using a computer
as a "black box", watching what the output that the computer generates
when fed with a certain syntax. But this still leaves something to be
desired. Our "mental" interpretation tends to be full of gaps and the
"black box" can't give the whole picture. So, the only way to really
fill in our knowledge of that important link is to study interpreters
themselves.
Newbie: What goes on in my head is that I visualize a memory space
containing a stack, a heap, registers and so on.
Hacker: Right, so you're simulating the hardware in your mind. This is
quite common, but not necessarily the most effective approach. It might
be more effective to translate into one or more "intermediary
languages". Translating to the machine level too early tends to create
a trees vs. forest problem, and doesn't help you deal with the
complexity in the most efficient way. Understanding abstraction layers
and the languages within each abstraction layer might help you control
complexity.
Newbie: Can you explain little more about these "intermediary
languages".
Hacker: It's easier to appreciate what's special in Lisp and Scheme by
shifting your viewpoint from a language user to a language designer -
the former is to accept the language features as given and
unchangeable, and tries to write all the solutions in it; while the
latter first look at the problem, think about what language is suitable
to describe the problem, then write the language using the base
language. In Lisp and Scheme programmers can become "language
designers" inventing the "language" for their specific problem. In that
way they get a very flexible program that can adapt easily when the
original problem changes.
Newbie: Is there other reasons that make Lisp and Scheme cool?
Hacker: I like the free SICP video lectures and the corresponding SICP
book. There's a very rich and exciting literature! For instance EoPL,
PLAI, On Lisp, and so on. I also like the intelligent conversations in
comp.lang.scheme and comp.lang.lisp.
Bob
Brian Harvey
jewel-like programming languages to keep on a shelf, rather than to
someone who's trying to get work done.
The first answer, and the only answer to give a "newbie", is to show him
six pages of Java code and the corresponding six lines of Lisp code.
Here's my favorite example:
(define (choices menu)
(if (null? menu)
'(())
(let ((smaller (choices (cdr menu))))
(reduce append
(map (lambda (item) (prepend-every item smaller))
(car menu))))))
(define (prepend-every item lst)
(map (lambda (choice) (sentence item choice)) lst))
(define (sentence a b) ; simplified version just for this example
(cond ((symbol? a) (sentence (list a) b))
((symbol? b) (sentence a (list b)))
(else (append a b))))
> (CHOICES '((SMALL MEDIUM LARGE)
(VANILLA (ULTRA CHOCOLATE) (RUM RAISIN) GINGER)
(CONE CUP)))
((small vanilla cone)
(small vanilla cup)
(small ultra chocolate cone)
(small ultra chocolate cup)
(small rum raisin cone)
(small rum raisin cup)
(small ginger cone)
(small ginger cup)
(medium vanilla cone)
(medium vanilla cup)
(medium ultra chocolate cone)
(medium ultra chocolate cup)
(medium rum raisin cone)
(medium rum raisin cup)
(medium ginger cone)
(medium ginger cup)
(large vanilla cone)
(large vanilla cup)
(large ultra chocolate cone)
(large ultra chocolate cup)
(large rum raisin cone)
(large rum raisin cup)
(large ginger cone)
(large ginger cup))
Lauri Alanko
Brian Harvey <b...@cs.berkeley.edu> wrote:
> Here's my favorite example:
>
> (define (choices menu)
> (if (null? menu)
> '(())
> (let ((smaller (choices (cdr menu))))
> (reduce append
> (map (lambda (item) (prepend-every item smaller))
> (car menu))))))
>
> (define (prepend-every item lst)
> (map (lambda (choice) (sentence item choice)) lst))
>
> (define (sentence a b) ; simplified version just for this example
> (cond ((symbol? a) (sentence (list a) b))
> ((symbol? b) (sentence a (list b)))
> (else (append a b))))
That seems a teensy bit complicated. You're essentially doing some
manual deforestation. This is the "unoptimized" version, which is IMHO
clearer, using fold-right and append-map from srfi-1 instead of
reduce. (Btw, your reduce doesn't seem to have a base case at all?)
(define (product lists)
(fold-right product2 '(()) lists))
(define (product2 l1 l2)
(append-map (lambda (x) (map (lambda (y) (append (listify x) y)) l2)) l1))
(define (listify x)
(if (pair? x) x (list x)))
Lauri
bobu...@yahoo.com
work done" and the ones that besides wanting to get the work done also
want to "synthesize what they already know", and build a rich
framework, that they can use as step stones for further learning and
for controlling complexity of their programs. A look at reviews of SICP
at Amazon reveals that the camp is divided into haters giving one star
and fans giving five stars. I think that the ones who give one star are
the ones that just "want to get work done". I'm not writing for them,
they are not my target group.
Maybe I'm an odd bird but the example you give wouldn't convince me to
care about Scheme. If I really wanted to "get the work done" (whatever
that means) and nothing more, I wouldn't choose Scheme to begin with,
but rather a language with rich library of functions that more or less
solved my problem.
On the other hand a statement like "Scheme is worth learning for the
profound enlightenment experience you will have when you finally get
it. That experience will make you a better programmer for the rest of
your days, even if you never actually use Scheme itself a lot." might
make me take a look at Scheme.
Bob
Brian Harvey
>There are different kind of newbies. The ones that "want to get the
>work done" and the ones that besides wanting to get the work done also
>want to "synthesize what they already know", and build a rich
>framework, that they can use as step stones for further learning and
>for controlling complexity of their programs.
Well, maybe you're right. I guess I felt that people who think in those
terms don't need a lot of convincing about Scheme -- its beauty just
hits them over the head by itself. :-)
bobu...@yahoo.com
--- snip ---
Newbie: What would you say is special about Lisp?
Hacker: Several things, for instance:
* programs-as-data
* manipulation of symbolic data
* procedures as first class objects
* variables are not typed
Newbie: can you explain programs-as-data?
Hacker: In Lisp programs are represented as lists which means that you
can manipulate programs as data. This is for instance important when
you write interpreters.
Newbie: Can you illustrate program-as-data with an example.
Hacker: Here's a simple example
> (car '(define (f x) (* x x)))
define
Now you know that the expression defines something ad so on.
Newbie: Can you illustrate what you mean by manipulation of symbolic
data.
Hacker: Let's say that the symbolic data is normal English words and
sentences. Here's a small program:
(define (exaggerate sent)
(map (lambda (wd)
(cond ((number? wd) (* 3 wd))
((eq? wd 'bad) 'gruesome)
((eq? wd 'big) 'gigantic)
((eq? wd 'like) 'love)
(else wd)))
sent))
; Test
(exaggerate '(I catched 2 big fishes))
; ==> (I catched 6 gigantic fishes)
--- snip ---
Hacker: Here's a simple procedure that takes a function as a parameter.
> (define (func g) (g 8 4))
> (func +)
12
> (func -)
4
As you can see the parameter of the procedure func is a procedure
called g. If we pass the procedure + to func we get 12, and if we pass
the procedure - to func we get 4.
Here's a slightly more complicated example:
> (define (compose f g)
(lambda (x y) (f (g x y) y)))
> ((compose + *) 2 3)
9
> ((compose * +) 2 3)
15
In the first case we multiply 2 and 3 and then we add 3. In the second
example we add 2 and 3 and then multiply by 3.
--- snip ---
All discussion about Python is deleted
--- snip ---
Newbie: Can you explain little more about these "intermediary
languages".
Hacker: Intermediary languages have to do with abstraction. It's easier
to appreciate what's special in Lisp and Scheme by shifting your
viewpoint from a language user to a language designer - the former is
to accept the language features as given and unchangeable, and tries to
write all the solutions in it; while the latter first looks at the
problem, thinks about what language is suitable to describe the
problem, then writes the language using the base language. In Lisp and
Scheme programmers can become "language designers" inventing the
"languages" for different abstraction levels. In that way they get a
very flexible program that can adapt easily when the original problem
changes.
Newbie: Is there other reasons why you like Lisp and Scheme?
Hacker: I think that automatic memory management and that you can have
interactive conversation with Scheme is nice. I also like the free
Lauri Alanko
<bobu...@yahoo.com> wrote:
> Newbie: can you explain programs-as-data?
>
> Hacker: In Lisp programs are represented as lists which means that you
> can manipulate programs as data. This is for instance important when
> you write interpreters.
This was true in old lisps. Not in Scheme or in Common Lisp. In
Scheme, code is code and data is data. Their relationship is simply
that:
* the syntax for expressions is similar to the syntax for literal
<datum>s in the grammar
* properly formed data structures can be evaluated as code with eval
The latter is not particular to Lisps: almost all scripting languages
support eval one way or the other, although most of them take strings
instead of tree structures.
The former is a consequence of the historical origins of Lisp, since
it was initially defined in terms of an eval function that takes list
structures as input.
Note that this code-looks-like-data relationship is not particular to
Lisp, either: in Prolog, as well, the syntax for code is exactly the
same as the syntax for data literals.
> Newbie: Can you illustrate program-as-data with an example.
>
> Hacker: Here's a simple example
> > (car '(define (f x) (* x x)))
> define
>
> Now you know that the expression defines something ad so on.
This doesn't demonstrate anything. You have a list structure that
happens to resemble the structure of a Scheme definition. This is not
program-as-data, this is just data-as-data, no more special than doing
> "int f(int x) { return x * x }".substring(0,3)
"int"
in Java, except that your '(define ...) has more structure than a
simple string.
Lauri
bobu...@yahoo.com
people have been saying to me for the last two weeks??
Lauri Alanko wrote:
> In Scheme, code is code and data is data. Their relationship is simply that:
>* the syntax for expressions is similar to the syntax for literal <datum>s in the grammar
>* properly formed data structures can be evaluated as code with eval
So what's this talk about code=data and the big thing as this being one
of the great strengths of Lisp?
Lauri Alanko wrote:
> > (car '(define (f x) (* x x)))
>This doesn't demonstrate anything. You have a list structure that
happens to resemble the structure of a Scheme definition. This is not
program-as-data, this is just data-as-data
Can you give me an example then that demonstrates this code-as-data
that people have been talking about.
I'm glad for your post Lauri because I've been little suspicous about
the whole thing myself. I hope that you and the other Schemers can
shed some light here.
Bob
Pascal Costanza
> In article <1137578367....@g14g2000cwa.googlegroups.com>,
> <bobu...@yahoo.com> wrote:
>
>>Newbie: can you explain programs-as-data?
>>
>>Hacker: In Lisp programs are represented as lists which means that you
>>can manipulate programs as data. This is for instance important when
>>you write interpreters.
>
>
> This was true in old lisps. Not in Scheme or in Common Lisp. In
> Scheme, code is code and data is data.
I don't know what you exactly mean by that, but I believe that in the
case of Common Lisp you're not right. (But this is probably not the
right place to discuss this.)
Pascal
--
My website: http://p-cos.net
Closer to MOP & ContextL:
http://common-lisp.net/project/closer/
ma...@wutka.com
> In article <1137578367....@g14g2000cwa.googlegroups.com>,
> <bobu...@yahoo.com> wrote:
>> Newbie: can you explain programs-as-data?
>>
>> Hacker: In Lisp programs are represented as lists which means that you
>> can manipulate programs as data. This is for instance important when
>> you write interpreters.
>
> This was true in old lisps. Not in Scheme or in Common Lisp. In
> Scheme, code is code and data is data. Their relationship is simply
> that:
>
> * the syntax for expressions is similar to the syntax for literal
> <datum>s in the grammar
> * properly formed data structures can be evaluated as code with eval
>
The fact that code and data are represented using the same syntax is
one of the things that makes macros so easy to do. You don't have to use
any special preprocessor or any special functions in order to write code
that generates or modifies code. That to me is what makes the Lisp
family exciting. Most of the other neat features I can also find in
various other languages, with the ML family having a lot of the same
features.
Mark
Nils M Holm
> > Hacker: Here's a simple example
> > > (car '(define (f x) (* x x)))
> > define
> >
> > Now you know that the expression defines something ad so on.
>
> This doesn't demonstrate anything. You have a list structure that
> happens to resemble the structure of a Scheme definition. This is not
> program-as-data, this is just data-as-data, no more special than doing
>
> > "int f(int x) { return x * x }".substring(0,3)
> "int"
>
> in Java, except that your '(define ...) has more structure than a
> simple string.
Here is a more elaborate example:
(define (unlabel x)
(letrec
((symlist (lambda (x r)
(cond ((null? x) (reverse r))
(#t (symlist (cdr x) (cons (caar x) r))))))
(vallist (lambda (x r)
(cond ((null? x) (reverse r))
(#t (vallist (cdr x)
(cons (_unlabel (cadar x)) r))))))
(term (lambda (x)
(_unlabel (caddr x))))
(make-lambda (lambda (x)
(append
(list (list 'lambda
(symlist (cadr x) '())
(term x)))
(vallist (cadr x) '()))))
(_unlabel (lambda (x)
(cond ((not (pair? x)) x)
((eq? (car x) 'quote) x)
((eq? (car x) 'let) (make-lambda x))
(#t (cons (_unlabel (car x))
(_unlabel (cdr x))))))))
(_unlabel x)))
Using UNLABEL, you can transform LET into LAMBDA:
(unlabel '(let ((a 3) (b 7)
(c 9) (d 4))
(let ((a-b (abs (- a b)))
(c-d (abs (- c d))))
(sqrt (+ (* a-b) (* c-d))))))
=> ((lambda (a b c d)
((lambda (a-b c-d)
(sqrt (+ (* a-b) (* c-d))))
(abs (- a b)) (abs (- c d))))
3 7 9 4)
Now I would like to see a similarly short example in Java that
performs the same transformation on a program contained in a
string.
--
Nils M Holm <n m h @ t 3 x . o r g> -- http://www.t3x.org/nmh/
Lauri Alanko
Pascal Costanza <p...@p-cos.net> wrote:
> Lauri Alanko wrote:
> > This was true in old lisps. Not in Scheme or in Common Lisp. In
> > Scheme, code is code and data is data.
>
> I don't know what you exactly mean by that, but I believe that in the
> case of Common Lisp you're not right. (But this is probably not the
> right place to discuss this.)
I'm no expert myself, but here's a quote from CLtL2, 2.13:
X3J13 voted in June 1988 (FUNCTION-TYPE) to revise these
specifications. The type function is to be disjoint from cons
and symbol, and so a list whose car is lambda is not, properly
speaking, of type function, nor is any symbol. However,
standard Common Lisp functions that accept functional
arguments will accept a symbol or a list whose car is lambda
and automatically coerce it to be a function; such standard
functions include funcall, apply, and mapcar. Such functions
do not, however, accept a lambda-expression as a functional
argument; therefore one may not write
(mapcar '(lambda (x y) (sqrt (* x y))) p q)
but instead one must write something like
(mapcar #'(lambda (x y) (sqrt (* x y))) p q)
This change makes it impermissible to represent a lexical
closure as a list whose car is some special marker.
Frankly, I'm not sure what this means, the "will accept a symbol or a
list whose car is lambda" and "do not ... accept a lambda-expression"
seem to me to be contradictory, but as I said, I'm no expert. I just
tried with CLISP:
[1]> (setq x '(lambda (x) (* x x)))
(LAMBDA (X) (* X X))
[2]> (funcall x 3)
*** - FUNCALL: argument (LAMBDA (X) (* X X)) is not a function.
To get a function in the current environment, write (FUNCTION ...).
To get a function in the global environment, write (COERCE '...
'FUNCTION).
So either I'm right or CLISP is wrong.
(Yeah, off-topic really, but experience has shown that the less
interaction there is between comp.lang.scheme and .lisp, the more
peaceful the world is...)
Lauri
Ulrich Hobelmann
> In article <1137578367....@g14g2000cwa.googlegroups.com>,
> <bobu...@yahoo.com> wrote:
>> Newbie: can you explain programs-as-data?
>>
>> Hacker: In Lisp programs are represented as lists which means that you
>> can manipulate programs as data. This is for instance important when
>> you write interpreters.
>
> This was true in old lisps. Not in Scheme or in Common Lisp. In
> Scheme, code is code and data is data. Their relationship is simply
> that:
>
> * the syntax for expressions is similar to the syntax for literal
> <datum>s in the grammar
> * properly formed data structures can be evaluated as code with eval
At least in Common Lisp macros are built this way: the macro gets the
original data (in list format) and can return another expression (in
list format).
Some Scheme macro systems are different.
Another nice thing is that Lisp/Scheme programs, and also those in other
languages, as well as configuration files etc. you might create can
simply be parsed by the builtin READ function.
Code isn't = data, but the syntactic representation is = data in Lisps.
--
The problems of the real world are primarily those you are left with
when you refuse to apply their effective solutions.
Edsger W. Dijkstra
Lauri Alanko
<bobu...@yahoo.com> wrote:
> So what's this talk about code=data and the big thing as this being one
> of the great strengths of Lisp?
Frankly, I'm not sure. It _might_ refer to eval, which _is_ handy
every now and then, and lisps and schemes arguably support it better
than other languages. Or it _might_ simply refer to the fact that
procedures (which contain code) are first-class objects and you can
manipulate them pretty much the same way as you can manipulate data
(except that you can't look inside them).
One could just say that Lisp has built-in data structures (lists and
symbols etc.) that are (sometimes) perfectly suited for representing
syntax trees of Lisp programs. The external appearance of Lisp
programs and the printed representation of the data structures happens
to be exactly the same, but I find this more confusing than convenient.
(I've discussed this before, see the thread starting at
<http://google.com/groups?selm=b40nnl$dga$1...@la.iki.fi>)
In addition to eval, some macro systems, too, treat list structures as
code, allowing the macro writer to operate on a list structure to
define a code transformation these kinds of macros are traditional in
older lisps, and are usually called "defmacro" -style macros:
> (defmacro (unless test result) (list 'if (list 'not test) result 0))
> (unless (= 0 1) (display "Eek\n"))
Eek
> (unless (= 1 1) (display "Eek\n"))
0
Here we define a macro named "unless" that, when called, gets the
unevaluated arguments (here a list beginning with the symbol '= and
another one beginning with the symbol 'display), and returns a list
beginning with the symbol 'if. Effectively, the result of the first
call is interpreted as the _code_ (if (not (= 0 1)) (display "Eek\n")
0), but this transformation between the code and the list structures
happens conceptually in the macro expander.
Note that although defmacro-style macros are traditional in Lisp, they
are not very popular in Scheme culture. In Scheme, one either uses
syntax-rules macros, which are plain pattern-matching, or syntax-case,
where one manipulates syntax objects which are _not_ lists but are
another first-class representation for code.
Lauri
Pascal Bourguignon
> Lauri, I'm totally confused now. I must have misunderstood everything
> people have been saying to me for the last two weeks??
>
> Lauri Alanko wrote:
>> In Scheme, code is code and data is data. Their relationship is simply that:
>>* the syntax for expressions is similar to the syntax for literal <datum>s in the grammar
>>* properly formed data structures can be evaluated as code with eval
>
> So what's this talk about code=data and the big thing as this being one
> of the great strengths of Lisp?
Scheme is a little special here.
In Common Lisp, we still have code=data:
(defmacro with-bindings (bindings expression)
(format t "in macro: data: expression=~S~% source code: " expression)
(print `(let ,bindings ,expression)))
[314]> (with-bindings ((x 2) (y 3)) (* x y))
in macro: data: expression=(* X Y) |printed
source code: | by the
(LET ((X 2) (Y 3)) (* X Y)) | macro
6 |result of the form
[315]> (macroexpand-1 '(with-bindings ((x 2) (y 3)) (* x y)))
in macro: data: expression=(* X Y) |printed
source code: | by the
(LET ((X 2) (Y 3)) (* X Y)) | macro
(LET ((X 2) (Y 3)) (* X Y)) ; |result of the
T | macroexpansion.
But there are also macros in Scheme, so we can say that code=data is
also demonstrated in Scheme macros.
Another example (implementation dependant; it wouldn't work with most
compilers, but it works with most interpreters):
[316]> (defparameter data (list '* 3 'x))
DATA
[323]> (defun exemple (x) (incf (second data)) #1=#.data)
EXEMPLE
[324]> (fdefinition 'exemple)
#<FUNCTION EXEMPLE (X) (DECLARE (SYSTEM::IN-DEFUN EXEMPLE))
(BLOCK EXEMPLE (INCF (SECOND DATA)) (* 3 X))>
[325]> (exemple 1)
4
[326]> (exemple 1)
5
[327]> (exemple 1)
6
[328]> (fdefinition 'exemple)
#<FUNCTION EXEMPLE (X) (DECLARE (SYSTEM::IN-DEFUN EXEMPLE))
(BLOCK EXEMPLE (INCF (SECOND DATA)) (* 6 X))>
[329]>
Here, the list referenced by data is also referenced as the last
expression in the body of EXEMPLE, ie. the same list is data and
_code_ at the same time. EXEMPLE is self-modifying: when the
interpreter executes (INCF (SECOND DATA)) it is modifying the second
element of the following expression in the source of the function.
This kind of self-modifying code is not specified, you will get
different results when compiling and in other implementations:
[329]> (compile 'exemple)
EXEMPLE ;
NIL ;
NIL
[330]> (exemple 1)
6
[331]> (exemple 1)
6
[332]> (exemple 1)
6
So this kind of code=data is frowned upon,
Such a function could be implemented as a closure instead:
(let ((factor 3))
(defun exemple (x)
(incf factor)
(* factor x)))
(list (exemple 1) (exemple 1) (exemple 1)) --> (4 5 6)
but the first EXEMPLE shows that code and data are the same in (some)
lisp interpreters.
Finally, since functions are first class object, you can consider that
code=data:
(define data (lambda (x) (+ 2 x)))
(let ((x data))
(if (eq data x) ; is it not data here?
(display "identical")
(display "something else")))
However:
(data 3) ; is it not code here?
More over, in Common Lisp, you can recover (if the implementation
allows it) the source of the code:
[340]> (defparameter data (lambda (x) (* 2 x)))
DATA
[341]> (let ((x data)) (format t "~:[something else~;identical~]~%" (eq x data)))
identical
NIL
[342]> (funcall data 3)
6
[345]> (function-lambda-expression data)
(LAMBDA (X) (* 2 X)) ;
#(NIL NIL NIL NIL ((DECLARATION XLIB::CLX-VALUES VALUES OPTIMIZE DECLARATION))) ;
:LAMBDA
;; even when compiled:
[350]> (compile nil data)
#<COMPILED-FUNCTION NIL> ;
NIL ;
NIL
[352]> (third (function-lambda-expression data))
(* 2 X)
[353]>
> Lauri Alanko wrote:
> > > (car '(define (f x) (* x x)))
>>This doesn't demonstrate anything. You have a list structure that
> happens to resemble the structure of a Scheme definition. This is not
> program-as-data, this is just data-as-data
>
> Can you give me an example then that demonstrates this code-as-data
> that people have been talking about.
See above. But not in Scheme.
> I'm glad for your post Lauri because I've been little suspicous about
> the whole thing myself. I hope that you and the other Schemers can
> shed some light here.
>
> Bob
--
__Pascal Bourguignon__ http://www.informatimago.com/
This universe shipped by weight, not volume. Some expansion may have
occurred during shipment.
Matthias Blume
> Lauri, I'm totally confused now. I must have misunderstood everything
> people have been saying to me for the last two weeks??
>
> Lauri Alanko wrote:
>> In Scheme, code is code and data is data. Their relationship is simply that:
>>* the syntax for expressions is similar to the syntax for literal <datum>s in the grammar
>>* properly formed data structures can be evaluated as code with eval
>
> So what's this talk about code=data and the big thing as this being one
> of the great strengths of Lisp?
It's just that: talk.
The fact that (source-)code looks like data mostly means that it
can be read (parsed) as data by a built-in function like Lisp's READ.
So it relieves the programmer from one fairly trivial aspect of
the problem, namely that of parsing.
The downside of this is that the datastructure used to represent code
(namely s-expressions) does not enforce many of the additional
syntactic constraints that govern code, so any user program
manipulating such data must either enforce these constraints
explicitly or be prepared to handle values that do not correspond to
valid code.
I think that the situation is at best a wash. Personally I don't mind
writing a parser every now and then if it let's me rely on statically
enforced invariants down the line.
> Lauri Alanko wrote:
> > > (car '(define (f x) (* x x)))
>>This doesn't demonstrate anything. You have a list structure that
> happens to resemble the structure of a Scheme definition. This is not
> program-as-data, this is just data-as-data
>
> Can you give me an example then that demonstrates this code-as-data
> that people have been talking about.
>
> I'm glad for your post Lauri because I've been little suspicous about
> the whole thing myself. I hope that you and the other Schemers can
> shed some light here.
It is a good thing to be suspicious here.
Cheers,
Matthias
Matthias Blume
> The fact that code and data are represented using the same syntax is
> one of the things that makes macros so easy to do.
... and so easy to get them wrong.
Matthias
Jens Axel Søgaard
> bobu...@yahoo.com writes:
>>So what's this talk about code=data and the big thing as this being one
>>of the great strengths of Lisp?
> It's just that: talk.
I agree.
--
Jens Axel Søgaard
Ulrich Hobelmann
Well, a macro can be seen as a small compiler. Some macros merely bind
variables, others implement whole parser generators.
Macros should only be used where functions are awkward or impossible to use.
Joe Marshall
Matthias Blume wrote:
... and I get to reply!
> The fact that (source-)code looks like data mostly means that it
> can be read (parsed) as data by a built-in function like Lisp's READ.
> So it relieves the programmer from one fairly trivial aspect of
> the problem, namely that of parsing.
It's certainly a trivial problem in Lisp (in the sense of it being
obvious and easy),
and it is a trivial problem in general (in the sense of having a very
simple solution: use Cambridge Polish notation. Problem solved.), but
it definitely non-trivial when some idiot decides that
context-sensitivity, arbitrary lookahead, and ambiguity is more
`natural looking'.
> The downside of this is that the datastructure used to represent code
> (namely s-expressions) does not enforce many of the additional
> syntactic constraints that govern code, so any user program
> manipulating such data must either enforce these constraints
> explicitly or be prepared to handle values that do not correspond to
> valid code.
>
> I think that the situation is at best a wash. Personally I don't mind
> writing a parser every now and then if it let's me rely on statically
> enforced invariants down the line.
I agree that we should move beyond QUOTE, CAR and CDR for
metalinguistic processing, but compared to languages that lack even
these primitive structures, they are amazingly powerful and useful.
Matthias Blume
Pascal Costanza
Try (setq x (coerce '(lambda (x) (* x x)) 'function)) or (setq x
(compile nil '(lambda (x) (* x x)))).
Ulrich Hobelmann
> The fact that (source-)code looks like data mostly means that it
> can be read (parsed) as data by a built-in function like Lisp's READ.
> So it relieves the programmer from one fairly trivial aspect of
> the problem, namely that of parsing.
But not having to write a parser makes experimentation easier. I'd
*love* some intermediate solution that doesn't involve parser generation
à la yacc/ml-yacc.
Scheme's syntax-rules are a little bit into that direction in that they,
unlike Lisp macros, pattern-match, both on the first symbol and on later
ones in the middle of an expression. But they can't generate code, only
rewrite it.
Maybe there shouldn't be categories like "parsing" and "macro rewrite
system".
> The downside of this is that the datastructure used to represent code
> (namely s-expressions) does not enforce many of the additional
> syntactic constraints that govern code, so any user program
> manipulating such data must either enforce these constraints
> explicitly or be prepared to handle values that do not correspond to
> valid code.
Yes, but often that's unavoidable for powerful code manipulation (think
any compiler, no matter how easy; there used to be attributes in
compilers that were directly part of the parsing stage, but nobody uses
them anymore, because it's not convenient nor modular). It'd be nice to
have a pattern-matcher that produces a meaningful error message at
times, like "THEN expected here". The one time I tried to put error
handling into a yacc/bison parser I was hopelessly helpless, and I'm not
at war with LR parsing. I simply couldn't figure out where I had to put
error rules so that they worked. Not at all intuitive, unlike the
understandable rest of LR.
> I think that the situation is at best a wash. Personally I don't mind
> writing a parser every now and then if it let's me rely on statically
> enforced invariants down the line.
Not if it were more convenient. I've only done yacc, ml-yacc and
hand-coded parsers, but even ml-yacc isn't exactly what I consider very
convenient. Lisp is, even though the lexical syntax is fixed.
Matthias Blume
> Matthias Blume wrote:
>> The fact that (source-)code looks like data mostly means that it
>> can be read (parsed) as data by a built-in function like Lisp's READ.
>> So it relieves the programmer from one fairly trivial aspect of
>> the problem, namely that of parsing.
>
> But not having to write a parser makes experimentation easier. I'd
> *love* some intermediate solution that doesn't involve parser
> generation à la yacc/ml-yacc.
For experimentation, working at the level of the algebraic datatype
(e.g., for ASTs or for IR) is usually sufficient. If this gets too
clumsy, writing a trivial, simple parser is easy.
> Scheme's syntax-rules are a little bit into that direction in that
> they, unlike Lisp macros, pattern-match, both on the first symbol and
> on later ones in the middle of an expression. But they can't generate
> code, only rewrite it.
What's the difference between "generate code" and "rewrite code"?
>> The downside of this is that the datastructure used to represent code
>> (namely s-expressions) does not enforce many of the additional
>> syntactic constraints that govern code, so any user program
>> manipulating such data must either enforce these constraints
>> explicitly or be prepared to handle values that do not correspond to
>> valid code.
>
> Yes, but often that's unavoidable for powerful code manipulation
> (think any compiler, no matter how easy; there used to be attributes
> in compilers that were directly part of the parsing stage, but nobody
> uses them anymore, because it's not convenient nor modular).
Yes, there are always tradeoffs, and I like to be somewhere on middle
ground.
> It'd be
> nice to have a pattern-matcher that produces a meaningful error
> message at times, like "THEN expected here". The one time I tried to
> put error handling into a yacc/bison parser I was hopelessly helpless,
> and I'm not at war with LR parsing. I simply couldn't figure out
> where I had to put error rules so that they worked. Not at all
> intuitive, unlike the understandable rest of LR.
Yes, yacc's "error" rules are evil. I prefer to work with something
like ml-yacc, where the error handling/error recovery does not require
these ad-hoc error rules. LL parsing, while less powerful, has the
advantage of being able to use RD parsers which usually have very
predictable behavior in terms of error reporting. For simple enough
languages (see: experimentation), I'd usuall make up some simple LL
language and write an equally simple RD parser.
>> I think that the situation is at best a wash. Personally I don't mind
>> writing a parser every now and then if it let's me rely on statically
>> enforced invariants down the line.
>
> Not if it were more convenient. I've only done yacc, ml-yacc and
> hand-coded parsers, but even ml-yacc isn't exactly what I consider
> very convenient. Lisp is, even though the lexical syntax is fixed.
I've been on both sides, too, and I take ml-yacc over car/cdr style
syntax analysis any day. But the real problem is that even after I've
done my car/cdr style analysis, I still can't encode the now enforced
invariants in the definition of the data structure, so I have to code
defensively against my own errors.
Matthias
Brian Harvey
>Lauri, I'm totally confused now. I must have misunderstood everything
>people have been saying to me for the last two weeks??
Here is the subtlety I think you're not getting: We don't all have the
same opinions about everything! :-)
There's no question that the code=data thing was crucial in getting Lisp
off the ground in the first place, based on McCarthy's reminiscing.
There's also no question that, as time has passed, people have learned
to use EVAL less, largely because of the way lexically scoped Lisps let
procedures encapsulate expressions-you-want-evaluated.
So, this leaves open to differences of opinion exactly where code-as-data
ranks among the coolnesses of Lisp. You're never going to get an
authoritative answer about that.
My own view is somewhere in the middle. As a teacher, I think it's great
that I can show freshmen a Scheme interpreter without having to spend any
time at all on parsing. But when it comes to other programming tasks,
I never eval a list -- the change from dynamic to lexical scope made that
*less* useful.
I do think it's pretty, though.
Ulrich Hobelmann
>> Scheme's syntax-rules are a little bit into that direction in that
>> they, unlike Lisp macros, pattern-match, both on the first symbol and
>> on later ones in the middle of an expression. But they can't generate
>> code, only rewrite it.
>
> What's the difference between "generate code" and "rewrite code"?
I meant that syntax-rules can't for instance, be something like yacc,
creating whole new code; they can only rewrite code that's already there
(as parameter), maybe add or subtract something.
Well, in theory you can do all kinds of programming with them, but not
in an reasonable way.
> Yes, yacc's "error" rules are evil. I prefer to work with something
> like ml-yacc, where the error handling/error recovery does not require
> these ad-hoc error rules. LL parsing, while less powerful, has the
> advantage of being able to use RD parsers which usually have very
> predictable behavior in terms of error reporting. For simple enough
> languages (see: experimentation), I'd usuall make up some simple LL
> language and write an equally simple RD parser.
Yes. I'm not even sure if there exist grammars that *need* LR. Mostly
you can create a good LL grammar for anything useful.
> I've been on both sides, too, and I take ml-yacc over car/cdr style
> syntax analysis any day. But the real problem is that even after I've
Well, mostly in Lisp you can have a CASE and then some
destructuring-bind; in Scheme two or three CAR/CDRs are fine, too. Of
course for a real parser (if you have to write one) you want some kind
of yacc.
> done my car/cdr style analysis, I still can't encode the now enforced
> invariants in the definition of the data structure, so I have to code
> defensively against my own errors.
Not sure what you mean with that. Even in ML you can't encode
invariants about abstract syntax, or I woulnd't know how. Syntax is
just ... syntax.
Pascal Bourguignon
CLtL2 is not the Common Lisp ANSI Standard. There have been changes
between CLtL2 and the ANSI Standard, including this point.
According to the standard, FUNCALL takes a _function designator_
and:
function designator n. a designator for a function; that is, an
object that denotes a function and that is one of: a symbol
(denoting the function named by that symbol in the global
environment), or a function (denoting itself). The consequences
are undefined if a symbol is used as a function designator but it
does not have a global definition as a function, or it has a
global definition as a macro or a special form. See also extended
function designator.
So, only symbols and function objects.
The other change is that in ANSI Common Lisp, LAMBDA is a macro:
(defmacro lambda (args &body body) `(function (lambda ,args ,@body)))
Therefore there's no need anymore for the ugly #' in front of (lambda ...):
[8]> (setq x (lambda (x) (* x x)))
#<FUNCTION :LAMBDA (X) (* X X)>
[9]> (mapcar x '(1 2 3))
(1 4 9)
[10]> (mapcar (lambda (x) (1+ x)) '(1 2 3))
(2 3 4)
[11]>
--
__Pascal Bourguignon__ http://www.informatimago.com/
What is this talk of 'release'? Klingons do not make software 'releases'.
Our software 'escapes' leaving a bloody trail of designers and quality
assurance people in it's wake.
Joe Marshall
Brian Harvey wrote:
> bobu...@yahoo.com writes:
> >Lauri, I'm totally confused now. I must have misunderstood everything
> >people have been saying to me for the last two weeks??
>
> Here is the subtlety I think you're not getting: We don't all have the
> same opinions about everything! :-)
Those of us who are right do. The other randoms, well.....
Marcin 'Qrczak' Kowalczyk
> Here is a more elaborate example:
Nice but wrong: mishandles quasiquote and macros.
--
__("< Marcin Kowalczyk
\__/ qrc...@knm.org.pl
^^ http://qrnik.knm.org.pl/~qrczak/
Nils M Holm
> Nils M Holm <before-2...@online.de> writes:
>
> > Here is a more elaborate example:
>
> Nice but wrong: mishandles quasiquote and macros.
I did not say that this example has any use in the real world.
This does not make it "wrong", though.