Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

Lisp's unique feature: compiler available at run-time

486 views
Skip to first unread message

Shayne Wissler

unread,
Jul 22, 2002, 2:03:09 PM7/22/02
to
At http://www.paulgraham.com/diff.html, Paul Graham says that the feature of
having the "whole language always available" is unique to lisp.

With lisp, he iterates that you can:
1) Compile and run code while reading
2) Read and run code while compiling
3) Read and compile code at run-time

Although I'm clear on what is technologically possible with respect to
having a compiler available at run-time, I'm not precisely clear on what
Paul means in the above. I *think* he means:

1. While parsing a lisp program, you can compile and invoke some of the code
you already parsed to help parse the subsequent code, allowing you to modify
the syntax on the fly. If this is correct, what are the limitations? Can I
write a lisp program that starts out as lisp and starts looking like C++?

2. While compiling a lisp program, you can execute special-purpose code that
makes semantic transformations of the lisp executable. When you see a given
input program, you can generate whatever code you want as a function of it.
This is called "macros" in lisp, but it is nothing like textual macros;
really it's a program generating another program at the binary or byte-code
level.

3. A running lisp program can read (or write), compile, and execute any
program, in the current program's execution context. I.e., if the newly
loaded program makes references to variables and functions that are only
defined in the original program, they are dynamically linked and accessible
as the program is compiled.

A few questions:
- Have I got it right?
- What are the limitations?
- What other languages have at least one of the above?
- Is it really true that no other language has all of the above?


Shayne Wissler

Thomas Bushnell, BSG

unread,
Jul 22, 2002, 2:09:28 PM7/22/02
to
"Shayne Wissler" <thal...@yahoo.com> writes:

> A few questions:
> - Have I got it right?
> - What are the limitations?
> - What other languages have at least one of the above?
> - Is it really true that no other language has all of the above?

Smalltalk, self, and other such systems have also always had those
capabilities.

Johan Kullstam

unread,
Jul 22, 2002, 2:45:51 PM7/22/02
to
"Shayne Wissler" <thal...@yahoo.com> writes:

> At http://www.paulgraham.com/diff.html, Paul Graham says that the feature of
> having the "whole language always available" is unique to lisp.
>
> With lisp, he iterates that you can:
> 1) Compile and run code while reading
> 2) Read and run code while compiling
> 3) Read and compile code at run-time
>
> Although I'm clear on what is technologically possible with respect to
> having a compiler available at run-time, I'm not precisely clear on what
> Paul means in the above. I *think* he means:
>
> 1. While parsing a lisp program, you can compile and invoke some of the code
> you already parsed to help parse the subsequent code, allowing you to modify
> the syntax on the fly. If this is correct, what are the limitations? Can I
> write a lisp program that starts out as lisp and starts looking like
> C++?

I am not sure that you could easily warp Lisp's reader into a C++
parser. Lisp has a very simple and regular syntax which makes reading
and parsing the Lisp source much easier than most languages.

> 2. While compiling a lisp program, you can execute special-purpose code that
> makes semantic transformations of the lisp executable. When you see a given
> input program, you can generate whatever code you want as a function of it.
> This is called "macros" in lisp, but it is nothing like textual macros;
> really it's a program generating another program at the binary or byte-code
> level.

Macros are text engines. Basically (someone correct me if I make a
technical error) a macro is a program (or function) which is called
right before compile time. Macros look like regular Lisp functions
(they appear at the front of a list) but are run as the source is
processed. C has a preprocessor which does much the same thing but
Lisp has 1) a very regular syntax 2) resonably good built-in
functionality for handling Lisp code (which look like nested lists)
and 3) can be extended by user functions and macros available at
compile time.

> 3. A running lisp program can read (or write), compile, and execute any
> program, in the current program's execution context. I.e., if the newly
> loaded program makes references to variables and functions that are only
> defined in the original program, they are dynamically linked and accessible
> as the program is compiled.

There isn't really any "linking" per se. That's a static language
concept. What Lisp will let you do is write a function and compile it
right there. From what I've encountered in a limited experience, two
kinds of this

1) closure

In this style you have most of the structure of the function but at
compile time do not know some variables. Later on, you can fill in
these parameters and compile a function with these hard-coded in. You
could have many functions with various parameter settings based off of
one general function form.

(defun make-adder (s)
(compile nil (lambda (x) (+ x s))))

calling make-adder will return a function which adds s.

this function creates and compiles another function at _run_ time
[4] (defun make-adder (s)
(compile nil (lambda (x) (+ x s))))
MAKE-ADDER

now attach functions to labels
[5]> (defvar foo (make-adder 3))
FOO
[6]> (defvar bar (make-adder 14))
BAR

call the functions we just created in steps 5 and 6
[7]> (funcall foo 39)
42
[8]> (funcall bar 12)
26

see what foo and bar, they are compiled functions
[9]> foo
#<COMPILED-CLOSURE NIL>
[10]> bar
#<COMPILED-CLOSURE NIL>


2) eval

In C and other languages it is easy to read, e.g., a number. C has
trouble with strings because it has a hard time with dynamic sized
objects but these are also very doable. Reading a _function_ at
run-time is hard. How do you supply
double func(double x) { return x * x; }
at _run-time_ in C or C++? You don't.

Lisp can read, evaluate and compile
(lambda (x) (* x x))
as easily as it can do 3.14.


> A few questions:
> - Have I got it right?
> - What are the limitations?
> - What other languages have at least one of the above?
> - Is it really true that no other language has all of the above?

I believe other languages have all the above. They are just not
called C or C++. Hope this helps.

--
Johan KULLSTAM <kulls...@attbi.com> sysengr

Topmind

unread,
Jul 22, 2002, 3:47:38 PM7/22/02
to

Most scripting languages that have operations like Eval(), Execute(),
and Include() also have a similar dynamic feel. (Some argue that
these lack closures and thus certain scoping power, but I don't want to
get into that argument yet again.)

-T-

Marco Antoniotti

unread,
Jul 22, 2002, 4:18:04 PM7/22/02
to

top...@technologist.com (Topmind) writes:

Yes. But the main difference with (Common) Lisp is that this has all
the mentioned features available all the time. AFAIK only Prolog and
Smalltalk come close. The so-called scripting languages cannot really
deal with their own code directly and as nicely as CL does, since
their basic data structure to manipulate this sort of things is a
string. Besides, the function `compile' in many CL implementations
produces x86, Sparc, MIPS, HPPA, PPC and other assembly du jour.

... of course, you can always take your favourite scripting language
and apply for the nth time Greenspun's Tenth Rule of Programming to
it. :)

Cheers

--
Marco Antoniotti ========================================================
NYU Courant Bioinformatics Group tel. +1 - 212 - 998 3488
719 Broadway 12th Floor fax +1 - 212 - 995 4122
New York, NY 10003, USA http://bioinformatics.cat.nyu.edu
"Hello New York! We'll do what we can!"
Bill Murray in `Ghostbusters'.

Dave Bakhash

unread,
Jul 22, 2002, 2:37:16 PM7/22/02
to
"Shayne Wissler" <thal...@yahoo.com> writes:

> With lisp, he iterates that you can:

> 3) Read and compile code at run-time

^^^^^^^^^^^


> - Have I got it right?

Not for the commercial implementations.

> - What are the limitations?

Some implementations (e.g. ACL, LispWorks) limit the use of the compiler
at run-time in delivered applications. You can compile a form, but not
file.

dave

Topmind

unread,
Jul 22, 2002, 5:30:26 PM7/22/02
to
>
> top...@technologist.com (Topmind) writes:
>
> > > "Shayne Wissler" <thal...@yahoo.com> writes:
> > >
> > > > A few questions:
> > > > - Have I got it right?
> > > > - What are the limitations?
> > > > - What other languages have at least one of the above?
> > > > - Is it really true that no other language has all of the above?
> > >
> > > Smalltalk, self, and other such systems have also always had those
> > > capabilities.
> > >
> >
> > Most scripting languages that have operations like Eval(), Execute(),
> > and Include() also have a similar dynamic feel. (Some argue that
> > these lack closures and thus certain scoping power, but I don't want to
> > get into that argument yet again.)
>
> Yes. But the main difference with (Common) Lisp is that this has all
> the mentioned features available all the time. AFAIK only Prolog and
> Smalltalk come close. The so-called scripting languages cannot really
> deal with their own code directly and as nicely as CL does, since
> their basic data structure to manipulate this sort of things is a
> string. Besides, the function `compile' in many CL implementations
> produces x86, Sparc, MIPS, HPPA, PPC and other assembly du jour.

Are you saying that it runs faster, or that it makes
for 'nicer code'?

>
> ... of course, you can always take your favourite scripting language
> and apply for the nth time Greenspun's Tenth Rule of Programming to
> it. :)
>
> Cheers
>
> --

-T-

Thaddeus L Olczyk

unread,
Jul 22, 2002, 6:02:57 PM7/22/02
to
On 22 Jul 2002 14:37:16 -0400, Dave Bakhash <ca...@alum.mit.edu>
wrote:

Actually, I believe that they require a different license ( to be read
as "more expensive" ) if you want to deliver the full compiler.


As to the original question, this feature is not unique. Read Eric
Raymonds support of Python, where he points this out as one of Pythons
better feature ( and gives the example of fetchmailrc ).

David Golden

unread,
Jul 22, 2002, 6:04:39 PM7/22/02
to
Shayne Wissler wrote:

> At http://www.paulgraham.com/diff.html, Paul Graham says that the feature
> of having the "whole language always available" is unique to lisp.
>

Hyperbole at best. Forth, for example, also has the whole language
available, including "parser", such as it is. Through CREATE DOES>
one can do a whole lot of funny business reminiscent of lisp, only
it will happily run on an 8-bit embedded microcontroller :-)

That said, I'd rather use Lisp for most things, but Forth is something
that programmers should be aware of, if only for the different
perspective...

See http://www.forth.org/


--
Don't eat yellow snow.

Thaddeus L Olczyk

unread,
Jul 22, 2002, 6:15:27 PM7/22/02
to

And to learn to think sideways.

Marco Antoniotti

unread,
Jul 22, 2002, 6:35:32 PM7/22/02
to

top...@technologist.com (Topmind) writes:

...

> > > Most scripting languages that have operations like Eval(), Execute(),
> > > and Include() also have a similar dynamic feel. (Some argue that
> > > these lack closures and thus certain scoping power, but I don't want to
> > > get into that argument yet again.)
> >
> > Yes. But the main difference with (Common) Lisp is that this has all
> > the mentioned features available all the time. AFAIK only Prolog and
> > Smalltalk come close. The so-called scripting languages cannot really
> > deal with their own code directly and as nicely as CL does, since
> > their basic data structure to manipulate this sort of things is a
> > string. Besides, the function `compile' in many CL implementations
> > produces x86, Sparc, MIPS, HPPA, PPC and other assembly du jour.
>
> Are you saying that it runs faster, or that it makes
> for 'nicer code'?

Both. First of all it runs as fast as any assembly code produced by a
native compiler (i.e. not a byte compiler, given all the necessary
qualifications). Second, (Common) Lisp machinery to deal with
Lisp-like languages (hence itself) is simply unparalled in other
environments (save maybe, AFAIK, Smalltalk and Prolog environments).

Current "scripting languages" simply do not come close both in
manipulation easiness and expressiveness *and* speed of the generated
code.

> > ... of course, you can always take your favourite scripting language
> > and apply for the nth time Greenspun's Tenth Rule of Programming to
> > it. :)

... and the above still stands :)

JRStern

unread,
Jul 22, 2002, 7:08:00 PM7/22/02
to
On Mon, 22 Jul 2002 18:03:09 GMT, "Shayne Wissler"
<thal...@yahoo.com> wrote:
>At http://www.paulgraham.com/diff.html, Paul Graham says that the feature of
>having the "whole language always available" is unique to lisp.
>
>With lisp, he iterates that you can:
> 1) Compile and run code while reading
> 2) Read and run code while compiling
> 3) Read and compile code at run-time
>
>Although I'm clear on what is technologically possible with respect to
>having a compiler available at run-time, I'm not precisely clear on what
>Paul means in the above. I *think* he means:
>
>1. While parsing a lisp program, you can compile and invoke some of the code
>you already parsed to help parse the subsequent code, allowing you to modify
>the syntax on the fly. If this is correct, what are the limitations? Can I
>write a lisp program that starts out as lisp and starts looking like C++?

No, it's not meant to modify the language on the fly.

As others have said, you just build up source text in a variable, then
Eval() the variable.

>2. While compiling a lisp program, you can execute special-purpose code that
>makes semantic transformations of the lisp executable. When you see a given
>input program, you can generate whatever code you want as a function of it.
>This is called "macros" in lisp, but it is nothing like textual macros;
>really it's a program generating another program at the binary or byte-code
>level.

Lisp has macros, too, but that's another subject.

>3. A running lisp program can read (or write), compile, and execute any
>program, in the current program's execution context. I.e., if the newly
>loaded program makes references to variables and functions that are only
>defined in the original program, they are dynamically linked and accessible
>as the program is compiled.

Some Lisps had that kind of dynamic scoping, others like Common Lisp
did not.

>A few questions:
> - Have I got it right?

Not especially.

> - What are the limitations?

As in any system, it's cheaper to compile once and run many times. In
the early 1960s it seemed like it might be useful to have
self-modifying code. However, this, IMHO, was an immature opinion
based on an insufficient understanding of how the Von Neuman computer
architecture works best, with separate code and data. It's hard for
us twenty-first century types to get the feel of what it was like to
do computers in the early days, not that we've come all that far,
maybe, but over the decades we've learned a few things!

> - What other languages have at least one of the above?

GWBasic could dynamically load source into a running program. Heck,
most scripting languages can write text to a disk file then invoke the
file, and it's not all that different.

> - Is it really true that no other language has all of the above?

No. Even most SQL implementations let you build and Eval() text at
runtime. The only thing Lisp has that other languages lack is a high
proportion of parenthesis.

Joshua Stern

Shayne Wissler

unread,
Jul 22, 2002, 7:25:54 PM7/22/02
to
JRStern wrote:

>> - What other languages have at least one of the above?
>
> GWBasic could dynamically load source into a running program. Heck,
> most scripting languages can write text to a disk file then invoke the
> file, and it's not all that different.

Hold on--the requirement is that the generated program can access the
variables and functions of the generating program. Otherwise C would count
as having this feature because you can generate C source, compile it, and
execute it from a C program.


Shayne Wissler

Greg Menke

unread,
Jul 22, 2002, 7:54:47 PM7/22/02
to

> >> - What other languages have at least one of the above?
> >
> > GWBasic could dynamically load source into a running program. Heck,
> > most scripting languages can write text to a disk file then invoke the
> > file, and it's not all that different.
>

Atari Basic did it a bit better, as I recall you can print new lines
of basic to the screen, then tweak the screen device driver to cause
the Basic listener to read and accept them. The result of which was
you could have your program rewrite parts of itself as desired without
terminating the running program- IIRC, all that was needed is a STOP
then a CONT after the new code to get back in. Debugging it was a
different story. Naturally all the syntax errors occured when the
tweaked driver was doing its thing, rendering the console unusable-
more or less requiring a reboot.

Greg Menke


Marco Antoniotti

unread,
Jul 22, 2002, 8:04:50 PM7/22/02
to

JXSternC...@gte.net (JRStern) writes:

> On Mon, 22 Jul 2002 18:03:09 GMT, "Shayne Wissler"

...

> >1. While parsing a lisp program, you can compile and invoke some of the code
> >you already parsed to help parse the subsequent code, allowing you to modify
> >the syntax on the fly. If this is correct, what are the limitations? Can I
> >write a lisp program that starts out as lisp and starts looking like C++?
>
> No, it's not meant to modify the language on the fly.

Actually, you can do that. You *can* modify th elanguage on
the fly.

> As others have said, you just build up source text in a variable, then
> Eval() the variable.

Yes. But the source text is totally unstructured (i.e. it is a string)
and not soething that you can manipulate easily and directly.
Moreover, the equivalent of the Compile() functionality in (Common)
Lisp gets you native code.

...

> Lisp has macros, too, but that's another subject.

Yes and no. (Common) Lisp Macros are more flexible and useful than
other construct present in other languages.

> >3. A running lisp program can read (or write), compile, and execute any
> >program, in the current program's execution context. I.e., if the newly
> >loaded program makes references to variables and functions that are only
> >defined in the original program, they are dynamically linked and accessible
> >as the program is compiled.
>
> Some Lisps had that kind of dynamic scoping, others like Common Lisp
> did not.

Common Lisp offers you both. You can mark variables to be looked up
in the dynamic environment via the `special' declaration.

>
> > - What are the limitations?
>
> As in any system, it's cheaper to compile once and run many times.

Let's define "run many times". Suppose I have a "small language
definition" that allows me to run a "function" over the elements of some
large data structure (say a graph). Suppose that such functions can
be built up in a web form.

Now, in a "regular" language, you have to (1) parse the string that
makes up the "function", (2) embed it in the "driver" (which may be
large) as text (the "driver" is text as well), (3) compile the
resulting text as a file, (4) run the newly compiled program.

In Common Lisp, you have your driver compiled and available all the
time. You just have to do (1) compile the function after minimal
manipulation and (2) run the driver which calls the compiled function
automagically.

Since the graph is large, in this case, you win by saving compilation
time.

> > - What other languages have at least one of the above?
>
> GWBasic could dynamically load source into a running program. Heck,
> most scripting languages can write text to a disk file then invoke the
> file, and it's not all that different.

I think the above example shows how different it is. Moreover, you
loose the compiler.

> > - Is it really true that no other language has all of the above?
>
> No. Even most SQL implementations let you build and Eval() text at
> runtime. The only thing Lisp has that other languages lack is a high
> proportion of parenthesis.

Plus the compiler, multiple method dispatch, plus read-write
equivalence, plus Greenspun's Tenth Rule. :)

Topmind

unread,
Jul 22, 2002, 8:23:56 PM7/22/02
to

Is this the article?

http://www.linuxjournal.com/article.php?sid=3882

I am not an email expert, but it looks like he is bragging about how easy
Python is at reinventing a database for user configurations. His classes
would be relational tables if I had my way (if anybody by chance cares).

Quote: "An important measure of effort in coding is the frequency with
which you write something that doesn't actually match your mental
representation of the problem, and have to backtrack...."

He is assuming everybody thinks like him. Common fallacy among
many programmers.

WRT LISP, it appears that he is complaining about the library
docs and user fragmentation, and not really the language itself.

-T-

Topmind

unread,
Jul 22, 2002, 8:26:57 PM7/22/02
to
>
> top...@technologist.com (Topmind) writes:
>
> ...
>
> > > > Most scripting languages that have operations like Eval(), Execute(),
> > > > and Include() also have a similar dynamic feel. (Some argue that
> > > > these lack closures and thus certain scoping power, but I don't want to
> > > > get into that argument yet again.)
> > >
> > > Yes. But the main difference with (Common) Lisp is that this has all
> > > the mentioned features available all the time. AFAIK only Prolog and
> > > Smalltalk come close. The so-called scripting languages cannot really
> > > deal with their own code directly and as nicely as CL does, since
> > > their basic data structure to manipulate this sort of things is a
> > > string. Besides, the function `compile' in many CL implementations
> > > produces x86, Sparc, MIPS, HPPA, PPC and other assembly du jour.
> >
> > Are you saying that it runs faster, or that it makes
> > for 'nicer code'?
>
> Both. First of all it runs as fast as any assembly code produced by a
> native compiler (i.e. not a byte compiler, given all the necessary
> qualifications). Second, (Common) Lisp machinery to deal with
> Lisp-like languages (hence itself) is simply unparalled in other
> environments (save maybe, AFAIK, Smalltalk and Prolog environments).


Do you have a specific semi-realistic example of
this by chance?


>
> Current "scripting languages" simply do not come close both in
> manipulation easiness and expressiveness *and* speed of the generated
> code.

I am not questioning the machine speed issue here.
I'll leave that to others who like to play with
stopwatches.

>
> > > ... of course, you can always take your favourite scripting language
> > > and apply for the nth time Greenspun's Tenth Rule of Programming to
> > > it. :)
>
> ... and the above still stands :)
>
> Cheers
>
> --
> Marco Antoniotti ========================================================
> NYU Courant Bioinformatics Group tel. +1 - 212 - 998 3488
> 719 Broadway 12th Floor fax +1 - 212 - 995 4122
> New York, NY 10003, USA http://bioinformatics.cat.nyu.edu
> "Hello New York! We'll do what we can!"
> Bill Murray in `Ghostbusters'.
>

-T-

Joel Ray Holveck

unread,
Jul 22, 2002, 8:04:48 PM7/22/02
to
>> 1. While parsing a lisp program, you can compile and invoke some of the code
>> you already parsed to help parse the subsequent code, allowing you to modify
>> the syntax on the fly. If this is correct, what are the limitations? Can I
>> write a lisp program that starts out as lisp and starts looking like
>> C++?
> I am not sure that you could easily warp Lisp's reader into a C++
> parser. Lisp has a very simple and regular syntax which makes reading
> and parsing the Lisp source much easier than most languages.

Yes, but the reader is completely modifiable in any arbitrary manner.
Some are easier than others, but there's been a number of posts to
comp.lang.lisp to create C-like syntaxes. C++ is not going to be a
trivial change, since it's a complicated syntax, but it's still
possible.

>> 2. While compiling a lisp program, you can execute special-purpose code that
>> makes semantic transformations of the lisp executable. When you see a given
>> input program, you can generate whatever code you want as a function of it.
>> This is called "macros" in lisp, but it is nothing like textual macros;
>> really it's a program generating another program at the binary or byte-code
>> level.
> Macros are text engines.

This is true in most languages, but not in Lisp.

Lisp macros operate on the structure, not on the text. The reader
will read the program fragment, then pass the already-parsed structure
into the macro handling function.

This has a few significant benefits. First, macros are much easier to
write, because you don't need to do any parsing at all: everything's
been done by the reader. You can make powerful macros with just a few
lines of code.

Second, if the reader is extended, macros still work. Since they are
called after the reader is finished, then syntactic extensions have
already been applied before the macro is called. The macro can be
blissfully unaware that it's operating on something that was created
with a syntactic extension.

>> 3. A running lisp program can read (or write), compile, and execute any
>> program, in the current program's execution context. I.e., if the newly
>> loaded program makes references to variables and functions that are only
>> defined in the original program, they are dynamically linked and accessible
>> as the program is compiled.
> There isn't really any "linking" per se. That's a static language
> concept. What Lisp will let you do is write a function and compile it
> right there. From what I've encountered in a limited experience, two
> kinds of this
> 1) closure

Closures are possible in statically linked languages, too, but aren't
often used. It's just like an indirected function call. Admittedly,
that's not static linking, but it's something that most statically
linked languages like C already have. You can actually find a paper
on how to add closures to C++ at
http://master.debian.org/~karlheg/Usenix88-lexic.pdf

(BTW: Your example called COMPILE on the closure. That's not normally
necessary: the compiler will create the compiled code when you compile
the outer function, and the runtime code binds the variables, so you
get a compiled closure.)

> 2) eval

This is a powerful feature of Lisp, but is only (properly) used in a
few circumstances.

Neither one of these is necessarily an enemy of static linking. The
basic complication is that functions can be redefined at runtime.

Here's a trivial example. Consider:

(export add-bias set-bias)
(defvar *bias* 0)
(defun set-bias (new-bias)
(setq *bias* new-bias))
(defun add-bias (sig)
(+ *bias* sig))

Here's an example of it in use:

* (add-bias 5)
5
* (set-bias 3)
3
* (add-bias 5)
8

Now, for rhetorical purposes, I'm going to rewrite this, so that it
does the same thing, but using a different technique:

(export '(add-bias set-bias))
(defun set-bias (new-bias)
(defun add-bias (sig)
(+ new-bias sig)))
(set-bias 0)

* (add-bias 5)
5
* (set-bias 3)
ADD-BIAS
* (add-bias 5)
8

Now, the point here isn't that the code created a closure, but that it
redefines the ADD-BIAS function.

So, why can ADD-BIAS not be statically linked? Because it can be
redefined at runtime.

In Lisp, functions are generally attached to symbols. The function
cell of a symbol can be changed at any time. (That's what DEFUN
does.) That's why static linking is not trivially feasible.

Now, this is a trivial example, that I wrote for rhetorical purposes.
A much more complex example may be a program that figures out
improvements to its own functions as it runs.

Hope this helps,
joelh

Topmind

unread,
Jul 22, 2002, 8:34:35 PM7/22/02
to

I know a few scripting languages that can execute code in
context.

if foo
include("myLines.txt")
end if

// Or:

myCode = "delete(scope='everything');listThem()...."
if foo
execute(myCode)
end if

As if the included code was already in there.
It can reference any vars in that existing
scope.

(I won't vouch for the speed, however. It is not
an issue most of the time.)

>
>
> Shayne Wissler
>
>

-T-

Christopher Barber

unread,
Jul 22, 2002, 8:46:12 PM7/22/02
to
"Shayne Wissler" <thal...@yahoo.com> writes:

> At http://www.paulgraham.com/diff.html, Paul Graham says that the feature of
> having the "whole language always available" is unique to lisp.
>
> With lisp, he iterates that you can:
> 1) Compile and run code while reading
> 2) Read and run code while compiling
> 3) Read and compile code at run-time
>

> ...


>
> - Have I got it right?
> - What are the limitations?
> - What other languages have at least one of the above?

Curl is one. Since Curl is a client-side language, there are some security
restrictions on what you can do while expanding a macro in Curl.

> - Is it really true that no other language has all of the above?

No, it's just hyperbole.

- Christopher

Bulent Murtezaoglu

unread,
Jul 22, 2002, 8:53:24 PM7/22/02
to
>>>>> "Topmind" == Topmind <top...@technologist.com> writes:

[... Attribution lost here by topmind, but probably Marco A. ]


>> Both. First of all it runs as fast as any assembly code
>> produced by a native compiler (i.e. not a byte compiler, given
>> all the necessary qualifications). Second, (Common) Lisp
>> machinery to deal with Lisp-like languages (hence itself) is
>> simply unparalled in other environments (save maybe, AFAIK,
>> Smalltalk and Prolog environments).


Topmind> Do you have a specific semi-realistic example of this by
Topmind> chance?


Several come to my mind, I hope others can come up with more:

Check out Paul Grahams On Lisp (free download)
http://www.paulgraham.com/onlisp.html for macrology. Chapter 25 builds a
toy OO-lisp with a few pages of code in Lisp.

For writing mini lispy languages which you then can compile, Peter Norvig's
Paradigms of Artificial Intelligence Programming has extensively worked out
examples. http://www.norvig.com/paip.html

As for calling the compiler at run time for efficiency, there should be
posted examples in the group but I only could find what I posted a while
back:

http://groups.google.com/groups?oi=djq&selm=an_269334305

cheers,

BM

Carl Shapiro

unread,
Jul 22, 2002, 9:22:08 PM7/22/02
to
Dave Bakhash <ca...@alum.mit.edu> writes:

> "Shayne Wissler" <thal...@yahoo.com> writes:

> > - What are the limitations?
>
> Some implementations (e.g. ACL, LispWorks) limit the use of the compiler
> at run-time in delivered applications. You can compile a form, but not
> file.

This is merely an economic issue, not a practical one.

JRStern

unread,
Jul 23, 2002, 1:27:22 AM7/23/02
to
On Mon, 22 Jul 2002 23:25:54 GMT, Shayne Wissler <thal...@yahoo.com>
wrote:

>> GWBasic could dynamically load source into a running program. Heck,
>> most scripting languages can write text to a disk file then invoke the
>> file, and it's not all that different.
>
>Hold on--the requirement is that the generated program can access the
>variables and functions of the generating program. Otherwise C would count
>as having this feature because you can generate C source, compile it, and
>execute it from a C program.

Well, if that's *the* requirement, ...

... maybe C does qualify, if crudely, since you can always persist the
state, gen the new program, initiate it, propagate the state, yada
yada. In SQL systems, most or all of the state resides in the
database, so generating and eval'ing SQL source on the fly is quite
useful.

Actually, I was confusing static scoping with the context for run-time
evaluation in my previous message. I don't recall just what Common
Lisp allowed, but I'm pretty sure that Interlisp allowed all the
dynamism you wanted.

Anyhow, GWBasic still seems to qualify.

Is there a practical or a principled motivation for these questions?
Pretty much whatever you can do by generating code and dynamically
loading and evaluating it, can be done pretty nearly the same way by
interpreting within a fixed body of code.

J.

Shayne Wissler

unread,
Jul 23, 2002, 1:58:06 AM7/23/02
to

"JRStern" <JXSternC...@gte.net> wrote in message
news:3d3ce837...@news.verizon.net...

> On Mon, 22 Jul 2002 23:25:54 GMT, Shayne Wissler <thal...@yahoo.com>
> wrote:
> >> GWBasic could dynamically load source into a running program. Heck,
> >> most scripting languages can write text to a disk file then invoke the
> >> file, and it's not all that different.
> >
> >Hold on--the requirement is that the generated program can access the
> >variables and functions of the generating program. Otherwise C would
count
> >as having this feature because you can generate C source, compile it, and
> >execute it from a C program.
>
> Well, if that's *the* requirement, ...
>
> ... maybe C does qualify, if crudely, since you can always persist the
> state, gen the new program, initiate it, propagate the state, yada
> yada.

C doesn't qualify.

> Is there a practical or a principled motivation for these questions?

Yes.

> Pretty much whatever you can do by generating code and dynamically
> loading and evaluating it, can be done pretty nearly the same way by
> interpreting within a fixed body of code.

Sure, and anything you can do in Lisp you can do in C.


Shayne Wissler

Software Scavenger

unread,
Jul 23, 2002, 2:54:13 AM7/23/02
to
Johan Kullstam <kulls...@attbi.com> wrote in message news:<m3d6tfm...@sysengr.res.ray.com>...

> Macros are text engines. Basically (someone correct me if I make a

In CL, macros are higher level than ordinary "text engine" macros. CL
code is represented as lists of lists, symbols, etc. You basically
have complete freedom and power to quickly and easily change the
programming language any way you want to. The same kinds of macros
you can write in CL easily are often very difficult or impossible in
C++.

Making good use of CL macros can make a big improvement in your
programming productivity, because you get so much bang for your buck,
so to speak.

Some people object to this on the grounds that changing the
programming language makes it harder for other programmers to
understand. But they're way off base. The hardest thing to
understand is the application. The technical details of the
application, and how those details are represented in the program, are
a much bigger factor in being able to understand the program than the
exact syntax is. By making good use of macros, you can make the
program much clearer, and make it a much closer match to the existing
technical jargon of the domain of the application, so you can
communciate fluently about the program and how it relates to the
application domain.

Rob Warnock

unread,
Jul 23, 2002, 5:22:04 AM7/23/02
to
JRStern <JXSternC...@gte.net> wrote:
+---------------

| As others have said, you just build up source text in a variable,
| then Eval() the variable.
+---------------

Correction: In Common Lisp, EVAL works on Lisp *objects*, not
test strings per se. Or more precisely, if you EVAL a text string,
you only get back the string itself (since strings are self-evaluating
literal objects):

(eval "(+ 2 3)") ==> "(+ 2 3)"

What you usually want to feed to EVAL is a *data structure* that
represents a valid Lisp program, e.g.:

(eval (list '+ 2 3)) ==> 5

It's the READ procedure (including variants such as READ-FROM-STRING
and the READ that's an implicit part of LOAD), with all of its
programmability (see CLHS "2.1.1 Readtables") that converts source
text into Lisp data structures.

Those resulting data structures might or might not get passed to EVAL,
depending on who did the READ and why. In the following case, it does:

(eval
(read-from-string
(coerce
(list #\( #\+ #\space #\2 #\space #\3 #\))
'string)))
==> 5

But in other cases, the application just wants to use the resulting
data structure for its own purposes.


-Rob

-----
Rob Warnock, 30-3-510 <rp...@sgi.com>
SGI Network Engineering <http://www.rpw3.org/>
1600 Amphitheatre Pkwy. Phone: 650-933-1673
Mountain View, CA 94043 PP-ASEL-IA

[Note: aaan...@sgi.com and zedw...@sgi.com aren't for humans ]

Christian Lynbech

unread,
Jul 23, 2002, 6:29:42 AM7/23/02
to

[ if anything is going to demonstrate lack of c.l.l maturity it is to
start a thread with that kind of subject :-) ]

The talk of lisp and unique features has prompted me to post my own
little pet theory as to what makes lisp the One True Programming Language.

Lisp in general and Common Lisp in particular has lots of features,
many of which can be found in other languages as well, including some
without direct heritage such as Dylan.

Its kind of like the reverse of Greenspuns 10. rule of programming (I
suggest we name it the Greenspun Corollary): Any programming language
can be turned halfway into an ad hoc informally-specified bug-ridden
slow version of Common Lisp through a lot of hard work.

So why can't one just ride all the way down that road and add stuff to
say Java and have the best of both worlds? What is lost when moving
features, one at a time from Lisp to other languages?

I believe that the essential quality that Lisp has, the one that all
the other features ultimately relies upon and contributes to,
is *linguistic flexibility*.

Think about the process of programming, what is it? It is a
conversation between the programmer and the machine in which the
programmer is explaining to the machine how it must go about solving a
particular problem, in other words a linguistic process.

This process is however very difficult because there is a huge gap
between the abstraction level of the programmers understanding of the
problem (it would be on par with "write me an accounting system") and
the level of abstraction the machine is able to understand. So not
only must the programmer figure out how to phrase the explanation, but
the programmer must at the same time enhance his or hers understanding
of the problem in order to be able to put the solution into the right
words.

In order to cross the gap of abstraction levels, a bridge must be
built, and the building blocks of such a bridge are abstractions.
These abstractions are essentially linguistic artifacts and thus the
linguistic power of the tool becomes the dominant (even if heavily
obscured) quality affecting the efficiency by which the programmer is
able to work.

Efficiency is important to remember here. Working in a language with a
low degree of linguistic power does not mean that the program cannot
be written, it only requires more resources to do so. This is simply
the Turing tar pit at play. Mathematically we know that all
programming languages are equally powerful, meaning that we know that
the *result* will be the same, no matter which tools we employ, but
the *efficiency* by which the result is reached may differ
radically. Our old math professors may not have cared much how we
arrive at programs, but the people that signs our paychecks definitely
will! (Alledgely, a mathematician at my old university should at one
point have dismmissed all of computer science as nothing more than
applied monad theory.)

The linguistic power of Lisp is however not very easy to transfer to
other programming languages without turning them into Lisp in the
process. It is not just macros or sequence functions or mapping
operators or higher order functions, but also symbols (what other
programming language would allow spaces in function names) and program
structure (as opposed to program text) and other more subtle instances
of the same.


------------------------+-----------------------------------------------------
Christian Lynbech | Ericsson Telebit, Skanderborgvej 232, DK-8260 Viby J
Phone: +45 8938 5244 | email: christia...@ted.ericsson.se
Fax: +45 8938 5101 | web: www.ericsson.com
------------------------+-----------------------------------------------------
Hit the philistines three times over the head with the Elisp reference manual.
- pet...@hal.com (Michael A. Petonic)

Nils Goesche

unread,
Jul 23, 2002, 8:42:02 AM7/23/02
to
Christian Lynbech <christia...@ted.ericsson.se> writes:

> [ if anything is going to demonstrate lack of c.l.l maturity it is to
> start a thread with that kind of subject :-) ]

I would be surprised if anybody disagreed with you, as you
(thankfully) didn't crosspost to comp.object ;-)

> The linguistic power of Lisp is however not very easy to transfer to
> other programming languages without turning them into Lisp in the
> process. It is not just macros or sequence functions or mapping
> operators or higher order functions, but also symbols (what other
> programming language would allow spaces in function names) and
> program structure (as opposed to program text) and other more subtle
> instances of the same.

I think this is true. One thing you didn't mention is object
identity, as measured by EQL, which seems to be regarded as very
important by Kent Pitman (``Identity, identity, identity!''). I have
a feeling that he is right on this, although I'll probably need quite
another while until I see clearly, why. Of course, we are identifying
symbols by their identity, but I find myself doing this with other
objects in Lisp, too, more and more often, and much more so than in
other languages, even functional ones, although I don't know yet, why.
This might also be the key for understanding that dynamic typing is
not only a matter of taste, as if you could remove it from Lisp
leaving all other things equal, but essential: Objects (with identity)
have types, not variables.

Just a thought.

Regards,
--
Nils Goesche
"Don't ask for whom the <CTRL-G> tolls."

PGP key ID 0x42B32FC9

Marco Antoniotti

unread,
Jul 23, 2002, 10:29:06 AM7/23/02
to

top...@technologist.com (Topmind) writes:

...

> > Both. First of all it runs as fast as any assembly code produced by a
> > native compiler (i.e. not a byte compiler, given all the necessary
> > qualifications). Second, (Common) Lisp machinery to deal with
> > Lisp-like languages (hence itself) is simply unparalled in other
> > environments (save maybe, AFAIK, Smalltalk and Prolog environments).
>
>
> Do you have a specific semi-realistic example of
> this by chance?

CLOS? :) Is that realistic enough?

> > Current "scripting languages" simply do not come close both in
> > manipulation easiness and expressiveness *and* speed of the generated
> > code.
>
> I am not questioning the machine speed issue here.
> I'll leave that to others who like to play with
> stopwatches.

To extend Python in any significant way, you need to hack in the
non-Python portions of the release. To extend (Common) Lisp you hack
(Common) Lisp itself.

A case in point. Not many people know Setl, but it is a very
interesting high level language that deals with "sets" and that gives
you a "set-based" notation to express your algorithms. E.g. you can
write things like

{x in [1 .. 100] | oddp(x) }

This is similar to "list comprehension" available in several older
functional languages (Haskell comes to mind), and recently adopted by
Python and (I beleive) Ruby. (Setl machinery is more powerful, but
that is beyond the point.) Now. To have this notation in your
scripting language du jour you have to go through a lot of hoops end
eventually hack the parser and the C/C++ code underneath.

The above notation is rendered in Common Lisp as

{x in (range 1 100) / (oddp x) }

A better example is a generator of the tuple (denoted by '[' and ']')
of prime numbers from 1 to `max'.

==============================================================================
(in-package "SETL-USER")

(defun primes (max)
[n in (range 3 max 2)
/ (not (exist m in (range 3 (min (1- n) (+ 2 (sqrt n))) 2)
/ (= (mod n m) 0)))])
==============================================================================

The above program is perfecly legal CL and it compiles (I repeat: it
compiles! And for people who did not know this: it compiles :) ) down
to x86, Sparc, MIPS and whatever CPU is currently supported by one or
the CL implementations.

The code to support this sort of extensions is pure Common Lisp and it
is (to my regret :} ) slightly more than 300 lines including comments.
Adding a few more lines will allow you to change the 'range'
expressions and to reuse the INFIX package.

So the bottom line is: to the best of my knowledge, only CL (and
probably Prolog and Smalltalk) gives you all you need all the time.
Not scripting language du jour gets you that much.

Having said so, do I program in CL all the time? No. I use what I have
to use given the task at hand. Only there are very few tasks unsuited
to CL :)

... and Greenspun's Tenth Rules! :)

Marco Antoniotti

unread,
Jul 23, 2002, 10:40:37 AM7/23/02
to

Nitpicking....

Joel Ray Holveck <jo...@juniper.net> writes:

> Here's a trivial example. Consider:
>
> (export add-bias set-bias)

The above is incorrect. `export' is a function that takes a list of
symbols.

(in-package "BIAS-PACKAGE")

(export (list 'add-bias 'set-bias))

is more correct.

Of course you could have written

(export* add-bias set-bias)

had defined

(defmacro export* (&rest symbols)
(assert (every #'symbolp symbols))
`(export ',symbols))

> (defvar *bias* 0)
> (defun set-bias (new-bias)
> (setq *bias* new-bias))
> (defun add-bias (sig)
> (+ *bias* sig))
>

Cheers

Marco Antoniotti

unread,
Jul 23, 2002, 10:43:11 AM7/23/02
to

"Shayne Wissler" <thal...@yahoo.com> writes:

>
> Sure, and anything you can do in Lisp you can do in C.

Tape and read/write heads also work. :)

JRStern

unread,
Jul 23, 2002, 11:03:31 AM7/23/02
to
Rob,

Thanks, I knew it wasn't really strings in Lisp, but between my fuzzy
memory of the right terminology and trying to simplify for the
discussion, I said whatever.

Can you clarify, in Common Lisp when you do the Eval(), does it have
access to the current execution context, or does it evaluate in a
separate context? I think it's in-context, ... but I never did that
much with CL, and it was a long time ago.

Joshua Stern


On 23 Jul 2002 09:22:04 GMT, rp...@rigden.engr.sgi.com (Rob Warnock)
wrote:

JRStern

unread,
Jul 23, 2002, 11:06:24 AM7/23/02
to
On Tue, 23 Jul 2002 05:58:06 GMT, "Shayne Wissler"
<thal...@yahoo.com> wrote:
>> ... maybe C does qualify, if crudely, since you can always persist the
>> state, gen the new program, initiate it, propagate the state, yada
>> yada.
>
>C doesn't qualify.

Party-pooper.

>> Is there a practical or a principled motivation for these questions?
>
>Yes.

Then just for the record, what is it?

>> Pretty much whatever you can do by generating code and dynamically
>> loading and evaluating it, can be done pretty nearly the same way by
>> interpreting within a fixed body of code.
>
>Sure, and anything you can do in Lisp you can do in C.

Anything along the lines of these interpretations. Anyway, it ain't
me telling anybody to use Lisp for anything.

J.

Marco Antoniotti

unread,
Jul 23, 2002, 11:27:04 AM7/23/02
to

top...@technologist.com (Topmind) writes:

> > JRStern wrote:
> >
> > >> - What other languages have at least one of the above?
> > >
> > > GWBasic could dynamically load source into a running program. Heck,
> > > most scripting languages can write text to a disk file then invoke the
> > > file, and it's not all that different.
> >
> > Hold on--the requirement is that the generated program can access the
> > variables and functions of the generating program. Otherwise C would count
> > as having this feature because you can generate C source, compile it, and
> > execute it from a C program.
>
> I know a few scripting languages that can execute code in
> context.
>
> if foo
> include("myLines.txt")
> end if

The above is equivalent in CL to.

(when foo (load "mySexprs.txt"))

however, it is not equivalent to CL's ability to extend itself.

>
> // Or:
>
> myCode = "delete(scope='everything');listThem()...."
> if foo
> execute(myCode)
> end if

The above is equivalent to

(let ((my-code '(delete-file "/usr/bin/vi")))
(eval my-code))

So the "scripting language" code above code is an application of
Greenspun's Tenth. (I am a Lisp zealot! What editor do you think I
use?!?)

Of course you can do things in CL in an easier way than you can in the
scripting language du jour.

(let ((my-code '(delete-file "/usr/bin/vi")))
(if (string= (second my-code) "/usr/bin/vi")
(error "Come on! Be nice! You are trying to delete ~S."
(second my-code))
(eval my-code)))

It is the `(second my-code)' form that gives away the power of CL.
You do not handle strings. You handle CL data here (i.e. `cons'
cells: programs are data). Hence this kind of "extension programming"
does not mix text and regular code in CL. Instead, you must do that in
any other environment (except Prolog and Smalltalk AFAIK). (And the
above gets compiled down to machine language).

The above would be equivalent in your SLDJ (scripting language du jour)
to

myCode = AST.function_call("delete_file",
[AST.string("/usr/bin/vi")]);
myStringCode = myCode.operator()
+ "("
+ myCode.arguments().first()
+ ")"
if myCode.arguments().first().equal("/usr/bin/vi")
error("Are you mad? Why would you delete "
+ myCode.arguments().first()
+ "? Are you a Lisper?");
else
execute(myStringCode);
endif

of course I am assuming that you have an AST package available (as you
do in some of the SLDJs out there).

.... get the message? :)

> As if the included code was already in there.
> It can reference any vars in that existing
> scope.

The notion of `scope' you seem to refer to here seem to reflect some
of the bizarre (thankfully semi-fixed now) scoping rules of Python.
I am not sure what you mean here.

> (I won't vouch for the speed, however. It is not
> an issue most of the time.)

Exactly. So why program in C/C++ when you can program in CL and get much
more speed out of your program than in SLDJ? :)

Jochen Schmidt

unread,
Jul 23, 2002, 11:29:03 AM7/23/02
to
JRStern wrote:

> Rob,
>
> Thanks, I knew it wasn't really strings in Lisp, but between my fuzzy
> memory of the right terminology and trying to simplify for the
> discussion, I said whatever.

Ok - but rendering them as strings takes away an important point about lisp
and is not really a valid simpification.

> Can you clarify, in Common Lisp when you do the Eval(), does it have
> access to the current execution context, or does it evaluate in a
> separate context? I think it's in-context, ... but I never did that
> much with CL, and it was a long time ago.

In CL EVAL evaluates in a lexical null-environment but in the actual dynamic
environment (CL has both - lexically and dynamically scoped "special"
variables).

(let ((a 1)) (eval '(+ a a)))
-> ERROR "unbound variable a"

(defvar *a* 1) ; A global special variable *a*

(let ((*a* 2))
(eval '(+ *a* *a*)))
-> 4

--
http://www.dataheaven.de

Shayne Wissler

unread,
Jul 23, 2002, 11:43:48 AM7/23/02
to

"JRStern" <JXSternC...@gte.net> wrote in message
news:3d3d7109...@news.verizon.net...

> >> Is there a practical or a principled motivation for these questions?
> >
> >Yes.
>
> Then just for the record, what is it?

I had come to the conclusion that the ideal language ought to be able to
seamlessly extend itself, dynamically. I read about Lisp, which sounded like
it had that feature, but it wasn't clear from my reading that it worked the
way I'd expect.


Shayne Wissler

Marco Antoniotti

unread,
Jul 23, 2002, 12:06:06 PM7/23/02
to

Jochen Schmidt <j...@dataheaven.de> writes:

> JRStern wrote:
>

...

> > Can you clarify, in Common Lisp when you do the Eval(), does it have
> > access to the current execution context, or does it evaluate in a
> > separate context? I think it's in-context, ... but I never did that
> > much with CL, and it was a long time ago.
>
> In CL EVAL evaluates in a lexical null-environment but in the actual dynamic
> environment (CL has both - lexically and dynamically scoped "special"
> variables).
>
> (let ((a 1)) (eval '(+ a a)))
> -> ERROR "unbound variable a"
>
> (defvar *a* 1) ; A global special variable *a*
>
> (let ((*a* 2))
> (eval '(+ *a* *a*)))
> -> 4

Just for the record. It is a (minor) misfortune that CL did not include
better "environment" handling alongside EVAL.

Secondly, the use of EVAL is highly discouraged in common CL
practice. As a matter of fact, you do not need it that much at all.
Essentially the macro system allows to work around EVAL shortcomings
and to produce better code. (Jochen, I know you know this. I just felt
it needed clarification).

Michael Sullivan

unread,
Jul 23, 2002, 1:32:04 PM7/23/02
to
Shayne Wissler <thal...@yahoo.com> wrote:

Well... C still counts as having this feature in a very roundabout way,
since your generator can publish an API to all it's relevant variables
and functions, and include headers for it with the source it produces,
compiles and runs.

The fact is, you can implement Lisp in any turing complete language, so
in some sense they all have this feature if you allow any amount of
program complexity to achieve it. But think about how much effort is
required to write a C program that does this. No matter how elegant
working this way makes your problem solution, the engineering required
to make this idea work in C (or most scriping languages) will likely
dwarf the dificulties in finding another way to represent the program
without doing this.

The fact is, to do this sort of thing requires a grammar generator for
the language. Lisp, having a very simple grammar and ready made tools
to call the compiler/interpreter, makes it easy. Other languages with
very simple grammars can (and sometimes do) also make it fairly easy.
Most common use languages that are not related to Lisp either have much
more complicated grammars or give up a great deal of expressiveness and
ability to handle abstraction (often both).


Michael

--
Michael Sullivan
Business Card Express of CT Thermographers to the Trade
Cheshire, CT mic...@bcect.com

JRStern

unread,
Jul 23, 2002, 7:56:51 PM7/23/02
to
On Tue, 23 Jul 2002 15:43:48 GMT, "Shayne Wissler"
<thal...@yahoo.com> wrote:
>I had come to the conclusion that the ideal language ought to be able to
>seamlessly extend itself, dynamically. I read about Lisp, which sounded like
>it had that feature, but it wasn't clear from my reading that it worked the
>way I'd expect.

Pretty much any language that supports functions or methods can be
looked at as extensible. C++'s overloaded operators are very powerful
that way. I keep wishing someone had a SQL that allowed overloaded
operators (does anybody?) for user-defined types, but I guess the
solution is to use Java for database procedures.

There was a book about a zillion years ago, "The Art of the Metaobject
Protocol", about using CLOS and the art and science of modifying its
basic syntax and semantics, whether on the fly or offline. I reviewed
the book for SIGArt and took the same line then, that the authors
never quite explained why it was a good idea to do that -- not just
extend the language, but warp even the basic features.

So, I guess I agree with you, but it's unclear to me that the language
has to be able to extend itself on the fly. More important to me is
the ability to change class definitions on the fly, which some
scripting languages now support.

Joshua Stern

JRStern

unread,
Jul 23, 2002, 7:58:39 PM7/23/02
to
On Tue, 23 Jul 2002 17:29:03 +0200, Jochen Schmidt <j...@dataheaven.de>
wrote:

>> Thanks, I knew it wasn't really strings in Lisp, but between my fuzzy
>> memory of the right terminology and trying to simplify for the
>> discussion, I said whatever.
>
>Ok - but rendering them as strings takes away an important point about lisp
>and is not really a valid simpification.

Mea maximum culpa.

>> Can you clarify, in Common Lisp when you do the Eval(), does it have
>> access to the current execution context, or does it evaluate in a
>> separate context? I think it's in-context, ... but I never did that
>> much with CL, and it was a long time ago.
>
>In CL EVAL evaluates in a lexical null-environment but in the actual dynamic
>environment (CL has both - lexically and dynamically scoped "special"
>variables).
>
>(let ((a 1)) (eval '(+ a a)))
>-> ERROR "unbound variable a"
>
>(defvar *a* 1) ; A global special variable *a*
>
>(let ((*a* 2))
> (eval '(+ *a* *a*)))
>-> 4

Thanks.

J.

Doug McNaught

unread,
Jul 23, 2002, 9:33:50 PM7/23/02
to
JXSternC...@gte.net (JRStern) writes:

> On Tue, 23 Jul 2002 15:43:48 GMT, "Shayne Wissler"
> <thal...@yahoo.com> wrote:
> >I had come to the conclusion that the ideal language ought to be able to
> >seamlessly extend itself, dynamically. I read about Lisp, which sounded like
> >it had that feature, but it wasn't clear from my reading that it worked the
> >way I'd expect.
>
> Pretty much any language that supports functions or methods can be
> looked at as extensible. C++'s overloaded operators are very powerful
> that way. I keep wishing someone had a SQL that allowed overloaded
> operators (does anybody?) for user-defined types, but I guess the
> solution is to use Java for database procedures.

PostgreSQL has this:

http://www3.us.postgresql.org/users-lounge/docs/7.2/postgres/sql-createoperator.html

-Doug

Jochen Schmidt

unread,
Jul 23, 2002, 9:34:59 PM7/23/02
to
JRStern wrote:

> On Tue, 23 Jul 2002 15:43:48 GMT, "Shayne Wissler"
> <thal...@yahoo.com> wrote:
>>I had come to the conclusion that the ideal language ought to be able to
>>seamlessly extend itself, dynamically. I read about Lisp, which sounded
>>like it had that feature, but it wasn't clear from my reading that it
>>worked the way I'd expect.
>
> Pretty much any language that supports functions or methods can be
> looked at as extensible. C++'s overloaded operators are very powerful
> that way. I keep wishing someone had a SQL that allowed overloaded
> operators (does anybody?) for user-defined types, but I guess the
> solution is to use Java for database procedures.

Or to use Common Lisp. OnShores "UncommonSQL" (called after Xanalys'
"CommonSQL" from which it took many ideas) is a really nice database
binding. It enables you for example to access tables as CLOS classes and to
create tables out of CLOS classes. It uses the MOP to do that.

> There was a book about a zillion years ago, "The Art of the Metaobject
> Protocol", about using CLOS and the art and science of modifying its
> basic syntax and semantics, whether on the fly or offline. I reviewed
> the book for SIGArt and took the same line then, that the authors
> never quite explained why it was a good idea to do that -- not just
> extend the language, but warp even the basic features.

It's not zillion years ;-) - mine (fourth print) is from 1995. The first
edition was printed 1991.
AMOP actually describes how CLOS can be implemented on itself and following
from that offering a protocol to extend/modify it's behaviour. The above
mentioned CommonSQL/UncommonSQL are based on that facility (the MOP).

> So, I guess I agree with you, but it's unclear to me that the language
> has to be able to extend itself on the fly. More important to me is
> the ability to change class definitions on the fly, which some
> scripting languages now support.

Saying "The language is able to extend itself" seems to be just to short to
show the whole consequences. It is not so much the "on the fly" part that
is so interesting. That would imply that one wants to extend the language
while some readily deployed application is running which _may_ be useful
too (Emacs, AutoCAD...) but is not the whole point. One important thing is
that Lisp development happens while what later will be the application is
already running (you incrementally add functionality to the running
system). The nice thing is, regardless of what you want to change - syntax,
code-generation, I/O, object-system... you can do it in Lisp itself having
the whole language available all the time.

ciao,
Jochen

--
http://www.dataheaven.de

JRStern

unread,
Jul 23, 2002, 10:20:43 PM7/23/02
to
On Wed, 24 Jul 2002 03:34:59 +0200, Jochen Schmidt <j...@dataheaven.de>
wrote:

>> There was a book about a zillion years ago, "The Art of the Metaobject
>> Protocol", about using CLOS and the art and science of modifying its
>> basic syntax and semantics, whether on the fly or offline....

>It's not zillion years ;-) - mine (fourth print) is from 1995. The first
>edition was printed 1991.

I think I was running a 25mhz system with 1mb RAM back then. That's
about the same technology as Fred Flintstone used.

When I graduated from college, there was no life on earth other than
blue-green algae, and 2 MIP mainframes from IBM with two megabytes of
2 microsecond ferrite memory.

J.

cr88192

unread,
Jul 24, 2002, 12:08:01 AM7/24/02
to
JRStern wrote:

> On Wed, 24 Jul 2002 03:34:59 +0200, Jochen Schmidt <j...@dataheaven.de>
> wrote:
>>> There was a book about a zillion years ago, "The Art of the Metaobject
>>> Protocol", about using CLOS and the art and science of modifying its
>>> basic syntax and semantics, whether on the fly or offline....
>
>>It's not zillion years ;-) - mine (fourth print) is from 1995. The first
>>edition was printed 1991.
>
> I think I was running a 25mhz system with 1mb RAM back then. That's
> about the same technology as Fred Flintstone used.
>

and what would that be? cpu's at that speed tended to have a bit more ram,
and systems with that amount of ram tended to run at slower speeds...

> When I graduated from college, there was no life on earth other than
> blue-green algae, and 2 MIP mainframes from IBM with two megabytes of
> 2 microsecond ferrite memory.
>

I get quite tired of that "oh, when from dinasaur time" type crap...
computers are better hardware wise but software now is crap, and if the
software wasn't crap then we could do a whole lot more with the hardware.

software in general (usefulness and raw power) has not improved much. is
"java" much better than anything that came before?

I have not even been to college yet, so really I should have more grounds
for this argument. however, I don't make this argument as I can appriciate
older technologies.
pc's were comming on strong by the time I was born, I spent much of the
first part of my life with "dinasaur" technologies, thus I can appriciate
the work that has gone into the hardware.
however the software industry is stagnent and backwater, just riding the
wave of hardware improvement...

I think it is just pitiful that hobbyists can nearly defeat the companies,
and even more pitiful how the companies respond, and this is from me, a
hobbyist.

so please, don't use those arguments...

--
<cr8...@hotmail.spam.com>
<http://bgb1.hypermart.net/>

Joel Ray Holveck

unread,
Jul 24, 2002, 1:22:41 AM7/24/02
to
> Just for the record. It is a (minor) misfortune that CL did not include
> better "environment" handling alongside EVAL.

Every now and then I come back and think about this. If you had The
Environment Protocol Of Your Dreams (including integration with eval,
macroexpand, etc, and whatever else you wanted), what would it
include? How would it help?

I'm asking more out of randomness than any particular objective here,
so feel free to blow the question off, but it's nice to know how my
favorite language is circumscribed.

joelh

Nicolas Neuss

unread,
Jul 24, 2002, 4:29:08 AM7/24/02
to
Marco Antoniotti <mar...@cs.nyu.edu> writes:

> It is the `(second my-code)' form that gives away the power of CL.
> You do not handle strings. You handle CL data here (i.e. `cons'
> cells: programs are data). Hence this kind of "extension programming"
> does not mix text and regular code in CL. Instead, you must do that in
> any other environment (except Prolog and Smalltalk AFAIK). (And the
> above gets compiled down to machine language).

I know that Prolog is as flexible or even more flexible than Lisp in
extending its Syntax. But could you point me to some link where this
is described for Smalltalk?

Thanks, Nicolas,

Hannah Schroeter

unread,
Jul 24, 2002, 10:19:29 AM7/24/02
to
Hello!

Joel Ray Holveck <jo...@juniper.net> wrote:
>[...]

>Closures are possible in statically linked languages, too, but aren't
>often used. It's just like an indirected function call. Admittedly,
>that's not static linking, but it's something that most statically
>linked languages like C already have. You can actually find a paper
>on how to add closures to C++ at
>http://master.debian.org/~karlheg/Usenix88-lexic.pdf

What about SML, ocaml, Haskell? All have and use closures quite much
and you're able to compile and link them statically.

Or current Java's (anonymous) inner classes?

Not all of the static world is C or C++ :-)

>[...]

Kind regards,

Hannah.

Marco Antoniotti

unread,
Jul 24, 2002, 10:47:50 AM7/24/02
to

Nicolas Neuss <Nicola...@iwr.uni-heidelberg.de> writes:

> Marco Antoniotti <mar...@cs.nyu.edu> writes:
>
> > It is the `(second my-code)' form that gives away the power of CL.
> > You do not handle strings. You handle CL data here (i.e. `cons'
> > cells: programs are data). Hence this kind of "extension programming"
> > does not mix text and regular code in CL. Instead, you must do that in
> > any other environment (except Prolog and Smalltalk AFAIK). (And the
> > above gets compiled down to machine language).
>
> I know that Prolog is as flexible or even more flexible than Lisp in
> extending its Syntax.

The Prolog 'reader' essentially has provisions to extend an infix
reader with operators. I.e. it gives you hooks to manipulates an
expression grammar. The CL reader coupled with the Macro system give
you more flexibility at a lower level but does not give you "high
level" control in the form of a "grammar parser" (please extrapolate
as necessary from my description).

> But could you point me to some link where this
> is described for Smalltalk?

I have no direct experience on this, but I have seen Smalltalk code
handling "blocks" of code in a rather straighforward way. A search
frm the Squeak pages will help. All in all, this may be akin to
having an AST manipulation library, like (I hear) is available in
Python. (Of course, I bet several minds in the Python/Ruby/SLDJ camp
are busy re-implementing these features, thus reconfirming
Greenspun's :) ).

Marco Antoniotti

unread,
Jul 24, 2002, 11:16:26 AM7/24/02
to

JXSternC...@gte.net (JRStern) writes:

> On Tue, 23 Jul 2002 15:43:48 GMT, "Shayne Wissler"
> <thal...@yahoo.com> wrote:
> >I had come to the conclusion that the ideal language ought to be able to
> >seamlessly extend itself, dynamically. I read about Lisp, which sounded like
> >it had that feature, but it wasn't clear from my reading that it worked the
> >way I'd expect.
>
> Pretty much any language that supports functions or methods can be
> looked at as extensible. C++'s overloaded operators are very powerful
> that way. I keep wishing someone had a SQL that allowed overloaded
> operators (does anybody?) for user-defined types, but I guess the
> solution is to use Java for database procedures.

There are degrees of extensibility. C++ operator overloading (sans
multiple dispatching, something you have in CLOS :) ) goes a long way
to allow you to "extend" the language. But, always given that Tape,
and Read/Write heads essentially allow for "extensibility", CL is
still unparalled in its ability to "extend itself".

Suppose you want to open a file and make sure you close it when you
are done, while handling errors.

The C++/Java/SLDJ (Scripting Language Du Jour) idiom to do this is

FILE f = NULL;
try
{
f = open("f.dat", "w");
print(f, "something");
}
finally
{
close(f);
}

apart from the fact that this sort of control structure was present in
Lisp and CL way before it appeared in C/C++ (hence Java and your SLDJ)
the CL equivalent is

(let ((f nil))
(unwind-protect
(progn
(setf f (open "f.dat" :direction :output))
(print "something" f))
(close f)))

Now, the interesting thing about CL is that you can write a macro that
wraps the above piece of code in a nice way.

(defmacro with-file-open ((fvar file &key (direction :output))
&body forms)
`(let ((,fvar nil))
(unwind-protect
(progn
(setf ,fvar (open ,file :direction ,direction))
,@forms)
(close ,fvar))))

after you have defined your macro you can say

(with-file-open (f "f.dat")
(print "something" f))

There is no way to do that as easily in any other language. Especially
your SLDJ! Certainly, you do not get it compiled down to native
machine code.

Essentially you have added a completely new construct to CL. To do so
in Java you need to write a complete parser (I know! I did it). Busy
minds are surely working right now to get this sort of things in your
SLDJ (thus reinforcing Greenspun's Tenth).

BTW. `with-open-file' is a standard ANSI CL macro.

Moreover! Do you know that you can write a `try' macro in CL to make
C++/Java/SLDJ programmers happy?

Since you are so interested in data base programming, can you see the
implications for a `with-transaction' macro?

> There was a book about a zillion years ago, "The Art of the Metaobject
> Protocol", about using CLOS and the art and science of modifying its
> basic syntax and semantics, whether on the fly or offline. I reviewed
> the book for SIGArt and took the same line then, that the authors
> never quite explained why it was a good idea to do that -- not just
> extend the language, but warp even the basic features.

You are entitled to this opinion. Not that I agree with you.
Especially when you do not explain what are "the basic features".

> So, I guess I agree with you, but it's unclear to me that the language
> has to be able to extend itself on the fly. More important to me is
> the ability to change class definitions on the fly, which some
> scripting languages now support.

Great! Another application of Greenspun's Tenth Rule of
Programming applied to SLDJ!

`change-class' has been part of Common Lisp for ages (since the
inception of CLOS). *And* you still get the calls to `change-class'
compiled to native machine language in Common Lisp.

Moreover, can you explain to us how you did not appreciate "The Art of
the MOP" and at the same time like one of the operations that fully
require all its power to be understood correctly?

Marco Antoniotti

unread,
Jul 24, 2002, 11:21:46 AM7/24/02
to

Joel Ray Holveck <jo...@juniper.net> writes:

> > Just for the record. It is a (minor) misfortune that CL did not include
> > better "environment" handling alongside EVAL.
>
> Every now and then I come back and think about this. If you had The
> Environment Protocol Of Your Dreams (including integration with eval,
> macroexpand, etc, and whatever else you wanted), what would it
> include? How would it help?

Maybe it would not help that much all the time. Essentially I see
EVAL very rarely in newer code and I practically do not use it at all.

Yet. There are times when it would turn out to be useful, and it
would add elegance to the language (of course, complicating its
implementation). So, I think it is a minor misfortune, and one I can
happily live with.

>
> I'm asking more out of randomness than any particular objective here,
> so feel free to blow the question off, but it's nice to know how my
> favorite language is circumscribed.
>

Well, I think I just blew the question off :)

Steven E. Harris

unread,
Jul 24, 2002, 11:47:11 AM7/24/02
to
Marco Antoniotti <mar...@cs.nyu.edu> writes:

> Suppose you want to open a file and make sure you close it when you
> are done, while handling errors.
>
> The C++/Java/SLDJ (Scripting Language Du Jour) idiom to do this is
>
> FILE f = NULL;
> try
> {
> f = open("f.dat", "w");
> print(f, "something");
> }
> finally
> {
> close(f);
> }

Please don't include C++ in your pejorative set. Its destructors take
care of your criticism:

std::ifstream f( "f.dat", ios_base::out );
f << "something";

The file is automatically closed in f's destructor, provided the file
had been opened successfully in the constructor. Omitted are the
checks whether the file was opened successfully and whether the write
succeeded. Failure in the former will of course manifest in the
latter.

--
Steven E. Harris :: seha...@raytheon.com
Raytheon :: http://www.raytheon.com

Steven E. Harris

unread,
Jul 24, 2002, 12:11:26 PM7/24/02
to
Steven E. Harris <seha...@raytheon.com> writes:

> std::ifstream f( "f.dat", ios_base::out );

^

And I should read my own writing. Correction:

std::ofstream f( "f.dat" );

Shayne Wissler

unread,
Jul 24, 2002, 12:15:52 PM7/24/02
to

"Steven E. Harris" <seha...@raytheon.com> wrote in message
news:877kjln...@harris.sdo.us.ray.com...

> Please don't include C++ in your pejorative set. Its destructors take
> care of your criticism:
>
> std::ifstream f( "f.dat", ios_base::out );
> f << "something";

I would say "touche", except for the fact that you have ignored the broader
point: with lisp, if one needs a new construct, one has the liberty to write
it.


Shayne Wissler

JRStern

unread,
Jul 24, 2002, 12:46:45 PM7/24/02
to
On 24 Jul 2002 11:16:26 -0400, Marco Antoniotti <mar...@cs.nyu.edu>
wrote:

>> Pretty much any language that supports functions or methods can be
>> looked at as extensible. C++'s overloaded operators are very powerful
>> that way. I keep wishing someone had a SQL that allowed overloaded
>> operators (does anybody?) for user-defined types, but I guess the
>> solution is to use Java for database procedures.
>
>There are degrees of extensibility. C++ operator overloading (sans
>multiple dispatching, something you have in CLOS :) ) goes a long way
>to allow you to "extend" the language. But, always given that Tape,
>and Read/Write heads essentially allow for "extensibility", CL is
>still unparalled in its ability to "extend itself".

OK, it's unparalleled.

>Now, the interesting thing about CL is that you can write a macro that
>wraps the above piece of code in a nice way.

...


>There is no way to do that as easily in any other language. Especially
>your SLDJ! Certainly, you do not get it compiled down to native
>machine code.

Wrappers are kewl.

Overloading operators are kewl.

Both are subject to abuse so that code becomes write-only. From this
observation, I surmise that there may be a theoretical issue at stake
which would suggest limiting such features. Until that is settled, I
maintain some skepticism about kewl features.

(Yes, yes, all features of all languages are subject to abuse, but
surely you know what I mean?)

>Essentially you have added a completely new construct to CL. To do so
>in Java you need to write a complete parser (I know! I did it). Busy
>minds are surely working right now to get this sort of things in your
>SLDJ (thus reinforcing Greenspun's Tenth).

Greenspun's Tenth Rule of Programming: "Any sufficiently complicated C
or Fortran program contains an ad-hoc, informally-specified bug-ridden
slow implementation of half of Common Lisp."

Yes, well, maybe, sort of. More like Turing's Ninth (which I invent
on the spur of the moment): Any sufficiently complex program contains
a half-assed version of a Turing machine.

Yes. Absolutely. And this is exactly what I *recommend* rather than
extend or warp the base language more than a very limited amount.

(Actually, both Greenspun's Tenth and Turing's Ninth are meaningless,
since virtually any interesting program is going to have internal
state and branching on data, and thus appears, from various improper
perspectives, to be exceeding its mandate to be a linear
transformation of input to output)

>Since you are so interested in data base programming, can you see the
>implications for a `with-transaction' macro?

Actually, no. You ever do any database programming?

>> There was a book about a zillion years ago, "The Art of the Metaobject
>> Protocol", about using CLOS and the art and science of modifying its
>> basic syntax and semantics, whether on the fly or offline. I reviewed
>> the book for SIGArt and took the same line then, that the authors
>> never quite explained why it was a good idea to do that -- not just
>> extend the language, but warp even the basic features.
>
>You are entitled to this opinion. Not that I agree with you.
>Especially when you do not explain what are "the basic features".

Hey, whatever. At some point, it goes too far. Redefining addition
to be a trinary operator, say. See previous points.

>> So, I guess I agree with you, but it's unclear to me that the language
>> has to be able to extend itself on the fly. More important to me is
>> the ability to change class definitions on the fly, which some
>> scripting languages now support.
>
>Great! Another application of Greenspun's Tenth Rule of
>Programming applied to SLDJ!

That's one view. Another is the question, as I think I've addressed
it several times now in this message, and was apparently Shayne's
original subject, of what might comprise the best set of basic and
fixed features in a (OO) language.

>`change-class' has been part of Common Lisp for ages (since the
>inception of CLOS). *And* you still get the calls to `change-class'
>compiled to native machine language in Common Lisp.
>
>Moreover, can you explain to us how you did not appreciate "The Art of
>the MOP" and at the same time like one of the operations that fully
>require all its power to be understood correctly?

I'm advocating you don't worry about changing the tires on a moving
car. I don't want to change anything meta -- I want to differentiate
between the meta and the actual. That's the basic name of my game.

You may or may not agree, but I think that's clear enough.

Joshua Stern

Hannah Schroeter

unread,
Jul 24, 2002, 12:42:38 PM7/24/02
to
Hello!

Marco Antoniotti <mar...@cs.nyu.edu> wrote:

>[...]

>Suppose you want to open a file and make sure you close it when you
>are done, while handling errors.

>The C++/Java/SLDJ (Scripting Language Du Jour) idiom to do this is

> FILE f = NULL;
> try
> {
> f = open("f.dat", "w");
> print(f, "something");
> }
> finally
> {
> close(f);
> }

C++ doesn't even have "finally".

Try things like:
FILE* f = 0;
try {
f = fopen(...);
fprintf(f, ...);
} catch (...) {
fclose(f);
throw;
}
fclose(f);

And even then, you're incorrect. The close is only correct iff
fopen() has succeeded. I.e.
FILE* f = 0;
f = fopen(...);
try {
fprintf(f, ...);
} catch (...) {
fclose(f);
throw;
}
fclose(f);

But, of course, you could also use destructors in this case, as
another post has mentioned.

>apart from the fact that this sort of control structure was present in
>Lisp and CL way before it appeared in C/C++ (hence Java and your SLDJ)
>the CL equivalent is

> (let ((f nil))
> (unwind-protect
> (progn
> (setf f (open "f.dat" :direction :output))
> (print "something" f))
> (close f)))

You'd rather do:

(let ((f (open ...)))
(unwind-protect
(... do something on f ...)
(close f)))

>Now, the interesting thing about CL is that you can write a macro that
>wraps the above piece of code in a nice way.

> (defmacro with-file-open ((fvar file &key (direction :output))
> &body forms)

>[...]

>There is no way to do that as easily in any other language. Especially
>your SLDJ! Certainly, you do not get it compiled down to native
>machine code.

Yes, you can. Try your favourite functional programming language,
you can define something like

onOpenFile fileName mode function =
...

in Haskell, SML, ocaml, ...

Usage is something like
onOpenFile fileName mode
(lambda fileHandle -> ... do something on fileHandle ...)

>[...]

>Moreover! Do you know that you can write a `try' macro in CL to make
>C++/Java/SLDJ programmers happy?

Is that really needed or do you just need to rename handler-case
into try? :-)

>Since you are so interested in data base programming, can you see the
>implications for a `with-transaction' macro?

Yep. And you might even abstract out the commonalities from the
individual macro definitions of with-open-file and with-transaction:
Some initialization, often with a binding, some main action, and
some cleanup which has to be executed always afterwards (i.e.
unwind-protect'ed). Macro writing macros are *cool*.

>[...]

>> So, I guess I agree with you, but it's unclear to me that the language
>> has to be able to extend itself on the fly. More important to me is
>> the ability to change class definitions on the fly, which some
>> scripting languages now support.

>Great! Another application of Greenspun's Tenth Rule of
>Programming applied to SLDJ!

>`change-class' has been part of Common Lisp for ages (since the
>inception of CLOS). *And* you still get the calls to `change-class'
>compiled to native machine language in Common Lisp.

I think the quote meant more something like defclass of an already
defined class name, including update-object-for-redefined-class :-)

>[...]

Kind regards,

Hannah.

Marco Antoniotti

unread,
Jul 24, 2002, 12:51:36 PM7/24/02
to

Steven E. Harris <seha...@raytheon.com> writes:

You are right and I of course should have mentioned that. However, my
point was about the "overall extensibility" of the language, not about
file open/close semantics and how it is implemented in any language.
The fact that the file open/close idiom with errors is such a useful
one to explain the effect of "extensibility" motivated my example.

To further comment on your note, I must point out that (as you imply)
since you do not check for errors the example is not completely
equivalent.

To be "equivalent" you have to do at least the following (of corse you
can do it differently)

{ // Note the block starting here to ensure the call of std::~ofstream()
std::ofstream f("f.dat", ios_base::out);

if (f && f.good())
{
f << "something" << endl;
}
else
{
; // Some error signaling condition I suppose.
}
}

which may be better than the `try ... catch ...' idiom, unless you
want to actually `throw' some exception in the `else' branch.

Nevertheless, you cannot pack all of this in C++ into a

with_open_file (f, "f.dat", std::ofstream)
{
f << "somthing" << endl;
}

unless you rewrite your C++ parser.

You can play tricks with CPP... (untested).

==============================================================================

#define with_open_file(_f_, _file_, _class_, _iosbase_flag_) \\
{ // Note the block starting here to ensure the call of std::~ofstream() \\
_class_ _f_(_file_, _class_, _iosbase_flag_); \\
\\
if (_f_ && _f_.good()) \\

#define fow \\
else \\
{ \\
; // Some error signaling condition I suppose. \\
} \\
}

==============================================================================

and the say

with_open_file(f, "f.dat", std::ofstream, ios_base::out)
f << "something" << endl;
fow;

but you do that with less guarantees of correctness (since you
essentially split up the block where the scope of '_f_' is defined)
and without the full power of the language (compiler included) at your
fingertips; and it is this power that allows you to insert syntactic
and semantics checks at "macro expansion" time. Finally, the above
does not look and feel like C++.

sv0f

unread,
Jul 24, 2002, 1:05:23 PM7/24/02
to
In article <y7clm81...@sindri.juniper.net>, Joel Ray Holveck
<jo...@juniper.net> wrote:

>Every now and then I come back and think about this. If you had The
>Environment Protocol Of Your Dreams (including integration with eval,
>macroexpand, etc, and whatever else you wanted), what would it
>include? How would it help?

Or: In what ways were the environment handling facilities proposed in
CLtL2 (pp. 207-214) but excluded from the ANSI standard deficient?

(This may be in the Hyperspec...)

Steven E. Harris

unread,
Jul 24, 2002, 1:02:51 PM7/24/02
to
"Shayne Wissler" <thal...@yahoo.com> writes:

> with lisp, if one needs a new construct, one has the liberty to
> write it.

Agreed. The closest we come in C++ is to emulate some "around"
behavior with paired ctor-dtor RAII types. That is, the simple
creation of an instance of a type with a nontrivial constructor and/or
destructor guarantees that some instructions will run.

Kent M Pitman

unread,
Jul 24, 2002, 1:28:36 PM7/24/02
to
no...@vanderbilt.edu (sv0f) writes:

They were removed because they were faulty. A number of problems were
discovered and we tried for a while to patch them. It became clear that
we would not finish in time, so we removed them rather than have a bunch
of buggy design be part of the unchangeable spec.

It was assumed that vendors might continue to work on these and come up
with a suitable substrate for next-round standardization. Not much seems
to have occurred in this way.

As a rule, though, please take note:

The things that are in CLTL2 were about MACROEXPAND and friends. As a
rule, I think there is very little dispute that environment management
facilities for macroexpansion are in order since these things contain
information known at compile time, and consequently aid efficient
compilation.

HOWEVER, run-time environments, such as are used by EVAL, are an entirely
different beast. Claiming you want runtime environment support pushes
almost to the point of saying you don't want efficient compilation, since
specifying what evaluation environment will be needed is very very hard
if you're talking about ever asking the question "what is my lexical
environment now?"

To see the issue, consider the following simple case:

(let ((x 5))
(values (+ x x) (get-lexical-environment)))

The question is: what is the value of X in the lexical environment
that GET-LEXICAL-ENVIRONMENT would want to return? In most high-quality
compilers, (let ((x 5)) (+ x x)) is going to compile to the constant 10,
with no runtime call to + and no binding of x. In systems where
you have access to the runtime lexical environment, you effectively have
to inhibit some number of compiler optimizations and transformations if
the runtime user is to have any expectation at all about what's going on.

If all you mean by runtime environment for EVAL is the ability to
construct a set of special variable bindings, you can already do this
with PROGV.

If what you mean by runtime environment for EVAL is the ability to add
FLET or MACROLET forms, you can of course, just use CONS (or backquote or
whatever) to wrap the form you want to evaluate with suitable extra context.

If you want something else, you're best to detail it so that people can
tell you if there's a better way to do this.

Marco Antoniotti

unread,
Jul 24, 2002, 1:32:25 PM7/24/02
to

han...@schlund.de (Hannah Schroeter) writes:

> Hello!
>
...

> C++ doesn't even have "finally".

It has `catch (...)'.

> Try things like:
> FILE* f = 0;
> try {
> f = fopen(...);
> fprintf(f, ...);
> } catch (...) {
> fclose(f);
> throw;
> }
> fclose(f);
>
> And even then, you're incorrect. The close is only correct iff
> fopen() has succeeded. I.e.
> FILE* f = 0;
> f = fopen(...);
> try {
> fprintf(f, ...);
> } catch (...) {
> fclose(f);
> throw;
> }
> fclose(f);

This is incorrect. The last `fclose' (assuming the C library
`fclose') generates an error if `fopen' failed. You need to at least
conditionalize the call to `fclose' to be reasonably correct.

if (f != NULL) fclose(f);

to which you may add that `fclose' itself may fail and so on and so on.

Note that my code was "pseudo" at some level. I wasn't writing C++ or
Java code.

>
> But, of course, you could also use destructors in this case, as
> another post has mentioned.

To which I just answered.

> >apart from the fact that this sort of control structure was present in
> >Lisp and CL way before it appeared in C/C++ (hence Java and your SLDJ)
> >the CL equivalent is
>
> > (let ((f nil))
> > (unwind-protect
> > (progn
> > (setf f (open "f.dat" :direction :output))
> > (print "something" f))
> > (close f)))
>
> You'd rather do:
>
> (let ((f (open ...)))
> (unwind-protect
> (... do something on f ...)
> (close f)))

No, you don't because in this case you miss the errors generated by
OPEN when called with :error as value to :if-exists or
:if-does-not-exist etc etc.

I admit that my code above is not completely correct. You really need

(let ((f nil))
(unwind-protect
(progn

(setf f (open ...))
(do-something f))
(when (and f (open-stream-p f)) (close f))))

> >Now, the interesting thing about CL is that you can write a macro that
> >wraps the above piece of code in a nice way.
>
> > (defmacro with-file-open ((fvar file &key (direction :output))
> > &body forms)
> >[...]
>
> >There is no way to do that as easily in any other language. Especially
> >your SLDJ! Certainly, you do not get it compiled down to native
> >machine code.
>
> Yes, you can. Try your favourite functional programming language,
> you can define something like
>
> onOpenFile fileName mode function =
> ...
>
> in Haskell, SML, ocaml, ...
>
> Usage is something like
> onOpenFile fileName mode
> (lambda fileHandle -> ... do something on fileHandle ...)

Sorry. This does not count. That is equivalent to passing functions
around and using the underlying exception semantics to deal with
exceptions themselves.

Try changing the `try <expression> with <match cases>' of Ocaml into
`handler_case <expression> cases <match cases>'. You need 'p4' to do
that. Nothing wrong with it, but it is not the same as in CL.

> >[...]
>
> >Moreover! Do you know that you can write a `try' macro in CL to make
> >C++/Java/SLDJ programmers happy?
>
> Is that really needed or do you just need to rename handler-case
> into try? :-)

Well, pretty much that :)

...

Marco Antoniotti

unread,
Jul 24, 2002, 1:34:00 PM7/24/02
to

Marco Antoniotti <mar...@cs.nyu.edu> writes:

> han...@schlund.de (Hannah Schroeter) writes:
>
> > Hello!
> >
> ...
>
> > C++ doesn't even have "finally".
>
> It has `catch (...)'.

... meaning that it comes close to `finally'.

Joe Marshall

unread,
Jul 24, 2002, 1:32:14 PM7/24/02
to

"Hannah Schroeter" <han...@schlund.de> wrote in message news:ahmldu$kqq$2...@c3po.schlund.de...

> Hello!
>
> Marco Antoniotti <mar...@cs.nyu.edu> wrote:
>
> > (let ((f nil))
> > (unwind-protect
> > (progn
> > (setf f (open "f.dat" :direction :output))
> > (print "something" f))
> > (close f)))
>
> You'd rather do:
>
> (let ((f (open ...)))
> (unwind-protect
> (... do something on f ...)
> (close f)))

If an interrupt occurs after the open, but before the unwind-protect
is entered, you could drop the stream. This is a bit better:

(let ((f nil)
(abort-p t))
(unwind-protect
(multiple-value-prog1
(progn (setq f (open "f.dat" :direction :output))
... do something ....)
(setq abort-p nil))
(when f
(close f :abort abort-p))))

But still, if an interrupt occurs after open constructs the stream,
but before the SETQ is performed, you can drop the stream. In addition,
an interrupt in the cleanup form could abort before the close completes.

To be even safer, you ought to do something like this:

(let ((interrupts-enabled-p #+allegro excl::*without-interrupts*
#+lispworks sys::*in-no-interrupts*)
(f nil)
(abort-p t)
;; Evaluate the arguments before entering the no interrupt context.
(open-argument-0 "f.dat")
(open-argument-1 :direction)
(open-argument-2 :output))
(multiple-value-prog1
(let (#+allegro (excl::*without-interrupts* t)
#+lispworks (sys::*in-no-interrupts* 1))
(unwind-protect
(multiple-value-prog1
(progn (setq f (open open-argument-0 open-argument-1 open-argument-2))
(let (#+allegro (excl::*without-interrupts* interrupts-enabled-p)
#+lispworks (sys::*in-no-interrupts* interrupts-enabled-p))
#+lispworks
(unless sys::*in-no-interrupts*
(system::without-interrupt-check-for-interrupts))
... do something ...))
(setq abort-p nil))
(when f
(close f :abort abort-p))))
#+lispworks
(unless sys::*in-no-interrupts*
(system::without-interrupt-check-for-interrupts))))

This ensures that the OPEN, CLOSE, and UNWIND-PROTECT cleanup forms cannot be
interrupted. The drawback to this approach is that if, for some reason, OPEN
or CLOSE take a long time, the Lisp system will not respond to anything.
If there is a way to prevent asynchronous aborts from happening without
disabling interrupts altogether, then you can prevent the stream from
being accidentally lost, but still retain responsiveness.

Christopher Barber

unread,
Jul 24, 2002, 1:33:06 PM7/24/02
to
Marco Antoniotti <mar...@cs.nyu.edu> writes:

> Now, the interesting thing about CL is that you can write a macro that
> wraps the above piece of code in a nice way.
>
> (defmacro with-file-open ((fvar file &key (direction :output))
> &body forms)
> `(let ((,fvar nil))
> (unwind-protect
> (progn
> (setf ,fvar (open ,file :direction ,direction))
> ,@forms)
> (close ,fvar))))
>
> after you have defined your macro you can say
>
> (with-file-open (f "f.dat")
> (print "something" f))
>
> There is no way to do that as easily in any other language.

You can do this this in any language with a sufficiently powerful macro
system, not just CL.

- Christopher

Shayne Wissler

unread,
Jul 24, 2002, 1:51:59 PM7/24/02
to

"Christopher Barber" <cba...@curl.com> wrote in message
news:psoy9c1...@boris.curl.com...

> > There is no way to do that as easily in any other language.
>
> You can do this this in any language with a sufficiently powerful macro
> system, not just CL.

What other languages have such a system (Curl, I presume?)?


Shayne Wissler

JRStern

unread,
Jul 24, 2002, 2:04:49 PM7/24/02
to
On Wed, 24 Jul 2002 17:51:59 GMT, "Shayne Wissler"
<thal...@yahoo.com> wrote:
>> > There is no way to do that as easily in any other language.
>>
>> You can do this this in any language with a sufficiently powerful macro
>> system, not just CL.
>
>What other languages have such a system (Curl, I presume?)?

Who cares?

You can do it in a method/function/procedure, instead of a macro in
virtually any language, or wrap it in a class, which is probably
neater still.

J.

Stephen J. Bevan

unread,
Jul 24, 2002, 2:11:09 PM7/24/02
to
Christopher Barber <cba...@curl.com> writes:
> Marco Antoniotti <mar...@cs.nyu.edu> writes:
> > after you have defined your macro you can say
> >
> > (with-file-open (f "f.dat")
> > (print "something" f))
> >
> > There is no way to do that as easily in any other language.
>
> You can do this this in any language with a sufficiently powerful macro
> system, not just CL.

Indeed. However, for this particular example, there isn't much
difference between using a macro and a higher order function. So,
the example could be recast in Scheme as :-

(with-open-file "f.dat"
(lambda (f)
(display "something" f)))

and in Smalltalk :-

"f.dat" withOpenFile: [ f | f print: 'something' ]

for suitable definitions of with-open-file and withOpenFile
respectively. The same approach could, of course, be used in CL.

Shayne Wissler

unread,
Jul 24, 2002, 2:17:01 PM7/24/02
to

"JRStern" <JXSternC...@gte.net> wrote in message
news:3d3eec2b...@news.verizon.net...

It takes some vision to see why one might want to tailor the semantics the
language designer happened to provide. Not much, but some.


Shayne Wissler

Marco Antoniotti

unread,
Jul 24, 2002, 2:34:55 PM7/24/02
to

JXSternC...@gte.net (JRStern) writes:

> On Wed, 24 Jul 2002 17:51:59 GMT, "Shayne Wissler"
> <thal...@yahoo.com> wrote:
> >> > There is no way to do that as easily in any other language.
> >>
> >> You can do this this in any language with a sufficiently powerful macro
> >> system, not just CL.
> >
> >What other languages have such a system (Curl, I presume?)?
>
> Who cares?

Some of us do.

>
> You can do it in a method/function/procedure, instead of a macro in
> virtually any language, or wrap it in a class, which is probably
> neater still.

Sure. It even become neater when you have multiple dispatching in
your Object System.

Cheers


--
Marco Antoniotti ========================================================
NYU Courant Bioinformatics Group tel. +1 - 212 - 998 3488

715 Broadway 10th Floor fax +1 - 212 - 995 4122

Shayne Wissler

unread,
Jul 24, 2002, 2:43:47 PM7/24/02
to

"Johan Kullstam" <kulls...@attbi.com> wrote in message
news:m3d6tfm...@sysengr.res.ray.com...

> > 1. While parsing a lisp program, you can compile and invoke some of the
code
> > you already parsed to help parse the subsequent code, allowing you to
modify
> > the syntax on the fly. If this is correct, what are the limitations? Can
I
> > write a lisp program that starts out as lisp and starts looking like
> > C++?
>
> I am not sure that you could easily warp Lisp's reader into a C++
> parser. Lisp has a very simple and regular syntax which makes reading
> and parsing the Lisp source much easier than most languages.

Suppose I'm writing a lisp program, and then hit a spot in the code where
I'm going to do a lot of arithmetic computations, and so I want to use the
standard mathematical syntax. In these expressions, I'm going to refer to
and define variables that are also defined/referred to in the lispy sections
of code.

Can I easily do that? What would the code look like?


Shayne Wissler

Jochen Schmidt

unread,
Jul 24, 2002, 3:17:03 PM7/24/02
to
Shayne Wissler wrote:

I never needed it but you can use the free "infix.cl" to do that.

(defun some-computation (a b)
(let ((c 2))
#I(
c += a[0,1]^^b,
a = floor(3.14),
c = c / a
)
c))

Where the part in the #I(...) gets translated to:

(PROGN
(INCF C (EXPT (AREF A 0 1) B))
(SETQ A (FLOOR 3.14))
(SETQ C (/ C A)))

ciao,
Jochen

--
http://www.dataheaven.de

Tim Bradshaw

unread,
Jul 24, 2002, 6:08:29 PM7/24/02
to
* JXSternChangeX2R wrote:

> You can do it in a method/function/procedure, instead of a macro in
> virtually any language, or wrap it in a class, which is probably
> neater still.

Yes, but can you write an iteration construct using this technique, or
at least one which is concise in use? WITH-x macros are kind of a
classic case of something which, in most cases you *can* implement
with a CALL-WITH-x function and first-class functions (which kind of
gives the lie to the `virtually any language' claim), without
introducing too much extra syntax. I don't think you could implement
LOOP that way.

Note the `too much extra syntax': arguments from Turing equivalence
are not very interesting.

--tim

Stephen J. Bevan

unread,
Jul 24, 2002, 7:07:33 PM7/24/02
to
Tim Bradshaw <t...@cley.com> writes:
> Yes, but can you write an iteration construct using this technique, or
> at least one which is concise in use? WITH-x macros are kind of a
> classic case of something which, in most cases you *can* implement
> with a CALL-WITH-x function and first-class functions (which kind of
> gives the lie to the `virtually any language' claim), without
> introducing too much extra syntax. I don't think you could implement
> LOOP that way.

I don't either. However, whether that is good or bad depends on
whether you think LOOP, or something similar, is a good way to do
iteration or not. Smalltalk uses blocks (closures) to do its
iteration with messages like collect: and reject: to some of the
things that LOOP does. Consequently, the inability to write an
equivalent of the LOOP macro in Smalltalk might not be particularly
convincing case for macros to a Smalltalker.

Thomas Bushnell, BSG

unread,
Jul 24, 2002, 7:06:03 PM7/24/02
to
JXSternC...@gte.net (JRStern) writes:

> Mea maximum culpa.

You mean "mea maxima culpa".

Joel Ray Holveck

unread,
Jul 24, 2002, 7:35:21 PM7/24/02
to
>> Closures are possible in statically linked languages, too, but aren't
>> often used. It's just like an indirected function call. Admittedly,
>> that's not static linking, but it's something that most statically
>> linked languages like C already have. You can actually find a paper
>> on how to add closures to C++ at
>> http://master.debian.org/~karlheg/Usenix88-lexic.pdf
> What about SML, ocaml, Haskell? All have and use closures quite much
> and you're able to compile and link them statically.
> Or current Java's (anonymous) inner classes?
> Not all of the static world is C or C++ :-)

Indeed. I just was using C as an example of a statically-linked
language. I didn't mean to imply that it was the only one. My point
was that closures are possible in statically-linked languages.

Thanks for the examples.

joelh

Kent M Pitman

unread,
Jul 24, 2002, 10:40:34 PM7/24/02
to
[ replying to comp.lang.lisp only
http://world.std.com/~pitman/pfaq/cross-posting.html ]

The paradox of LOOP is that if you confine yourself only to the loops
that look stylistically clean and simple (which are the ones that Smalltalk
probably does well), you get a subset that would never have caused LOOP
to come into existence in the first place.

Like it or not, LOOP was invented to allow you to write the complex stuff
that is hard to express in terms of simple functional iterators because
the control structure is a tangle. LOOP's real power comes when you are
mixing multiple sources and collection styles at once, and I daresay that
for all these may look contorted in LOOP, they probably look worse in
Smalltalk.

The world is not always pretty. When it is, one must not blame the lack
of prettiness on the language that tries to emulate it. Sometimes tangled
webs of reality lead to tangled webs in programs. Saying that Smalltalk
programmers might be content not to deal with such cases resonates in
my brain as if you're saying Smalltalk programmers might content not to deal
with such cases of reality. Your mileage will doubtless vary.

Christopher Barber

unread,
Jul 24, 2002, 10:59:02 PM7/24/02
to
"Shayne Wissler" <thal...@yahoo.com> writes:

Yes, Curl is one, although macros are new with the latest release (macros have
actually been part of the internal implementation of Curl since the beginning
-- much of the standard syntax is actually implemented using macros -- but the
ability for user code to define macros was not supported until we had time to
clean the feature up for public use).

Dylan and Scheme are other language with macros. Of course, the C
preprocessor also provides a crude, inelegant and yet still useful macro
system.

Many scripting languages have macros or macro-like capabilities. For
instance, it is fairly easy to implement arbitrary syntax extensions in Tcl,
although I don't think it has formal macros.

There are also extensions to various non-macro languages providing macro
capabilities.

Finally, aren't CL macros non-hygienic?

- Christopher

Stephen J. Bevan

unread,
Jul 24, 2002, 11:26:47 PM7/24/02
to
Kent M Pitman <pit...@world.std.com> writes:
> The world is not always pretty. When it is, one must not blame the lack
> of prettiness on the language that tries to emulate it. Sometimes tangled
> webs of reality lead to tangled webs in programs. Saying that Smalltalk
> programmers might be content not to deal with such cases resonates in
> my brain as if you're saying Smalltalk programmers might content not to deal
> with such cases of reality. Your mileage will doubtless vary.

I didn't say that Smalltalk programmers are content not to deal with
the cases. I said that the block/closures approach gets you a long
way without having to create a language for iteration. For the cases
that are left then it is a judgement call whether using LOOP or
defining your own special purpose iterator macro is better than
griding it out using the facilities that you do have. The more and
more times you have to deal with such complex loops the more
attractive LOOP (and the ability to define your own iteration macros)
becomes. However, for those not familiar with LOOP and/or defining
special purpose iteration macros (which shouldn't be anyone now that
this has been restricted to comp.lang.lisp) then simply mentioning
LOOP or macros for iteration isn't necessarily going to mean much.
For them, concrete examples are needed, preferably ones that aren't
easy to do using a blocks/closure approach to iteration.

Daniel Barlow

unread,
Jul 24, 2002, 6:28:15 PM7/24/02
to
JXSternC...@gte.net (JRStern) writes:

> Hey, whatever. At some point, it goes too far. Redefining addition
> to be a trinary operator, say. See previous points.

* (+ 1 2 3)
6

"Too far" is a matter of personal taste and/or project standards, of
course, but I find it hard to think of a situation where that usage
would be frowned on


-dan

--

http://ww.telent.net/cliki/ - Link farm for free CL-on-Unix resources

Paolo Amoroso

unread,
Jul 25, 2002, 6:46:09 AM7/25/02
to
[Followup to comp.lang.lisp only]

On Tue, 23 Jul 2002 23:56:51 GMT, JXSternC...@gte.net (JRStern) wrote:

> There was a book about a zillion years ago, "The Art of the Metaobject
> Protocol", about using CLOS and the art and science of modifying its
> basic syntax and semantics, whether on the fly or offline. I reviewed
> the book for SIGArt and took the same line then, that the authors
> never quite explained why it was a good idea to do that -- not just
> extend the language, but warp even the basic features.

This paper discusses some applications of a MOP:

Open Implementations and Meta Object Protocols
http://www.parc.xerox.com/spl/groups/eca/pubs/papers/Kiczales-TUT95/for-web.pdf

I don't know whether they qualify as language extensions the way you mean
them.


Paolo
--
EncyCMUCLopedia * Extensive collection of CMU Common Lisp documentation
http://www.paoloamoroso.it/ency/README

Ingvar Mattsson

unread,
Jul 25, 2002, 7:44:45 AM7/25/02
to
JXSternC...@gte.net (JRStern) writes:

One macro I wrote recently (in patr of implementing an NNTP library
for Common Lisp, it's not ready yet, the previous version was *very*
much too icky to let anyone look at) is:

(defmacro prefixcase (var &rest body)
(let ((lenvar (gensym))
(gvar (gensym)))
`(let* ((,gvar ,var)
(,lenvar (length ,gvar)))
(cond
,@(loop for expr in body
if (listp (car expr))
collect `((or ,@(loop for str in (car expr)
collect `(string= ,gvar ,str :end1 (min ,le
nvar ,(length str)) :end2 (min ,lenvar ,(length str))))) ,@(cdr expr))
else
collect `((string= ,gvar ,(car expr) :end1 (min ,lenvar ,(length
(car expr))) :end2 (min ,lenvar ,(length (car expr)))) ,@(cdr expr)))))))

This is basically used to do something similar to:
(prefixcase header
("Subject: " ...)
("From: " ...)
...)

in order to simplify access of some (not all) headers in a news
article.

Now, I could have done one of "write the code out in full", "hunted
down a more suitable pre-existing macro" (CASE? Does it take a
specific test function, if so I could've just written a string-prefix=
and gone from there, I guess) or used a function taking (say) a
header and a list of prefix/lambda pairs.

I can't see how wrapping this in an object would be clearer and I do
think this is more easy to read than calling a separate function to do
the checking.

//Ingvar
--
When C++ is your hammer, everything looks like a thumb
Latest seen from Steven M. Haflich, in c.l.l

Hannah Schroeter

unread,
Jul 25, 2002, 8:38:54 AM7/25/02
to
Hello!

Loop is one thing. And in Lisp, there's map and reduce and remove-if
and so on, too, which roughly matches Collection do: aBlock, etc.
of Smalltalk.

How about seamless integration of e.g. scanner or parser generators
into the used programming language?

I could in fact envision e.g. an LALR(1) generator, written as functions,
loaded in at compile time, interfaced in with a frontend macro,
and thus seamlessly integrated into the rest of CL compilation.

Kind regards,

Hannah.

Knut Arild Erstad

unread,
Jul 25, 2002, 9:46:58 AM7/25/02
to
[Johan Kullstam]
:
: (defun make-adder (s)
: (compile nil (lambda (x) (+ x s))))
:
: calling make-adder will return a function which adds s.

Or you can simply do

(defun make-adder (s)
(lambda (x) (+ x s)))

and if make-adder is compiled, the returned function will also be
compiled. In fact, there is hardly ever any reason to call 'compile'
explicitly except during development (and with a proper IDE, not even
then).

Actually, I have used 'compile' in my code, *once*. It involved
generating optimized code for spline functions and compiling them
on-the-fly. It was kind of neat. :)

--
Knut Arild Erstad

But if less is more, then just think how much more more will be.
-- from "Frasier"

Nils Goesche

unread,
Jul 25, 2002, 10:18:09 AM7/25/02
to
han...@schlund.de (Hannah Schroeter) writes:

> I could in fact envision e.g. an LALR(1) generator, written as
> functions, loaded in at compile time, interfaced in with a frontend
> macro, and thus seamlessly integrated into the rest of CL
> compilation.

That's just what Lispworks' DEFPARSER macro does, BTW.

Regards,
--
Nils Goesche
"Don't ask for whom the <CTRL-G> tolls."

PGP key ID 0x42B32FC9

Stephen J. Bevan

unread,
Jul 25, 2002, 10:34:08 AM7/25/02
to
han...@schlund.de (Hannah Schroeter) writes:
> How about seamless integration of e.g. scanner or parser generators
> into the used programming language?
>
> I could in fact envision e.g. an LALR(1) generator, written as functions,
> loaded in at compile time, interfaced in with a frontend macro,
> and thus seamlessly integrated into the rest of CL compilation.

Agreed. A (very common) alternative in macro-less languages is to
define the parser generator language and write a compiler for it
(either from scratch, using an existing parser generator or build it
around some "generic" format such as XML). Of course, this doesn't
have the seamless integration that macros can provide.

A less common alternative is not to use a parser generator at all and
use parser combinators which are just higher order functions. This
gives you all the seamless integration of the macro approach though at
the cost of restricting yourself to top-down parsing.

So while both approaches solve many problems neither quite give you
what macros in Common Lisp can provide. Therefore this shows that
macros are a more powerful, general solution. Where it becomes
subjective is whether each person thinks that macros provide them with
something that is significantly better than they've already got.
The key words being "significantly better". Like it or not, people
are doing to disagree on that.

JRStern

unread,
Jul 25, 2002, 11:02:22 AM7/25/02
to
On 24 Jul 2002 22:59:02 -0400, Christopher Barber <cba...@curl.com>
wrote:

>Finally, aren't CL macros non-hygienic?

Could you expand on that?

J.

Marco Antoniotti

unread,
Jul 25, 2002, 11:10:41 AM7/25/02
to

Daniel Barlow <d...@telent.net> writes:

> JXSternC...@gte.net (JRStern) writes:
>
> > Hey, whatever. At some point, it goes too far. Redefining addition
> > to be a trinary operator, say. See previous points.
>
> * (+ 1 2 3)
> 6
>
> "Too far" is a matter of personal taste and/or project standards, of
> course, but I find it hard to think of a situation where that usage
> would be frowned on

Hey, (Common) Lispers go even further. '+' is at least a 50-ary
operator :) Actually I am wrong. '+' is also a zero-ary (for lack of a
better word) operator.

cl-prompt> (+)
0
cl-prompt> (*)
1

Marco Antoniotti

unread,
Jul 25, 2002, 11:15:27 AM7/25/02
to

The complete sentence should be

"mea culpa, mea culpa, mea maxima culpa"

:)

Michael Hudson

unread,
Jul 25, 2002, 11:14:25 AM7/25/02
to
Christopher Barber <cba...@curl.com> writes:

> Finally, aren't CL macros non-hygienic?

Yes, but I singularly fail to understand why this is considered a
problem so long as you aren't really crap at writing macros...

Cheers,
M.

--
I've even been known to get Marmite *near* my mouth -- but never
actually in it yet. Vegamite is right out.
UnicodeError: ASCII unpalatable error: vegamite found, ham expected
-- Tim Peters, comp.lang.python

Johan Kullstam

unread,
Jul 25, 2002, 10:57:33 AM7/25/02
to
knute...@ii.uib.no (Knut Arild Erstad) writes:

> [Johan Kullstam]
> :
> : (defun make-adder (s)
> : (compile nil (lambda (x) (+ x s))))
> :
> : calling make-adder will return a function which adds s.
>
> Or you can simply do
>
> (defun make-adder (s)
> (lambda (x) (+ x s)))
>
> and if make-adder is compiled, the returned function will also be
> compiled. In fact, there is hardly ever any reason to call 'compile'
> explicitly except during development (and with a proper IDE, not even
> then).

Thanks. I was unaware of this.

> Actually, I have used 'compile' in my code, *once*. It involved
> generating optimized code for spline functions and compiling them
> on-the-fly. It was kind of neat. :)

Nod. I am currently working with arbitrary trellis code. I want to
generate a function (at run time) from a trellis code table. Then,
rather than using a bunch of loops, I will unroll them. I guess this
is a kind of run-time rather than compile-time macro. The result, I
want to eventually compile for speed.

Closing over

(let ((s 4))
(lambda (x) (* x s)))

is different from closing over

(lambda (x) (.....))

where i fill ..... with + and 4 x-es.

(lambda (x) (+ x x x x))

the 4 will be constant for a particular code, but unknown in the
general case. I am still early in this effort, but I suspect I will
learn something from it. What something is, I do not yet know. ;-)

--
Johan KULLSTAM <kulls...@attbi.com> sysengr

Paul D. Lathrop

unread,
Jul 25, 2002, 11:47:02 AM7/25/02
to
Christopher Barber <cba...@curl.com> wrote in
news:psou1mo...@boris.curl.com:
> Finally, aren't CL macros non-hygienic?
>
> - Christopher

Sorry to sound ignorant, but what does non-hygienic mean in
this context?

Paul D. Lathrop

Seth Gordon

unread,
Jul 25, 2002, 12:39:36 PM7/25/02
to
Johan Kullstam wrote:

>
> "Shayne Wissler" <thal...@yahoo.com> writes:
>
> > 1. While parsing a lisp program, you can compile and invoke some of the code
> > you already parsed to help parse the subsequent code, allowing you to modify
> > the syntax on the fly. If this is correct, what are the limitations? Can I
> > write a lisp program that starts out as lisp and starts looking like
> > C++?
>
> I am not sure that you could easily warp Lisp's reader into a C++
> parser. Lisp has a very simple and regular syntax which makes reading
> and parsing the Lisp source much easier than most languages.

See Henry Baker's "Pragmatic Parsing in Common Lisp":
http://home.pipeline.com/~hbaker1/Prag-Parse.html

--
"I haven't convinced myself that it is in the best
interest of our shareholders." --Scott McNealy,
CEO of Sun, on a requirement that CEOs attest to
the accuracy of their company's financial statements
// seth gordon // wi/mit ctr for genome research //
// se...@genome.wi.mit.edu // standard disclaimer //

Jochen Schmidt

unread,
Jul 25, 2002, 1:41:35 PM7/25/02
to
JRStern wrote:

The (rather dumb) name "hygienic" in this context means that the macro
system does not (not to say "is not able" to ;-) capture variables in the
context where the macro gets expanded.

Lets look at an example:

(defmacro foo (a)
`(+ ,a b))

This macro called with (foo 3) would expand to (+ 3 b) and therefore would
capture any variable "b" in the context of the expansion.

(let ((b 1))
(foo 3))

==

(let ((b 1))
(+ 3 b))

While such behaviour can sometimes be what one wants there is the
possibility that one captures a variable without wanting so.

(defmacro dovector ((item vector) &body body)
`(let ((vector ,vector))
(dotimes (index (length vector))
(let ((,item (aref vector index)))
,@body))))

(dovector (i #(1 2 3)) (print i))

would expand to

(LET ((VECTOR #(1 2 3)))
(DOTIMES (INDEX (LENGTH VECTOR))
(LET ((I (AREF VECTOR INDEX)))
(PRINT I))))

As you can see there are three variables captured in the body of dovector -
the symbol given as the argument "item", vector and index. Only the one
given through "item" is really wanted.
Now imagine you have a variable binding with one of those names around a
dovector.

(let ((index 7))
(dovector (i #(1 2 3))
...))

Now the programmer might be confused that within the dovector his index is
shadowed.

We can solve this problem by creating symbols with GENSYM for this purpose
here:

(defmacro dovector ((item vector) &body body)
(let ((vector-sym (gensym))
(index-sym (gensym)))
`(let ((,vector-sym ,vector))
(dotimes (,index-sym (length ,vector-sym))
(let ((,item (aref ,vector-sym ,index-sym)))
,@body))))

Now neither index nor vector are captured any longer.

In "Hygienic" macrosystems such variable capture cannot occur but you buy
that opportunity with a _huge_ reduction in flexibility in the macrosystem.

The "Hygienic" macrosystem issue is actually a very old point of hot
discussions between some Common Lisp and Scheme people. In Scheme being a
Lisp dialect with one namespace for functions and values unintented
variable capture is much more critical than in CL (which has distinct
namespaces). Actually it seems as if the "Hygienic Macros" issue does not
seem to be a real problem for Common Lisp as it is for Scheme...

Jochen Schmidt

unread,
Jul 25, 2002, 1:47:20 PM7/25/02
to
Jochen Schmidt wrote:

> (defmacro dovector ((item vector) &body body)
> (let ((vector-sym (gensym))
> (index-sym (gensym)))
> `(let ((,vector-sym ,vector))
> (dotimes (,index-sym (length ,vector-sym))
> (let ((,item (aref ,vector-sym ,index-sym)))
> ,@body))))

I missed a closing paren here

Thomas Bushnell, BSG

unread,
Jul 25, 2002, 2:24:56 PM7/25/02
to
Marco Antoniotti <mar...@cs.nyu.edu> writes:

> tb+u...@becket.net (Thomas Bushnell, BSG) writes:
>
> > JXSternC...@gte.net (JRStern) writes:
> >
> > > Mea maximum culpa.
> >
> > You mean "mea maxima culpa".
>
> The complete sentence should be
>
> "mea culpa, mea culpa, mea maxima culpa"
>

Well, sure;

Christopher Barber

unread,
Jul 25, 2002, 2:48:23 PM7/25/02
to
Jochen Schmidt <j...@dataheaven.de> writes:

<excellent description of macro hygiene ...>

> In "Hygienic" macrosystems such variable capture cannot occur but you buy
> that opportunity with a _huge_ reduction in flexibility in the macrosystem.

Not necessarily. As long as the macro system lets you capture variables when
you want to, you lose no flexibility. The question is really whether code
expanded in macros should be hygienic by default.

- Christopher

Jochen Schmidt

unread,
Jul 25, 2002, 3:04:39 PM7/25/02
to
Christopher Barber wrote:

To me the question is more like - "Is this really a problem?". At least for
CL I would say it is no real problem and there are means that make it easy
to cope with such things.

Bob Bane

unread,
Jul 25, 2002, 3:09:53 PM7/25/02
to

Hannah Schroeter wrote:


>
> Or current Java's (anonymous) inner classes?
>

For me, Java's closure-like anonymous classes have been more of an
annoyance than either no closures at all, or better closures like CL.
The annoyance is that Java's inner classes will only close over local
variables that are declared "final" - this is apparently to make life
easier for the JVM implementors, who thus don't have to worry about
cross-thread local variable access.

A typical worse-is-better decision, trading implementor effort for user
effort. Bleah.

JRStern

unread,
Jul 25, 2002, 3:23:26 PM7/25/02
to
On Thu, 25 Jul 2002 19:41:35 +0200, Jochen Schmidt <j...@dataheaven.de>

wrote:
>> On 24 Jul 2002 22:59:02 -0400, Christopher Barber <cba...@curl.com>
>> wrote:
>>>Finally, aren't CL macros non-hygienic?
>>
>> Could you expand on that?
>
>The (rather dumb) name "hygienic" in this context means that the macro
>system does not (not to say "is not able" to ;-) capture variables in the
>context where the macro gets expanded.
...

Very clear explanation, thanks.

J.

PS - I'm reading this on comp.object, and your message follow-up seems
to be set only to comp.lang.lisp. Haven't seen many recent
discussions cross those two groups, it's refreshing.

Hannah Schroeter

unread,
Jul 25, 2002, 7:09:15 PM7/25/02
to
Hello!


I wrote:

>> Or current Java's (anonymous) inner classes?

Bob Bane <ba...@removeme.gst.com> wrote:

>For me, Java's closure-like anonymous classes have been more of an
>annoyance than either no closures at all, or better closures like CL.
>The annoyance is that Java's inner classes will only close over local
>variables that are declared "final" - this is apparently to make life
>easier for the JVM implementors, who thus don't have to worry about
>cross-thread local variable access.

That I didn't know. If you just want a normal function closure,
those inner classes are annoying anyway.

>A typical worse-is-better decision, trading implementor effort for user
>effort. Bleah.

Seems so.

Kind regards,

Hannah.

Thomas Bushnell, BSG

unread,
Jul 25, 2002, 11:22:52 PM7/25/02
to
Michael Hudson <m...@python.net> writes:

> Christopher Barber <cba...@curl.com> writes:
>
> > Finally, aren't CL macros non-hygienic?
>
> Yes, but I singularly fail to understand why this is considered a
> problem so long as you aren't really crap at writing macros...

Um, this is the argument that was used to explain why FORTRAN wasn't
an improvement on assembly, why Algol wasn't an improvement on
FORTRAN, etc., etc.

The whole problem is that people *do* make silly stupid mistakes all
the time with the most trivial of programming tasks, and that higher
level languages do a great service by preventing the need for us to
keep bothering with thinking about how to code bignum multiplication.

I'm surprised especially to see the argument on a Lisp newsgroup that
there is no real advantage to high-level facilities, since they are
only needed if you are "crap" at writing things yourself.

It is loading more messages.
0 new messages