Packages

217 views
Skip to first unread message

Erann Gat

unread,
Mar 16, 2002, 12:31:27 PM3/16/02
to

In my formative years I was a fan of T, which for those of you who don't
know, is a member of the Lisp family of languages ;-) that resembles
Scheme more than Common Lisp. It is a Lisp-1, but it had a lot of
features not found in Scheme, like an object system, and a setf-like
feature called "setters". ("SETTER" was a function which when called on
another function would return a function that when called would set the
location referred to by the second function. Whew! e.g.: ((setter car)
x 1) == (replaca x 1)

One of T's built-in features was first-class environments, which are data
structures that map symbols to values. Environments were designed to
serve the same purpose as packages in CL: to prevent name collisions
between program modules. The difference between packages and environments
is that packages allow you to have multiple symbols with the same print
name, while environments let you have multiple values associated with a
single symbol. (This is not to be confused with returning multiple values
from a function, which is a completely different issue.)

A mentor of mine at the time (1988 or so) once expounded on the virtues of
environments and the evils of packages. He made an offhand comment that I
still remember to this day: "Common Lisp" he said "really has a flat
namespace." What he meant was that there was little effective difference
between packages as they exist in CL, and a Lisp with only one package and
a reader hack that attached the prefix "package:" (where "package" is
replaced with the value of the global variable *package*) to all symbol
names that didn't already have colons in them before interning them. Now,
this isn't quite right, of course, because of import, export, and
use-package, but his fundamental point has stuck with me through the
years, that discriminating between multiple interpretations of a symbol
name at read time rather than at eval time was the Wrong Thing. (This is
not to say that I necessarily agree with this point of view, just that I
remember it.)

Through the years I have always found the package system to be pretty
annoying. I often find myself doing the following:

> foo
ERROR: foo is unbound ; Doh! Forgot that foo is in foo-package
> (use-package 'foo-package)
ERROR: symbol FOO-PACKAGE:FOO conflicts with symbol FOO ; Arrggh!!!


So I've always considered packages as more of a necessary evil than a
feature. I was told that this view was common in the CL world, as
reflected in one of Guy Steele's many subtle jokes in CLTLn: in both
editions of the book the section dealing with packages is "chapter 11",
which in the American legal system refers to a set of laws dealing with
bankruptcy.

But in the recent discussion of what "first class symbol" means a number
of people have made statements that at least imply (and sometimes
explicitly state) that they like packages, that they think they really are
a cool feature. For those of you who feel this way I have the following
question: were you aware of the eval-time alternative for resolving name
conflicts provided by environments? And if you were, what is it that
makes you prefer packages?

Thanks,
E.

Christopher C. Stacy

unread,
Mar 16, 2002, 1:53:25 PM3/16/02
to
>>>>> On Sat, 16 Mar 2002 09:31:27 -0800, Erann Gat ("Erann") writes:
Erann> In my formative years I was a fan of T, which for those of you
Erann> who don't know, is a member of the Lisp family of languages
Erann> ;-) that resembles Scheme more than Common Lisp.

The original description of T (as told to me by the implementors
when they had just started) was: "industrial strength Scheme".

Dorai Sitaram

unread,
Mar 16, 2002, 2:23:38 PM3/16/02
to
In article <gat-160302...@192.168.1.50>,

Erann Gat <g...@jpl.nasa.gov> wrote:
>
>Through the years I have always found the package system to be pretty
>annoying. I often find myself doing the following:
>
>> foo
>ERROR: foo is unbound ; Doh! Forgot that foo is in foo-package
>> (use-package 'foo-package)
>ERROR: symbol FOO-PACKAGE:FOO conflicts with symbol FOO ; Arrggh!!!

I sympathize, but I expect to get singed on the
REPL all the time, even in Scheme. I mainly use the
REPL to observe how things crash and burn.

>So I've always considered packages as more of a necessary evil than a
>feature. I was told that this view was common in the CL world, as
>reflected in one of Guy Steele's many subtle jokes in CLTLn: in both
>editions of the book the section dealing with packages is "chapter 11",
>which in the American legal system refers to a set of laws dealing with
>bankruptcy.

:-) :-) All these years with that book, and I
didn't realize that till now.

>But in the recent discussion of what "first class symbol" means a number
>of people have made statements that at least imply (and sometimes
>explicitly state) that they like packages, that they think they really are
>a cool feature. For those of you who feel this way I have the following
>question: were you aware of the eval-time alternative for resolving name
>conflicts provided by environments? And if you were, what is it that
>makes you prefer packages?

Environments are forbiddingly expensive? I mean,
Scheme, whose midwives were well aware of first-class
environments (SICP 1/e mentions them) would have
standardized on them in a jiffy otherwise, no?

--d

Rahul Jain

unread,
Mar 16, 2002, 4:43:43 PM3/16/02
to
g...@jpl.nasa.gov (Erann Gat) writes:

> One of T's built-in features was first-class environments, which are data
> structures that map symbols to values. Environments were designed to
> serve the same purpose as packages in CL: to prevent name collisions
> between program modules.

WRONG! Just because you don't value symbolic processing doesn't mean
that others can't. PLEASE notice that symbols in Lisp ARE VALUES and
can be used as such. Therefore, when passing symbols around, one MUST
be sure that application X's symbol is not the same as application Y's
symbol due to some unintentional name-clash. That is what packages
solve.

> A mentor of mine at the time (1988 or so) once expounded on the virtues of
> environments and the evils of packages.

Amazing that of two things that can cooperate, one is evil and one is good.

> Through the years I have always found the package system to be pretty
> annoying. I often find myself doing the following:

> > foo
> ERROR: foo is unbound ; Doh! Forgot that foo is in foo-package
> > (use-package 'foo-package)
> ERROR: symbol FOO-PACKAGE:FOO conflicts with symbol FOO ; Arrggh!!!

So choose the restart to unintern it from the current package.

> But in the recent discussion of what "first class symbol" means a number
> of people have made statements that at least imply (and sometimes
> explicitly state) that they like packages, that they think they really are
> a cool feature. For those of you who feel this way I have the following
> question: were you aware of the eval-time alternative for resolving name
> conflicts provided by environments?

No, since environments map symbols to values, not strings to symbols.

--
-> -/ - Rahul Jain - \- <-
-> -\ http://linux.rice.edu/~rahul -=- mailto:rj...@techie.com /- <-
-> -/ "Structure is nothing if it is all you got. Skeletons spook \- <-
-> -\ people if [they] try to walk around on their own. I really /- <-
-> -/ wonder why XML does not." -- Erik Naggum, comp.lang.lisp \- <-
|--|--------|--------------|----|-------------|------|---------|-----|-|
(c)1996-2002, All rights reserved. Disclaimer available upon request.

Tim Bradshaw

unread,
Mar 16, 2002, 5:48:46 PM3/16/02
to
* Erann Gat wrote:

> A mentor of mine at the time (1988 or so) once expounded on the virtues of
> environments and the evils of packages. He made an offhand comment that I
> still remember to this day: "Common Lisp" he said "really has a flat
> namespace." What he meant was that there was little effective difference
> between packages as they exist in CL, and a Lisp with only one package and
> a reader hack that attached the prefix "package:" (where "package" is
> replaced with the value of the global variable *package*) to all symbol
> names that didn't already have colons in them before interning them. Now,
> this isn't quite right, of course, because of import, export, and
> use-package, but his fundamental point has stuck with me through the
> years, that discriminating between multiple interpretations of a symbol
> name at read time rather than at eval time was the Wrong Thing. (This is
> not to say that I necessarily agree with this point of view, just that I
> remember it.)

It's quite possible to build a system on top of the `flat' package
namespace which allows it to be quite non-flat. I have a system
called conduits (and there is at least one other such system I think),
which adds a couple of options to DEFPACKAGE which let you define a
package as `extending' another one. For a long time I didn't really
use this in anger but I have done so now, and the package structure of
the system I'm working on now is layered quite deeply.

I have a whole bunch of low-level packages, and then something like:

(defpackage :weld-low
(:use)
(:extends ...n packages...))

and then further up

(defpackage :weld-crs-implementation
(:use :cl :weld-low))

...

(defpackage :weld
(:use)
(:extends :weld-low :weld-crs-implementation ...))

others Of course it's `just a hack' and not hygenic or anything, so
doesn't really qualify as a proper program in our brave, pure, new
world. It also doesn't address the heirarchical package *name* issue
at all which I think ACL does for instance.

Still, it's very useful, I find.

--tim

Erik Naggum

unread,
Mar 16, 2002, 7:00:55 PM3/16/02
to
* Erann Gat

| A mentor of mine at the time (1988 or so) once expounded on the virtues
| of environments and the evils of packages. He made an offhand comment
| that I still remember to this day: "Common Lisp" he said "really has a
| flat namespace." What he meant was that there was little effective
| difference between packages as they exist in CL, and a Lisp with only one
| package and a reader hack that attached the prefix "package:" (where
| "package" is replaced with the value of the global variable *package*) to
| all symbol names that didn't already have colons in them before interning
| them.

I begin to see where you come from and where you went wrong, Erann. This
is such a stupid thing to say that I marvel at the respect you must have
felt for this mentor. I hope _I_ shall never have a student who does not
have in him the ability to say "that's a load of crap, Erik" if I make a
similarly retarded point and expect people to believe it, not laugh.

How about the function and variable values of a symbol? It is just as if
we had used Scheme evaluation rules, but had a reader hack that prefixed
the first position of a form with "operator-" and every other symbol with
"variable-". See? No material difference between Scheme and Common Lisp
-- they are equivelant modulo a reader hack. Right? If you see how
_retarded_ it would be to explain the difference this way, which I
suspect you will, since I am not your highly respected mentor and you are
free to think instead of believe, it should help you realize just how
retarded the explanation that you heard about packages was, too.

| Now, this isn't quite right, of course, because of import, export, and
| use-package, but his fundamental point has stuck with me through the
| years, that discriminating between multiple interpretations of a symbol
| name at read time rather than at eval time was the Wrong Thing.

This is just too incredible to be for real. There is no discrimination
between multiple interpretation of a symbol at read time. That is just
_so_ confused, no wonder you have serious problems. Have you actually
ever understood Common Lisp at all, Erann? I have to wonder. At read
time, a string is interned in only one package to return a symbol. There
_is_ no symbol without the package. There are only the characters in a
symbol's name.

Again and again, you show us that you are _horribly_ confused and that
much of your dislike of Common Lisp comes not from Common Lisp, but from
some staggeringly stubborn misconception and refusal to listen to others.

| (This is not to say that I necessarily agree with this point of view,
| just that I remember it.)

The more I read from you, the more I think you are a Scheme person.
This is not a compliment.

| Through the years I have always found the package system to be pretty
| annoying. I often find myself doing the following:
|
| > foo
| ERROR: foo is unbound ; Doh! Forgot that foo is in foo-package
| > (use-package 'foo-package)
| ERROR: symbol FOO-PACKAGE:FOO conflicts with symbol FOO ; Arrggh!!!

Such are the results of incompetence at work. And how typical to react
emotionally at something so fundamentally silly. How come I never have
this problem? Could it be that I use symbol completion in Emacs? Could
it be that when I once in a while do get an _error_ like this, one of the
available restarts is to unintern the old symbol and proceed, so I have
no _problem_ with it? Could it be that I build my packages with care and
actually want to know which package a symbol is in -- _because_ I do not
consider packages and symbols evil?

| So I've always considered packages as more of a necessary evil than a
| feature.

Yop, Scheme is for you, this is becoming clearer and clearer.

| I was told that this view was common in the CL world, as reflected in one
| of Guy Steele's many subtle jokes in CLTLn: in both editions of the book
| the section dealing with packages is "chapter 11", which in the American
| legal system refers to a set of laws dealing with bankruptcy.

I find this unfunny, not because it is offensive, but because it is so
stupid. It could have been a cute pun if it it has been rephrased as
something like "Common Lisp packages are in Chapter 11", but as stated it
is just dumb, mostly because "Chapter 11" is nothing more than a silly
coincidende and it has to be in an expression that relates it to the
U.S. Bankruptcy Code to make any sense.

But your interpretation of what "Chapter 11" sort of "means" is actually
a very good example of how _wrong_ it is to do eval-time interpretation
instead of read-time. It may make for puns (good, bad, atrocious), but
not understanding. Unless one wants to publish books without a Chapter
11 like some hotels do not have a 13th floor because of a similarly bad
case of numerology and mysticism, there will have to be a Chapter 11 in
books with more than 10. E.g., exceptions are bankrupt in Java in your
"interpretation" because they are in Chapter 11 in both editions of "The
Java Language Specification", also written by Steele, et al, so surely
this is not accidental, either. I am sure this is "fun" to people who do
eval-time interpretation with random environments, but I find it _stupid_.

BTW, Chapter 11 of the U.S. Bankruptcy Code (Title, not Chapter, 11 of
the United States Code) is concerned with reorganization. It applies
when your creditors have some hope that they will recover their money if
they can (forcibly) reorganize the company. Chapter 7 (liquidation)
applies when there is no hope and the creditors accept whatever share
they get from the proceeds of its liquidation. This should have been
common knowledge to anyone who has any interest in referring to the
U.S. legal system. A remarkable level of intellectual sloppiness is
necessary to make this connection and to think it has any meaning.

| But in the recent discussion of what "first class symbol" means a number
| of people have made statements that at least imply (and sometimes
| explicitly state) that they like packages, that they think they really
| are a cool feature.

You know, there are some people who move to other countries, or other
places in the same country, whatever, and who manage to _like_ the place
they have moved to, even though it has lots of flaws and problems and
unlikable people and funny smells and such, probably just like the place
they moved from, which they probably also made up their mind to like, but
it may not have worked out, or they wanted something _different_, not for
lack of like. Then there are people who know what they will and will not
like and who are _never_ satisfied or happy with anything, and who are
more than likely to _avoid_ doing anything than doing something they do
not like through and through. Scheme has a history of not doing anything
because they _feared_ that would not like it, or that somebody had made
up their mind they did not like it even before they tried.

I think the ability to make up your mind to like something you _want_ to
use is kind of crucial to survival. There is so much wrong with the
world that one has to accept flaws and imperfections and just get on with
one's purpose in life. People who _have_ no purpose, do not need this.
There is no such thing as a necessary evil. Thinking in such terms is
itself destructive, not only to one's ability to do what is necessary,
but to appreciate the results. If you have deal with necessary evils all
the time, you will be in a state of permanent frustration, and what
better way of expressing this than "I like Common Lisp, but ...", and of
becoming so frustrated with Common Lisp that one spends one's time
griping about the language instead of just getting on with one's life.

But more importantly, the attitude problems that result in one's
inability to "like" something one has to use or do because of many other
factors, such as liking _them_, is actually pandemic to a personality.
People who have to set _all_ the rules, cannot _follow_ any rules. We
have seen this in people who publicly favor going behind their clueless
boss instead of working _with_ him, we have seen this is people who make
their _living_ from other people's trust that they can follow a standard
specification go public with statements that it is "braindamaged".
However, there are people who are actually able to live and prosper in a
system where respecting authorities, the rule of law, your boss and the
chain of command, etc -- one does not _have_ to be a criminal to make
money or become rich, but it helps some, like Bill Gates and Enron and
such, but by far most criminals end up rather destitute or dead at an
early age. Protesting against authority is an immature youth thing, part
of breaking free from your parents, but some people actually figure out
that authority itself is not what made it necessary to break with your
parents, and they do not have to break with every other authority the
same way.

| For those of you who feel this way I have the following question: were
| you aware of the eval-time alternative for resolving name conflicts
| provided by environments?

Yes, we call them hashtables using strings for keys. Oh, right, Scheme
does not have hashtables. Common Lisp also supports hashtables that use
keywords for keys.

| And if you were, what is it that makes you prefer packages?

This question makes absolutely no sense. If I am aware of the existence
of white meat, why do I prefer red meat? I like both, I eat both, but on
different occasions. I use hashtables with string keys for some things
and keyword for others. I sometimes use the package system because the
reader is such a convenient function that I do not want to reimplement
it, and the package system has some really nifty features that, if you
can come to like what you have, you can find good use for.

If you cannot like what you have freely accepted, there is something
wrong with you. If you do not accept something, but you cannot break
free from it, there is something wrong with you. You have but one life
to live, and it is too short to remain too long with your mistakes and
too long _not_ to leave them behind in time.

///
--
In a fight against something, the fight has value, victory has none.
In a fight for something, the fight is a loss, victory merely relief.

Kent M Pitman

unread,
Mar 16, 2002, 8:03:58 PM3/16/02
to
g...@jpl.nasa.gov (Erann Gat) writes:

g...@jpl.nasa.gov (Erann Gat) writes:

> In my formative years I was a fan of T, which for those of you who don't
> know, is a member of the Lisp family of languages ;-) that resembles
> Scheme more than Common Lisp. It is a Lisp-1, but it had a lot of
> features not found in Scheme, like an object system, and a setf-like
> feature called "setters". ("SETTER" was a function which when called on
> another function would return a function that when called would set the
> location referred to by the second function. Whew! e.g.: ((setter car)
> x 1) == (replaca x 1)

...


> But in the recent discussion of what "first class symbol" means a number
> of people have made statements that at least imply (and sometimes
> explicitly state) that they like packages, that they think they really are
> a cool feature. For those of you who feel this way I have the following
> question: were you aware of the eval-time alternative for resolving name
> conflicts provided by environments?

Being one of the central parties making this noise, I'll do something I'm
not big on and actually cite my credentials to show that I am not blathering
out of ignorance:

(1) I, along with Jonathan Rees and Norman Adams, was one of a 3-person
team that kicked of the T effort at Yale. I did the initial language
prototype. I basically wanted to design a language but not a compiler,
and Jonathan wanted to design a compiler but not a language, which is
how he and I teamed up. (Norm was new to it all then and his major
contributions I think came after I left.)

I'm personally responsible for the creation of the creation of the
"locale" concept because Jonathan didn't like the "global" notion
and yet I was convinced people wanted the metaphor even if it had
problems in practice. So we made locales, which were, as I initially
described them, "locally global" workspaces that could serve as
bottom-out points for lexicality to home all otherwise-free references.

I'm also the person who, at Jonathan's request (i.e., it was his idea
but I validated its feasibility), first did the hack, in my Maclisp-based
emulator of making message-sending functions basically use their own
identity as the value to be sent. This is a huge awful pun that could
only work cleanly in a Lisp1 (and we knew that) but basically we did
something like (if I recall the technique, though I've forgotten the
operator names--I only know we spelled out whole words and didn't use
abbrevs like I've used here):

(define (make-op)
(letrec ((op (lambda (self . args) (apply send self op args))))
op))

(define kar (make-op)) ;Make a KAR operation
(define kdr (make-op)) ;Make a KDR operation

So that the handler for an object doing this message would handle
a message whose name was the function that sent the message. (Weird.)

So one day I changed over the operation handlers from something that
did (cond ((eq? msg 'kar) ...) ...) to (cond ((eq? msg kar) ...) ...)
which had the neat [lisp1-only] property that if you renamed the kar
message operation, you also renamed its key. So someone in another
environment used to calling it FIRST and LAST could do
(cond ((eq? msg first) ...) ...)
and didn't have to know that in the sender's environment the name of the
message was KAR, not FIRST. Further, since SETTER is an object message
itself, a renaming of KAR means that SETTER gets called on the
right thing and so (SETTER FIRST) in the environment with FIRST renamed
does exactly the right thing.

It's all very cool. But, then again, it's one of those oh-too-cute
kludges that they're referring to when they say "try not to be
too clever". It's a hack. A hack that works. But a serious
pun. It isn't something that falls out of the beauty of Lisp1. It's
a kludge constructed tenuously from the felicity ("accidental truth")
of Lisp1 and other accidental factors--like that you can compare
procedures with EQ? ... there was a move afoot in Scheme a while back
to say that EQ? should not be a "total" function; that is, that it
should be an error to call it on certain types, functions among them.
Had that been so, the entire kludge that was that cool thing in T
would have not worked, Lisp1 or not. Message objects would have had to
be some other datum than the function itself. But it didn't come out
that way and the punsters of the world (Jonathan and I) won out that day.
I _like_ puns mind you, and I don't feel guilty. I loved
this pun in particular and am glad you remember it fondly. But I
have a real laugh when people talk about Lisp2 as if they are the
home of all puns and Lisp1 is free somehow of the urge to pun.
In any language, people will make use of the semantics to their
advantage. Where namespaces are folded on themselves, people
will use that fact; where they are not, people will use that.
We're all sinners. Let he who thinks he is not cast the first
stone.. (Oh, ouch, that's right--they have. Ok. You can stop now.)

(2) I am among the authors of the Scheme report. I don't get my way a
lot in certain votes, but I like to think that it sometimes matters
that I am there. And in any case, I am aware of how the Scheme
people think about modules. It's been discussed many times among
the members over the years.

(3) I have used Scheme in workplace settings for paid work. I do know how
Scheme works from a practical point of view.

(4) The ISLISP committee, for which I was for a time US Representative from
(X3)J13 and Project Editor. [I continue as Project Editor although not
a J13 member because this is a neutral, non-technical role and does not
require me to be a technical participant nor national representative,
just a helpful guy.] Modules and packages are among the topics
that we discussed some years back but deferred for "future work".

In short, if your question was, am I just making up my preference for packages
because I am ignorant of the alternatives, the answer is: No.

> And if you were, what is it that makes you prefer packages?

Do you want me to repeat the stuff I mentioned under the data hygiene thread?

The short answer is that non-symbols are fine, but they do not a symbolic
processing language make. Some days I don't mind passing around Scheme's
symbols (with their characteristic "no associated meaning") + an environment
(to give the symbols meaning). Other days, I like the power of being able
to have symbols like in CL, where the symbol means something all by itself.
(Most days, in fact.) Passing non-symbols is fine computation, but is not
the essence of symbolic computation.

Erann Gat

unread,
Mar 16, 2002, 10:36:57 PM3/16/02
to
In article <sfwg030...@shell01.TheWorld.com>, Kent M Pitman
<pit...@world.std.com> wrote:

> Being one of the central parties making this noise, I'll do something I'm
> not big on and actually cite my credentials to show that I am not blathering
> out of ignorance:

I won't follow your example because I don't know if I'm blathering out of
ignorance or not. (That's what I'm trying to figure out.) But if you
want to see my credentials my CV is on the Web. Suffice it to say I've
built a successful career in part by writing Common Lisp code. I feel
fairly confident that I'm sufficiently non-idiotic that if someone can't
explain something in this area so that I can understand it there's a good
chance that it's their fault and not mine.

> > And if you were, what is it that makes you prefer packages?
>
> Do you want me to repeat the stuff I mentioned under the data hygiene thread?

No, but I would like you to expand on something:

> I cannot later 'relate', at the symbol
> level, your person-1 with my person-1, without doing a global renaming.

What does that mean? Can you restate this (which seems to be the crucial
point) without scare quotes and without squishy phrases like "at the
symbol level"?

> The short answer is that non-symbols are fine, but they do not a symbolic
> processing language make. Some days I don't mind passing around Scheme's
> symbols (with their characteristic "no associated meaning") + an environment
> (to give the symbols meaning). Other days, I like the power of being able
> to have symbols like in CL, where the symbol means something all by itself.
> (Most days, in fact.) Passing non-symbols is fine computation, but is not
> the essence of symbolic computation.

Suppose CL had no symbols. It seems I could easily add them by defining
two classes: "symbol" with slots name, value, plist, etc. and "package"
that's really just a hash table that maps strings onto instances of the
symbol class. Probably 20-30 LOC at most. Would that be sufficient? Or
would I still be missing some essential capability? What would it be?

Maybe the problem is that I don't really understand what people mean by
"symbolic processing". A concrete non-contrived example of "symbolic
processing" that's easy in CL and hard in Scheme would probably help.

E.

Joe Marshall

unread,
Mar 16, 2002, 10:13:56 PM3/16/02
to

"Erann Gat" <g...@jpl.nasa.gov> wrote in message
news:gat-160302...@192.168.1.50...

>
> Through the years I have always found the package system to be pretty
> annoying. I often find myself doing the following:
>
> > foo
> ERROR: foo is unbound ; Doh! Forgot that foo is in foo-package
> > (use-package 'foo-package)
> ERROR: symbol FOO-PACKAGE:FOO conflicts with symbol FOO ; Arrggh!!!
>
>
> So I've always considered packages as more of a necessary evil than a
> feature.

Since everyone else is weighing on this I might as well offer my uninformed
opinion.

In the years I have been hacking lisp, I have on many occasions cursed
the package system, mainly for making me restart my lisp to undo whatever
random damage happened when I read something in the wrong package.
On the other hand, I have *never once* had the occasion to say
`Wow, damn glad that package system saved my ass on *this*!'

Nils Goesche

unread,
Mar 17, 2002, 12:24:07 AM3/17/02
to
"Joe Marshall" <prunes...@attbi.com> writes:

It is hard to tell if you are serious on this but do you actually
mean that? Are you aware of the fact that it might be a benefit
of the package system that you had no reason so far to complain
when you used different packages from different parties for your
own projects? Or is the above in fact an example of very fine
irony which wanted to say this very thing? In that case I feel
very embarrassed for bothering you...

Regards,
--
Nils Goesche
Ask not for whom the <CONTROL-G> tolls.

PGP key ID 0xC66D6E6F

Kenny Tilton

unread,
Mar 17, 2002, 12:33:36 AM3/17/02
to

> And if you were, what is it that
> makes you prefer packages?

Good things come in small packages.

When I broke up a monolithic wadge of code into reasonable packages (for
no particular reason other than it seemed The Right Thing), to get clean
compiles I had to refactor -- not too much, but enough to realize the
code was being improved in each case by the exercise.

Note that this was not about name conflicts, just classic stuff like a
utility package calling functions from a higher level package it should
not have known about. less so the higher-level package going after
unexported implementation stuff.

--

kenny tilton
clinisys, inc
---------------------------------------------------------------
"Well, I've wrestled with reality for thirty-five years, Doctor, and
I'm happy to state I finally won out over it." Elwood P. Dowd

Nils Goesche

unread,
Mar 17, 2002, 12:51:37 AM3/17/02
to
g...@jpl.nasa.gov (Erann Gat) writes:

> In article <sfwg030...@shell01.TheWorld.com>, Kent M Pitman
> <pit...@world.std.com> wrote:
>
> > Being one of the central parties making this noise, I'll do something I'm
> > not big on and actually cite my credentials to show that I am not blathering
> > out of ignorance:
>
> I won't follow your example because I don't know if I'm blathering out of
> ignorance or not. (That's what I'm trying to figure out.) But if you
> want to see my credentials my CV is on the Web. Suffice it to say I've
> built a successful career in part by writing Common Lisp code. I feel
> fairly confident that I'm sufficiently non-idiotic that if someone can't
> explain something in this area so that I can understand it there's a good
> chance that it's their fault and not mine.

Absolutely true. It is certainly not your fault if people cannot
(so far) explain in all detail how they arrived at what they are
talking about now. Everybody is aware of your impressive
credentials, and you know much, much more about Lisp than I do.
But I do know something about ideas. What you are doing now
looks a bit unfair to me -- Kent seems to have had a sudden,
epiphanic, idea on what might be the source, the root of the
difference in character between Lisp and Scheme; why don't you
give this idea a chance? Of course he cannot prove it formally,
let alone right now. But isn't it worth considering for a while?

Nils Goesche

unread,
Mar 17, 2002, 1:29:02 AM3/17/02
to
Nils Goesche <n...@cartan.de> writes:

Sorry for following up to myself, but I'd like to add something:
Why don't you try to prove it yourself? I have often found that
it is a good way, even if you don't believe in an idea, to try to
prove it nevertheless: You'll find out what the major obstacles
are and thus gain valuable insight -- that would be much more
productive than demanding a formal prove from others.

Nils Goesche

unread,
Mar 17, 2002, 1:30:35 AM3/17/02
to
Nils Goesche <n...@cartan.de> writes:

> productive than demanding a formal prove from others.

proof

Darn, sometimes even English spelling is hard. Sorry.

Erann Gat

unread,
Mar 17, 2002, 1:34:21 AM3/17/02
to
In article <32253120...@naggum.net>, Erik Naggum <er...@naggum.net> wrote:

> I begin to see where you come from and where you went wrong, Erann. This
> is such a stupid thing to say that I marvel at the respect you must have
> felt for this mentor. I hope _I_ shall never have a student who does not
> have in him the ability to say "that's a load of crap, Erik" if I make a
> similarly retarded point and expect people to believe it, not laugh.

That's a load of crap, Erik. Having respect for someone doesn't mean that
you believe everything they say.

> Have you actually ever understood Common Lisp at all, Erann?

I don't know. That's what I'm trying to figure out. You're not being
much help.

> I have to wonder. At read
> time, a string is interned in only one package to return a symbol. There
> _is_ no symbol without the package.

Uninterned symbols have no package. Have you ever understood Common Lisp
at all, Erik?

> Again and again, you show us that you are _horribly_ confused

That's true. I am horribly confused. Whether that's my fault or the
fault of the people trying to explain things to me is unclear.

> and that much of your dislike of Common Lisp

That is false. I like Common Lisp very much.

> comes not from Common Lisp, but from
> some staggeringly stubborn misconception and refusal to listen to others.

Do you even realize that earlier in this very same post you berated me not
for my refusal to listen, but for having listened?

> | Through the years I have always found the package system to be pretty
> | annoying. I often find myself doing the following:
> |
> | > foo
> | ERROR: foo is unbound ; Doh! Forgot that foo is in foo-package
> | > (use-package 'foo-package)
> | ERROR: symbol FOO-PACKAGE:FOO conflicts with symbol FOO ; Arrggh!!!
>
> Such are the results of incompetence at work.

One could respond the same way to every frustration anyone has ever had
with any programming language. On this view every problem anyone has
with, say, C++, is not a problem with C++ but merely incompetence on the
part of the user. Some people actually believe this.

E.

Rahul Jain

unread,
Mar 17, 2002, 2:28:06 AM3/17/02
to
g...@jpl.nasa.gov (Erann Gat) writes:

> In article <32253120...@naggum.net>, Erik Naggum <er...@naggum.net> wrote:
> > I begin to see where you come from and where you went wrong,
> > Erann. This is such a stupid thing to say that I marvel at the
> > respect you must have felt for this mentor.

> That's a load of crap, Erik. Having respect for someone doesn't


> mean that you believe everything they say.

Then it was worse, it was blind faith.

> > Have you actually ever understood Common Lisp at all, Erann?

> I don't know. That's what I'm trying to figure out. You're not
> being much help.

It helps if you try to understand what he's telling you and stop
bringing up rants from misguided people who confuse orthogonal
concepts.

> > I have to wonder. At read time, a string is interned in only
> > one package to return a symbol. There _is_ no symbol without
> > the package.

> Uninterned symbols have no package. Have you ever understood Common
> Lisp at all, Erik?

It's a good thing you removed (from your mind?) the rest of the
context in order to make it look wrong. You were talking about
choosing the right package to intern into at read time.

> > Again and again, you show us that you are _horribly_ confused

> That's true. I am horribly confused. Whether that's my fault or the
> fault of the people trying to explain things to me is unclear.

That's easy. If you don't understand something that many others
understand and use effectively, they obviously learned it somehow. Why
don't you try learning it, too, instead of trying to figure out whom
to blame next?

> > and that much of your dislike of Common Lisp

> That is false. I like Common Lisp very much.

You seem to complain about it more than enough and intentionally
misunderstand it enough. And as one of the basic principles of Common
Lisp, a dislike of packages implies a dislike of Lisp and symbolic
processing itself.

> > comes not from Common Lisp, but from some staggeringly stubborn
> > misconception and refusal to listen to others.

> Do you even realize that earlier in this very same post you berated
> me not for my refusal to listen, but for having listened?

Blind faith involves the listener consciously choosing to stop
listening at that point.

> > | Through the years I have always found the package system to be pretty
> > | annoying. I often find myself doing the following:
> > |
> > | > foo
> > | ERROR: foo is unbound ; Doh! Forgot that foo is in foo-package
> > | > (use-package 'foo-package)
> > | ERROR: symbol FOO-PACKAGE:FOO conflicts with symbol FOO ; Arrggh!!!
> >
> > Such are the results of incompetence at work.
>
> One could respond the same way to every frustration anyone has ever had
> with any programming language. On this view every problem anyone has
> with, say, C++, is not a problem with C++ but merely incompetence on the
> part of the user. Some people actually believe this.

If someone has a problem with understanding what a feature of C++
means, it's the fault of that person for not understanding it, not the
fault of the language for including it. Maybe it was designed for
people with different goals. In this case, it's clearly a result of
the fact that you forgot to think about what you were doing before
doing it. If you realized that packages formed contexts within which
symbols were interpreted, you wouldn't have had this problem. It's
like looking in the wrong module for a value in a scheme system. Is it
scheme's fault that you chose to look where you didn't want to?

Thomas Bushnell, BSG

unread,
Mar 17, 2002, 2:39:45 AM3/17/02
to
Rahul Jain <rj...@sid-1129.sid.rice.edu> writes:

> You seem to complain about it more than enough and intentionally
> misunderstand it enough. And as one of the basic principles of Common
> Lisp, a dislike of packages implies a dislike of Lisp and symbolic
> processing itself.

Geez! Erann actually *likes* CL, probably more than Scheme, but
(I'm guessing) doesn't happen to think it is Absolutely Perfect. As a
result, he perhaps complains about things once in a while.

And anyone who complains must therefore dislike CL!?

One must think CL packages are really the best thing ever, or else one
not only dislikes CL in general, but also Lisp, and "symbolic
processing itself"?

Puhleez.

> If someone has a problem with understanding what a feature of C++
> means, it's the fault of that person for not understanding it, not the
> fault of the language for including it.

Actually, that's not quite true. One of the tasks of a language
designer is to make the language easy to understand. C++ more or less
fails on that score (at least as judged by the conceptual elegance of
both CL and Scheme). This is a fault to be chalked up to the language
designer as much as to the people who have trouble understanding the
result.

Thomas

Rahul Jain

unread,
Mar 17, 2002, 2:56:23 AM3/17/02
to
tb+u...@becket.net (Thomas Bushnell, BSG) writes:

> Rahul Jain <rj...@sid-1129.sid.rice.edu> writes:
>
> > You seem to complain about it more than enough and intentionally
> > misunderstand it enough. And as one of the basic principles of Common
> > Lisp, a dislike of packages implies a dislike of Lisp and symbolic
> > processing itself.
>
> Geez! Erann actually *likes* CL, probably more than Scheme, but
> (I'm guessing) doesn't happen to think it is Absolutely Perfect. As a
> result, he perhaps complains about things once in a while.

He seems to not like actually using it, then.

> One must think CL packages are really the best thing ever, or else one
> not only dislikes CL in general, but also Lisp, and "symbolic
> processing itself"?

CL packages are merely an implementation of namespacing for
symbols. Claiming that they should be replaced with a totally
orthogonal system is claiming that symbols should not be namespaced,
which means that having multiple symbolic processing systems operating
on the same data is subject to name clashes.

> Actually, that's not quite true. One of the tasks of a language
> designer is to make the language easy to understand. C++ more or less
> fails on that score (at least as judged by the conceptual elegance of
> both CL and Scheme). This is a fault to be chalked up to the language
> designer as much as to the people who have trouble understanding the
> result.

Except that this is the sole feature which allows people to use the
language in it's professed primary purpose, symbolic processing. If
you misunderstood what the point of the "new" operator in C++ was and
claimed that it should be replaced with interned strings, would that
be a failing of C++?

Thomas Bushnell, BSG

unread,
Mar 17, 2002, 3:13:32 AM3/17/02
to
Rahul Jain <rj...@sid-1129.sid.rice.edu> writes:

> tb+u...@becket.net (Thomas Bushnell, BSG) writes:
>
> > Rahul Jain <rj...@sid-1129.sid.rice.edu> writes:
> >
> > > You seem to complain about it more than enough and intentionally
> > > misunderstand it enough. And as one of the basic principles of Common
> > > Lisp, a dislike of packages implies a dislike of Lisp and symbolic
> > > processing itself.
> >
> > Geez! Erann actually *likes* CL, probably more than Scheme, but
> > (I'm guessing) doesn't happen to think it is Absolutely Perfect. As a
> > result, he perhaps complains about things once in a while.
>
> He seems to not like actually using it, then.

So he must like everything about it, or he's against it?

There are lots of things about my Debian GNU/Linux systems that I
complain about. It does not follow that I don't like Debian, or that
I think Posix is a waste.

> CL packages are merely an implementation of namespacing for
> symbols. Claiming that they should be replaced with a totally
> orthogonal system is claiming that symbols should not be namespaced,
> which means that having multiple symbolic processing systems operating
> on the same data is subject to name clashes.

Erann didn't make any such claim at all. He didn't express an
opinion; he just said that packages got in his way once or twice, and
he'd heard from really well connected people that they were (at least
once) regarded as rather a necessary wart on Common Lisp than a great
feature. So he was wondering the reasons why they were now being
toutes as a great feature.

He didn't claim that they should be replaced at all.

> Except that this is the sole feature which allows people to use the
> language in it's professed primary purpose, symbolic processing.

I think this is a great misunderstanding of what "symbolic processing"
is about. However, I'm not interested in having that conversation
with you. (Kent, if you find it worth discussing, then I am
interested in having that conversation with you.)

Thomas

Rahul Jain

unread,
Mar 17, 2002, 4:58:39 AM3/17/02
to
tb+u...@becket.net (Thomas Bushnell, BSG) writes:

> Rahul Jain <rj...@sid-1129.sid.rice.edu> writes:
> > tb+u...@becket.net (Thomas Bushnell, BSG) writes:
> > > Rahul Jain <rj...@sid-1129.sid.rice.edu> writes:

> > > Geez! Erann actually *likes* CL, probably more than Scheme, but
> > > (I'm guessing) doesn't happen to think it is Absolutely Perfect.
> > > As a result, he perhaps complains about things once in a while.

Once in a while? What else has he done in c.l.l in the entire time
I've been reading it?

> > He seems to not like actually using it, then.

> So he must like everything about it, or he's against it?

No, but he doesn't seem to like anything about it that he couldn't get
elsewhere, along with other features that he would like better.

> There are lots of things about my Debian GNU/Linux systems that I
> complain about. It does not follow that I don't like Debian, or that
> I think Posix is a waste.

But do you spend most of your posts complaining about debian on
debian...@l.d.o? Not that I've seen. Note that Erann hasn't made
any suggestions on how to accomodate symbol namespacing without CL
packages yet, and neither have you.

> Erann didn't make any such claim at all. He didn't express an
> opinion; he just said that packages got in his way once or twice, and
> he'd heard from really well connected people that they were (at least
> once) regarded as rather a necessary wart on Common Lisp than a great
> feature. So he was wondering the reasons why they were now being
> toutes as a great feature.

> He didn't claim that they should be replaced at all.

He claimed that he agreed with someone who said that they should be
replaced. I don't see what the material difference is unless you're
claiming that he didn't know what he was agreeing with.

> > Except that this is the sole feature which allows people to use the
> > language in it's professed primary purpose, symbolic processing.

> I think this is a great misunderstanding of what "symbolic processing"
> is about.

Then you misunderstand that name clashes are a fatal situation in
symbolic processing. It's like doing side-effects while ignoring
structure-sharing.

> However, I'm not interested in having that conversation with you.
> (Kent, if you find it worth discussing, then I am interested in
> having that conversation with you.)

I agree that Kent is much more knowledgable than I am. I don't see how
that makes my input useless, but I guess that's your personal bias.

Thomas Bushnell, BSG

unread,
Mar 17, 2002, 5:19:38 AM3/17/02
to
Rahul Jain <rj...@sid-1129.sid.rice.edu> writes:

> No, but he doesn't seem to like anything about it that he couldn't get
> elsewhere, along with other features that he would like better.

The fact that he may complain about features X, Y, and Z does not mean
that he doesn't like A, B, and C.

> But do you spend most of your posts complaining about debian on
> debian...@l.d.o? Not that I've seen. Note that Erann hasn't made
> any suggestions on how to accomodate symbol namespacing without CL
> packages yet, and neither have you.

I can't speak for Erann. I've already said several times that, in the
general framework of Common Lisp, I think that Common Lisp packages
are the best way I know of to get what someone would want from them.

But that doesn't mean that they always do just what I wish; there are
things about them that I wish were different, but seem essentially
required by the way they have to work to get the other things that are
important, and are well done. This is, I suspect, why Guy Steele also
thought they were a little wartish.

> Then you misunderstand that name clashes are a fatal situation in
> symbolic processing. It's like doing side-effects while ignoring
> structure-sharing.

You are confusing one implementation of symbolic processing with the
very concept of symbolic processing.

Erik Naggum

unread,
Mar 17, 2002, 7:21:13 AM3/17/02
to
* Erann Gat

| That's true. I am horribly confused. Whether that's my fault or the
| fault of the people trying to explain things to me is unclear.

It is your fault. Confusion can never be blamed on anyone else. If you
think it can, you are horribly confused.

* Erik Naggum


> Such are the results of incompetence at work.

* Erann Gat


| One could respond the same way to every frustration anyone has ever had
| with any programming language.

Well, there is a crucial distinction between incompetence at the designer
end of the language and incomptence at the user end of the language. You
seem to conclude thet there can be incompetence only at one end, despite
your deriding Common Lisp "flaws". I wonder how you reconcile this with
yourself, but you probably do not bother.

Erik Naggum

unread,
Mar 17, 2002, 8:27:03 AM3/17/02
to
* Thomas Bushnell, BSG

| So he must like everything about it, or he's against it?

It is the other way around. Only those who are against it, express their
dislike of it. Those who are for it, think in a very different way: "It
would be even better if ...". The difference between dislike and like
less than some hypothetical situation can seem hard to grasp, but think
of a line, with zero in the middle. Dislike is on the left of the zero.
Like is on the right of the zero. "Like less than some hypothetical"
ought to have both on the right of the zero, and that is certainly the
case for "it would be even better if ...". But in effect, yes, one's
opinion of all features of something one has chosen to work with and
wishes to become proficient and productive in should all be on the
positive side. That does mean that one has to defend them all or that
one does not prefer one way over another, perhaps never using a feature,
but active dislike is so psychologically destructive that you have people
constantly complain about it instead of just getting over it and on with
their lives. You and Erann Gat keep bad-mouthing Common Lisp all the
time. To those who wish to become proficient and productive in Common
Lisp, this is actually really destructive. It is like going to a bar
with your friends and haveing to suffer someone who berates something or
other about your girlfriend or wife or kids or cat or perhaps your car or
your job or your house or whatever _all_ the time. This pisses most
people off _tremendously_, to the point where people actually resort to
violence to make the shitheads shut up or leave. For a community like
ours, it is like having someone come to a bar regularly only to whine
about the beer or the hygiene or the furniture or the decorations or the
bartender or whatever. That someone would be beaten up if he refused to
leave the others in peace to enjoy themselves. Common decency strongly
suggests that such naysayers and shitheads just get the hell out of the
place, but for some reason, shitheads like you and Erann Gat keep coming
to the same bar and keep deriding and denigrating something that a lot of
the people there actually come there _because_ they like. Now, to keep
this metaphor, everybody will appreciate a suggestion to make something
better if they perceive it to be in good will, i.e., with respect for
what has already been done, but if your suggestion is not adopted, it is
very _rude_ to keep suggesting it over and over (no matter how politely
it has been suggested) To change the scene a little: If someone should
argue that your cat would look a lot better with short hair, the only
appropriate answer is that then it would have to be a different cat. To
really make you destructive shitheads realize what you are doing, suppose
you talked to a black guy in the highly racially sensitized United States
and told him that he would probably be a much better person if he were
white. That probably overstates the case a little, but having you and
Erann Gat and shitheads like you around here to rag Common Lisp all the
time is extremely unwelcome. And the message I keep trying to send to
you obnoxious assholes is just that: Get the fuck out of here if all you
come here to do is deride and berate and denigrate the _one_ common
interest to the people in this forum. People will generally listen to
your problems and try to help you if you have a constructive purpose to
some frustration or other, but if all you want to do is complain that
what you want to do is not perfectly acceptable and you do not _want_ a
solution, people _will_ tire of you, some more quickly than others.

Erik Naggum

unread,
Mar 17, 2002, 8:37:03 AM3/17/02
to
* Erann Gat

| I feel fairly confident that I'm sufficiently non-idiotic that if someone
| can't explain something in this area so that I can understand it there's
| a good chance that it's their fault and not mine.

This arrogance is probably the reason why you never _listen_ to anyone.
There is probably no way to make you revert to a mental state where you
realize that you know less than those who try to tell you something.

You have established that you will only listen to very particular points
that people raise, and not listen to their "philosophical" answers. You
even ridicule me for trying to answer your philosophical _question_. In
order for you to stop being such a prick, you have to snap out of the "I
know all I need to know, thank you" attitude. Your confusion comes from
thinking you know something you do not in fact know. The only possible
solution for you is to stop thinking you know, but judging from how you
respond to every suggestion I make, you will take this to mean that you
should be humiliated and ridiculed because that is what happens when you
flout your arrogance. However, if you could start to think and listen,
and actually try to _learn_ something from what people tell you, instead
of going on your fucking annoying fault-finding missions _all_ the time,
you would be unconfused in no time.

I would actually like to help you and any other victim of confusion, but
when you are so dead certain that you know everything you need to know
and can _arbitrarily_ decide that "I don't get it, and it's his fault",
there is no incentive to try to explain anything to you at all. Only the
people who might read your incessant whining are worth talking to.

Erik Naggum

unread,
Mar 17, 2002, 8:46:37 AM3/17/02
to
* "Joe Marshall" <prunes...@attbi.com>

| In the years I have been hacking lisp, I have on many occasions cursed
| the package system, mainly for making me restart my lisp to undo whatever
| random damage happened when I read something in the wrong package. On
| the other hand, I have *never once* had the occasion to say `Wow, damn
| glad that package system saved my ass on *this*!'

How could it? You have obviously never actually set it up so it _could_
have saved your ass.

But what I keep wondering, however, is why "save my ass" is a criterion
for the value of something. I have heard a few people argue that if they
are bumbling fools and resoundingly incompetent, they are "saved" by some
feature or another, and this is somehow valuable. I think the opposite:
I think if people are bumbling fools and resoundingly incompetent, they
should crash and burn, the sooner the better, because saving them does
only one thing: it makes it possible for them to continue to be bumbling
fools and resoundingly incompetent.

Have you ever heard anyone say "Man, I am so glad I have studied
mathematics and acquired a mathematical way of thinking! That has
_really_ saved my ass!"? If not, let us tear down the mathematics
department at every university and burn the math books!

How would you rate a medical doctor who exclaimed "I'm really happy I
know the spleen. That really saved my ass!". I would rate him as
dangerously incompetent, just as I rate your comment.

But you are a Scheme freak, are you not?

Samuele Pedroni

unread,
Mar 17, 2002, 8:53:14 AM3/17/02
to

Erann Gat <g...@jpl.nasa.gov> wrote in message
gat-160302...@192.168.1.50...

>
> Suppose CL had no symbols. It seems I could easily add them by defining
> two classes: "symbol" with slots name, value, plist, etc. and "package"
> that's really just a hash table that maps strings onto instances of the
> symbol class. Probably 20-30 LOC at most. Would that be sufficient? Or
> would I still be missing some essential capability? What would it be?
>
> Maybe the problem is that I don't really understand what people mean by
> "symbolic processing". A concrete non-contrived example of "symbolic
> processing" that's easy in CL and hard in Scheme would probably help.
>

A question: what is for you the relation between "L has X",
and "X can be implemented (with effort, without effort) in L"?


Tim Bradshaw

unread,
Mar 17, 2002, 8:41:25 AM3/17/02
to
* Thomas Bushnell wrote:

> I can't speak for Erann. I've already said several times that, in the
> general framework of Common Lisp, I think that Common Lisp packages
> are the best way I know of to get what someone would want from them.

Well, as someone who seems to have been occasionally tarred with the
`thinks CL is perfect' brush, I'll say that I think this is not so. I
think the CL package system is really not that great. *However* I
don't have any coherent suggestions about how it could be made
compatibly better, so I don't tend to complain too vociferously about
it.

Here are some of the things I'd like to see be different:

Locks on packages (how to do them)?

User-defined external-only packages and automatic-self-binding
packages (like KEYWORD).

Package inheritance (like my conduits system, except better thought
out). This can be done as a user, but it really needs system-level
support to make sure all the errors that should be signaled really
are.

Hierarchical names for packages and some notion of a `current package
directory' and per-directory nicknames. If I'm in the `COM.CLEY'
context I want to be able to say that, as well as WELD refering to
COM.CLEY.WELD, NET refers to ORG.ALU.NET-COMPATIBILITY or something.

User-controlled interning. I want to be able to step in at the point
where the reader has got a string which is a symbol candidate and tell
it what to do. This makes it much easier to use READ in `unsafe'
contexts without interning a mass of symbols that you can't later
find.

The only thing I've even thought about hard is the third of these: I
think that, even if this is an uncontentious set of things people
would like (unlikely) that there is a lot of fairly serious design
work here (not to mention looking at existing implementations).

--tim


Kent M Pitman

unread,
Mar 17, 2002, 2:17:38 PM3/17/02
to
"Joe Marshall" <prunes...@attbi.com> writes:

> In the years I have been hacking lisp, I have on many occasions cursed
> the package system, mainly for making me restart my lisp to undo whatever
> random damage happened when I read something in the wrong package.
> On the other hand, I have *never once* had the occasion to say
> `Wow, damn glad that package system saved my ass on *this*!'

I have. More times than I can count.

And, in general, I have a meta-rule (as in "rule of thumb", not "rule of law"):

If person X perceives a problem and person Y doesn't perceive a problem,
then unless you have reason to believe person X to be clinically nuts,
if you know no other piece of information about the problem, then person
X is probably right and person Y is probably wrong.

For example:

(a) Mary complains that Joe sexually harrassed her.
Sally says, "gosh, Joe never sexually harrassed me."
Has Sally refuted Mary? No, Sally has said "I wasn't present
and have no information."

(b) If Fred says "The thing I like about Kathy is that she always praises
people for the good work they do." and Joe says "I've never heard
her do this for me." and Fred says "I've heard her praise you."
Is it more likely that (1) Kathy said it and Joe didn't hear or
(2) Fred is a pathological liar and just made this up to irritate
or confuse Joe. Obviously either are possible but I tend to defaultly
assume (1) unless shown strong evidence of (2).

So given the choice between thinking I'm looney for liking packages or
you're oblivious for not understanding the value that packages offer you,
I'm reluctantly pressed into the bad position of suggesting you may be
creating a situation where to continue productively in the conversation,
I have to call you oblivious. It's not my first choice to approach a
discussion by addressing a speaker's personal characteristics since
ordinarily that is an ad hominem, but in this particular case you have
created what I guess I'll call a "passive ad hominem" (in honor of the
notion of "passive aggressive") by observing a personal characteristic
in yourself and then citing it as a reference for a technical point. There
is really no other way of proceeding according to proper rules of procedure
in an evidentiary hearing than to examine the credibility of the witness.

And besides, the package system is _designed_ not to overwhelm you with
the sense that it is there at all, so you're not _supposed_ to notice
when it saves you. Its whole purpose in life is to give you the "I own
the whole universe feel." If we didn't want to do this, we'd have just
made whose actual name was TV:FOO (6 letters) rather than omittable
package names because then sharing would always be explicit. The whole
notion of omitting the package is intended to draw attention away from
notions of sharing, even though those notions of sharing are often
absolutely critical to the correct function of your application. It's
not that you're not supposed to know; it's only that you're not supposed
to focus on that at every instant.

Joe Marshall

unread,
Mar 17, 2002, 2:19:43 PM3/17/02
to

"Erik Naggum" <er...@naggum.net> wrote in message
news:32253616...@naggum.net...

> * "Joe Marshall" <prunes...@attbi.com>
> | In the years I have been hacking lisp, I have on many occasions cursed
> | the package system, mainly for making me restart my lisp to undo
whatever
> | random damage happened when I read something in the wrong package. On
> | the other hand, I have *never once* had the occasion to say `Wow, damn
> | glad that package system saved my ass on *this*!'
>
> How could it? You have obviously never actually set it up so it _could_
> have saved your ass.

I don't really know *how* to set it up so that it could save my ass.
This isn't to say that I don't how the package system works, it is that
I think the particular tools that the package system provides are
rather crude, and although they can be used as a servicable namespace
partitioning, they quickly reveal limitations.

> But what I keep wondering, however, is why "save my ass" is a criterion
> for the value of something. I have heard a few people argue that if
they
> are bumbling fools and resoundingly incompetent, they are "saved" by
some
> feature or another, and this is somehow valuable. I think the opposite:
> I think if people are bumbling fools and resoundingly incompetent, they
> should crash and burn, the sooner the better, because saving them does
> only one thing: it makes it possible for them to continue to be bumbling
> fools and resoundingly incompetent.
>
> Have you ever heard anyone say "Man, I am so glad I have studied
> mathematics and acquired a mathematical way of thinking! That has
> _really_ saved my ass!"?

I have occasionally thought that myself.

> If not, let us tear down the mathematics
> department at every university and burn the math books!

I wouldn't suggest an all-or-nothing solution. I happen to dislike the
package system, but that doesn't mean that I'm going to abandon Lisp
and embrace Visual Basic.

> How would you rate a medical doctor who exclaimed "I'm really happy I
> know the spleen. That really saved my ass!". I would rate him as
> dangerously incompetent, just as I rate your comment.

I think I would prefer him over a medical doctor that said
``Oh. What's this organ then?''

But perhaps I should have been less colloquial. What I was implying
is that the package system has caused me an inordinate amount of
difficulty compared to the problem it is designed to solve. `Saving
my ass' should be read as hyperbole, not as criterion.

> But you are a Scheme freak, are you not?

I am tempted to quote scripture: It is you who say that I am.
I like Scheme, and enjoy programming in it. On the other hand, I like
Common Lisp, too, and have written orders of magnitude more code
in Common Lisp than I have in Scheme. I feel comfortable with either
language.

Joe Marshall

unread,
Mar 17, 2002, 2:28:08 PM3/17/02
to

"Kent M Pitman" <pit...@world.std.com> wrote in message
news:sfwd6y3...@shell01.TheWorld.com...

> "Joe Marshall" <prunes...@attbi.com> writes:
>
> > In the years I have been hacking lisp, I have on many occasions cursed
> > the package system, mainly for making me restart my lisp to undo
whatever
> > random damage happened when I read something in the wrong package.
> > On the other hand, I have *never once* had the occasion to say
> > `Wow, damn glad that package system saved my ass on *this*!'
>
> I have. More times than I can count.
>
> So given the choice between thinking I'm looney for liking packages or
> you're oblivious for not understanding the value that packages offer you,
> I'm reluctantly pressed into the bad position of suggesting you may be
> creating a situation where to continue productively in the conversation,
> I have to call you oblivious. It's not my first choice to approach a
> discussion by addressing a speaker's personal characteristics since
> ordinarily that is an ad hominem, but in this particular case you have
> created what I guess I'll call a "passive ad hominem" (in honor of the
> notion of "passive aggressive") by observing a personal characteristic
> in yourself and then citing it as a reference for a technical point.
There
> is really no other way of proceeding according to proper rules of
procedure
> in an evidentiary hearing than to examine the credibility of the witness.

Oh, I don't mind being called oblivious. Erann Gat solicited opinions
and I gave what I explicitly said was my `uninformed opinion'.

> And besides, the package system is _designed_ not to overwhelm you with
> the sense that it is there at all, so you're not _supposed_ to notice
> when it saves you. Its whole purpose in life is to give you the "I own
> the whole universe feel."

I understand this. However, I have found that the package system does
a poor job of this. If I *never* noticed it, that would be fine, but
I notice it far more frequently than I'd like, and when I *do* notice it,
I find that it can be difficult to get it back to a useful state
without rebooting.

If you would like, I can give you several technical examples where the
package system has gotten in the way.

Joe Marshall

unread,
Mar 17, 2002, 1:49:33 PM3/17/02
to

"Nils Goesche" <n...@cartan.de> wrote in message
news:87henf2...@darkstar.cartan...

> "Joe Marshall" <prunes...@attbi.com> writes:
> >
> > In the years I have been hacking lisp, I have on many occasions cursed
> > the package system, mainly for making me restart my lisp to undo
whatever
> > random damage happened when I read something in the wrong package.
> > On the other hand, I have *never once* had the occasion to say
> > `Wow, damn glad that package system saved my ass on *this*!'
>
> It is hard to tell if you are serious on this but do you actually
> mean that?

Yes, in this case I am not being facetious.

> Are you aware of the fact that it might be a benefit
> of the package system that you had no reason so far to complain
> when you used different packages from different parties for your
> own projects?

Obviously there is a need for *some* mechanism to allow people
to develop code independently without them having to worry
about what names other people might want to use in their code.
However, I think the package system is the wrong way to go about
this.

Kent M Pitman

unread,
Mar 17, 2002, 3:09:42 PM3/17/02
to
g...@jpl.nasa.gov (Erann Gat) writes:

> In article <sfwg030...@shell01.TheWorld.com>, Kent M Pitman
> <pit...@world.std.com> wrote:
>
> > Being one of the central parties making this noise, I'll do
> > something I'm not big on and actually cite my credentials to show
> > that I am not blathering out of ignorance:
>
> I won't follow your example because I don't know if I'm blathering out of
> ignorance or not. (That's what I'm trying to figure out.) But if you
> want to see my credentials my CV is on the Web.

I'm not questioning your credentials. I thought you had asked effectively
for mine since your question seemed to ask "is it just the case that the
other side is speaking out of ignorance". Anyone can chime in and tell me
if my credentials seem insufficient--I'm not going to do like VeriSign and
offer a self-validating credential where I both assert credentials and tell
you that you can trust them--but I'm comfortable with them for myself and
willing to have them open to inspection. I _think_ I know what the other
options are. I _wish_ I could feel comfortable using the other options,
but I have repeatedly tried and found both Lisp1 and module-based solutions
inadequate to my personal needs.

> Suffice it to say I've
> built a successful career in part by writing Common Lisp code.

No quibble with your credentials here. I'm not suggesting you are not
entitled to engage in this conversation in any way. I'm just suggesting
you're wrong in point of fact if you think there is no valid defense of a
package system. Packages and modules are orthogonal structures and both
have a place. I'm not anti-module, I'm anti-module-qua-panacea.

> I feel fairly confident that I'm sufficiently non-idiotic that if
> someone can't explain something in this area so that I can
> understand it there's a good chance that it's their fault and not
> mine.

I use this same rule of thumb myself, so I can relate. Then again, I
think you should not conclude from someone's inability to articulate a
natural langauge rule to explain a philosophically complex behavior
that they do not have deep-seated and real beliefs, nor even perhaps
well-founded beliefs. Sometimes people understand but cannot
describe. Sometimes adequate description requires the person
receiving the description to 'fess up enough data about their
confusions that a custom explanation can be rendered adequate to their
needs. I have no doubt you are trying. I have neither any doubt that
my belief in packages is technically, not just emotionally or
historically, well-founded. I'm happy to engage in this discussion
with you on that basis. I'm even happy for you to conclude "Because
Kent and others cannot say what I need to hear, I will continue to
believe what I believe." (That is, I'm happy to live with the belief
that you are wrong. Heh... Which incidentally is on a statement that
you are wrong, it is just a statement about my belief structure.) I'm
not happy for you to assert (and I hope you are not asserting, I'm
just checking) that "The inability of various people to articulate a
defense is a de facto proof that no such defense exists." or that kind
of thing.

> > > And if you were, what is it that makes you prefer packages?
> >
> > Do you want me to repeat the stuff I mentioned under the data
> > hygiene thread?
>
> No, but I would like you to expand on something:
>
> > I cannot later 'relate', at the symbol
> > level, your person-1 with my person-1, without doing a global renaming.
>
> What does that mean? Can you restate this (which seems to be the crucial
> point) without scare quotes and without squishy phrases like "at the
> symbol level"?

I'll be as concrete as I can. I'm trying to be brief, too, so I may have
to add detail later if I've left something out, but here's a sketch of a
sample scenario I intended to address. Examples have the virtue of having
detail that abstract discussions do not. Re-generalize anything you see here
in concrete form back to the abstract when done; my point is not to say
"Here is the problem that worries me and if you fix it the world is well."
My point is rather to say, "My belief that there is a problem would lead
me to hypothesize a specific set of concrete and observable design problems.
Here is an example of one." Further, you should understand that when I say
"design problem" I don't mean "There is no way to compute an answer to this
problem i another system." since we are not talking about computability.
The Turing Equivalence of Scheme and CL is not in question. Rather, we are
talking about the use of a specific paradigm to express a particular problem;
so if in "solving" the technical problem, you switch paradigms, then you have
made my case. The intent of this examples is specifically to show a use of
symbols, so saying "we can substitute non-symbols to solve this" make my case,
it does not refute it... Ok, enough prelude. Here we go...

CL allows me, simply in the symbol data, to control whether I'm doing various
kinds of sharing. For example, I can separate my spaces, as in:

(in-package "FLOWER")
(setf (get 'rose 'color) 'red)
(setf (get 'daisy 'color) 'white)

(defun colors-among (things)
(loop for thing in things
when (get thing 'color)
collect thing))

(in-package "PEOPLE")
(setf (get 'rose 'color) 'white)
(setf (get 'daisy 'color) 'black)

(defun colors-among (things)
(loop for thing in things
when (get thing 'color)
collect thing))

(in-package "OTHER")

(flower:colors-among '(flower:rose people:rose flower:daisy))
=> (FLOWER:RED FLOWER:WHITE)

(flower:colors-among '(flower:rose people:daisy people:rose))
=> (FLOWER:RED)

(people:colors-among '(flower:rose people:rose flower:daisy))
=> (PEOPLE:WHITE)

(people:colors-among '(flower:rose people:daisy people:rose))
=> (PEOPLE:BLACK PEOPLE:WHITE)

Some examples of the value here:

- The information is manifest in symbols, a primitive data structure
in the language
- Both the objects and the properties can be named by symbols,
which are easy to type.
- When debugging either the PEOPLE or FLOWER package, you don't worry
about symbols and just type short names.
- In OTHER contexts, when you want to mix people and flowers in the
same list, it's easy to do without perturbing already chosen symbolic
data structures and without yielding confusion.

> > The short answer is that non-symbols are fine, but they do not a symbolic
> > processing language make. Some days I don't mind passing around Scheme's
> > symbols (with their characteristic "no associated meaning") + an environment
> > (to give the symbols meaning). Other days, I like the power of being able
> > to have symbols like in CL, where the symbol means something all by itself.
> > (Most days, in fact.) Passing non-symbols is fine computation, but is not
> > the essence of symbolic computation.
>
> Suppose CL had no symbols. It seems I could easily add them by defining
> two classes: "symbol" with slots name, value, plist, etc. and "package"
> that's really just a hash table that maps strings onto instances of the
> symbol class. Probably 20-30 LOC at most. Would that be sufficient?

No. The critical value is the ability to TYPE the symbol without
doublequotes or other impediment in most cases. Yes, you have to write :
but only in the case where an ambiguity is possible to resolve that
ambiguity. In Scheme, you also don't write : in the unambiguous cases but
you have no notation to fall back on to resolve the ambiguous cases.

You could create a notation wherein for Scheme doing TV:FOO would refer to
(GET-FROM-ENVIRONMENT 'FOO TV). And I vaguely recall in T (Yale Scheme)
we had talked about this notation: #>foo>bar>baz to mean
(GET-FROM-ENVIRONMENT 'BAZ
(GET-FROM-ENVIRONMENT 'BAR
(GET-FROM-ENVIRONMENT 'FOO
(GET-FROM-ENVIRONMENT ROOT-ENVIRONMENT))))
though I don't recall the name of the operator for GET-FROM-ENVIRONMENT.
It probably had "LOCALE" in its name. Anyway, though, the point is that
what this syntax would resolve to is not a symbol, nor is the data that it
would retrieve. This is a computationally ok answer, but it is a refutation
of any claim that the language is using "symbol" processing at that point.
Symbols are used incidentally in here (since they are one of the arguments
to some of the functions involved, but saying that means it's symbol processing
is like saying that numerical processing is what's going on in:
(defun find-longest-name (array-of-symbols)
(let ((longest (aref array-of-symbols 0))) ;!!! assumes at least one elt
(loop for i from 1 below (length array-of-symbols)
for contender = (aref array-of-symbols i)
when (> (length (string longest))
(length (string contender)))
do (setf longest contender))
longest))
Yes, numbers are involved, but that's not the essence of
"numerical processing". I think of numerical processing as something where
the core data are encoded as numbers. I think of symbolic processing as
something where the core data are encoded as symbols.

> Or
> would I still be missing some essential capability? What would it be?

You'd have a symbol datatype but not a serviceable one for me.
Even ignoring lack of packages, this is the reason I can't stand Java
symbols--the syntax is too cumbersome.

FOO is a symbol. foo is a symbol. I don't mind typing 15\r or |15r|
to preserve case. I don't mind typing tv:foo if I'm not in package TV
to get to that symbol. But basically I want unimpeded access to the name
as part of my syntax for literals if it's going to be what I think of as
a symbol. I don't mind quoting symbols. In Macsyma, I could do
X:10;
F(X,Y):=X+Y;
F(X+3,'X+3);
and get back X+16 which contains one symbol and one number. It doesn't
matter to me what the surrounding syntax is when talking about whether it's
symbol processing. I consider infix/prefix to be orthogonal, and lousy
though I find dylan syntax to be I can't see why they couldn't have
made the above kind of syntax work for them.

> Maybe the problem is that I don't really understand what people mean by
> "symbolic processing". A concrete non-contrived example of "symbolic
> processing" that's easy in CL and hard in Scheme would probably help.

The above programs are contrived, but I hope show enough meat that you
can see the problem. Try casting them into Scheme in whatever way suits you
and we can proceed to the next conversational step from there.

Kent M Pitman

unread,
Mar 17, 2002, 3:42:31 PM3/17/02
to
Rahul Jain <rj...@sid-1129.sid.rice.edu> writes:

> tb+u...@becket.net (Thomas Bushnell, BSG) writes:
>
> > However, I'm not interested in having that conversation with you.
> > (Kent, if you find it worth discussing, then I am interested in
> > having that conversation with you.)

Thomas, I'm enjoying chatting with _both_ of you guys, first of all.

But, just fyi, a quick scan of the last 100 messages shows that you
have replied to nearly every single message of the 20 or so Rahul has
sent in the last day. If you want not to engage him in conversation,
the conventional tool is not to hit "r" when viewing one of his
posts. ;)



> I agree that Kent is much more knowledgable than I am. I don't see how
> that makes my input useless, but I guess that's your personal bias.

I don't personally like mustard or olives or horseradish. That doesn't
make those things without worth. Someone is still buying them from the
grocery store even when I don't.

Worth is a complex commodity and people spend too much time here
debating the worth of others as if any finite number of people were
even in a position to judge this, or as if having judged it there were
any materially interesting action that could then be taken that was
justified by such a judgment. I recommend simply ignoring those you
don't find of worth. Others may still find them of worth, or not, but
the outcome will likely be the best.

Conversations are usefully made of people of all skill levels. Even
people with no skill come with intuitions and these are often
conversationally useful among people who engage in respectful conversation.

Incidentally, Thomas, my entire rant on "data hygiene" was a _direct_
response to something Rahul said. Had he not posted his remarks, I
wouldn't have had occasion to sit for a while pondering that answer,
which I'm quite happy to have penned. I'm not saying Rahul's posts
have no content value, but even if they accomplished only catalyzing
the speech of others, I would not discount that as conversational
value. We each offer value in our own way and at our own time.

Further, I think a much neglected value metric in this forum is "what it
takes to keep the 'interesting' people here". Everyone has their notion
of who's interesting and who's not, so I'm not trying to single anyone
out, but I often feel as if people think that they want the newsgroup
to have only the interesting people and to have the others stand down.
But sometimes the interesting people are here only because of others who
may be disliked, or cause a stir, or be otherwise problematic. In fact,
this is an ecology, and I urge everyone not to think that killing the
mosquitos (whoever you think those people might be) will have no effect
on the food chain. Just bite into the food you find tasty and avoid
the things that taste bad or sting and be happy you live in the richness
of the jungle instead of the arctic where few things grow ...

Kent M Pitman

unread,
Mar 17, 2002, 4:17:27 PM3/17/02
to
"Joe Marshall" <prunes...@attbi.com> writes:

> If you would like, I can give you several technical examples where the
> package system has gotten in the way.

I'm sure there are examples of places where where modern medicine gets in
the way, too. But that isn't a refutation of its many benefits.

Nevertheless, since I'm curious what in the world you're talking about,
yeah, I'd be interested to hear...

Rahul Jain

unread,
Mar 17, 2002, 5:36:24 PM3/17/02
to
tb+u...@becket.net (Thomas Bushnell, BSG) writes:

> Rahul Jain <rj...@sid-1129.sid.rice.edu> writes:

> > No, but he doesn't seem to like anything about it that he couldn't get
> > elsewhere, along with other features that he would like better.

> The fact that he may complain about features X, Y, and Z does not mean
> that he doesn't like A, B, and C.

He can get A, B, and C without X, Y, and Z from a different community,
from what I've seen about A, B, C, X, Y, and Z.

> > Then you misunderstand that name clashes are a fatal situation in
> > symbolic processing. It's like doing side-effects while ignoring
> > structure-sharing.

> You are confusing one implementation of symbolic processing with the
> very concept of symbolic processing.

The concept of avoiding name clashes is an implementation of symbolic
processing?

You are confusing constructive criticism with whining. Now figure out
what you really want from symbol namespaces and explain how to get
that or accept them as they are and either use them or don't.

Rahul Jain

unread,
Mar 17, 2002, 5:46:22 PM3/17/02
to
Tim Bradshaw <t...@cley.com> writes:

> Well, as someone who seems to have been occasionally tarred with the
> `thinks CL is perfect' brush, I'll say that I think this is not so.

CL is by no means perfect, it's just a good compromise for the
features that the Lisp community wanted at the time.

> I think the CL package system is really not that great.

I think it's more than good enough for what it does. Most of what you
talk about below are layers on top of packages or extra features in
other systems which use packages and symbols.

> *However* I don't have any coherent suggestions about how it could
> be made compatibly better, so I don't tend to complain too
> vociferously about it.

Yes, that's what I'm trying to explain to Erann and Thomas. They're
merely complaining about a feature without providing suggestions for
improvement.

> Here are some of the things I'd like to see be different:

> Locks on packages (how to do them)?

ACL has an implementation of this, no? I think it's time to
standardize on an API for it. With more and more Lisp applications out
there, we'll need more and more to ensure that they don't accidentally
do nasty things to symbols they don't own.

> User-defined external-only packages and automatic-self-binding
> packages (like KEYWORD).

That's a good idea. I could especially use the external-only
packages. Not needed, but a nice convenience. Hmm, but there are
issues with local variables used in the defintions of symbols in that
package. I'm not exactly sure how to effectively handle that
situation.

> Hierarchical names for packages and some notion of a `current package
> directory' and per-directory nicknames. If I'm in the `COM.CLEY'
> context I want to be able to say that, as well as WELD refering to
> COM.CLEY.WELD, NET refers to ORG.ALU.NET-COMPATIBILITY or something.

ACL and CMUCL implement the same API for hierarchial names, but I
don't think they have per-directory nicknames. That would be a nice
addition.

> User-controlled interning. I want to be able to step in at the point
> where the reader has got a string which is a symbol candidate and tell
> it what to do. This makes it much easier to use READ in `unsafe'
> contexts without interning a mass of symbols that you can't later
> find.

This is one feature of my generic-reader which is a subproject of
DefDoc. I designed it both so that I can check package access controls
as well as read normal text. All the usual caveats about time apply.

Rahul Jain

unread,
Mar 17, 2002, 5:52:27 PM3/17/02
to
"Joe Marshall" <prunes...@attbi.com> writes:

> What I was implying is that the package system has caused me an
> inordinate amount of difficulty compared to the problem it is
> designed to solve.

So how would you design a namespacing system for symbols which would
cause less difficulty?

Joe Marshall

unread,
Mar 17, 2002, 6:14:58 PM3/17/02
to

"Rahul Jain" <rj...@sid-1129.sid.rice.edu> wrote in message
news:87u1re2...@photino.sid.rice.edu...

> "Joe Marshall" <prunes...@attbi.com> writes:
>
> > What I was implying is that the package system has caused me an
> > inordinate amount of difficulty compared to the problem it is
> > designed to solve.
>
> So how would you design a namespacing system for symbols which would
> cause less difficulty?

I'd start by researching the state of the art vis-a-vis module systems....

But seriously, I think a lot of the issues I have with the package
system could be ameliorated if 1. read didn't cause visible side effects
and 2. symbol resolution was deferred until runtime

Joe Marshall

unread,
Mar 17, 2002, 5:42:18 PM3/17/02
to

"Kent M Pitman" <pit...@world.std.com> wrote in message
news:sfwy9gq...@shell01.TheWorld.com...

READ causes noticable side effects. This can create dependencies
on the order in which things are read. In the `normal' case of
reading a file in order to compile it, (or the analagous case of
interning symbols in a fasl file), there are other dependencies
that also have to be obeyed (like macros being defined before
use). So this isn't so much of a day-to-day problem.

Where it *was* a problem was on the Lisp Machine. When you opened
a file in Zmacs, it would parse the file mode and it would
parse some of the file contents. It would attach properties to
the symbols mentioned in the file.

When you were browsing code, you might open a file that was
in a package that was not yet defined. Well, the package became
defined and a slew of symbols from the file would get interned
in it. Unfortunately, it was unlikely that the package was
correctly set up, so when you went to evaluate any code it
wouldn't work, or when you went to fix the packages, you'd run
into thousands of name conflicts.

Additionally, if you accidentally fasloaded a file before the
packages were set up, you would end up with functions pointing
at the wrong symbols. Patching up the packages by uninterning
the conflicting symbols didn't fix this, it just made the functions
point at UNINTERNED wrong symbols.

Now there is a `standard' mode of operating that avoids most of
this. You make *absolutely sure* that the *very first thing* you
load into any lisp is a set of defpackage forms that set up the
*entire* package structure. You also make sure that this file
is loaded both at compile time *and* at run time. This isn't hard
to do if you put it in your `init' file.

So, on to more obscure problems. You can't use `forward references'
to symbols that don't exist in packages that don't exist (yet).
In your init file, you'd *like* to be able to customize some
parameters before you start building the system. Perhaps you have
some macros that are parameterized, whatever. As an example, I
once worked at a place that had a `verbosity' switch that I wanted
set.

The obvious thing is to write (setq utility::*verbosity* t)
but the utility package isn't loaded yet. Never mind that the
utility package *will* be there when I call this function, it
isn't there when I *read* the code. Instead I write:
(let* ((utl-pkg (find-package :utility))
(verbosity (intern (symbol-name :*verbosity*) utl-pkg)))
(setf (symbol-value verbosity) t))

The same goes for functions that I want to call. I can't
write (progn (load "java-tools.lisp") (java-tools:make-all))

The way around this problem is to write a function that
defers interning until runtime. Then you can write
(progn (load "java-tools.lisp") (funcall-in-package :java-tools :make-all))

The package system allows me to `split' names. I can have
my symbol FOO be different from your symbol FOO. It doesn't
allow me to `join' names. I cannot have my symbol BAR be
the same as your symbol FOO. (Or perhaps more illustrative,
I can't have `JRM:ORIGINAL-MULTIPLE-VALUE-BIND' be the same
as `CL-USER:MULTIPLE-VALUE-BIND'.) On the lisp machine, you
actually *could* do this by forwarding the memory, and it
comes in handy on occasion.

Another example: when writing a tiny language, it is nice to
embed that language in Lisp. You can make a package containing
your language constructs, and use macros to `translate' them
into standard lisp for compilation. However, CL:IF is a special
form `known' to the compiler. FOO:IF is not. The compiler will
not generate a conditional unless you use the symbol in the CL
package. On the other hand, the point of putting the symbols
into your own package was to *not* use the same symbols.

Finally, a really nasty one. The `topology' of the package
structure is a global resource. When we were building a new
Lisp machine, we had a brand new compiler. Obviously, it goes
in the `compiler' package. Well, on the old machine, there
already was a compiler in the compiler package. So we had
to load the compiler into a different package. But the
compiler had to `appear' as if it were in the `compiler'
package in the fasl file. We modified the package inheritance
in the new machine. A number of symbols were moved to
different packages, but were to remain visible. The problem
was that the current machine wouldn't work if you did this.
Our `solution' was a rather hairy thing that twiddled the
guts of the package system on the fly to satisify whatever
compiler happened to want to be using it. It was a nightmare.

Now I realize that name conficts are a problem (look at emacs).
But I don't think the solution is to hack the READER. With
a little careful thought, I'm sure that a better way could be
found that defers identifier->symbol mapping until later in
the game.

Rahul Jain

unread,
Mar 17, 2002, 6:57:41 PM3/17/02
to
"Joe Marshall" <prunes...@attbi.com> writes:

> "Rahul Jain" <rj...@sid-1129.sid.rice.edu> wrote in message
> news:87u1re2...@photino.sid.rice.edu...

> > So how would you design a namespacing system for symbols which would


> > cause less difficulty?
>
> I'd start by researching the state of the art vis-a-vis module systems....

Module systems (according to any definition I've seen) aren't
namespaces for symbols, but first-class environments, where symbols
have local mappings to values.

> But seriously, I think a lot of the issues I have with the package
> system could be ameliorated if 1. read didn't cause visible side effects
> and 2. symbol resolution was deferred until runtime

That wouldn't make sense. How would I use a symbol if it didn't exist
when I was trying to use it?

Thomas Bushnell, BSG

unread,
Mar 17, 2002, 7:06:27 PM3/17/02
to
Rahul Jain <rj...@sid-1129.sid.rice.edu> writes:

> Yes, that's what I'm trying to explain to Erann and Thomas. They're
> merely complaining about a feature without providing suggestions for
> improvement.

I guess I have to say it again:

I don't have any complaints about the Common Lisp package system.
It's not perfect, but I can't see any better way to solve the problems
that it does solve for Common Lisp. And, from what I understand, that
was Guy Steele's feelings about it too.

Thomas

Joe Marshall

unread,
Mar 17, 2002, 7:11:29 PM3/17/02
to

"Rahul Jain" <rj...@sid-1129.sid.rice.edu> wrote in message
news:87wuwaz...@photino.sid.rice.edu...

> "Joe Marshall" <prunes...@attbi.com> writes:
>
> > "Rahul Jain" <rj...@sid-1129.sid.rice.edu> wrote in message
> > news:87u1re2...@photino.sid.rice.edu...
>
> > > So how would you design a namespacing system for symbols which would
> > > cause less difficulty?
> >
> > I'd start by researching the state of the art vis-a-vis module
systems....
>
> Module systems (according to any definition I've seen) aren't
> namespaces for symbols, but first-class environments, where symbols
> have local mappings to values.
>
> > But seriously, I think a lot of the issues I have with the package
> > system could be ameliorated if 1. read didn't cause visible side
effects
> > and 2. symbol resolution was deferred until runtime
>
> That wouldn't make sense. How would I use a symbol if it didn't exist
> when I was trying to use it?

Depends on the use. The vast majority of uses of symbols is as identifiers
in programs. These symbols simply don't need to be interned.

Rahul Jain

unread,
Mar 17, 2002, 7:16:56 PM3/17/02
to
"Joe Marshall" <prunes...@attbi.com> writes:

> Where it *was* a problem was on the Lisp Machine. When you opened
> a file in Zmacs, it would parse the file mode and it would
> parse some of the file contents. It would attach properties to
> the symbols mentioned in the file.

Yes, I think that's a dangerous way to deal with editing source. The
editor should never use INTERN, only FIND-SYMBOL.

> Additionally, if you accidentally fasloaded a file before the
> packages were set up, you would end up with functions pointing
> at the wrong symbols. Patching up the packages by uninterning
> the conflicting symbols didn't fix this, it just made the functions
> point at UNINTERNED wrong symbols.

Wouldn't the IN-PACKAGE form at the top of the file cause an error,
preventing the file from being loaded? (or asking if you would like to
implicitly create it, etc)

> So, on to more obscure problems. You can't use `forward references'
> to symbols that don't exist in packages that don't exist (yet).

Again, this should cause an error to be signalled, at which point, you
should load the definition of the other package or just implicitly
create it.

> In your init file, you'd *like* to be able to customize some
> parameters before you start building the system.

So you load the defpackage form(s) before playing with symbols in
those packages.

> The package system allows me to `split' names. I can have
> my symbol FOO be different from your symbol FOO. It doesn't
> allow me to `join' names.

Yes, a sort of symbol-alias would be nice. I suppose the symbol could
print as its name in its home package without any real loss.

> Another example: when writing a tiny language, it is nice to
> embed that language in Lisp. You can make a package containing
> your language constructs, and use macros to `translate' them
> into standard lisp for compilation. However, CL:IF is a special
> form `known' to the compiler. FOO:IF is not. The compiler will
> not generate a conditional unless you use the symbol in the CL
> package. On the other hand, the point of putting the symbols
> into your own package was to *not* use the same symbols.

Right, so either define FOO:IF as a macro that expands to CL:IF or
import CL:IF. The package system doesn't help you when you don't tell
it to do what you want it to do.

> Finally, a really nasty one. The `topology' of the package
> structure is a global resource.

Yes, having unnamed packages and dynamic name->package bindings might
be a useful addition, but it's so uncommonly needed that I doubt
people have really thought about how to do it so that the resulting
packages are actually useful. I guess these would be called
"first-class packages".

Hmm, actually, that might be simple to add to my generic-reader.
Simply have the token-interpretation-fn be a closure around some
name->package EQUAL hashtable. Or have a *packages* hashtable that can
be dynamically shadowed. Probably providing both options would be a
good idea.

> Now I realize that name conficts are a problem (look at emacs).
> But I don't think the solution is to hack the READER. With
> a little careful thought, I'm sure that a better way could be
> found that defers identifier->symbol mapping until later in
> the game.

Identifiers in Lisp are symbols. Symbol->value mapping is what occurs
later. Use the right abstraction at the right time. It's just a matter
of learning when each of these conversions take place and choosing the
conversion that matches your desired semantics.

Hartmann Schaffer

unread,
Mar 17, 2002, 7:23:23 PM3/17/02
to
In article <87lmcrs...@photino.sid.rice.edu>,

Rahul Jain <rj...@sid-1129.sid.rice.edu> writes:
> tb+u...@becket.net (Thomas Bushnell, BSG) writes:
>
>> Rahul Jain <rj...@sid-1129.sid.rice.edu> writes:
>> > tb+u...@becket.net (Thomas Bushnell, BSG) writes:
>> > > Rahul Jain <rj...@sid-1129.sid.rice.edu> writes:
>
>> > > Geez! Erann actually *likes* CL, probably more than Scheme, but
>> > > (I'm guessing) doesn't happen to think it is Absolutely Perfect.
>> > > As a result, he perhaps complains about things once in a while.
>
> Once in a while? What else has he done in c.l.l in the entire time
> I've been reading it?

couldn't that be because he thinks ngs are for a little bit more than
mutual administration societies? something like "i like cl, vut have
problems with this feature, so what do others think is good about it
or what alternatives are there"?

> ...

hs

--

don't use malice as an explanation when stupidity suffices

Rahul Jain

unread,
Mar 17, 2002, 7:38:33 PM3/17/02
to
h...@heaven.nirvananet (Hartmann Schaffer) writes:

> In article <87lmcrs...@photino.sid.rice.edu>,
> Rahul Jain <rj...@sid-1129.sid.rice.edu> writes:
> > tb+u...@becket.net (Thomas Bushnell, BSG) writes:
> >> Rahul Jain <rj...@sid-1129.sid.rice.edu> writes:
> >> > tb+u...@becket.net (Thomas Bushnell, BSG) writes:

> >> > > Geez! Erann actually *likes* CL, probably more than Scheme, but
> >> > > (I'm guessing) doesn't happen to think it is Absolutely Perfect.
> >> > > As a result, he perhaps complains about things once in a while.

> > Once in a while? What else has he done in c.l.l in the entire time
> > I've been reading it?

> couldn't that be because he thinks ngs are for a little bit more than
> mutual administration societies?

But they are for a little bit more than people just complaining that
"you lied to me!".

> something like "i like cl, vut have problems with this feature, so
> what do others think is good about it or what alternatives are
> there"?

Erann never asked questions about what is better, he complained to us
that Python is better than CL, and it's our fault that he thought
otherwise. And then he claimed that packages should be removed and
replaced with modules. That is not a useful idea for people who need
to use symbols as objects, a common occurence in the Lisp community,
from all that I can gather.

Rahul Jain

unread,
Mar 17, 2002, 8:10:35 PM3/17/02
to
Kent M Pitman <pit...@world.std.com> writes:

> Incidentally, Thomas, my entire rant on "data hygiene" was a _direct_
> response to something Rahul said. Had he not posted his remarks, I
> wouldn't have had occasion to sit for a while pondering that answer,
> which I'm quite happy to have penned. I'm not saying Rahul's posts
> have no content value, but even if they accomplished only catalyzing
> the speech of others, I would not discount that as conversational
> value. We each offer value in our own way and at our own time.

Thanks, Kent. :)

By the same logic, Thomas and Erann are at least as valuable to this
thread as I am, and I would have never enumerated my list of features
I find useful in CL if it weren't for their remarks. So thanks to
them, too. :)

Thomas Bushnell, BSG

unread,
Mar 17, 2002, 9:28:55 PM3/17/02
to
Rahul Jain <rj...@sid-1129.sid.rice.edu> writes:

> By the same logic, Thomas and Erann are at least as valuable to this
> thread as I am, and I would have never enumerated my list of features
> I find useful in CL if it weren't for their remarks. So thanks to
> them, too. :)

Yeah, and to join the love fest, I was over-harsh to Rahul in my
stream of recent messages. He too has stimulated my thinking in
productive ways. One thing I can say is that the whole conversation
has certainly helped me understand the whole durn thing better than I
did before, and that's a good thing, all round.

Thomas

Joe Marshall

unread,
Mar 17, 2002, 8:45:49 PM3/17/02
to

"Rahul Jain" <rj...@sid-1129.sid.rice.edu> wrote in message
news:87sn6yz...@photino.sid.rice.edu...

> "Joe Marshall" <prunes...@attbi.com> writes:
>
> > Additionally, if you accidentally fasloaded a file before the
> > packages were set up, you would end up with functions pointing
> > at the wrong symbols. Patching up the packages by uninterning
> > the conflicting symbols didn't fix this, it just made the functions
> > point at UNINTERNED wrong symbols.
>
> Wouldn't the IN-PACKAGE form at the top of the file cause an error,
> preventing the file from being loaded? (or asking if you would like to
> implicitly create it, etc)

I don't think fasl files have an `in-package' form at the top.

> > So, on to more obscure problems. You can't use `forward references'
> > to symbols that don't exist in packages that don't exist (yet).
>
> Again, this should cause an error to be signalled, at which point, you
> should load the definition of the other package or just implicitly
> create it.

The error is the problem. I want to refer to a symbol that I *know*
will exist a short time in the future. I have no intention of
using or manipulating the symbol until it exists and the package
system is set up, I just want to refer to the name. Even so, I'm
out of luck.

> > In your init file, you'd *like* to be able to customize some
> > parameters before you start building the system.
>
> So you load the defpackage form(s) before playing with symbols in
> those packages.

In general, init files are loaded *before* anything else. It is
hard to get the defpackage forms to load prior to the init forms.

> > Another example: when writing a tiny language, it is nice to
> > embed that language in Lisp. You can make a package containing
> > your language constructs, and use macros to `translate' them
> > into standard lisp for compilation. However, CL:IF is a special
> > form `known' to the compiler. FOO:IF is not. The compiler will
> > not generate a conditional unless you use the symbol in the CL
> > package. On the other hand, the point of putting the symbols
> > into your own package was to *not* use the same symbols.
>
> Right, so either define FOO:IF as a macro that expands to CL:IF or
> import CL:IF. The package system doesn't help you when you don't tell
> it to do what you want it to do.

I can't tell it to do what I want it to do. I *want* to hand forms
to the compiler of the sort '(if (foo) (bar)), but I *don't* want
the symbol IF in my package to be CL:IF (for various reasons, perhaps
I'm mucking around with the function cell). Now I could walk the form
replacing FOO:IF with CL:IF (and so forth for all the special forms),
but this is idiotic. The fact of the matter is that the compiler
is not using the value cell, the function cell, the plist, the package,
or the print name of 'CL:IF. It is only using it as a unique name.
But nonetheless, if I want to use it the same way the compiler uses it,
I have to drag along all the other baggage that comes with it.

> > Finally, a really nasty one. The `topology' of the package
> > structure is a global resource.
>
> Yes, having unnamed packages and dynamic name->package bindings might
> be a useful addition, but it's so uncommonly needed that I doubt
> people have really thought about how to do it so that the resulting
> packages are actually useful. I guess these would be called
> "first-class packages".
>
> Hmm, actually, that might be simple to add to my generic-reader.
> Simply have the token-interpretation-fn be a closure around some
> name->package EQUAL hashtable. Or have a *packages* hashtable that can
> be dynamically shadowed. Probably providing both options would be a
> good idea.

I wouldn't recommend that. The first thing we found when we tried that
was that we needed `package closures', i.e., a way to bind a function
to a particular package topology.

> > Now I realize that name conficts are a problem (look at emacs).
> > But I don't think the solution is to hack the READER. With
> > a little careful thought, I'm sure that a better way could be
> > found that defers identifier->symbol mapping until later in
> > the game.
>
> Identifiers in Lisp are symbols.

Yes, but the vast majority need not be. If I write this:

(defun fact (x) (if (zerop x) 1 (* x (fact (1- x)))))

The identifier X is interned as a symbol, yet the code does not use
the value cell, function cell, print name, package, or plist. Not
only that, but the identity of the identifier need not even last
beyond the compilation! However, were I to enter that function,
and then attempt to import a symbol named X from another package,
I'd get a name conflict. What is conflicting?

> Symbol->value mapping is what occurs later.

Yes. It occurs as late in the process as possible: just before you
need the value for a computation.

But identifier->symbol mapping occurs extremely early in the process,
where it isn't obvious that the symbol will be needed at all.


Kent M Pitman

unread,
Mar 17, 2002, 10:01:30 PM3/17/02
to
"Joe Marshall" <prunes...@attbi.com> writes:

> READ causes noticable side effects. This can create dependencies
> on the order in which things are read.

EVAL and/or COMPILE-FILE causes noticeable side-effects too. This
creates dependencies on the order in which Scheme expressions can be
processed.

Moreover, the Scheme environment is among the harshest I've ever seen
for a programming language because it tends to take the attitude that
if your program is in error, all bets are off. The Scheme spec doesn't
waste any time at all trying to tell you that you have any expectation
of hope if your program ever makes a mistake. And certainly among all
of that is that you are expected to have the initial environment set up
perfectly for your initial work. No advice about how to avoid problems.
No operators or paradigms for correction if problems occur.

And it's CL you're complaining about? CL has terminology for talking
about these problem states and correcting some of them, as well as for
inspecting and manipulating the environment to determine whether
you're going to get stung. Hmm.

> Where it *was* a problem was on the Lisp Machine. When you opened
> a file in Zmacs, it would parse the file mode and it would
> parse some of the file contents. It would attach properties to
> the symbols mentioned in the file.

I'm familiar with the problem you're talking about, but you can't
blame this on the package system. I don't think this is a fair
criticism. The Zmacs editor chose to put things on the property list
of symbols prematurely and got in trouble. (Had it put them on the
property list of Scheme-like symbols, the same problems would have been
even worse).

> When you were browsing code, you might open a file that was
> in a package that was not yet defined. Well, the package became
> defined and a slew of symbols from the file would get interned
> in it. Unfortunately, it was unlikely that the package was
> correctly set up, so when you went to evaluate any code it
> wouldn't work, or when you went to fix the packages, you'd run
> into thousands of name conflicts.

Just because someone can design a problem for which symbols ought not
be used and use symbols does not make symbols a bad datastructure.

I likewise had problems with what Zmacs did, but I attribute the
problem very, very differently.

> Additionally, if you accidentally fasloaded a file before the
> packages were set up, you would end up with functions pointing
> at the wrong symbols.

Just as if you accidentally load Scheme code when things are not fully
set up, you can get confusing effects. I don't see your point here.

> Patching up the packages by uninterning
> the conflicting symbols didn't fix this, it just made the functions
> point at UNINTERNED wrong symbols.

Bad defaults? When I get package errors in LispWorks, I usually take
the default correction and I'd say 95% of the time I don't have to do
anything more than press RETURN. Not that I'd think it awful if I had
to think about or intervene more than this.



> Now there is a `standard' mode of operating that avoids most of
> this. You make *absolutely sure* that the *very first thing* you
> load into any lisp is a set of defpackage forms that set up the
> *entire* package structure.

Or you code your editor to not require this. I'm not comfortable
blaming this "requirement" on the design of the package system when
it's very plainly a problem with the Zmacs architecture.

> You also make sure that this file
> is loaded both at compile time *and* at run time. This isn't hard
> to do if you put it in your `init' file.
>
> So, on to more obscure problems. You can't use `forward references'
> to symbols that don't exist in packages that don't exist (yet).
> In your init file, you'd *like* to be able to customize some
> parameters before you start building the system. Perhaps you have
> some macros that are parameterized, whatever. As an example, I
> once worked at a place that had a `verbosity' switch that I wanted
> set.
>
> The obvious thing is to write (setq utility::*verbosity* t)
> but the utility package isn't loaded yet. Never mind that the
> utility package *will* be there when I call this function, it
> isn't there when I *read* the code. Instead I write:
> (let* ((utl-pkg (find-package :utility))
> (verbosity (intern (symbol-name :*verbosity*) utl-pkg)))
> (setf (symbol-value verbosity) t))
>
> The same goes for functions that I want to call. I can't
> write (progn (load "java-tools.lisp") (java-tools:make-all))
>
> The way around this problem is to write a function that
> defers interning until runtime. Then you can write
> (progn (load "java-tools.lisp") (funcall-in-package :java-tools :make-all))

I usually did:

(load "java-tools.lisp")
(java-tools:make-all)

as two different forms. In the few cases where this wasn't possible, I did:

(progn (load "java-tools.lisp")
(eval (read-from-string "(java-tools:make-all)")))

or

(progn (load "java-tools.lisp")
(funcall (or (find-symbol "MAKE-ALL" "JAVA-TOOLS")
(error "JAVA-TOOLS:MAKE-ALL is not defined."))))

Hardly a major problem. And it doesn't require use of any non-standard
operators.

> The package system allows me to `split' names. I can have
> my symbol FOO be different from your symbol FOO. It doesn't
> allow me to `join' names.

Scheme doesn't either, btw.

> I cannot have my symbol BAR be
> the same as your symbol FOO. (Or perhaps more illustrative,
> I can't have `JRM:ORIGINAL-MULTIPLE-VALUE-BIND' be the same
> as `CL-USER:MULTIPLE-VALUE-BIND'.) On the lisp machine, you
> actually *could* do this by forwarding the memory, and it
> comes in handy on occasion.

This is a legitimate gripe. However, I believe CL can rise above it.
I see it as an API problem, not a data structure problem.

> Another example: when writing a tiny language, it is nice to
> embed that language in Lisp. You can make a package containing
> your language constructs, and use macros to `translate' them
> into standard lisp for compilation.

By this I assume you mean:

(defmacro if (test then &rest else)
`(cl:if ,test ,then ,else))

> However, CL:IF is a special
> form `known' to the compiler. FOO:IF is not. The compiler will
> not generate a conditional unless you use the symbol in the CL
> package.

Well, it's supposed to macroexpand your form before reaching
that conclusion.

> On the other hand, the point of putting the symbols
> into your own package was to *not* use the same symbols.

> Finally, a really nasty one. The `topology' of the package
> structure is a global resource. When we were building a new
> Lisp machine, we had a brand new compiler. Obviously, it goes
> in the `compiler' package. Well, on the old machine, there
> already was a compiler in the compiler package. So we had
> to load the compiler into a different package.

I don't see any problem here that adding more power to the package
system wouldn't fix. In particular, original Zetalisp had
hierarchical packages. Symbolics Genera usefully expanded on this by
having "package universes" associated with Syntax. Symbolics Genera
had about 5 to 7 syntaxes (package universes) loaded into the same
core image at the same time, depending on the configuration. I wrote
the code that maintained that separation. I never got complaints from
amyone saying the implementation had major technical problems. Most
people were surprised that Zetalisp and Common Lisp were able to be
co-resident at the same time, given they had needs for packages to be
called different things.

I certainly don't by any stretch think this problem would have been
fixed by removing complexity of packages and using Scheme-style
packages.

> But the compiler had to `appear' as if it were in the `compiler'
> package in the fasl file. We modified the package inheritance in
> the new machine. A number of symbols were moved to different
> packages, but were to remain visible. The problem was that the
> current machine wouldn't work if you did this. Our `solution' was a
> rather hairy thing that twiddled the guts of the package system on
> the fly to satisify whatever compiler happened to want to be using
> it. It was a nightmare.

I'm probably missing something without more details. I think the term
"overconstrained" existed before Lisp, much less its package system.
And I'm not surprised that it's possible to overconstrain problems in
Lisp since it's possible to overconstrian them in life. I'm hard
pressed to say this is a package system problem.

> Now I realize that name conficts are a problem (look at emacs).
> But I don't think the solution is to hack the READER. With
> a little careful thought, I'm sure that a better way could be
> found that defers identifier->symbol mapping until later in
> the game.

Yes, those may be possible workarounds in some of the cases you've
mentioned. I'm not sure they exhaust the space of possible workarounds,
nor that the presence of such a workaround is a proof that what CL does
is bad.

I have not at any time said modules are something people should not use.
I've said that packages are something that people can and do useful use.

What you're relating is very much akin to the issue of early vs late
binding in declarations. Sometimes one adds a declaration because they
know something about data and they want early (compile time) processing
of that information. Sometimes one knows something about a name and
one uses packages to carry this information. Sometimes one doesn't know
what the name-to-package association is, and there are ways of accomodating
that. But often one does know.

In
(defmacro foist (x) `(car ,x))
there is no material change in knowledge between read time and macro
expansion time such that delaying the semantic attachment of CAR's meaning
will do anything other than make it more difficult to figure out what symbol
is what. So most of the problems to do with symbols and macros might as well
be sorted out at read time as at macro expansion time.

In the case you describe, if I follow you correctly, and I kind of
doubt I do totally, you have some weird situation where your compile
time packages and runtime packages are not aligned. (Note that CL
says this is not valid CL, so you are on your own, since every time
Scheme says you are doing something not valid the Scheme folks are
happy to say you're SOL. I don't know why a double standard of how
much help you should expect should apply.) In general, I think the
problem is more that CL does not seek to handle this problem than that
CL would not be capable of handling this problem with available data
structures if it chose to provide you the necessary operators. I don't
see a data structure problem here; perhaps an API weakness. But new APIs
can be written without changing the language spec.

Rahul Jain

unread,
Mar 17, 2002, 10:10:06 PM3/17/02
to
"Joe Marshall" <prunes...@attbi.com> writes:

> "Rahul Jain" <rj...@sid-1129.sid.rice.edu> wrote in message
> news:87sn6yz...@photino.sid.rice.edu...

> > Wouldn't the IN-PACKAGE form at the top of the file cause an error,


> > preventing the file from being loaded? (or asking if you would like to
> > implicitly create it, etc)

> I don't think fasl files have an `in-package' form at the top.

Hmm, yeah. :)

That's what defsystem is for anyway...

> > > So, on to more obscure problems. You can't use `forward references'
> > > to symbols that don't exist in packages that don't exist (yet).

> > Again, this should cause an error to be signalled, at which point, you
> > should load the definition of the other package or just implicitly
> > create it.

> The error is the problem. I want to refer to a symbol that I *know*
> will exist a short time in the future.

So load the definition that allows the name to exist.

> > So you load the defpackage form(s) before playing with symbols in
> > those packages.

> In general, init files are loaded *before* anything else. It is
> hard to get the defpackage forms to load prior to the init forms.

Couldn't you load it in the init file?

Or do something like (setf (symbol-value (find-symbol ...)) ...)?

> I can't tell it to do what I want it to do. I *want* to hand forms
> to the compiler of the sort '(if (foo) (bar)), but I *don't* want
> the symbol IF in my package to be CL:IF (for various reasons, perhaps
> I'm mucking around with the function cell). Now I could walk the form
> replacing FOO:IF with CL:IF (and so forth for all the special forms),
> but this is idiotic. The fact of the matter is that the compiler
> is not using the value cell, the function cell, the plist, the package,
> or the print name of 'CL:IF. It is only using it as a unique name.
> But nonetheless, if I want to use it the same way the compiler uses it,
> I have to drag along all the other baggage that comes with it.

Right, that's why you should do

(defmacro foo:if (&rest args)
`(cl:if ,@args))

If you don't want the whole symbol, you'll have to define which exact
parts you want to take from the other symbol anyway.

> > Hmm, actually, that might be simple to add to my generic-reader.
> > Simply have the token-interpretation-fn be a closure around some
> > name->package EQUAL hashtable. Or have a *packages* hashtable that can
> > be dynamically shadowed. Probably providing both options would be a
> > good idea.

> I wouldn't recommend that. The first thing we found when we tried that
> was that we needed `package closures', i.e., a way to bind a function
> to a particular package topology.

That sounds like modules to me...

> > Identifiers in Lisp are symbols.

> Yes, but the vast majority need not be.

They need to be because that's how Lisp code is defined.

> The identifier X is interned as a symbol, yet the code does not use
> the value cell, function cell, print name, package, or plist.

But it uses the symbol X in the code to indicate what identifier you
are using. That way the evaluator can compare two symbols and
determine if they are the same identifier or not.

> Not only that, but the identity of the identifier need not even last
> beyond the compilation! However, were I to enter that function, and
> then attempt to import a symbol named X from another package, I'd
> get a name conflict. What is conflicting?

The names you used. If you want to have lexical-extent symbols, I
think I can understand, but I can't think of a practical way of
actually doing it.

> > Symbol->value mapping is what occurs later.

> Yes. It occurs as late in the process as possible: just before you
> need the value for a computation.

> But identifier->symbol mapping occurs extremely early in the process,
> where it isn't obvious that the symbol will be needed at all.

If the symbol is used, it's needed. I don't see how this can be
removed, resulting in a language that can really be called
"Lisp". This distinction of read, compile, eval, macroexpand, load,
etc times is one of the distinctive, important, and useful features of
lisp. If you don't like those features, you'll have to form a
different (possibly overlapping) community, since many of the current
leaders of the Lisp community seem to prefer dealing with the times
the way we do now. I think it just comes down to being comfortable
with the way they are defined and being precise about what times
things occur when you write your code.

Essentially, this distinction is one of the axioms of Lisp, as I see
it.

Kent M Pitman

unread,
Mar 17, 2002, 10:08:24 PM3/17/02
to
"Joe Marshall" <prunes...@attbi.com> writes:

> "Rahul Jain" <rj...@sid-1129.sid.rice.edu> wrote in message
> news:87sn6yz...@photino.sid.rice.edu...
> > "Joe Marshall" <prunes...@attbi.com> writes:
> >
> > > Additionally, if you accidentally fasloaded a file before the
> > > packages were set up, you would end up with functions pointing
> > > at the wrong symbols. Patching up the packages by uninterning
> > > the conflicting symbols didn't fix this, it just made the functions
> > > point at UNINTERNED wrong symbols.
> >
> > Wouldn't the IN-PACKAGE form at the top of the file cause an error,
> > preventing the file from being loaded? (or asking if you would like to
> > implicitly create it, etc)
>
> I don't think fasl files have an `in-package' form at the top.

Well, they are required to bind *PACKAGE* around the entire load.
And they have to give that value a binding. They may not do
IN-PACKAGE per se, but it's going to amount to the same.

> > > So, on to more obscure problems. You can't use `forward references'
> > > to symbols that don't exist in packages that don't exist (yet).
> >
> > Again, this should cause an error to be signalled, at which point, you
> > should load the definition of the other package or just implicitly
> > create it.
>
> The error is the problem. I want to refer to a symbol that I *know*
> will exist a short time in the future. I have no intention of
> using or manipulating the symbol until it exists and the package
> system is set up, I just want to refer to the name. Even so, I'm
> out of luck.

Its name is a string. You can certainly refer to its name. ;)

...


> But identifier->symbol mapping occurs extremely early in the process,
> where it isn't obvious that the symbol will be needed at all.

No, you mean where it happens not to be needed in this case.
You want it delayed until load time.

But in normal compilation, the identifier to symbol mapping _is_ done
at compile time, in order to process semantics, so it is not normall
delayed that long. It is typical of interactive and bootstrap code
that this problem comes up, but interactive and bootstrap code is not
typical of real code.

Matthias Blume

unread,
Mar 17, 2002, 10:02:02 PM3/17/02
to

[ Apologies if this gets posted twice, or if a garbled version of it got posted
first... ]

g...@jpl.nasa.gov (Erann Gat) writes:

> A mentor of mine at the time (1988 or so) once expounded on the virtues of
> environments and the evils of packages. He made an offhand comment that I
> still remember to this day: "Common Lisp" he said "really has a flat
> namespace." What he meant was that there was little effective difference
> between packages as they exist in CL, and a Lisp with only one package and
> a reader hack that attached the prefix "package:" (where "package" is
> replaced with the value of the global variable *package*) to all symbol
> names that didn't already have colons in them before interning them. Now,
> this isn't quite right, of course, because of import, export, and
> use-package, but his fundamental point has stuck with me through the
> years, that discriminating between multiple interpretations of a symbol
> name at read time rather than at eval time was the Wrong Thing. (This is
> not to say that I necessarily agree with this point of view, just that I
> remember it.)

Erann,

your mentor had it right, basically. Let me explain why.

First, we should say what a "symbol" really is: A symbol is a little
data structure, a record if you will. (The data structure contains a
number of slots. One slot, for example, could hold the symbol's
package (if it is interned), another its name within that package,
another its global value, another its function binding, another its
property list. I do not claim completeness here.) What a symbol
really is is not important for the purpose of this discussion -- only
that it exists and has an identity that is independent of the name
that refers to it.

Second, what is a package? A package is a structure that (at least)
consists of a finite map mapping names (strings of characters) to
existing (interned) symbols (where by "symbol" I mean those little
data structures introduced above).

For the time being, let's say that a package *is* its map from names to symbols.
(Of course, this is not the whole picture as far as CL is concerned. But it
will illustrate the idea without getting bogged down in unimportand but tedious
technicalities.)

If we call the domain of names "name", the domain of symbols "symbol",
and the domain of packages "package", then we can write the domain equation:

package = name |--> symbol

Now, packages themselves also have names, which means that there is a
global finite map PACK (that is also mutable, but, we postpone this issue)
that maps strings to packages. So, in a fashion analogous to the
above, we say that if m is the name of an existing package p, then

PACK(m) = p

or

PACK : name |--> package

or

PACK : name |--> (name |--> symbol)

In other words, to look up a symbol given the name m of its package and
the name n of the symbol within the package named by m, we simply do

PACK(m)(n)

(I am sure you recognize PACK: it is CL's "find-package", only grossly simplified
for the purpose of this discussion.)

--------

At this point, let's step back for a moment and look at total maps.
For every total map f that maps A's to total maps from B's to C's:

f : A --> (B --> C)

there is a total map, let's call it "uncurry(f)" from pairs (x, y) \in A * B
to C:

uncurry(f) : A * B --> C

Moreover, for every g : A * B --> C there is a curry(g) : A --> B -->
C. Finally, both "curry o uncurry" and "uncurry o curry" are
identities, so "uncurry" (like "curry") is an isomorphism. (And, not
unimportantly, computability is also preserved by "curry" and
"uncurry".)

Some minor but annoying technical difficulties arise once we try to
apply the same reasoning to partial (in particular: finite) maps
because the domains

A |--> B |--> C and A * B |--> C

are not isomorphic. The problem is that in the curried case an
application can fail at either of the two stages while in the
uncurried case these two failure modes are collapsed into one.
Translated to the PACK case this would mean that one couldn't
distinguish between looking up a non-existing symbol in an existing
package and looking up anything in a non-existing package.

But notice that these issues *can* be finessed, e.g., by "lifting" the
domains (supplying an explicit failure result -- nil in the case of
find-package) and lifting the second stage lookup function
accordingly. On the uncurried side this leads to the need for an
auxiliary map from package names to booleans (i.e., effectively a set
of package names) to keep track which packages exist and which ones
don't.

The point is: There /is/ an isomorphism, and where there is an
isomorphism between two explanations (here: curried and uncurried
lookup), one can say that one can be /viewed in terms of the other/.

Of course, one has to be careful: Once you decide which view to take,
you have to be consistent. Curried and uncurried functions are not
the same, they are merely related by an isomorphism. If one gets
mixed up between the different views, arbitrary nonsense occurs.

---------

Anyway, coming back to your mentor's "offhand comment":

- The members of the cross-product of two string domains could be
represented as the concatenation of the original strings -- provided there
is some way of recognizing the boundary so that the original two strings
can be recovered. (In CL there is such a way.)

- Thus, an uncurried lookup function (which is the essence of a "flat" domain)
could, indeed, take the symbol name with the package name prefixed.

- This is a plausible way of viewing how things work. Of course, it
does not mean that things actually /do/ work that way in terms of how
they are implemented. They probably are not (although I would claim --
with enough finesse they could be, no matter how pointless that would probably
be).

- I do not see how "import", "export", or "use-package" pose insurmountable
difficulties with the flat model.

---------

Finally, how to deal with the mutability? Well, it is actually not that
hard. Essentially the same kind of isomorphism like the one above can
be constructed, only the details are a bit messier. That's why I leave the
construction to the interested reader.... :-)

Matthias

Thomas Bushnell, BSG

unread,
Mar 17, 2002, 10:23:25 PM3/17/02
to
Kent M Pitman <pit...@world.std.com> writes:

> Moreover, the Scheme environment is among the harshest I've ever seen
> for a programming language because it tends to take the attitude that
> if your program is in error, all bets are off.

This is pretty much normal, I think. Nearly every modern language
specification says that an undefined behavior is, well, undefined.
This is especially true for errors which are not required to be
caught, and it seems to me that Scheme is conservative about requiring
compilers to catch runtime errors in order to enable them to produce
maximally efficient code.

I think this is, in effect, the same for Common Lisp; if you do
something undefined, the result is undefined.

> I have not at any time said modules are something people should not use.
> I've said that packages are something that people can and do useful use.

Hrm. If I have a good module system, and symbols are "lightweight",
that is, they don't have plists or dynamic values, then what
advantages would be gained by adding packages? Maybe I've missed it,
but all the advantages seem to depend on either plists or dynamic
values (which are, it seems to me, really examples of the same
thing). I fully grant that if you have either of those, then packages
become an absolute necessity. But if you don't then what would
packages give you that a module system doesn't?

Thomas

Rahul Jain

unread,
Mar 17, 2002, 10:38:57 PM3/17/02
to
tb+u...@becket.net (Thomas Bushnell, BSG) writes:


> I fully grant that if you have either of those [plists or dynamic
> values], then packages become an absolute necessity. But if you


> don't then what would packages give you that a module system
> doesn't?

Is this a trap? I could just imagine the response I'd get from you if
I actually answered this question yet again.

Thomas Bushnell, BSG

unread,
Mar 17, 2002, 10:44:01 PM3/17/02