Until recently I thought this was the dominant view in the CL community,
but I have recently discovered this is not so.  Many people believe that
packages are a feature in and of themselves, that they provide some
benefit independent of any legacy considerations, that if they had it to
do over again they would still incorporate packages into the design.  I am
now trying to understand that point of view.

So far, the arguments I have heard advanced for the second point of view
have seemed to me to be more philosophical than technical, going something
like: "You need packages in order to support symbolic processing," where
"symbolic processing" is *defined* as computing with
common-lisp::symbols.  I grant that argument, but it just begs a different
question: why is "symbolic processing" thus defined a good idea?  What
does common-lisp::symbolic-processing buy you that
scheme::keyword-processing does not?

Note that by restating the question I do not mean to imply that it hasn't
already been answered.  Kent has written at length about this, but it's
taking me some time to digest everything he's had to say.  (I still have
to get some actual work done between newsgroup postings.)

Maybe what I've been doing these many years that I have always thought of
as symbolic processing is really scheme::keyword-processing (except that
I've been doing it in Common Lisp and not Scheme).

  I marvel at the psychological problems that produce your need to respond.

| Being lectured on arrogance by Erik is so funny that words fail me.

  I would much prefer they _actually_ did.

| Erik, you should really stop confusing not listening to *you* with not
| listening to anyone.

  You take the time off from your productive life to complain that people
  say bad things about you and you do not find _yourself_ ridiculous when
  you spend an _enormous_ amount of time badmouthing me to the point that
  people who do not even know me cannot figure out what I could possibly
  have done to cause your psychotic break and the scary monsters you see.

  What if you tried to understand what I say, Erann?  What dangers are your
  mind protecting you from when it shut down and you consistently fail to
  listen?  How much truth have I _really_ said about you that you need to
  pretend so much?  It is obvious that I have really freaked you out --
  into some kind of self-protective position where you _have_ to pretend
  that what I say is so "ridiculous" and "funny" that you do not have to
  look too closely at it.  These are quite obviously defense mechanisms.

  Just cut it out.  Get over whatever it was that hurt so badly and move
  the hell _on_, dude.  Whatever it is that I can do that hurts so much
  only gets worse every time you stupidly decide to attack me to feel
  better.  If not, it would have worked and you would have quit, right?

Is that *really* a common expression, somewhere?

Scheme always shares all symbols with all others having the same name.
CL allows you to chose whether you will or not.

Being pro-choice doesn't mean being pro-abortion.  It means reserving the
right to take one or the other action.  The pro-life term "pro-abortion"
incorrectly characterizes the others' movement as always favoring abortion
even though some pro-choicers have never had an abortion and never would;
they simply tolerate choice.  Ironically, the pro-choice movement applies
the term "anti-choice" which does, I think, objectively describe their
political opponents.

Being pro-choice in symbol separation doesn't mean either always
sharing your symbols with others or always separating them.  It means
getting the choice on a case-by-case basis.  Don't make the mistake of
thinking that the opposite of pro-share [the Scheme point of view,
with alias "anti-choice"] is pro-separation.  Nothing about the CL
approach keeps you from doing what Scheme does.  But the Scheme
approach keeps you from doing what CL does, at least with any data
type called symbol.

And, as often happens, it becomes a war over who gets legitimate use
of the word.

It's worth noting, going back to the abortion debate, that the
argument is so intense that there have been reports of advocacy
organizations purchasing and otherwise influencing dictionaries in
order to get subtle changes in the definition of "life" so that they
can influence the debate on this matter, since some people actually
reason based on the words, rather than on their target meanings.

Words matter.  Refined control of when they bind is critical.

There was an occurrence of this name-bonding thing in the design of
ISLISP actually.  When WG16 was first working on an international
standard, we got together and talked about what its nature should be.
One national representative (I'll spare that person the embarrassment
of mentioning his country of origin) said to me privately "I don't
care what's in the standard.  I care only that the standard is called
Common Lisp."  "Why?" I asked, puzzled.  "Because I am a consultant
and have told a client that Common Lisp is the future.  If the
standard we make is called Common Lisp, whatever its content, then I
have told the truth.  It doesn't matter what's in the language, only
that the words I said were true.  If what we make has the semantics of
Common Lisp but is called something else, I will have given bad
advice."  So you see, late-binding the meaning of a name ("Common
Lisp" in this case) may be helpful to some (this representative), but
is not helpful to all (his client, the committee, ...).  It's a point
of view thing.

I know I shouldn't be continuing on this thread, but....

Why does symbolic processing require the namespacing of symbols?
Because we need to allow different applications to use their natural
terminology, and manage conflicts and similarities among them.

"Pierre R. Mai" <p...@pmsf.de> wrote in message
news:87r8mhfnae.fsf@orion.bln.pmsf.de...

(defun build-java ()
  (load "java-tools.lisp")
  (java-tools:make-all))

What needs do you have for packages and symbols that is not being net?  I do
not have any additional needs so there is no driving force for me to do
anything.  What is driving you?

From your original post I saw:

  But if you typed it into the REPL, you must have been in the wrong
  package to begin with.  Why is that?  My guess is that you were in the
  common-lisp-user package and called use-package instead of changing the
  current package to a well-defined package that uses the packages you have
  defined.  I think you should be using in-package instead of use-package.

  In my opinion, you are using the package-system counter to its design
  when you randomly call use-package, regardless of whether it is because
  of this "error".  I tend to think it is just plain wrong to use the
  package operations interactively unless you are working to piece together
  a package in order to dump it to file prior to defining anything that
  depends on it having properly been set up.  In other words, you have a
  broken package definition to begin with when you need to call use-package
  to "fix" it.  And this is most likely the wrong "fix", too.

  You have probably thought that use-package is something very different
  than it is and are now frustrated that it is not what you think it is,
  but what you think and what it is are probably close enough not to give
  you a strong enough hint that you need to revise your concept of what it
  is.  May I suggest that you do a refresh read of the package concepts in
  the standard to see that your assumptions are actually valid?

  I tried to remember when I ran into problems like you describe, but it is
  quite hazy to me.  I must have changed the way I use Allegro CL compared
  to how I used CMUCL.  In particular, if I try to evaluate an unbound
  variable in ACL, I get these restarts:

(52) cl-user
foo
Error: Attempt to take the value of the unbound variable `foo'.
  [condition type: unbound-variable]

Restart actions (select using :continue):
 0: Try evaluating foo again.
 1: Use :foo instead.
 2: Set the symbol-value of foo and use its value.
 3: Use a value without setting foo.
 4: Return to Top Level (an "abort" restart).
 5: Abort entirely from this process.

  CMUCL is quite deficient in the restart department:

* foo

Error in KERNEL::UNBOUND-SYMBOL-ERROR-HANDLER:  the variable FOO is unbound.

Restarts:
  0: [ABORT] Return to Top-Level.

Debug  (type H for help)

(EVAL FOO)
Source:
; File: target:code/eval.lisp

; File has been modified since compilation:
;   target:code/eval.lisp
; Using form offset instead of character position.
(SYMBOL-VALUE EXP)

  I had to go through a manual apropos call to get the equivalent of
  restart 1 in ACL above.  That would be annoying, but this is a quality of
  implementation issue with CMUCL, not a design issue with packages.

  CMUCL does not by default print the package name, and there is no
  convenient way to change the package, so it seems to me that your having
  adopted this abuse of use-package may well be an history artifact and
  that you should now use in-package, instead.

* Kent Pitman

  Really?  This would only be true of all symbol names were always distinct
  regardless of the package system.  Since this is obviously false and a
  use-package therefore can signal an error caused by two well-defined
  symbols in both packages, there is something wrong with your premises.

  I think your _premise_ is that the package system is broken and now you
  are only trying to find ways to "prove" it by doing things that would
  break _any_ working system.  I consider this prejudicial.

| No, I don't want a DWIM.  What I do want is something that is a bit more
| clever.  Consider this behavior: when I make a typo and get an unbound
| variable error, why not have the system unintern the symbol just created
| if and *only if* it was created by the typo itself.

  Would you want the reader to record the second value of intern for each
  symbol it has interned and make them available to the evaluator so it can
  safely unintern a symbol that causes a symbol and search all packages for
  a symbol with the same name and suggest them to you in case the user is
  unable to set up the package-system so it produces user-correct results?

  This sounds like a really bad solution to the wrong problem.

| These two behaviors (and note that I'd like that latter to be a user
| controlled switch so no one *has* to monitor the load process
| looking for these things) would greatly improve my relationship with the
| package system.

  The user-controlled switch exists and is called defpackage.  If you do
  not use an in-package form in your compiled source files, I think you
  deserve to lose, and if your package definitions are out of sync with
  respect to their actual interdepencies, even more so.

One of the great things about CL (or at least MCL, which is the CL
environment I've used since the mid-80's) is that recompiling small parts
of the code has essentially zero cost.  (I believe this is true of CL in
general, but I don't have enough experience in other environments to be
sure.)  Because of this, perhaps, I do not perceive having to recompile
some code when I change a design decision as a problem.  (And I make a
*lot* of design changes while coding.  In fact, one of the sources of
friction I have with some of my colleagues is that I think that design and
coding ought to be much more finely interleaved than they do.)

It's a good read even today.

Hm.  Must sleep on it.

Though the design was initially bad having only those stupid package
operations (which should always have been only subprimitive to DEFPACKAGE
and its ilk, to error correction, etc.) so I don't know if counter to its
design is quite how I'd say it.  I'd say "counter to what has become
good practice" ;)

But interning somehow is the CRITICAL part.  Otherwise, (LIST 'A 'A)  
becomes (LIST '#:A '#:A) which is a very different thing because it
contains no duplicates and hence no info shared between list elements.

But, moreover, for building embedded rule-driven languages, there is simply
no need to manipulate compiled code.

And anyway, no matter how easy it is, no one but the program maintainers has
any business manipulating the code in my package.  But I may publish data
descriptions they can manipulate.

Also symbols externalize even if functions don't.
(And if functions externalize, it's only because secretly they use names
of symbols internally, because otherwise if you tug on certain functions,
all of creation goes out to a file...)

Actually, in Zetalisp, pathnames also get interned.

So while I'm here, since I exist only in worlds where software for fee is
a given, let's hypothesize this as a given.  If you want to start a thread
on why we don't need both software for pay and also Kent, that's fine.

I've heard the atom bomb addendum, as well.

[sorry for continuing off-topic; couldn't resist ... ]

 >> To quote a childhood mantra: "almost only counts in horseshoes and
 >> hand grenades"

 Bruce> I know that from my childhood as well, but only because Col. Potter said
 Bruce> it a lot.

 Bruce> Is that *really* a common expression, somewhere?

Regs, Pierre.

1. How can anyone do a reasonable Lisp development without
   understanding AND ACCEPTING the package mechanism which you started
   a discussion about?

2. How can anyone work successfully together with other people at JPL
   or at Google, when he has such difficulties in understanding other
   people?  What I hoped with my question was that someone who
   actually worked with you (e.g. Norvig at Google) could clarify
   things.

3. Why do you have to follow up every damned message you do not
   understand (and these are plenty) asking for further elaboration?
   Why can't you simply accept answers people give here, ponder them
   several days (at least) and WORK FOR YOURSELF to acquire sufficient
   knowledge for continuing.  As much as I remember you told us that
   you do not actively work with CL anymore, so I get the impression
   that you want the people here to teach you without the pondering
   and working step you should do.  I do not see how I could work
   together with someone doing that constantly, so I hoped that some
   colleagues of yours could explain it to me.

Now several months later I write a third program that uses both packages, and I
suddenly discover that there is a name conflict between 'wh:a' and 'lh:a'.  I don't
use the property list, value cell, or function cell of either symbol, so what I'd
really like to say is, "Treat these as the same symbol; they will never occur in the
same context, so there will never be any ambiguity about which is meant." Obviously,
this example is contrived, but this sort of thing has happened to me more than once.

Another example is where package 1 defines a macro that uses various symbols as
"local syntax" markers (such as 'for', 'in', when', etc. in '(loop for thing in
things when (get thing 'color) collect thing))), and package 2 defines another macro
that happens to use 'for' and 'collect' as local syntax markers.  Now I write a
program that uses both packages.  There seems to be no reason in the world to have
to write

(loop pkg1:for thing in things
   when (get thing 'color)
   pkg1:collect thing)

and

(wait pkg2:for (event e ...)
  then pkg2:collect  e)

but that's what I have to do.

Of course, in both these cases I can decide to make package "PKG1" or package "LH"
primary, and have "PKG2" and package "WH" import the conflicting symbols from their
partner before re-exporting them, or set up a third package to play the role of
dispenser of awkward symbols, but then this breaks all the applications that just
want to use, say, 'wordhack' without 'letterhack.'  (In the actual example above,
'for' and 'collect' happen to be in the "CL" package, so the problem, by
happenstance, wouldn't arise; I just chose 'loop' so I could use a familiar macro.)

For these and other reasons, over the years my Common Lisp style has evolved away
from being "symbol-dependent."  Whenever possible, I make sure my macros use local
syntax markers from the keyword package only.  I always use tables instead of
property lists.  I often define variant symbol types that print something like
symbols, and when appropriate read something like symbols, but differ from symbols
in various ways (such as not being interned until after the first time they are
printed).

However, symbols are a wonderful thing.  Note that the whole reason symbols and
packages come up as an issue in the Scheme-Lisp world, and not in the Haskell-ML
world is that the language uses the same reader that user code uses.  That makes
macros possible, but it also raises all these complications about binding time.  On
balance I think I would prefer a module system to the package system, but it's not a
strong preference.

And now apparently I have to tack on some verbiage about how I love
"symbol-processing languages," and I despise Scheme (which, believe it or not, I do,
pretty much), and I'm competent to talk about all this because I once coauthored a
book on Lisp programming.  Really, the tone on this newsgroup often sounds like a
meeting of a Communist Party cell circa 1935, with the main item on the agenda being
how to detect the Trotskyites among us.

It's all about identity.  Not identity in the sense of eq, but in the
sense of how objects are named, i.e. how the programmer identifies
things.  I'll speak of naming rather than identity to avoid the
confusion about all objects having identity in the sense of eq.

Kent is pointing out that it's not a bug, but a feature that symbols are
renamed by the package system.  Merely renaming functions to be exported
would not be enough.

The issues regarding naming are the same, whether it's a programmer
naming a function, or a program naming some object with a symbol.
Keeping the names straight in different contexts is what the package
system does.  A module system would only help keep the function names
straight.

I thought Kent's answer was clear, but perhaps the 3-paragraph summary
above will help.

I'm satisfied with Kent's answer as to why the CL package system is a
good thing.  However, what I want is an answer to a different question.
I tried to ask this question in a previous post, but it wasn't clear, so
I'll try again with some background.

Scheme does not have a module system.  Different implementations have
module systems as extensions, but there is no standard Scheme module
system, and I think some implementations don't even have one.

When using programs from different sources, you rely on luck to avoid
naming conflicts, or on uniqueness of prefixes chosen by the programmer
(e.g. Dorai's pregexp), much as you do with C.  However, nobody claims
that C and R5RS Scheme are not examples of programming languages, even
though there is a serious naming issue that affects programming.

However, now that Kent has discovered that there's a serious naming
issue that affects symbolic processing, he's ready to conclude that
Scheme is not an example of a symbolic processing language.  Sure, you
can do symbolic processing on a small scale, but start to integrate
symbolic processing code from different sources and you'll hit naming
conflicts.

Note that in the above paragraph, you could substitute any style of
programming for "symbolic processing", the only difference being that
the naming conflicts in question would be function names rather than
symbol names.  Shall we go on to say that Scheme is not an exemplar of
*any* style of programming, at least until such time as it gets a
standardized module system?

Each category can do something significant the other
cannot.  The "happy" identification of them that Kent
is suggesting and Erann is agreeing to requires an
amount of stretching that may not just be unhappy
but impossible.