Thanks,
abe

But this is EXACTLY the point people are trying to make.

The Scheme community (and, I would say, the FP community in general)
wants to DEFINE what is natural and easy by a simplicity metric they
pulled out of a hat, as if the very meaning of natural/easy is
uniquely determined/predicted by that simplicity metric.  It is not.
That's a possible theory of simplicity, but it is (a) not proven to in
fact make things simpler for most people and (b) not proven to be the
only way to make things simpler for most people.

The Scheme community then works to minimize the number of premises
they work from, as if minimizing this one axis in what is obviously a
multivariate system will yield a consequent minimization of the other
axes by magic.  I don't buy that.  First, I see language size and
isolated sentence size as inversely correlated.  The bigger the
language, the less you have to say.  THat is, a language that has has
all 64 crayola crayon colors as primitive concepts is at a marked
advantage for describing a sunset over one that has only the primary
colors plus the modifier words "saturated", "bright", "dark" and
adverbials like "not" and "very".  So it may please some teacher of
this Esperanto-like "simple" language to to know he "empowered" people
generally to describe ANYTHING with his tools, but it won't
necessarily please the reader of a fine novel to see the sunset
described as a "very, very, light, unsaturated mix of blue and red" if
what is meant is "lavender".  The reason "lavender" works, even though
it takes a long time to learn all the subtle color names, is that your
brain wetware learns to make a primitive connection from this word to
this color and doesn't have to waste compute time simulating things.
Moreover, it can be compared in more "satisfying" ways to other colors
like "orchid" than can "very, very, light ..." be compared to just
"very light..."  Sure, you can make all the claims you want about the
extra word "very" being missing giving the same sense, but that still
requires you to remember the *input* string and deal with it as a
formal algebraic string because the human brain is not a precise
enough processor to actually carry the consequence of such thought
exercises primitively.  I know my girlfriend often orders a medium
coffee extra extra extra light from Dunkin Donuts but I've seen them
sometimes give her a medium coffee extra extra light, which both
suggests that people receiving such data are bad at holding even that
long a string in their head and that she's unable to hold the precise
notion of that concept in her head.

Now, second, the Scheme community has another very unwarranted premise
underlying a great deal of its teaching, and it overflows sometimes
into design: that's that if they can find a simpler way to express
things (using their simplicity metric: smaller language) that this
will autmatically lead something to be easier to learn.  This makes
unstated assumptions about how people think and how people learn.  I
claim, without enumerated proof here, but with a belief that at least
I could dredge up proof if I had to because psychologists do research
these things all the time, that the human brain is not like an Intel
box, and that people do not operate on a RISC instruction set.  I
think people have a highly parallel associative mechanism capable of
efficiently managing an enormous amount of special cases at the same
time.  All naturally-designed human languages that I know of, not
counting the ones that were designed by professional logicians or
computer people, have tons of special cases, have context, etc.  Of
course, human languages are all very different, each with their own
idiosyncracies.  But what this tells you is that it's ok to have some
variation here.  Humans will cope.

I think the fact is that the reason to keep programming languages
simple is not for the sake of humans, but for the sake of code.  So
yes, for the sake of code, not making the language too difficult will
make programming easier.  But on that we have a 25-or-so year history
during which both Scheme and CL have existed, and in no cases have I
ever seen or heard of someone tearing out their hair and leaving the
CL community saying "I just can't build complex programs because this
NOT/NULL [or false/empty-list] thing is making it too hard to write
metaprograms".  It just doesn't happen.  So no one ever points to that
as the reason for splitting false/empty-list.  They point instead into
the murky depths of the human brain, citing simplicity without
defining their simplicity metric, citing how hard it is for students
to learn (which it is, in the abstract, but mostly because students
are often ill-trained to learn).  I sometimes would go so far as to
think some students of Scheme are indoctrinated by some teachers to
actually reject, almost as a political action, the possibility of
receiving certain kinds of learning because they recognize it as not
simple enough.  Yet they have somehow managed to recognize and adopt
this whole anti-complexity complexity when it would be simpler to just
learn things than to apply political philosophy to everything you
learn.  The only other place I've ever seen that kind of resistance is
in recent converts to this or that church, who are never sure if they
should try to directly understand a truth you're trying to offer them
because they're not sure it's offered to them in a form that it would
be good for them to receive it in.

But to me, the issue of what makes a language easy/hard to use is the
dissonance between the mental representation of the user and the
manifest structure of the program.  I have some theories about that,
but I acknowledge this as a hard problem.  I'm not trying to argue CL
is an utterly simple language to learn and use.  But people do often
claim that Scheme is simple to learn and use, and also that it is
simpler to learn and use than CL, and I'm arguing against accepting
such claims without better proof.  If someone doesn't bring either
serious case studies or serious psychological evidence of mental
models into play in the discussion, I think it's worth questioning all
these unstated simplicity metrics that are advanced instead.

On learning and understanding, I think this: Years ago, I taught
myself Portuguese in anticipation of a couple of weeks vacation I was
going to spend in Brasil, it didn't occur to me that it was going to
be pronounced differently than Spanish.  When I got to Rio, I found
some people who could understand me and some who couldn't.  It wasn't
smart people who understood me and stupid people who didn't.  It
crossed those lines.  The line I eventually drew was that people who
wanted to understand me (i.e., those who stubbornly resisted
entertaining the implausible notion that what was coming from my mouth
might not be "an attempt at language") had no trouble understanding
me.  But those who didn't want to understand me, or didn't mind not
understanding me, looked at me with blank stares.  I'm convinced those
people actually had allowed themselves to think "perhaps this isn't
speech but just some kind of useless babble", and the believe that
this "simpler" answer could be true allowed them to rest confident
that not understanding me was the simplest answer to their
conversational dilemma.  I really think the same of Lisp.  Some people
see the NOT/NULL thing or the false/empty-list thing and decide "it
probably means the langauge is incapable of expressing anything
plainly".  And voila, they find proof of their claim in the fact that
they cannot express themselves.  Or else they decide that it's just an
artifact of the language that can/must be worked with, and, wonder of
wonders, it never becomes a problem.  But the dividing line is not
that language feature.  

  I think it is.  Some people insist on being inexperienced despite their
  experience.  I think this is a stupid kind of stubbornness, but we find
  it in a number of people.

| But our current conception of evolution could very well be off, and I
| don't think it takes a genius to figure out that English is a crazy (if
| practical) language.

  But it takes a really smart person to accept it for what it is and not
  nurture a desire to change the grammar.  The same is true for medicine.
  It is pretty easy for anyone who is studying anatomy to figure out ways
  that the human body could be improved.  Fortunately, the ethical
  standards of the medical discipline tend to discourage such desires.

| I *really* just wanted to refute the idea that it was nuts to separate
| NIL and false.

  It _is_ nuts.  False is defined as nil.  Just as true is defined as t.

| For all the arguments in that direction, the obvious usability of Common
| Lisp means that it's also reasonable to conflate them, despite any
| conceptual messiness that might (or might not) entail.

  They are not conflated.  There is no conceptual messiness.  False is
  defined as nil.  True is defined as t.  That is how it is.  If you cannot
  deal with this, it is only your problem.  Augustine's prayer may apply:
  God grant me serenity to accept the things I cannot change, courage to
  change the things I can, and wisdom to know the difference.

         A Plan for the Improvement of English Spelling
                          by Mark Twain

        For example, in Year 1 that useless letter "c" would be dropped
to be replased either by "k" or "s", and likewise "x" would no longer
be part of the alphabet.  The only kase in which "c" would be retained
would be the "ch" formation, which will be dealt with later.  Year 2
might reform "w" spelling, so that "which" and "one" would take the
same konsonant, wile Year 3 might well abolish "y" replasing it with
"i" and Iear 4 might fiks the "g/j" anomali wonse and for all.
        Jenerally, then, the improvement would kontinue iear bai iear
with Iear 5 doing awai with useless double konsonants, and Iears 6-12
or so modifaiing vowlz and the rimeining voist and unvoist konsonants.
Bai Iear 15 or sou, it wud fainali bi posibl tu meik ius ov thi
ridandant letez "c", "y" and "x" -- bai now jast a memori in the maindz
ov ould doderez -- tu riplais "ch", "sh", and "th" rispektivli.
        Fainali, xen, aafte sam 20 iers ov orxogrefkl riform, wi wud
hev a lojikl, kohirnt speling in ius xrewawt xe Ingliy-spiking werld.

--
Janis Dzerins

It seems to me that *every* datatype deserves to potentially have a
"don't know" or "doesn't exist" value.  Like NULL in SQL.  An integer
field in a database can allow NULL, or can be declared to be NOT NULL
and this is an important distinction.  In any complex program some code
needs to be written to cope with NULL values, and some code wants to
assume that there is always valid data.

It's bad to use a value of zero in an integer variable to mean NULL.  
It's bad to use a value of -1 in an integer variable to mean NULL.  It's
also bad, I think, to have a null pointer value in every pointer data
type.  How often do C programs fail because some function gets passed a
null pointer that didn't expect it?

It seems to me that using an empty list to represent NULL is just as bad.

Seems very strange when you only need one bit to represent that second
value, and the language you are using has *already* made provision to
incorporate that sort of information in every value.  Every value, that
is, except for the empty list.

  Please stop thinking of it as a "confusion", and you may get it.

| suppose you have a function (foo name) that returns a list of all the
| classes being taken by a student.  If the function returns NIL, does that
| mean that the student is taking no classes at the moment, or does it mean
| that the student doesn't exist?

  Well, read the documentation of the function to find out.  What else can
  there possibly be to say about this?  This is a non-existing problem.

  Since you think SQL's NULL is so great, do you get a NULL if the student
  whose classes you ask for does not exist?  Why would you ask for the
  classes of a non-existing student?   Not to mention, _how_?  Is a student
  somehow magically adressable?  In other words, what did you give that
  function as an argument to identify the student?

| It seems to me that using an empty list to represent NULL is just as bad.

  If you need to distinguish the empty list from NULL (nil) in the same
  function, your design sucks because you are confused, not the language.

| No matter what set someone is returning, there's always a desire to squeeze
| in just one extra element ... and often it defeats the whole point of having
| picked the size of data storage one has chosen.  Just [...] look at the
| opposite outcome in the utterly stupid Base64 MIME encoding, which requires
| 65(!) characters to correctly represent it, so never really compacts up
| quite right.

I don't quite understand the relevance of this example, since the
purpose of using 64 characters for the Base64 encoding is not, I
believe, compactness of representation, but simplicity of coding and
decoding.  But it's true that the "=" padding character is not needed
for Base64 decoding: Just decode the input in groups of 4 characters
each producing 3 octets.  Then, when all these groups are used up, you
will be left with 0, 2 or 3 characters which decode into 0, 1 or 2
octets respectively.  So in this sense I can concur with your
characterization of Base64 as stupid.  But compared to the use of NUL
to terminate C strings it seems to me a minor stupidity indeed.

If the author thinks that those cases need to be distinguished, he has
any number of possible ways of communicating the difference.  For
example he can use mulitple values, with the secondary value
indicating whether the student existed at all.

But mostly, if the difference between non-existing and
existing-but-course-less students is really relevant, I'd specify foo
to signal an error if passed the name of a non-student.

Unless someone can demonstrate real problems when programming within
CL that are caused by (eq 'NIL '()), I consider it a worse problem
that so much time is wasted on discussing that non-issue, than any
problems that (eq 'NIL '()) could imaginably create, if it were indeed
a problem at all, which I don't see.

I think such issues should be decided based on some form of real-life
data, and not on armchair musing.

I have yet to see one bug that was caused by (eq 'NIL '()) tripping up
some minimally-competent programmer (i.e. one that didn't just start a
CL implementation by accident, when he really was expecting Scheme, or
some other language), or forcing someone to write noticably more
convoluted code because of it.

Regs, Pierre.

If that is your attitude about error checking, than I don't think that
(eq 'nil '()) is something you need to fret about, for a looong time.

Furthermore why care about the cost of exceptions in such a situation?
Unless your program is buggy as hell (i.e. calls foo with non-student
names all of the time, in-spite of foo's specification), that is.  If
it is important to differentiate between non-students and students
with no courses, _you don't call foo with an unchecked name_.  Period.

You write instead:

(defun bar (some-random-name)
  (cond
   ((student-p name)
    (let ((courses (foo name)))
      ;; Do something silly here...
      ))
   (t
    ;; Do something differently silly here...
    )))

The only thing that a non-nil "NULL" value would give you here, is the
ability to write instead

(defun bar (some-random-name)
  (let ((courses-or-student-status (foo name)))
    (cond
     ((not courses-or-student-status)
      ;; Do something differently silly here...
      )
     (t
      ;; Do something silly with courses-or-student-status here, but
      ;; we now know it actually is a list, not some overloaded
      ;; other return value-type...
      ))))

And that's the point where I'm thanking the powers that were, that we
don't see this kind of code all over the place.  FOO should never have
been specified like this in the first place, if there really is a
difference between non-students and students with no courses.  What
would be a descriptive name for foo?  STUDENT-COURSES-AND-STUDENT-P?

(eq courses-or-student-status #f)

instead of

(not courses-or-student-status)

Not that I would consider code that did this to be well written code.
It actively encourages the same kind of lossage you get in C, where we
have all of those "failure" indicating values, that you have to
manually check, and if you don't (who does), work very nicely to
increase the time between error occurrance and error detection.

Regardless of any (eq 'NIL '()) issue, I can't see any justifiable
reason (senseless obsession about speed isn't one), for a function
student-courses that returns anything but a list (or other collection
datastructure).  Either you define non-students names to be valid
arguments to that function, then you should just return the empty list
(collection, whatever) for non-students, or you say "student-courses
is actually not defined for non-students", in which case you signal an
error when called with a non-student name.

I'm even critical of a multiple-value version that _additionally_
indicates whether we're really talking about a student.  But in cases
where this is justified, that additional value doesn't say "the
primary value is valid or not".  The primary value is always valid,
and has the normal meaning.  The secondary value just _adds_
additional information.  Doing anything else just opens the door for
undetected error situations.  We could then just as well be doing
errno in Common Lisp...

Doing anything else is just plain bad code in my book.

Regs, Pierre.

- you should be able to have a binding that you know has a valid value
of the appropriate type

- you should be able to have a binding that might contain a valid value,
or might contain the NULL value

- it's handy if the NULL value tests as false in conditional expressions

- I don't really care whether there is a different (NULL-OF FOO) for
each type, but I'd suggest that it's fine to have them all be the same,
e.g. #F.  More efficient to test for, too.

This is all fine in CL already, *except* that a the list type is (OR
CONS NULL) and (NULL-OF (OR CONS NULL)) is NIL, which is a valid list.  
That's why it would be better if there was a distinct "false" value
which wasn't the same as an empty list.

  The way you deal with these things for efficiency is to accept arguments
  of any complex (or ...) type you want, but then you something like this

(typecase <arg>
  (<type-1>
    (locally (declare (type <type-1> <arg>))
      ...))
  (<type-2
    (locally (declare (type <type-2> <arg>))
      ...)))

  This particular situation may actually be pre-optimized by your compiler
  with appropriate locally forms and declarations inserted for you.

| This is all fine in CL already, *except* that a the list type is (OR
| CONS NULL) and (NULL-OF (OR CONS NULL)) is NIL, which is a valid list.
| That's why it would be better if there was a distinct "false" value
| which wasn't the same as an empty list.

  "Better" in the absence of a context or a purpose renders the whole
  statement completely meaningless.  Most of the time, context-free
  "better" simply means "better for me, regardless of consequences or what
  other people need", and such statements should simply be ignored.  I
  would say they are arbitrary (which is even worse and more misleading
  than if they were false) because of the absence of specific meaning.

  I believe the only productive way to learn a new skill is to open one's
  mind to the superior knowledge of those who already know it well and
  really listen to their tales of what they went through to get where they
  are today.  If you come from somewhere else and have a different history
  behind you, whatever you come to will look strange, but if you think what
  you came from must always be more important than what you are going to,
  and some people mysteriously believe this _unconditionally_, it will be
  too hard for them to get into anything new, so they give up, and instead
  go on and on about how wrong what they came to is.  There are immigrants
  in every culture who keep longing for their past and denouncing their new
  living conditions for their entire life, but yet do not return.  I do not
  understand what is so special about what one accidentally met _first_
  that makes everything one meets later on _productively_ judged by it.

  struct foo {int x};
  foo *a = 0;
  foo &b = *a;

  b.x = 13;  // oops!

This can't happen in, say, Dylan because there are no null pointers.  If
you have something like...

  define class foo(<object>)
     slot x :: <integer>;
  end;

  let a :: <foo> = ...;

  a.x := 13;

... then there is guaranteed to be no error in the assignment of 13 to
a.x.

If you actually *want* to have a null value then you do it explicitly:

  let a :: type-union(singleton(#"null-value"), <foo>) = ...;

Conventionally the null value used is always #f -- the canonical "false"
-- and there is a standard function that makes it more convenient to
create these union types:

  let a :: false-or(<foo>) = ....;

  when (a)
    a.x := 13
  end

Furthermore, I have a lot of experience in a language in which a false
value is not a list, so it's hardly armchair musing when I say that it's
a damn good idea to separate the two notions.

Cool.

Don't worry.  CL is totally perfect just as it is.  No other language,
past, future or present ever had any idea better than the ideas already
present in CL.  Where CL is more flexible than other languages it's
because such complexity is necessary.  Where CL is more restrictive than
othe languages it is because more generality is unnecessary.

I get it now.

Everything else is fine.

Furthermore, it's not even a *correct* representation of the situation.  
I learned Lisp 1.5 long before I learned Dylan.  Contrary to your "baby
duck syndrome" supposition, I saw that very many things in Dylan are
done far better than in Lisp 1.5, separation of the concepts of false
and empty list being just one of them.

And that's not even counting the various other journeys.  My first
programming language was FORTRAN IV.  I saw that Pascal was bette, and
moved to that, and then later saw that Modula-2 was better and moved to
that.  Then I learned C and saw that it was better than Pascal but worse
than Modula-2 and stayed with Modula-2.  Then C++ came along and I saw
that it was better than Modula-2 and moved to it.  Then Java came along
and I saw that it was worse than C++ + Boehm so I stayed with C++.  Oh,
and I didn't mention the Data General machine I used in 1985 - 1986
which had only FORTRAN, COBOL and PL/1 available and so I choose to use
PL/1, that being the best of a bad lot.  Or probably another dozen or
more languages along the way that I've learned, evaluated, and either
used or discarded.

One of the things I sense about Lisp is that it has developed for a very
long time under the hands of folks who are insanely obsessed with doing
the Right Thing, and I think a large part of the determination of what
is the Right Thing comes from What Would A Reasonable Person Expect. And
so a Reasonable Person programs Lisp like a hot knife through warm
butter. But I digress.

A good example was my shift from MCL to ACL. My first reaction to ACL
was yechhh. But I figured it was just because I was used to MCL. Sure
enough, I now have no problems with ACL (and would probably have trouble
getting back into MCL).

So the point is, if you think the folks who created X gave it some
thought, take a few months to get into X before you dis it.

What is the fix? Instead of

(defun foo (x)
  (bar (list-or-nil-returner x)))

(defun bar (list)
  (dolist (elem list) (print elem)))

you want to see either

(defun foo (x)
  (let ((result (list-or-nil-returner x)))
    (bar (if (valid-p result) ;; Could be only result if NIL != ()
             result
             '()))))

(defun bar (list)
  (dolist (elem list) (print elem)))

or

(defun foo (x)
  (bar (list-or-nil-returner x)))

(defun bar (list-or-nil)
  (let ((list (if (valid-p list-or-nil) list-or-nil '())))
    (dolist (elem list) (print elem))))

I think the first replacement is the better one, if bar can be
specified to just handle lists.  But the fact of the matter is that
the original code is completely equivalent to that replacement, not
only in effect, but also in communicated intent.

So your experience with Dylan can support the claim "There is at least
one language where completely separating false and every other
datatype worked out great".  Since I don't have enough experience with
Dylan, I can neither refute nor confirm that claim for Dylan, but I
have used a number of other programming languages that did this, and
didn't find any problems with this approach.

What your experience doesn't support is the claim "(eq NIL ()) is a
problem in Common Lisp", or the broader claim "Identifying the false
value and some value of another datatype is wrong or problematic in
any language".

What my experience with CL does support is that (eq NIL ()) is not a
problem I have ever encountered in serious use of Common Lisp:

If I were to start programming in Dylan, and stumbled across the
"problem" that Dylan seals its GFs by default, since I, not having
read anything about Dylan and blissfully assuming that this works
just like in CL, ran into large numbers of errors.  Now tell me, is my
having problems indicative of Dylan having a problem, or is it not
indicative of _me_ having a serious problem?

So we are not speaking about normal users having problems, we are
speaking about misguided individuals having problems.

<quote hyperspec>
  loop [name-clause] {variable-clause}* {main-clause}*
</quote>

Since termination tests are part of the main-clauses, you can't write

(loop for x = (foo ...)
      while x
      for y = (do-something-with-non-null x)
      for z = (bar y)
      do
      ...)

This in itself causes one to write more convoluted code in such
situtions, especially if one can't fold the binding of y and z into a
LET construct, because one wants to use them in further loop clauses.

One of the better possibilities is

(loop for x = (foo ...)
      for test = (unless x (loop-finish))
      for y = (do-something-with-non-null x)
      for z = (bar y)
      do
      ...)

[Thanks Kent for reminding me about this possibility]

As I think you'll agree this is ugly, and non-obvious.

Furthermore this problem is aggravated by the fact that nearly all
LOOP implementations _do_ support this kind of intermingling, but
often without specifying the exact semantics of such a thing.  This
leads many people astray, blissfully assuming that it is well
specified portable code they are writing, until some new LOOP
implementation coughs at their code, hopefully producing warnings, but
possibly producing false iteration code instead.

And it isn't a problem that only seriously misguided people
encounter.  Even people well versed in the language make that slip
from time to time, as can be seen by the fact that Kent posted an
erroneous example only recently.  Same thing happened to me lots of
times, until I have become paranoid enough that I check every loop
form I stumble across for that mistake.  This is clearly an indication
that something is amiss here.

How to solve that problem is another question, of course.   Reopening
the standard is out of the question for some time to come, so some
other solution will have to be found.

That is IMHO completely wrong.  Many, many decisions of languages have
to be judged subjectively by that language's community.

For example, Dylan is more static than CL in a number of areas, and
defaults to the static choice in a number of places (e.g. sealing GFs
by default).  So tell me, which is the better language, which made the
better choices?  I don't think this is a sensible question.  I think
CL made the correct choices with regards to the CL community, and
Dylan did so with regards to the Dylan community.  Neither is a priori
better than the other, they are different, because the members of
their communities value things differently, and likely also write
different programs.

That doesn't mean that it isn't possible to criticize a language, it
just means that you have to do it from the point of view of someone
writing non-trivial programs in it.

What would you think of a German, who came to England, and started to
criticize the English language based on the little he knows about it,
and the many great ideas that German qua language embodies?  The fact
that you can't just directly transplant language features from German
to English, and that things that would have been problems in German
aren't in English (and vice-versa), doesn't mean that one can't engage
in informed criticism of the language, it just means that one has to
inform oneself about the language, as used by the people, before one
can do so.

Regs, Pierre.