I've recently come to notice strange EVAL behaviour in several Common
Lisp implementations, particularly, CMU Lisp (on a DEC Alpha), CLisp
and GNU CL (both under Linux on an x86 machine).  For instance:

* (let ((x 1)) (eval 'x))

In: LET ((X 1))
  (LET ((X 1))
    (EVAL 'X))
Warning: Variable X defined but never used.

Error in KERNEL::UNBOUND-SYMBOL-ERROR-HANDLER:  the variable X is unbound.

Restarts:
  0: [ABORT] Return to Top-Level.

Debug  (type H for help)

(EVAL X)
0]

This seems counterintuitive to me.  Is EVAL just using the global
environment?  Is that what the language spec says it should be?  If
that is the case, how could the following behaviour be explained:

* (defvar x 100)

X
* (let ((x 1)) (eval 'x))

1
* x

100
*

Note that the eval only seems to work after a *GLOBAL* variable 'x'
has been defined.  I consider this very strange, but it seems very
improbable that the authors of these implementations would overlook
such an obvious bug.  So, what's the explanation?

In article <4utk8d$...@net.auckland.ac.nz> mcha...@cs26.cs.auckland.ac.nz (Malcolm Chiang Yoen Chan) writes:

   This seems counterintuitive to me.  Is EVAL just using the global
   environment?  Is that what the language spec says it should be?

Not exactly, it is using the null lexical environment and the current
dynamic environment.

Malcolm Chiang Yoen Chan asks whether EVAL is just using the global
environment, and the answer is yes, it does, and that's what what the
language specification says it should do.

http://www.harlequin.com/books/HyperSpec/Body/fun_eval.html

Malcolm then asks about (let ((x 1)) (eval 'x)) after (defvar x 100).  as
the above web page will tell you, `eval' "evaluates the form in the current
dynamic environment and the null lexical environment".  `defvar' makes the
variable dynamic.  `let'-bindings on dynamic variables do not make them
lexical.

hope this helps.

Tim Chippington Derrick asks for *good* reasons why `eval' does not access
the lexical environment of the calling function.  I think it would be
illuminating to think of what it would take to actually access it.  hint:
`eval' is a normal function call, and lexical environments are not passed
in functions calls -- that's _the_ point.

Tim also says that it "does sort of feel counterintuitive".  I have had
many occasions to upgrade my intuition when working with Lisp.  once, I
comforted a friend who was very frustrated with `eval-when', and said that
it becomes intuitive only when it has ceased to be an impenetrable mystery.

In article <4utk8d$...@net.auckland.ac.nz>,
Malcolm Chiang Yoen Chan <mcha...@cs.auckland.ac.nz> wrote:

    ? (defun foo () x)
    ;Compiler warnings :
    ;   Undeclared free variable X, in FOO.
    FOO
    ? (let ((x 1)) (foo))
    ;Compiler warnings :
    ;   Unused lexical variable X, in an anonymous lambda form.
    > Error: Unbound variable: X
    > While executing: FOO

If you really need to get the value of X from inside FOO, you can
make FOO a macro.  This works because the compiler actually places
the macro's code inside of the LET form, rather than just calling
the code from inside the LET form:

    ? (defmacro foo () 'x)
    FOO
    ? (let ((x 1)) (foo))
    1

Lexical scoping is important, not only for making compilers efficient
but also for certain kinds of reasoning about programs.  For example,
consider the following simple version of MAPCAR:

    (defun mapper (f l)
      (do ((in l (rest in))
           (out nil (cons (funcall f (first in)) out)))
          ((endp in) (nreverse out))))

Note that it uses the destructive (and normally dangerous) function
NREVERSE on the value of OUT.  This can be done with absolute
safety, because the binding of OUT is lexical and therefore the
function F can't access OUT, not even if F makes use of EVAL.

    ? (setq x 100)
    100
    ? (let ((x 1)) (eval 'x))
    ;Compiler warnings :
    ;   Unused lexical variable X, in an anonymous lambda form.
    100

Robert Munyer says that `setq' at top-level establishes a global variable
without making it special.  there is really no such thing.  a variable is
either lexical or it is not.  the "not lexical" in Common Lisp is
"special".  for more information on the environments in Common Lisp, see
the Common Lisp Hyperspec at Harlequin, Inc, section 3.1.1 Introduction to
Environments.

    http://www.harlequin.com/books/HyperSpec/Body/sec_3-1-1.html

Erik Naggum wrote (15-Aug-96 13:00:12):

        First of all, I'd like to ask anyone to explain what are those dynamic
and lexical enviroments. The idea that I have is that the lexical enviroment
includes local variables created by "let" and lambda variables.

        In this perspective I understand that a (let ((x 1)) ...) does some
sort of a local (setf x 1). If x already exists, it just sets it, otherwise it
creates the variable. However this is somehow wrong, since
                (setf y 123)
                (let ((y 1)) (eval 'y))
returns 123 instead of 1. Therefore there must be a difference between x and
y. This is confirmed by a (describe 'x) that returns
                This is the symbol X, a variable declared SPECIAL.
and a (describe 'y) returns
                This is the symbol Y, a variable.
I suppose that these different variable status is the key to the problem. Yet
I don't like to see exceptions in a apparently "clean" language as LISP was
supposed to be. In a general way, I guess that any exception to a rule is
dangerous.

        Regards,

--

Trying to discuss these issues without terminology misunderstandings
is a bit like trying to walk on eggshells without breaking them.
I think part of the problem is that it's very difficult to get
everyone using the same definition for "variable."  Since you
referred to the HyperSpec, I'll use its definition for now, so that
we'll be able to communicate without misunderstandings.  But at
the end I'll explain why I'm not really fond of that definition.

The HyperSpec says that "variable" just means "variable binding."
Using that definition:

In article <3049395203874...@arcana.naggum.no>,

The point I was making for Mr. Chan is that DEFVAR has two different
side effects: (A) it assigns a global value to a name, and (B) it
also declares that all uses of that name will be dynamic.  Mr. Chan
observed an effect that DEFVAR had on his code, which he thought
was due to A; I was explaining that it was really due to B.

Having said all that, I should point out that the HyperSpec's
definition of "variable" as meaning "variable binding" is not
exactly standard.  It doesn't mesh well with the following from
CLtL2 section 5.1.2:

    The value a special variable has when there are currently no
    bindings of that variable is called the global value of the
    (special) variable.  A global value can be given to a variable
    only by assignment, because a value given by binding is by
    definition not global.

Thus using CLtL2 terminology, the following code:

    (defvar x 3)
    (+ x x)

does not involve any bindings.  So if (as defined by the HyperSpec)
a variable is just a variable binding, then the code above does
not involve any variables!

I don't blame the authors of the HyperSpec for this; I'm not sure
it's even possible to come up with consistent definitions that
don't conflict with CLtL2.

While I'm on the subject, here is something else I don't like about
the HyperSpec definition of "variable:"

    (terpri *query-io*)

I don't like to say that the above code refers to different variables
at different times, depending on whether *query-io* has been rebound.
I would rather say that it refers to the same variable every time
it is called, but not necessarily to the same binding.  I can't
say this if "variable" and "variable binding" are the same thing.

I suppose if I want to say that and be consistent, I will probably
end up having to espouse the following definition of "variable:"

  * Each individual lexical binding is an individual variable.
  * All dynamic bindings which share the same name, along with
    the global value assigned to that name, together refer to a
    single dynamic variable.

In other words, a program can have many lexical variables named X
but only one dynamic variable named X.

But given all the confusion, perhaps it would be better just not
to use the word "variable" without qualifying it, e. g. "variable
name," "variable binding," "variable reference," etc.

Well, there it is.  Sorry it got so long.  I guess I got dragged
into something much bigger and more involved than what I expected
when I first replied to Mr. Chan.

Comments, anyone?

Robert Munyer offers the following pearl for our enjoyment: "I should point
out that the HyperSpec's definition of "variable" as meaning "variable
binding" is not exactly standard."  matter of fact is: it _is_ exactly
standard.  the HyperSpec contains the text from ANSI X3.226-1994.

I don't think there is anything I can do to help Robert with the confusion
over environments, since he seems to ignore what I have already said, so
I'll leave it at that.

You're confused because the word "binding" is used in a number of different
ways in CLtL and the Common Lisp spec.

As a noun, "binding" means any association between a name and its value.

As a verb, "to bind" usually refers to establishing a *local* binding.

Thus, (defvar x 3) creates a binding, but we don't say that the value was
given "by binding".  (let ((x 3)) ...) gives the variable a value by
binding.

]I don't blame the authors of the HyperSpec for this; I'm not sure
]it's even possible to come up with consistent definitions that
]don't conflict with CLtL2.

The HyperSpec makes no attempt to be consistent with CLtL2.  The HyperSpec
contains the text of the ANSI specification.

]While I'm on the subject, here is something else I don't like about
]the HyperSpec definition of "variable:"
]
]    (terpri *query-io*)
]
]I don't like to say that the above code refers to different variables
]at different times, depending on whether *query-io* has been rebound.
]I would rather say that it refers to the same variable every time
]it is called, but not necessarily to the same binding.  I can't
]say this if "variable" and "variable binding" are the same thing.
]
]I suppose if I want to say that and be consistent, I will probably
]end up having to espouse the following definition of "variable:"
]
]  * Each individual lexical binding is an individual variable.
]  * All dynamic bindings which share the same name, along with
]    the global value assigned to that name, together refer to a
]    single dynamic variable.

If we were defining the terms using more formal, computer science terms, we
would say that a binding refers to the association between a name and a
location where a value can be stored.  When a local binding is established,
a new association is created.

Rodrigo Ventura also asks about environments.  I posted the URL to the
Common Lisp Hyperspec the other day, and it bears repeating:

    http://www.harlequin.com/books/HyperSpec/Body/sec_3-1-1.html

this section contains what you should need to know about environments.

In article <4vahg9$...@tools.bbnplanet.com>,

I hope this doesn't sound petulant, but...  Please re-read your
article carefully.  The word "bindings" in the second line of the
excerpt from CLtL2 is clearly a noun.  I was not confused about
any of this [*]; I was merely pointing out a small disagreement
between CLtL2 and the HyperSpec.

[*] Actually I might be confused about the authorship of the
HyperSpec.  It is not clear to me exactly how much difference there
is between the HyperSpec and the ANSI standard.  The HyperSpec
itself only claims that it is "derived from" the ANSI document, so
I thought it might include some original material not from ANSI
(just as Franz Inc.'s CLtR was "derived from" CLtL, but included
original material authored by Franz).

At any rate, I regret having implied that CLtL2 is somehow "more
standard" than the HyperSpec.  I hereby make the following correction
to my article of 8/19/96: replace the words "exactly standard" with
the words "consistent with CLtL2."

In article <3049468889902...@arcana.naggum.no>,

  "The Common Lisp HyperSpec is not the ANSI Common Lisp standard"

When I noticed that the HyperSpec was inconsistent with CLtL2, I
thought it would be a good idea to attribute the inconsistency to
the document where I actually saw it, rather than to the document
from which it was "derived."

I haven't seen the non-HyperSpec version of the ANSI document, but
I will take your word for it that the definition of variable is
the same in both.  So, OK, I will use that definition.  And I regret
having implied that CLtL2 is somehow "more standard" than the
HyperSpec.  I hereby make the following correction to my article
of 8/19/96: replace the words "exactly standard" with the words
"consistent with CLtL2."

If you think I said something wrong, please tell me what it is;
don't just hint that I am wrong by saying that I "seem to ignore"
you.  I haven't ignored anything you have said.  If there is
something I don't understand, I want to be corrected.  But you
haven't corrected me or supplied any specific details; you have
only posted a vague hint that I am wrong.  Hints won't do me (or
anyone else) any good.

You seem to have in mind a specific misconception under which you
think I am laboring.  If so, please state this misconception
explicitly -- or better yet, post a snippet of code that you think
I will interpret incorrectly.

Just to make this absolutely explicit:  I believe my articles of
8/18 and 8/19 were 100% correct if you make the following changes:
  * 8/18: change "it" to "its name."
  * 8/19: change "exactly standard" to "consistent with CLtL2."

I don't think you have said anything that conflicts with this
belief.  If you (or anyone else) disagree, please tell me why.

Rodrigo> I don't like to see exceptions in a apparently "clean"
Rodrigo> language as LISP was supposed to be. In a general way,
Rodrigo> I guess that any exception to a rule is dangerous.

LISP is a language family, not a language.  It accomodates many points
of view, even those that are mutually exclusive, by allowing different
dialects which permit and rule out various things.  Beware of speaking
overbroadly about what "it" is, or of inferring what any dialect's
goals were without doing a historical search to find that out.

DEFVAR proclaims a name SPECIAL.  When proclaimed SPECIAL, the effect
is pervasive (affects bindings of that name).  Some of the reason for
Common Lisp's semantics are historical, having allowed for the
evolution of the language and some of its large programs from a time
when there was no lexical binding at all.

Common Lisp is a commercially powerful dialect--its goal is to work
reliably, predictably, and usefully.  It does these things.  Aesthetics
are important to the CL community, but they are formally (as a matter of
design principle) secondary/subordinate to practicalities such as cost
of change of existing compilers or user programs.

That Common Lisp is sometimes messy at the edges is a sign of the fact
that people have used it enough to care that their programs not be
broken over time.  The design of Common Lisp was done with the
assumption that there were people in the community who used the
language it referred to and who would want to continue throughout the
long process to be using that language without undue economic
hardship.  Even small changes to a language can mean many $10K's of
real dollar cost to a company to hire a consultant to fix old code, and
we tried in our work to be mindful of this.

Indeed, languages that can freely tolerate arbitrary changes for the
sake of being "elegant" or "clean", no matter how many (or how few)
users are affected will have trouble attracting a certain commercial
community to rely on them as a stable base for building programs that
will last into the future.  (They may indeed find OTHER commercial
communities that prefer aesthetics and tolerate breakage that occurs
toward that end, but that is not a disproof of my claim that not ALL
companies want aesthetics over all else.)

Professional CL programmers will tell you that good style is to ALWAYS
put *'s around the name of a variable you will proclaim SPECIAL
(whether explicitly or through DEFVAR or DEFPARAMETER).  In that way,
you will always be reminded when binding it that you are going to get
a SPECIAL, and you effectively (though not formally) get a separate
namespace.

You might ask why not just make it a separate namespace.  The answer is
in the history...

First, consider the printer control variables *PRINT-LENGTH* and
*PRINT-LEVEL*.  It would be quite cumbersome in practice to have to
bind them by

 (LET ((*PRINT-LENGTH* 3) (*PRINT-LEVEL* 3))
   (DECLARE (SPECIAL *PRINT-LENGTH* *PRINT-LEVEL*))
   ...)

to have the binding "take the obvious effect".  Program errors would
just about always be indicated if someone just wrote

 (LET ((*PRINT-LENGTH* 3) (*PRINT-LEVEL* 3))
   ...)

and if the semantics were not to produce a special binding automatically.
Yes, it's true, we could have made a DYNAMIC-BIND form like EuLisp later
did.  One reason, for better or worse, that we did not is that it made
the transition from older dialects of Lisp more difficult.  It used to be
in some older dialects that all variables were defaultly SPECIAL.

That brings me to my second example.  Many people bridged the gap by
adding a few PROCLAIM's at the front of their program and then writing
"the old way".

 (PROCLAIM '(SPECIAL A B C D))

 (DEFUN FOO (A B C D)  ...)

Then, one-by-one as they could prove that A,B,C, and/or D was not
needed to be special because they'd fully tracked the data flow and
seen that no special references were still being made, they could
simply remove the SPECIAL proclamation for that one ... until one day
their program ran with no special proclamations.  If instead CL had
REQUIRED a new form, like DYNAMIC-BIND, to do special binding,
it would have meant all such forms would have been rewritten AND
re-shaped, as in:

 (DEFUN FOO (A0 B0 C0 D0)
   (DYNAMIC-BIND ((A A0) (B B0) (C C0) (D D0))
      ...))

This would have made code hard to read including even simpler things
like pushing things farther and farther to what might have been an
already cramped right margin in the editor (a practical consideration
we've all struggled with in the Lisp community).  Also you'd have had
to do something even more complicated to have transitioned old code
that did

 (PROCLAIM '(SPECIAL A B))

 (DEFUN FOO (A &OPTIONAL (B (F))) ...)

where (F) might have referred to the SPECIAL variable A, and where:

 (DEFUN FOO (A0 &OPTIONAL (B0 (F)))
   (DYNAMIC-BIND ((A A0) (B B0))
     ...))

was NOT a correct rewrite because (F) would happen before A was bound
as a dynamic variable and since F might refer specially to A, you
needed to arrange for the A to be bound before F was called.  To
correctly rewrite such a function using DYNAMIC-BIND would have
required:

 (DEFUN FOO (&REST ARGS)
   (LET ((N-ARGS (LENGTH ARGS)))
     (UNLESS (OR (= N-ARGS 1) (= N-ARGS 2)) (ERROR ...))
       (DYNAMIC-BIND ((A (POP ARGS)))
         (DYNAMIC-BIND ((B (IF (= N-ARGS 2) (CAR ARGS) (F))))
            ....))))

Like it or not, this degree of reshaping would not have supported the
smooth transition of old programs into the future at the time lexical
scoping (an undeniably good idea we had to implement, in spite of the
cost) was introduced.  So CL adopted the convention of allowing one to
individually declare variables locally or globally in various ways that
supported the transition we were undergoing.

Of course, the real power of CL is that if you don't like what it
gives you, you can pretty much make your own way of doing things. e.g.,

 (DEFMACRO DYNAMIC-BIND (VARS-AND-BINDINGS &BODY FORMS)
   `(LET ,VARS-AND-BINDINGS
      (DECLARE (SPECIAL ,@(MAPCAR #'CAR VARS-AND-BINDINGS)))
      ,@FORMS))

 (DEFMACRO DYNAMIC (NAME)
   `(LOCALLY (DECLARE (SPECIAL ,NAME)) ,NAME))

and then just always use DYNAMIC-BIND to bind specials and DYNAMIC to
reference them.

If you do this, then you won't get any surprising effects with EVAL
because only the things you dynamic-bound will be visible to EVAL.

Or if you use the standard convention of putting *'s around special
variable names, you won't get surprising effects either.  No experienced CL
programmer is surprised by

  (LET ((*PRINT-LEVEL* 3)) (EVAL '*PRINT-LEVEL*)) => 3

because they know this is a dynamic binding.  They ARE surprised by

  (LET ((X 3)) (EVAL 'X)) => 3

Robert Munyer seems to want to maintain that CLtL2 is his authority in the
presence of ANSI Common Lisp, and he seems to take delight in the legalese
phrase where the HyperSpec says that is not the Real Thing, which is just
saying that "if you need to be absolutely certain (probably because you're
a lawyer), you have to buy the Real Thing for USD 350 from ANSI".  if the
HyperSpec had not had that disclaimer, somebody could be in trouble for
trying to be more authoritative then they were.  in other words, if some
electronic document says it's not the real thing, you can bet several
kilobits per second that it it's exactly the same as the official document,
except you won't be able to say it is in a court of law.  arguments to the
effect that people write their own stuff into standards documents are
specious at best.  I cannot think of anybody who would actually do that --
temporarily ignoring the immense stupidity of such an undertaking, it would
be so easy to prove that tampering had taken place and the credibility of
whever did it would be destroyed right there.  such tampering is much
harder to spot in paper standarsd, except that they can ordered fresh from
some authority.

note that CLtL2 was not an official document; CLtL1 was; ANSI X3.226-1994
Common Lisp is.  the HyperSpec at Harlequin is Kent Pitman's formatting
into HTML from the exact same source text that went into the final text of
the standard that was later published by ANSI.  Kent Pitman was the editor
of the standard.  see the Credits section in the standard or the HyperSpec.

let's leave this much ado about nothing to the lawyers should the need to
compare the HyperSpec to the Real Paper Thing come up.  (part of) Robert's
answer was wrong, relied on implementation specifics at best, and is no
better off after his attempted rescue missions.  end of story.

# Note to comp.lang.lisp subscribers: if any of you are still
# reading this thread, the "good stuff" (i. e. real technical
# discussion) in this article is at the bottom.  You may want
# to skip the first half.

In article <3049576919413...@arcana.naggum.no>,

Because of this I carelessly implied that the HyperSpec definition
was somehow less standard than the one used by CLtL2.  People kindly
corrected me and I retracted that statement about 26 hours after
I posted it.  You posted your snide and disingenuous reply AFTER
my retraction.

We might as well stop talking about this, because it really has
nothing to do with the subject of this thread.  The only reason I
brought up the authorship of the HyperSpec was to discuss the
meaning of the word "variable," and the only reason I brought up
the meaning of the word "variable" was that I thought the difference
in definitions might be the reason for your complaint about my
original article.  But it turns out that that was NOT the reason,
because you replied that I was ignoring you.  Therefore discussion
of the meaning of "variable" is just a useless side issue, and
discussion of the authorship of the HyperSpec is just a side issue
on a side issue.  So let's get back to the main issue, now that
I've found that the meaning of "variable" has nothing to do with
your complaint.

A pattern seems to be emerging here.  Every time I try to get you
to be more specific, you become even more vague.  On 8/18 I posted
an article to explain scoping to Mr. Chan; you jumped in and said
I was wrong, and mentioned a URL to a HyperSpec page that does not
conflict with what I wrote.  On 8/19 I tried to find out if your
unspecified complaint had something to do with the definition of
"variable;" you replied with a vague comment that I was "confused
about environments" and "ignoring what you already said."  On 8/20
I challenged you to (finally) make your point, with a specific
complaint or better yet code snippets.  You replied with the
amazingly evasive comment above: "(part of) Robert's answer was
wrong..."  How much more vague could you possibly get?

A pattern of increasing evasiveness is a sign of a person who
doesn't have the slightest idea what he is talking about.  I hate
to say anything insulting like that, but you forced me into it.
If you don't want me to say things like that about you, then just
quit evading and (at long last) make your point.

Please, respond to the challenge at the bottom of my 8/20 article.
Do not simply evade it as you did on 8/21.

Or better yet, I will put a new challenge below.  Please respond
to this one.  If you ignore both challenges again, and reply as
you did before, that will make two cycles in a row in which you
have refused to answer my questions.  If you do this I will have
to assume that you have some ulterior motive for refusing to answer.
I know it's not nice to say that about someone, but how else could
I explain your refusal?

Below I will give you a DAG of numbered statements, all of them
explaining a difference between SETQ and DEFVAR.  Most of these
statements include "pointers" to other statements, from which I
claim they follow necessarily.  For each of these statements, please
tell us whether you agree or disagree.  If you choose to dispute
some statement N but not to dispute statement M from which I claim
N follows, then you must dispute my claim that N follows from M.
And you must dispute statements #1 and #2, if you wish to justify
your articles of 8/18 and 8/19 respectively.

Statement #1 is my original statement, to which you objected.  I
claim it is technically correct, but it is not as well phrased or
as meaningful as I would have liked, because I was being a little
careless in my use of the words "variable" and "it."  After all,
I was just explaining EVAL and scoping to a beginner.  A more
reasonable reader probably would have understood what I meant by
reading the code snippet.

Statement #2 is my original statement after I corrected it on 8/19.
(Apparently you still object to it in this corrected form, because
on 8/19 you said I was ignoring what you said on 8/18, and on 8/21
you said my "rescue missions" were unsuccessful.)

I have prepared this DAG in excruciating detail, with very fine
grain, because I am trying very hard to get your complaint narrowed
down to something that is explicit enough to be worth discussing.

1. Look what would happen if you used SETQ rather than DEFVAR, to
   establish a global variable without declaring it to be special:
     [In the context of my original article, and using my old
      (informal and outdated) definition of "variable," my intention
      was for statement #1 to follow from #2 or #3.  Using the
      standard definition of "variable" (on which we now all agree),
      #1 is still technically correct because it follows from #6.
      The binding created by statement #1 is special, but it's not
      special by declaration; it's special simply by lack of an
      enclosing lexical binding.  Yes, I know I'm being a bit of
      a Philadelphia lawyer here, but you forced me into it.  And
      remember, to justify the abusive articles you have posted,
      you must dispute statement #2 below as well as #1.]

2. Look what would happen if you used SETQ rather than DEFVAR, to
   establish a global variable without declaring its name to be
   special: [in the context of my original article, this statement
   follows from statement #3].

3. When you use SETQ at the top level on a previously unused variable
   name, you are not declaring that variable name to be special.
   [this follows from statement #4 or #5].

4. When you use SETQ at the top level on a previously unused variable
   name, you are not declaring that variable name to be special,
   and you will be able to use that name for lexical variables.

5. When you use SETQ on a variable name, you are not declaring that
   variable name to be special.  [this follows from statement #6].

6. When you use SETQ on a variable name, you are not declaring
   anything to be special.  [this follows from statement #7].

7. When you use SETQ on a variable name, you are not declaring
   anything at all.  [this follows from statement #8].

8. When you use SETQ you are not making a declaration.  Unlike
   DEFVAR, SETQ does not include an implicit PROCLAIM.

                    2---3---4
                         \
                          5---6---7---8
                             /
                            1

[ Yes folks, I know this is a ridiculous level of detail.  But I
  can't think of any other way to narrow down the complaint and
  finally get an answer out of this guy. ]

This article is a final attempt to get you to say something useful.
If you brush this one off too, I will just have to assume that you
are a "flame baiter" who likes to start arguments but is not
interested in serious discussion.

-- Robert

P. S.  Of course, others are welcome to contribute.  I know there
are plenty of geniuses who read this newsgroup.  If I have really
made a mistake in this chain of reasoning, perhaps one of you will
step forward and explain where and why.  I promise I won't flame
you, and even graciously accept being wrong, if you will just be
reasonably explicit rather than saying something like "duh, part
of it is wrong."  As I said before, snippets of code which demonstrate
your point are especially welcome.

That was a great article.  Thank you for writing it.

I have one small comment to add:

In article <KMP.96Aug20222...@romulus.harlqn.co.uk>,

If Scheme eventually starts to get "messy at the edges" because of
the quantity of existing code and the high cost of changing it,
anyone in the Lisp community can easily create "Yet Another Little
Lisp" and experiment with new elegant features.

In article <4ve824$...@tools.bbnplanet.com>,

Robert Munyer's exceedingly lengthy diatribe seems to hinge on one issue:
whether the semantics of `setq' at top-level of a variable that has not
been declared special is defined by Common Lisp.  I can't see that it is.
I'd like Robert to show me how he found the semantics of this to be defined
by the language specification since he makes so many conclusion based on
his assumption that its semantics _is_ defined.

barring any overlooked clauses which Robert will bring to my attention,
whatever it is that he's trying to distinguish between `setq' and `defvar'
at top-level is purely a matter of how _implementations_ differ in
implementing undefined semantics.

I'm sorry to see that Robert incessantly perceives any pointers to a
specification that does not support his claims, together with a refusal to
speak on behalf of or for those who defined the standard, as "vagueness".
I do not find it to be my job to over-interpret a standard in the face of
unwillingness, or even lack of interest, to read it.  If a standard cannot
speak for itself, it should be improved until it does, but it is hopeless
to attempt to let things speak for themselves if the audience does not
share its context.  I specifically do not claim that I understand the
totality of Common Lisp, which is why I point to authoritative sources that
I find to contradict non-authoritative examples that do not appears to show
an interest in the language, but rather in implementations, which concern
me only inso far as they implement the specification.  otherwise, they have
bugs that should be reported.  if legitimate disagreement over some clause
in the standard is found, the right thing is to submit a request for
clarification to ANSI about it -- there are procedures for this, and the
committee should be prepared to handle such requests.  a prerequisite to
all this is of course that all parties involved in the disagreement know
precisely what the standard says on the issue (or not, as the case may be).

finally, this is not a question of whether _I_ agree or disagree with
anything Robert says or implies.  the question is: is the basic assumption
on which Robert seems to build his entire case _valid_ in the context of
the language definition?  if not, his case be dismissed.

I was trying to point Robert in the direction of realizing that he had no
foundation for what he was claiming with such authority.  I still see no
foundation.  I see no realizing he has no foundation, either, and that is
what really irks me in this case.  to have the gall to conjure up some
elaborate "DAG" of arguments when his basic assumption is in question!

Robert, the HyperSpec is there for your perusal.  I suggest you use to
support your case.  I regret the need to play standards lawyer with you,
but you compel me to it.  (compare that with what you feel forced to.)

the question is: does ANSI Common Lisp (or CLtL2 for that matter) define
the semantics of `setq' at top-level, including whether the symbol should
be declared or proclaimed special?  this is a very specific question, and I
will hear no more about Robert's delusions of "vagueness" in response.

In article <3049754333170...@arcana.naggum.no>,
Erik Naggum  <e...@naggum.no> wrote:

The most common behavior is to treat any top-level form that contains a
free reference to an undeclared variable as if it were locally declared
special, i.e.

(setq foo 3)

is treated as

(locally (declare (special foo))
  (setq foo 3))

When they do this, compilers generally print a warning, e.g. "Free variable
FOO being declared SPECIAL".

As an aside, Maclisp, the Lisp dialect that CL was mostly derived from, had
the behavior that a free reference caused the variable to be declared
globally special thereafter.  This was because lexical variables only
existed in the compiler; the interpreter only supported dynamic scoping.
Automatically declaring variables special was a heuristic to deal with the
fact that programmers frequently failed to put special declarations in
their code.  You could still be screwed if code that binds the variable
textually precedes the code that contains the free reference, e.g.

(defun foo1 ()
  (let ((*x* 3))
    (bar)
    *x*))

(setq *x* 0)

(defun bar ()
  (incf *x*))

(defun foo2 ()
  (let ((*x* 3))
    (bar)
    *x*))

Now, if someone claims that the ANSI spec, or the HyperSpec, is
inconsistent with CLtL 2nd edition, that's not very surprising in
itself.  CLtL 2 is, in effect, a snapshot of how things stood
at a certain point during the development of the standard.
The ANSI standard reflects a later point (indeed, the final
point, for that standard) and is (at least largely) separately
written text.

But is the "inconsistency" being discussed here among the actual
inconsistencies between CLtL 2 and the standard or HyperSpec?
If someone wants to show it is a real inconsistency, then, given
that the claim seems to be controversial, I wish they would do
it as follows:

  Say something like "CLtL 2 says the following is true:", followed
  by some text, which I'll call T.  Then present the evidence that
  CLtL 2 says T is true.  Finally, present the evidence that the
  standard (or the HyperSpec) says T is false.

When doing this, equivocation is _not_ allowed.  You're not to
give any word in T one meaning when arguing that CLtL 2 says T
is true and a different meaning when arguing that the standard
or Spec says T is false.

Another claim we've seen in this discussion is that Common Lisp
does not define semantics for global variables that have not been
introduced by defvar (or something like that).  If so, it's an
oversight and, in effect, an error in the standard.

To perhaps save time, let me make it clear that the only thing that
would convince me (as a former member of X3J13, etc) that it was
not an oversight (if the standard somehow fails to specify the
semantics) would be an explicit statement in the standard
that the semantics was not defined or an explicit startement in
something passed by X3J13 (and not modified or overturned later
on) that said the semantics was not defined.

  A dynamic variable can be referenced at any time in any program;
  there is no textual limitation on references to dynamic variables. At
  any given time, all dynamic variables with a given name refer to
  exactly one binding, either in the dynamic environment or in the
  global environment.

  ...

  A dynamic variable can be referenced outside the dynamic extent of a
  form that binds it. Such a variable is sometimes called a ``global
  variable'' but is still in all respects just a dynamic variable whose
  binding happens to exist in the global environment rather than in some
  dynamic environment.

Yet in the glossary we find:

  dynamic variable n. a variable the binding for which is in the
  dynamic environment. See special.

Nothing about the global environment _there_.

3.1.2.1.1.2 also says

  A variable is a dynamic variable if one of the following conditions hold:

    * It is locally declared or globally proclaimed special.

    * It occurs textually within a form that creates a dynamic binding
      for a variable of the same name, and the binding is not shadowed[2]
      by a form that creates a lexical binding of the same variable name.

The 1st item just above suggests that Eric may be right, so far as the
ANSI standard is concerned.  However, "if" is not the same as "only if"
or "if and only if", so perhaps things aren't as clear here as they
might be.  "No textual limitation" and the (so far as I know) lack
of anything that says the semantics are _not_ defined, suggests that
free refs are refs to global special variables.  (N.B. special but
not _proclaimed_ special, an important distinction.  Special
proclamations make all bindings of the name special, declarations don't.)

In 3.1.2.1.1, Symbols as Forms, we find:

  If a form is a symbol that is not a symbol macro, then it is the
  name of a variable, and the value of that variable is returned. There
  are three kinds of variables: lexical variables, dynamic variables,
  and constant variables. A variable can store one object. The main
  operations on a variable are to read[1] and to write[1] its value.

There's no hint here (or anywhere else that I've found) that's
another case: variables that aren't lexical or constant yet not
dynamic either because there's hasn't been a special proclamation
or declaration.  Moreover, the description of setq uses examples such
as the following without any declarations or proclamations:

 ;; A simple use of SETQ to establish values for variables.
 (setq a 1 b 2 c 3) =>  3
 a =>  1
 b =>  2
 c =>  3

CLtL 2 is perhaps clearer.  Section 5.1.2 (p 70) says

  ... if the symbol occurs textually within a program construct
  that creates a _binding_ for a variable of the same name, then
  the reference is to the variable specified by the binding; if
  no such program construct textually contains the reference,
  then it is taken to refer to the special variable of that name.

The description of setq (p 121) is less clear than it might be,
since it says only that "the usual rules" determine whether
the variable being assigned to is lecical or dynamic.  But I
think it's fair to conclude at least that there's not meant
to be a difference between the rules for assignment and the
rules for reference.

Section 7.1.1, Reference (p 114) says that whether the variable
is special or lexical depends on the presence of absence of a
special declaration; but I don't think that is meant to contradict
page 70.

However, the situation is muddued by the pactice in many
implementations of giving warnings on references to undeclared
free variables.  Language such as "assumed to be special" is
often used.  This is because implementors suspect that most
such cases are unintentional and indicate mistakes in the code.

* (setq x 1)
Warning:  Declaring x special.
1

* (setq x 2)
2

* (defun fx () x)

* (fx)
2

For instance, from PROCLAIM-LEXICAL version 2:

Description of problem:

  CLtL pp. 55-56 implies that if a name (symbol) is not proclaimed or
  declared special, then a free reference to that name is a reference to
  the special variable of that name, while a LAMBDA-binding of that name
  indicates a binding of the lexical variable of that name.  This would
  mean that the following program is legal and that (TST) => 4:

    (defun tst ()
      (setq x 3)
      (funcall (let ((x 4)) #'(lambda () x))))

  However, if you feed this program to many Common Lisp compilers
  (including Symbolics's and DEC's), a warning message will be
  produced for the SETQ, saying something like "Warning: X not
  declared or bound, assuming special."

  These warnings, unlike the annotations of undefined functions (which
  occur only at the end of a compilation), are presented so
  prominently that a user would be hard put to say that a program
  which elicited such warning messages was "correct" in that
  implementation.  Unlike the situation with unused variables, there is
  no possible declaration one can write which suppresses the warning
  messages.

  This disagreement between theory and practice should be mended
  somehow.

Proposal (PROCLAIM-LEXICAL:CURRENT-PRACTICE):

  Change the language definition (page 55?) to say that it is an error
  for there to be a free reference or assignment to a name unless a
  SPECIAL proclamation or declaration is in effect for that name.

  This would legitimize the behavior of current implementations.

Proposal (PROCLAIM-LEXICAL:BY-THE-BOOK):

  Shame implementors into going by the book.  Implementations should
  simply stop intimidating users who want to write code like this.
  "Apparently unbound variable" warnings should be given the same
  purely advisory status that "apparently undefined function" warnings
  now enjoy.  The exact meaning of this is of course implementation-
  dependent.

[... omitting, among other things, other proposals ...]

Cost of adopting change:

  CURRENT-PRACTICE: Ostensibly none, although implementations which
  don't signal this as an error should be explicitly encouraged
  to do so.  It ought to be signalled in interpreted code as well.

  BY-THE-BOOK: This would be an easy fix to the error reporting and
  bookkeepping components of existing compilers.  Of course it is not
  a change to the language, so there is no impact on portable code.

(Later versions of the proposal use more neutral language and
went into more technical detail about the proposed semantics.
But this version is better than later ones as a guide to how
the issue was seen and the motivations for change.)

# Note to comp.lang.lisp subscribers: the first half of this article
# is just "flame war self defense."  There's some real technical
# discussion in the second half.

In article <3049754333170...@arcana.naggum.no>,

    The effect of SETQ at the top level, on a variable that has
    not been declared special, is not defined.  Please see the
    HyperSpec page on SETQ.

Instead, you said something like this:

    a variable is either lexical or it is not.  the "not lexical"
    in Common Lisp is "special".  for more information on the
    environments in Common Lisp, see [the HyperSpec page "Introduction
    to Environments."]

The HyperSpec page to which you referred does not say anything
about situations in which the effect of SETQ is undefined, and it
is not even the place in which one would expect to find such material
if it existed.

If your 8/18 article was really an attempt to describe your current
complaint, then it was a monumental botch, because it was not even
close to this subject -- and your abusive articles of 8/19 and 8/21
were fruitless attempts to defend this original botched article.

Since you have brought up your complaint about SETQ so late in the
discussion, I suspect that it was not your original complaint at
all.  I think you probably realized you were wrong and gave up on
your original complaint -- and then tried to take advantage of your
own vagueness by secretly switching complaints in midstream.

But that's OK with me, because your new complaint is also based on
your incomplete understanding of Common Lisp.  You are still wrong.

         (SET 'X 100)
    and  (SETF X 100)
    and  (SETF (SYMBOL-VALUE 'X) 100)

are free to cause this undocumented side effect, as well as

         (SETQ X 100)  ?

You never said you were questioning my "basic assumption," nor did
you even state what you thought my basic assumption to be, until now!

    (setq a 100)
    (let ((a '(this list is not a number)))
      (eval '(1+ a)))

Which of the following best describes this code?

    (a) It is legal Common Lisp and it will return 101.
    (b) It is not legal Common Lisp, and it will signal an error.
    (c) It is not legal Common Lisp, and its effect is undefined.

If you say (b) or (c), I will rebut.  If you say (a), I will agree
with you but I will wonder what the heck you have been complaining
about!