I've been watching the recent threads on special variables, top-level
setq-s and the like, and I've come to the conclusion that the current
situation is badly broken.  I have a proposal on how to fix it that I'd
like some feedback on.

The intractable problem is that beginners are chronically confused by
the fact that Lisp does not make an adequate syntactic distinction
between two completely different ways of binding variables.  Imagine
you are reading some code and you encounter the following two functions:

(defun foo (x) (blarg x))

(defun baz (y) (blarg y))

Do FOO and BAZ do the same thing?  Well, they might or they might not.
It depends.  It depends on whether X or Y have been DEFVARed.  It depends
on whether there are any free references to X or Y in any function called
by BLARG.  It depends on whether you are trying to write thread-safe code.
It depends on whether performance differences matter.  It depends on whether
your code is interpreted or compiled.  And if you want to explain the
reasons it depends on all these things you have to start talking about
the differences between symbols and variables, what bindings are, and
that there's nothing at all special about "special" variables, it's just
an ancient piece of Lisp terminological baggage that means nothing more
than "dynamic scope."

A beginner's initiation into this state of affairs invariably comes when
he or she one day as an expedient types (defvar x 1) and then spends the
rest of the day (or week) trying to figure out why her code is suddenly
exhibiting mysterious bugs.  The admonition to always use *...* notation
for DAFVARed variables is a temporary fix, but it generally takes a very
long time for a typical programmer to wrap their brains around what is
really going on.

The problem is pervasive and subtle.  Consider:

(funcall (lambda (x y) (lambda () ... x ... y ...)) 1 2)

Does this return a lexical closure over X and Y?  Well, it might, or it
might not.  It depends.

Now, this has not been much of an issue because artificially separating
the name spaces of lexical and dynamic variables using the *...* convention
generally works to keep the problem at bay.  But the trouble is that this
solution only works if you use it, and you can only use it if you are
aware of it.  And even if you are aware of it, your collaborator might
not be.  *You* might not have DEFVARed X, but how do you know that Joe
didn't?

The solution to this problem is to change the language so that the
distinction between dynamic and lexical bindings is locally manifested.
This is technically simple, but politically complex because it means
either adding some kind of lexical declaration or getting rid of pervasive
SPECIAL declarations (e.g. getting rid of DEFVAR, (PROCLAIM (SPECIAL ...))
etc.)  My personal preference is the latter.  Top-level declarations that
radically change the semantics of code in ways that are not lexically
apparent are EVIL!

Even local special declarations are IMO evil.  The semantics of a piece
of code ought not to depend on forward references.  It's also very much
against the spirit of a declaration.  Declarations are for providing extra
information for the compiler.  They are not for changing the semantics of
the program.  It should be possible to remove all declarations without
changing anything about the program except how fast it runs.

The solution is obvious.  Special variables are nothing more than the
symbol-value slot of some symbol.  Why not make that simple fact manifest
itself in the syntax?  For example:

(let ( (x 1)                   ; A lexical binding
       ((symbol-value x) 2) )  ; A dynamic binding

While we're at it we could fold in the binding of multiple values and
destructing into one uniform syntax:

(superlet ( (x 1)                   ; A lexical binding (guaranteed)
            ((symbol-value x) 2)    ; == (let ((x 2)) (declare (special x))
            ((values a b c) (foo))  ; == (multiple-value-bind (a b c) (foo)
            ((values a (symbol-value b) c) (baz))
                                    ; As above, but with (declare (special b))
            ((a b c) (foo)) )       ; A DESTRUCTING-BIND

A simple reader macro helps make things a lot less wordy:

$x == (symbol-value 'x)

So now we can write:

(let ( (x 1) ($y 2) ) ; X is a lexical binding, Y is a dynamic binding

This would work at top-level too:

(setf $x 1) ; Set the global X with no warnings and no semantic ambiguity
(setf x 1) --> Warning! No lexical variable named X visible here.  Setting
$X instead.

To implement this proposal completely requires the cooperation of Lisp
implementors because you have to change the way lambda lists are
processed.  But once that's done you can write (lambda (x $y) ...)
instead of (lambda (x y) (declare (special y)) ...), or
(multiple-value-bind (x $y) (foo) ...) instead of (multiple-value-bind
(x y) (foo) (declare (special y)) ...) or (defun foo (x $y) ...) instead
of (defun foo (x y) (declare (special y))  ...)

As a prototype I have implemented a macro called BIND that works as
described above.  BIND binds lexical and dynamic variables, and handles
multiple values and destructuring.  It also uses a paren-less LOOP-like
syntax.  I present it as both an illustration of how simple my proposal
would be to implement properly, and as an illustration of how easy it is
to change Lisp completely within the standard.

The syntax for BIND is:

(BIND var [=] value [and] ... in &rest body)

var : symbol | (symbol-value symbol) | (values var var ...) | list

Keywords in square brackets are optional.

Example: (BIND x = 1 and (z ($y) q) = (foo) and (values $a b c) = (baz) in ...)

expands to:

(let ( (x 1) )
  (destructuring-bind (z (y) q) (foo)
    (declare (special y))
    (multiple-value-bind (a b c) (baz)
       (declare (special a))
       ...

I conjecture that beginners at least will find BIND easier to deal with
than the things it replaces.

Erann Gat
g...@jpl.nasa.gov
---

(set-macro-character #\$
                     (lambda (s c)
                       (declare (ignore c))
                       (if (whitespacep (peek-char nil s))
                         '|$|
                         `(%symval ,(read s))))
                     t)

(defmacro %symval (x) `(symbol-value ',x))

(defun process-vartree (vl)
  (let ( (specials '()) )
    (setf vl
          (iterate loop1 ( (v vl) )
            (cond ( (atom v) v )
                  ( (eq (car v) '%symval)
                    (push (second v) specials)
                    (second v) )
                  (t (cons (loop1 (car v)) (loop1 (cdr v)))))))
    (values vl specials)))

(defmacro bind (&rest forms)
  (cond ( (null forms) (error "Missing IN keyword") )
        ( (eq (car forms) 'in)
          `(progn ,@(cdr forms)) )
        ( (eq (car forms) 'and)
          `(bind ,@(cdr forms)) )
        (t (let ( (var (pop forms))
                  (val (pop forms)) )
             (if (eq val '=) (setf val (pop forms)))
             (cond
              ( (atom var)
                `(let ( (,var ,val) ) (bind ,@forms)) )
              ( (eq (car var) '%symval)
                (setf var (second var))
                `(let ( (,var ,val) )
                   (declare (special ,var))
                   (bind ,@forms)) )
              ( (eq (car var) 'values)
                (receive (vars specials) (process-vartree (cdr var))
                  `(multiple-value-bind ,vars ,val
                     (declare (special ,@specials))
                     (bind ,@forms))) )
              (t (receive (vars specials) (process-vartree var)
                   `(destructuring-bind ,vars ,val
                      (declare (special ,@specials))
                      (bind ,@forms)))))))))

#|
; Example:
(defun baz () (values 1 2 3 4))

(defun foo ()
  (declare (special x y))
  (list x y $z $w))     ; Look Ma, no warnings!

(setf x nil y nil z nil w nil)