(let ((n 0))
  (defun next ()
    (incf n))
  (defun reset ()
    (setf n 0)))

the outer let might create a big lookup table used only by the
internal defuns, or just might be some state variables that the
internal defuns share.

now, i've certainly not read a great deal of lisp code in my time, but
it seems that i rarely see anything like this, except in books.  is
there some reason for this?

 * This used to not compile well and many old-time lisp programmers
   are not in the habit of doing it.  And in some implementations it
   might not still compile well, so some programmers nervously avoid
   it as what they perceive as a gratuitous risk if they port their
   code.

 * Some people (myself included) think this is symptomatic of a major
   split between Lisp and Scheme.  Lisp is capable of doing what you
   describe and you should feel free to do it if you like that
   style, but many Lisp programmers prefer not to because they prefer
   to have named structures for easier debugging.  What you suggest
   is done really a lot in Scheme, and long ago when I used to program
   in Scheme it was one of those things that routinely drove me nuts.
   The issue is this:

   Since there are no closure accessors, it's hard to debug the above
   code (e.g., to extend to add another operation, to view the
   counter, or to reset the local variable to value not planned for)
   without some powerful tool capable of violating a closure's internal
   data structures in ways that no tool really ought to have the right
   to do.

   The more conventional CL style is:

   (defvar *counter* 0)
   (defun next () (incf *counter*))
   (defun reset () (setf *counter* 0))

   The effective only difference between these two approaches is that
   the counter is named, and many find that important.  Before you
   think this is silly, think about the fact that you could have written:
    (let ((n 0))
      (labels ((next () (incf n))
               (reset () (setf n 0)))
        ... code that uses next and reset ...))
   and really the only reason you don't is that you find it useful to
   sometimes name the functions NEXT and RESET from outside that
   context (e.g., in another file or in interactive debugging).
   So you're just drawing the line at a different place, and the
   language isn't stopping you.  Other people draw it differently,
   and for what have often been seen as good reasons.

 * There's also some messiness if you need to redefine/recompile functions
   such as the ones above.  They have to be recompiled as a pair.
   Consider this scenario:

    (a) You execute your original code above.
    (b) You do store #'next in the heap somewhere, e.g., by doing
        (defvar *saved* #'next)
    (c) You later recompile the original code you gave.

   Voila'.  You now have a bogus version of next stored in *saved*.
   And while it's true that if you had changed the definition of next
   incompatibly you'd STILL have a bogus version, at least a recompile
   without changing wouldn't hurt if you used my revised definitions.

   This is, in a sense, just a variant of the inability to name
   things, but it basically means that incremental development tools
   are as crippled as you when trying to keep everything up to date
   if you use this coding style.

Hope that helps.  I don't mean to suggest you have to change your
coding style.  You should do what makes you the most comfortable.
But I do think the people who use other coding styles are not crazy
for it...

Btw, some might argue the use of specials here is overkill.  The
reasonable compromise would be to use global lexicals, which Scheme
has and Common Lisp doesn't. e.g.,

 (define n 0)
 (define (next) (set! n (+ n 1)))
 (define (reset) (set! n 0))

  I regard this as an optimization technique, not as normal coding style.
  accessing a special variable is generally more expensive than accessing a
  closure variable and this may matter in some circumstances.  sometimes,
  you have to perform a number of sanity checks on global variables because
  they may get nuked by the user or set to weird values.  in such cases,
  you can save yourself a lot of trouble by inhibiting direct access to the
  variable and performing the sanity checks in the setter function, knowing
  that there are no other ways to set the variable.

  normally, I regard this kind of "information hiding" to be anathema to a
  good software development environments because recompiling the closure
  or re-loading the file they are contained in irrevocably destroys the
  encapsulated information, so it should be employed only when you know
  that the closures are loaded only once, and I prefer that to be before
  dump time for the Lisp image.

  when implementing a new fully table-driven approach for international
  character sets in Allegro CL, I found, after heavy profiling, that so
  much time was wasted in accessors into the character set definition
  objects and special variables that it payed well to make a giant closure
  with all the functions accessing pre-accessed slots from the objects as
  closure variables, set by the sole setter function for the character set.
  since this is a fundamental framework that when changed (almost) requires
  re-dumping the Lisp image, anyway, I thought this was a reasonable cost
  to obtain the needed safety and speed.

  in other words, I think this approach is fine when the code is known not
  to be changed in the life-time of the application and no access to the
  encapsulated information will ever be needed, i.e., when the code is
  completely static.  this happens less often than one might think.

Note that calling a lexical closure is often more expensive than calling a
plain function, because an implicit environment argument may have to be
passed to the function.

Your point about application-level checking that may be necessary for
special variables is well taken, though.  This is also the justification
for information hiding in most object oriented languages -- by limiting
access to internal state, you can often prove invariants about that state.

In my opinion, this style of using and depending on global state
is not the best.  It is difficult to maintain, and generally will fail
miserably in a parallel processing / distributed memory environment.
I suggest an object approach:

(defclass counter ()
 ((value :initform 0 :accessor counter.value)))

(defmethod next ((c counter)) (incf (counter.value c)))
(defmethod reset ((c counter)) (setf (counter.value c) 0))

;; and then you pass around a counter object:

(let cnt = (make-instance 'counter)
 do
  ... (reset cnt) )

[ ... Well-considered treatise on tradeoffs pro and con local lexical
  closure variables vs globals vs ... ]

(let ((n 0))
  (declare (optimize (debug 2)))
  (defun next ()
    (break "next1 break")
    (incf n))
  (defun reset ()
    (setf n 0)))

In Allegro CL 5.0:

USER(1): (setq *load-local-names-info* t)
T
USER(2): :cl xxx
;;; Compiling file xxx.cl
;;; Writing fasl file xxx.fasl
;;; Fasl write complete
; Fast loading /force/duane/acl51/src/xxx.fasl
USER(3): (next)
Break: next1 break

Restart actions (select using :continue):
 0: return from break.
 1: Return to Top Level (an "abort" restart)
[1c] USER(4): :loc
Compiled lexical environment:
0(LOCAL): SYSTEM::.LAMBDA-LEXICAL-ENV.: #(0)
1(LOCAL): N: 0
[1c] USER(5): :loc n
0
[1c] USER(6): :set-loc n 10
[1c] USER(7): :cont 0
11
USER(8):

Assuming aggressive optimzation settings:

For a global variable access, the name of the variable must be loaded
into a register from the function-environment, then an access of the
value slot gets the value.

For a lexical closure variable, the lexical environment must be ensured
to be in a register, then accesses to any number of the variables in
that same closure contour can take place one-load-per-instruction, assuming
the lexical environment is vector-like.

For the global accesses there is no setup of the function-environment; it
usually comes for free as part of the calling sequence.  But an instruction
scheduler at the peephole phase must take care of the back-to-back loads
in order to break any instruction pipeline interlocks.  In other words,
global accesses have relatively constant overhead, as contrasted in the
next paragraph.

For lexical closure variables, there is at least a several instruction
extra setup required per function, and the compiler must ensure that the
environment is close-at-hand for any accesses.  If the choice is made
to use a register for the lexical environment, then it might be loaded
very early (and then again after each call that the function makes), but
once the lexical environment is in a register, there would tend to be
fewer instruction pipeline interlocks, especially if many operations on
many variables at the same closure level are performed.  In other words,
lexical closure variables tend to have higher setup cost for faster
access time once the setup is done.

In conclusion, If closed-over variables are used "enough" times within
a function to offset the initial setup overhead, then it can be faster
than a global access.

Of course, if the closure structure becomes complex (e.g. multiple
variables homed in different closure levels), then the cost of closure
setup and access climbs.

Personally, I think depending on the debugger is just wrong.  You are
welcome to disagree, but here are my reasons for what they are worth:

 - A special variable offers program adaptability even in highly
   optimized code because the actual language semantics requires
   that adaptability.

 - A special variable can be program-modified in a patch file.

 - A compiler doing data flow and/or dead-code analysis might
   determine a side-effect isn't going to happen and optimize away
   the variable.    This might be inconvenient for debugging.

 - Some environments in some implementations have no debugger, or
   may have an insufficient  debugger; a conforming implementation
   say simply type "core dumped" rather than implement a debugger.
   I figure the market will sort that out; nevertheless, there is
   value in not depending on the debugger for all your help.

Items 2 and 3 of the above are things you usually care to affect AFTER
you write your code--it's not part of the original design.  So you either
plan in that kind of flexibility or risk getting screwed for having
failed to.  The debugger can be a help in cases where you failed to
do it yourself--no doubt about that.  But, IMO, nothing about the
debugger is linguistic; and, perhaps because of my background, I see
programming as a linguistic activity--saying what I want to be true.
Requesting dynamic programming flexibility is properly said by naming
things in publicly accessible ways, not by naming things privately
and hoping for a tool that will violate that privacy.  Again, it's
not that there shouldn't be that tool--it's just that I don't rely
on it as a first line of defense.

"Show and tell" about LW debugging follows...

- - - - -

This demo is from LW 3.2.2 on Unix; we have newer Unix releases but this
was what was handy and we've had this feature for a while, so...

Given a file containing:
  (let ((n 0))
    (defun foo () (break "Foo") n)
    (defun bar () n)
    (defun baz () (incf n)))
here's what it looks like in ours:

CL-USER 19 > (load (compile-file "/u/kmp/temp.lisp"))
;;; Compiling file /u/kmp/temp.lisp ...
;;; Safety = 3, Speed = 1, Space = 1, Float = 1, Interruptable = 0
;;; Compilation speed = 1, Debug = 2, Fixnum safety = 3, GC safety = 3
;;; Source level debugging is on
;;; Source file recording is  on
;;; Cross referencing is on
; (TOP-LEVEL-FORM 1)
; (TOP-LEVEL-FORM 2)
; Loading fasl file /nfs/holmes/u/ldisk/kmp/temp.wfasl
#P"/nfs/holmes/u/ldisk/kmp/temp.wfasl"

CL-USER 20 > (bar)
0

CL-USER 21 > (foo)

Foo
  1 (continue) Return from break.
  2 (abort) Return to level 0.
  3 return to top loop level 0.

Type :c followed by a number to proceed or type :? for other options

CL-USER 22 : 1 > :n
Call to (SUBFUNCTION FOO (TOP-LEVEL-FORM 1))

CL-USER 23 : 1 > :v
Call to (SUBFUNCTION FOO (TOP-LEVEL-FORM 1))
   N : 0

CL-USER 24 : 1 > (setq n 3)
3

CL-USER 25 : 1 > :c
3

CL-USER 26 > (bar)
3

CL-USER 27 > (baz)
4

But the same will happen if the two processes try to step or to
reset the same instance cnt of the counter which you defined below.

The above version would have to be changed for multiprocessing
to something like

(defun makecounter ()
  (let ((n 0))
     (cons
        #'(lambda () (incf n))
        #'(lambda () (setq n 0)))))

(setq cnt (makecounter))
((car cnt)) ; counts
((cdr cnt)) ; resets

which is what a class is syntactical sugar for.
(I did mostly scheme for the last months, so I may have too few or too
many "#'"s or "funcall"s here.)

A great deal of Common Lisp's design intentionally supports incremental
development--a concept that has no place if you take the static view
that all there is to a program is what it is at a single static point
in time when seen by a compiler.

Incidentally, in Common Lisp I would typically have written:
 (defun make-counter ()
   (let ((n 0))
     (values #'(lambda () (incf n))
             #'(lambda () (setq n 0)))))
and then later
 (multiple-value-bind (next-counter reset-counter)
     (make-counter)
   ...
   (funcall next-counter) ; counts
   ...
   (funcall reset-counter) ;resets
   ...)

I use the let for another purpose, perhaps there is a more correct
way:

(let ((ar (make-array 100)))
  (defun foo ()
       ... do some calculations that require the array ar
           to hold intermediate values ...))

Presumably, if I put the creation of ar inside defun, an array
will be created each time the function is executed, which isn't
good if the function is called a lot, as it is in my simulations.
For this particular function, it makes no sense to make ar a global
variable as its contents are of only fleeting interest within the
context of foo.

Is there a way to create a "static" array inside a function?  I assume
something like

(defun foo ()
  (let ((ar #1A(0 0 ...) ))
        ...))

would do this, but this approach is impractical for large arrays.

Any thoughts?

You want LOAD-TIME-VALUE.  When its second arg, read-only-p, is NIL
(the default value, so it may be omitted), then arg1 is evaluated only
once in the runtime environment and the result is still allowed to be
modified:

 (defun foo ()
   (let ((arrray (load-time-value (make-array 100))))
     ...array...))

Normally, I'd still use

 (defun *foo-array* (make-array 100))
 (defun foo () ... *foo-array* ...))

but there are times when it's better not to.  One example might
be in the implementation of a hypothetical class system where
one might turn:

 (defmessage (frob :area) () (+ (slot x) (slot y)))

into

 (setf (class-message frob :area)
       #'(lambda () (+ (slot-ref frob x)
                       (slot-ref frob y))))

where, without working this in detail, you can imagine things
like (slot-ref class x) turning into

 (cdr (load-time-value (lookup-value-cell-cons :class 'frob
                                               :slot 'x)))

so that at runtime the references to slots were as fast as
(car value-cell) and (setf (car value-cell) new-value).
Here again, you could make a special variable for each slot
but it's ill-advised since it would mean a lot of clutter
to the symbol namespace for little value.

A similar strategy, using define-symbol-macro and load-time-value can
be used to implement global lexicals and "bindable constants"
needed for ISLISP.

Did you mean to say DEFVAR here?

i asked why closures didn't seem to be a common idiom in common lisp,
and gave the following trivial example:

(let ((n 0))
  (defun next ()
    (incf n))
  (defun reset ()
    (setf n 0)))

the most important point seems to be that code like this is hard to
redefine dynamically, since recompiling or reloading the code
necessarily blows away the closed-over state.  this is philosophically
at odds with the notion of easy incremental development, and of
easy dynamic re-definition.

it seems to me that common lisp exposes things a lot more than other
languages do.  the package system comes to mind---even if a symbol is
not exported, you can still get at it with the pkg::foo syntax.  so,
if you're used to a more fascist language, CL's refusal to enforce
information hiding may leave you feeling kind of naked.

it occurs to me that lisp fosters an environment which is well-suited
for a single person or a small team working uno animo.  it seems to
enable the programmer to be a spider at the center of a web of
thoughts and abstractions, all of which are interconnected to each
other and easily accessible.  this is in distinct contrast to the
gospel of strictly defined interfaces that some software engineering
people preach.  perhaps this is why the idea of the single-user lispm
was so attractive.

sorry if this is all obvious.

comp.lang.lisp has to be the best newsgroup on the net.

some say the US is a good place to live.  it's relatively free what
you can do.  there exist societies where you can get thrown in prison
for littering.  no one in the US is arguing that they want littering.
but sometimes the cure for a bad behavior is worse than the problem.
some programming languages don't let you deviate from standard
practice--and for some people that is necessary or desirable.  but
for others it is crippling of imagination.    flexibility is not
without risk (of abuse).  but then, neither is security (of lost
freedom or creativity).  

people tend to reject what most gets in their way and what most
has recently been a burden to them.  to those tired of
crime and vandalism, a protective society seems good.  to those
tired of government running their lives, a free society seems
good.  in my opinion, governments allowed to run people's lives
usually opt to do so and i consider that risk ultimately greater
than any risk brought on by individuals.  but that's just me.

likewise, to people who haven't been good at disciplining
programs, or who have been around others who haven't, it's easy
to see why a programming language that imposes structure is good.
but having myself seen programs that were wondrous and could only
be written in lisp because other programming languages are too
oppressive, the idea of forced conformance pains me.

it may be that lisp isn't good for some because maybe some need more
structure than lisp imposes.  i don't know.  i like to think the
environment can offer enough structure that this isn't an issue.  but
classically, lisp has solved the hardest, the most complex, the most
elusive of programming problems.  and often it takes the most creative
minds to do that.  tying their hands is not first on my list of tools
to enable such people.

Languages like Lisp and Perl, which provide extreme flexibility and little
enforced structure, assume that the user is more of an architect than a
worker.  Exploratory and evolutionary programming are the opposite of the
above style of rigid design, but it's mainly appropriate for experienced,
creative software designers, not software technicians.

Regs, Pierre.

But you're right that the discipline doesn't have to be part of the
language definition.  C isn't very strict (esp. K&R C, which didn't have
prototypes), which is why lint was created.  But if you can embed the rules
that a lint-like program enforces into the language definition, you don't
have to worry about people who forget to run lint and aren't working for an
organization that automates it for them.

Literate programming languages also embody a similar philosophy.  Their
designers realized that programmers lazy about documenting what how the
program works.  But if the language makes it easy to write the
documentation while you're writing the program, it might lower the bar.
This never really took off, though.

    http://www.genworks.com/idl-shareware/doc/catalog2oracle.html

If anyone is interested in the source for the function documentation writer,
I have it (since I wrote it)(it's only a couple lines of code). The defpart
documentation writer is probably less useful for most people, but I do have a
shareware version of that as well if anyone is interested.

By the way, I find cl-http to be very useful in developing documentation, since
you can just keep recompiling code in the Emacs buffer and reloading the doc
page from the virtual URL defined in cl-http (i.e. you don't have to keep
writing out a bunch of physical files to disk - the web pages are just computed
on a demand-driven basis).

1ICAD is a macro extension of Allegro Common Lisp, used for ``Knowledge-based
Engineering'' work (traditionally it's used mostly for generative geometrical
applications and geometric reasoning, but I use it for pretty much everything
(I'm spoiled)).

--
   "The Lisp ignorants will come up with an amazing array of bullshit
    to support their ignorance. They are like racists that way"

Note to C++ gurus: I know this violates the ODR and evokes undefined
behaviour but it has a fair chance of working on the more common
compilers.

In general the only system I have ever worked with that tried to
enforce this discipline full force ( an incredibly bad Ada-83
implementation with a library manager from hell ) just had as effect
everyone in our group had the passwords of all others and that we
disabled some general security features in order to get anything done
with the beast.

% It may not be exactly what Literate Programming is after, but isn't that
% what JavaDoc accomplishes?  And some IDE tools (e.g., IBM's VA for Java)
% will write a skeleton of the JavaDoc while they create your routine
% template.  I think we'll be seeing more of this in the years ahead.

What happens if the function starts to do something different than it
was originally declared to do because the design requirements changed?
I see so many cases of the comments not being updated and the function
name no longer being descriptive.

--
David Steuber (ver 1.31.2a)
http://www.david-steuber.com
To reply by e-mail, replace trashcan with david.