The only TAIL-CALL operator I've ever been familiar with was in Teco
and it was used to allow you to "GO" into another macro without popping
the q-register state, so would correspond more to:

 (defun foo1 () n)
 (defun foo (n) (tail-call (foo1)))
 (foo 3) => 3

I can't see a tail-call in CL meaning *that*, so I don't know what it would
mean.  What if I wrote:

 (defun factorial (x)
   (if (zerop x) 1
       (* x (tail-call (factorial (1- x))))))

The only thing that can be tail called here is the "*".  More than that
-- it *will* be tail called by any implementation that does tail-call
optimization (if it doesn't optimize even further and inline it).

You can't tail-call "factorial" because the result of the enclosing
function isn't identical to the result of the call of "factorial".

I once had a love-affair with named-LET-style iteration (I'm all cured
now, I use LOOP like Real Men).  CMUCL has an ITERATE form which does
this sort of thing, and I wrote a clone using the obvious expansion to
LABELS. The problem is when you use an implementation which doesn't do
tail-call elimination, of course.  So I hacked away some more and I
made my macro convert these `recursive' calls to GOs.  Except now it
breaks if they aren't tail-calls, so I did some yet further hack which
required the name of the local function to have the string "LOOP" in
it, and only in that case would iterative code be compiled.  Very
shortly after this I realised that this was all just stupid, and that
the only way to do this right was to do the whole analysis and
actually compile GO iff the thing was a tail call, and that doing
that would require me to write a reasonable chunk of a CL compiler.
This was the moment when I realised that I should have used LOOP all
along.

So, what is the difficulty in deciding whether or not a function should
be tail called?  It should be tail called If and Only If the result of
the current function is exactly the result of the function to be called.  
That is, no calculations are to be done using the result of the function
being called and no environment cleanup is required (exception handlers,
destructors in languages which have them, etc...).

It seems to me that the only problems that arise are:

1) the actual mechanics of doing the tail-call at the machine level.  
This can require a certain amount of stack-munging to pop off the return
address and old arguments, push the new arguments and re-push the return
address, and then jump to the tail-called function.  This is easier with
e.g. RISC CPUs which don't use the stack for function calls and have
arguments in registers and the return address in a special (or at least
fixed) register.  It's virtually impossible with anything that uses a
caller-cleanup function calling sequence (most C compilers used to do
that by default in order to support varargs, newer ones tend to use
callee-cleanup when they can) unless the current function and the
function being tail-called have the same number of bytes of arguments.

2) optimization problems.  Things which are prima facie tail calls are
easy, but Kent is appearing to indicate that users want compilers -- and
even interpreters -- to do sufficient analysis to recognize some things
that are NOT tail calls but that can be *optimized* into tail calls.

I guess Kent's example is a good one:

 (define (factorial x n)
   (if (zero? x) n
       (let ((result (factorial (- x 1) (* x n))))
         result)))

He says some (undefined) people would expect the recursive call to
factorial to be a tail call even though it's not prima facie a tail call.

Expanding the definition of "let" this actually means...

 (define (factorial x n)
   (if (zero? x) n
       ((lambda (result) result)
        (factorial (- x 1) (* x n)))))

... which is ...

 (define (factorial x n)
   (if (zero? x) n
       (anon-lambda (factorial (- x 1) (* x n)))))

 (define (anon-lambda result)
   result)

Either way it's pretty clear that you need some pretty serious compiler
work to implement "inlining" in order to allow the call to factorial to
be a tail-call.  And even then it would *not* be a tail call if there
was another expression in the body of the lambda/let ... especially one
with side-effects such as something like (format "factorial %d = %d\n" x
result).

I don't have a problem with saying that this is too complicated a
situation to expect an interpreter or simple compiler to analyse and
that if something isn't prima facie a tail-call then you can't *count*
on it being optimized as one.

But you might get lucky with a sufficiently smart compiler.

My main point is not that it can't be done, but that the RIGHT way to
do it is for a vendor to do it, present it to users, let it get
pounded on for a while, and then report EXPERIENCE.  Vendors are equipped
to sense whether risks like this are worthwhile and to track the response.
If no vendor wants to do it (and I included "free software" makers as
vendors here, since they aren't selling for money but they still are
putting their reputation on the line, and risking their email box will fill),
there's little point to bugging the community as a whole to do it.
Standards work is not a place to do design; it is a place to consolidate
design among vendors that have either all done the same thing or have made
divergent solutions to a common problem that need tweaking in order to get
in line.

KMP> I honestly don't undersatnd what this TAIL-CALL is going to do.

Well, I didn't propose it, and I wasn't particularly arguing for it
(just that it's not the same as DO), but I can actually think of a
nice use for it: request that the compiler eliminates that tail call.
Obviously it would need all the capabilities of a compiler that does
it more or less quietly.  If it could not eliminate the call, it could
signal an error when compiling, and it would not eliminate tail calls
that weren't marked for elimination.

Actually, I kind of like that idea.

 [...]

A function call is either the last function call in the body of the
function or else it is not.  If it's last then tail-call it (with a
"jump"), if it's not last then call it the regular way (with a "jump to
subroutine").

"Last" means either the last expression evaluated in the function.  This
means either being physically last, or else being the last expression in
an IF or COND (etc) that is physically last in the function.

  But this is not quite as trivial as people seem to think.  There are all
  kinds of low-level cleanup issues that may not be visible or even known
  to the programmer and which the compiler writer may need to be told about
  (either through a formal specification of a requirement or by invoking a
  local declaration) before it makes sense to allow for making a tail call
  into a jump.  Also note that the callee needs to believe that it was
  called, unless the language has specified semantics to this effect, too.

| "Last" means either the last expression evaluated in the function.  This
| means either being physically last, or else being the last expression in
| an IF or COND (etc) that is physically last in the function.

  I think you might mean the form that evaluates to the value of the
  function.  It might, for instance, be possible to unwind special bindings
  and unwind-protect forms in such a way that you could jump to a function
  with it returning to the cleanup forms, so the notion of "last" might be
  very confusing and counterproductive.

Actually, so as to eliminate confusion, instead of a TAIL-CALL special form,
or of TAIL-APPLY and TAIL-FUNCALL special forms, I think it might be better
to extend the RETURN and RETURN-FROM forms with a keyword argument :TAIL-CALL
or to provide TAIL-RETURN and TAIL-RETURN-FROM as in

(defun mult-fact (x n)
   (declare (type unsigned-byte n))
   (if (< n 2) n
       (return (mult-fact (* x n) (1- n)) :tail-call t)))

  <bruce-76CFD2.22564601102...@news.akl.ihug.co.nz>

Anything subject to unwind protect or the like is prima facie NOT in
tail position.  It is possible that a smart compiler may be able to
optimize things so that it can be moved into tail position, but that's
not something you could depend on.

It seems especially dangerous to move something out from the body of an
unwind-protect -- you'd have to prove first that it was impossible for
it to throw.

* Bruce Hoult
| I covered this in that previous message as well.
|
| Anything subject to unwind protect or the like is prima facie NOT in
| tail position.  It is possible that a smart compiler may be able to
| optimize things so that it can be moved into tail position, but that's
| not something you could depend on.

  If you had covered it, you would have known that this is not simply an
  "optimization", but a design decision that would have to be taken if you
  wanted true support for tail calls.  Both special bindings and unwind-
  protect are so common in Common Lisp programs that separating them from
  the call stack would have been a _necessary_ design decision, in order
  that tail calls actually produce useful results.  Since you said you had
  covered this, contrary to my usually flawless short-term memory (I study
  SQL intensely right now, so there may be some irregular lossage :), I
  looked for where you had, but I cannot find it.

| It seems especially dangerous to move something out from the body of an
| unwind-protect -- you'd have to prove first that it was impossible for it
| to throw.

  This is utter nonsense.  Please try to understand how a system that does
  unwind forms _must_ set up these things.  The body forms in practice _do_
  return, throws or not, through a chain of unwind form, including special
  bindings, and whether you do this by setting up in-band catcher code or
  through a separate stack of unwind-forms-to-be-executed, is immaterial.
  If you do the latter on a separate stack, there is no difference between
  a normal return from the code that sets it upand a return from somewhere
  else: the setup has already been done and the execution is performed as
  part of the return procedure.

  If you decide to mandate tail-call optimization into jumps, this design
  would have to be "required" of an implementation because refusing to do
  tail calls simply in the face of special bindings would just be stupid --
  having an elegant solution as it does.  If you do not mandate it, you can
  get away with only "simple" tail-call optimization, and you can get an
  even more inexpensive tail-call technique when you get it, if you get it.

  In essence, I believe "properly tail-recursive" is a design decision that
  affects how the call stack must be designed (and continuations fall right
  out of a heap-allocated call frame and chain design) and how you can deal
  with special bindings and unwind-protect (it is no accident that Scheme
  does not), while tail-call _optimization_ is possible even in languages
  such as ANSI C.  I therefore also believe that it is a _disservice_ to
  the community to require "properly tail-recursive" because it requires
  too many negative and unwanted properties of the implementation, but that
  it is perfectly legitimate to ask the implementations for such a feature.
  Amazingly, this is the actual situtation: E.g., Allegro CL does a very
  good job of optimizing tail calls to both self and non-self if you ask
  for it in their proprietary way, but optimization _is_ a "proprietary"
  process, anyway.  I think some language designers fail to understand this
  and believe in _mandated_ optimization, which only limit and restrict how
  their languages may be implemented to the techniques known at the time
  their favorite optimization theory was in vogue, or techniques developed
  along that evolutionary branch.  The less you prescribe in terms of how
  to optimize, the more you can take advantage of free-ranging research in
  optimization, both software and hardware.  This is why C is _still_ best
  optimized for PDP-11-style processors.

Or, rather, you *don't* want it and so insist that if it can't work in
every conceivable situation then it isn't worth having at all.

So you can't do useful tail call elimination on things which are wrapped
inside "clean up after use" constucts?  I agree and that's just fine by
me.  Why do you think such constructs are a useful target for tail call
optimizations?  Do you frequently write recursive functions or sets of
mutually-recursive functions in which each call invokes another layer of
cleanup?  Can you give an example?  In such situations it's always
seemed considerably cleaner to me to put the cleanup constructs at an
outer level and let some inner stuff recurse away merrily sans
intermediate cleanup.

One of the main reasons to support tail call optimization is so that you
can CPS-convert code in certain circumstances.  Such code certainly
never has cleanup after the tail call -- the tail calls of the
continuation are *never* going to return.  That's the point.

The same goes for loops of various sorts expressed as recursive
functions or sets of mutually-recursive functions.  Think of them as a
different way of expressing iteration.  Would you want to insert another
layer of cleanup code every time you executed the start of a loop?  Why?

  Huh?  I have argued for a differentiation of "properly tail-recursive"
  and "tail call optimization".  What I want from tail call optimization is
  meant to be a severe restriction of what Scheme freaks want from their
  undesirable "properly tail-recursive" concept.

| Or, rather, you *don't* want it and so insist that if it can't work in
| every conceivable situation then it isn't worth having at all.

  Are you nuts?  Where did you get this garbage?  Of course I want it.
  However, I do not want it mandated by the standard, because that has a
  lot of negative aspects to it.  Hell, if I want a standard that says
  something so annoying as requiring a properly tail-recursive execution
  model, I would use Scheme.  I loathe Scheme.  Get a grip, Bruce, quit
  pretending that you are fighting an enemy you are not.  Please realize
  that there is a difference between a desire for a feature and a desire
  for a law (or specification) that require others to provide a feature.
  You seem to confuse the two, or at least not allow other people (Kent and
  me?) to make a distinction, and get all flustered because you think we do
  not want what we do not want there to be a law to _require_.

  I believe the rest of your message rests on these false premises, too.

| No it is not.

  Christ, Bruce, THINK!  I have no patience with this crap.  Go back and
  read what I wrote.  You have clearly not understood what I said, and now
  you argue against something you have misunderstood.  Quit that stupidity.

| You'd have to prove a bunch of other things as well, of  course, but that
| one alone is sufficient to show that it's very very  difficult to
| impossible to convert something inside an unwind-protect  into something
| that can be tail-called.

  This is just simply wrong.  _Listen_, now, OK?  

| Indeed.  Which is precisely why using a tail-call for things inside an
| unwind-protect is a nonsense.

  You obviously fail to think about this, and that annoys me.  Consider
  this simple fact: What a function returns to is _not_ under its control.
  If a function returns to its caller, which does some work before it also
  returns, it could as well return to a different function that does that
  work.  This is the core of the argument for tail-call optimzation, is it
  not?  Why does it _not_ apply to unwind forms?  _Think_ about this, OK?

| The whole *point* of the tail call is that it doesn't return

  Huh?  What is this nonsense?  On the contrary, the whole point is that it
  returns to the caller of the caller in place of the caller, which has
  _redirected_ it to return elsewhere from what _its_ caller said it should
  return to.  The whole *point* with tail calls is that there is a simple
  semantic equivalence between call/return/return and jump/return, and that
  this is completely independent of _both_ what you jump _and_ return to.
  What I suggest is an _expansion_ of the mere implementation of tail-call
  optimization such that you can call functions for their side-effect after
  a call that returns a value simply by returning to them instead of
  callling them.  That is well within the conceptual framework of what you
  seem to be clamoring for.  Why do you not get this simple application of
  your concept of tail-call optimization?

| Anything that you have to come back and do later (e.g. unwind) totally
| nullifies that.

  _Wrong_.  _THINK_ about this, will you?  _How_ you arrange for something
  to be done after you have returned is ORHTHOGONAL to arranging to do it.

| This is true, but it's pretty pointless.  In a recursive call that
| contains an unwind-form you're going to grow your stack of
| unwind-forms-to-be-executed without bound.

  So you think tail-call optimization is _only_ for recursion?  What _is_
  this?  Bruce, you _are_ not the idiot you seem to be insisting on showing
  me right now.  What has gotten into you?  The Scheme freaks want a
  properly tail recursive execution model for a number of reasons.  Tail
  call optimization is a much weaker concept, and it will actually save a
  lot of stack space -- namely the stack space consumed by arguments and
  call frames -- _even_ if you have special bindings and unwind-protect
  forms.  _Those_ are the worth-while savings.  And what is this silly
  thing about "without bound"?  There will be no more and no less bound on
  the recursion regardless of tail-call optimization!  If you can save on
  _some_ resources, but not all, why are _you_ opposed to that?  Was it not
  _you_ who accused Kent and me of the following just a few lines up?

| Or, rather, you *don't* want it and so insist that if it can't work in
| every conceivable situation then it isn't worth having at all.

  Now _you_ are arguing this silly position which nobody else holds.  Quit
  the stupidity and _think_ about what you are saying, damn it!

| Sure, you'll save a little memory because the structure that you put on
| that stack each time is probably a little smaller than the entire stack
| frame for the function would be, but that's just a savings of a constant
| factor -- it doesn't make a tail-recursive function (or set of mutually
| tail-calling functions) execute in constant space, which is the purpose
| of the optimization.

  Why do you restrict an optimization to such a purpose?  Why do you need
  there to be "constant space" when there is a clear and present benefit
  from not over-using another resource?  And why do you drag in the _size_
  of call frames?  Christ, Bruce, that is _so_ beside the point.  Call
  frames can be *large* these days.  An unwind-protect/special stack can be
  very small.  Now _you_ want to discard the stack space savings because
  you will still need special/unwind-protect space?  What _is_ this?

  Whatever _are_ you defending, Bruce Hoult?  Whatever it is, I can assure
  you that it is _not_ under attack.  Just _think_ about the arguments you
  get and you might learn something that you are currently insisting on
  _not_ seeing.

The fact is that the language isn't changing any time soon.

But the fact is also that when the language does change, there are certain
"reasonable care" requirements that implicitly apply.  Users often don't
mind if all kinds of issues are unresolve--most users only use 90% of the
language anyway, and would kill for us to offer only 90% of a certain
functionality and toss the rest.  Like XML without DTD's for example.
A user can write a library that only implements 90% of the spec and it can
be very popular because they have to answer to no one for not doing the rest.
"Fine, don't use it if you don't like it" is all they have to say.   A vendor,
on the other hand, has a higher degree of care required because he might
advertise an XML implementation and people would take them "more seriously"
and be more surprised if it wasn't complete, even if not advertised as such.
Or they might want it to be more complete next release because they assume
they're buying into a process, whereas user libraries are often one-off
take it or leave it.  Language standards are even more complex because the
designers are NOT gating the introduction of extensions which can already
mostly be done by vendors but RATHER are imposing requirements on vendors.

In spite of how it seems to users,more like ain the US where we have
the all-encompassing Federal government and a local State government
both affecting individuals, and the Federal goverment is often
creating what are called "unfunded state mandates", where they require
the State government to do something but don't say where the money is
supposed to come from or how the program is to be implemented in
finite resources.  There is, or should be at least, a large burden on
the people who impose such burdens to really show that it can be done.
So in a sense, and as a result the Federal government's key "design
job" should be to "be skeptical about the inclusion of anything
because it's so hard to tell that it will be universally a good idea.
One state may love it and already implement it, while another state may
be full of people who left that state to escape it!  

I hope you can see that this analogy is true of language design as well.
It works better on the first round, when no one has anything and you
impose a bunch of things because you're in "attract mode" and you hope
people will gravitate to your language.  But then when they have done it
you have a more complex problem because people have already adhered
"geographically" to vendors they like and now if you mix the soup, you
are not just affecting your "attraction pitch" but you are potentially
driving people away  from something they have grown to like.  People don't
like having to up and leave a country they've settled just because some
big government suddenly decides the local laws aren't good enough.

As such, I see my role as a language designer is to keep track of lots
of facts and ideas and preconceptions that people have and regurgitate
them at the right time so that a later discussion on an idea is not left
with the simple feeling that just because the people who argued about
it before are out of the room, it has no opposition.  It is sometimes,
therefore, the case that I will raise an issue in a slightly ill-formed
way because I can't quite remember how it was best expressed, whether
it was me or someone else who expressed it.  I'd rather you beat me up
for being vague or confused than have someone else beat me up for having
forgotten they cared about some issue at all.  

And, in general, whenever I have marked an issue as "complex and
controversial" in my mind, I try not to let anyone get away with
convincing me it's "simple".  It might be simple.  And I might
eventually be convinced such.  But I have an obligation because of how
I see my role to be considerably more obstinate than others in the
same discussion might be unless I'm sure that all parties with the
relevant concerns are properly represented by competent members of the
discussion.

So please don't oversimplify my position as to say "oh, you just don't
want this".  The fact is that I know there are a lot of people who think
that this is not a no-brainer, and I regard standards-making as something
that has to involve consensus-building, so the solutions and techniques
I seek are based on the standard ways of doing that.

In the past, "lexical scoping", "CLOS", the "CL condition system" and
even (I kid you not) "base 10 numbers" (it was base 8 in Maclisp, and
it was a BIG fight to get it changed to base 10 by default) were
controversial.  Sometimes people worry over things that are either
nothing to worry about at all (like "lexical scoping") or things that
even though controversial are too big a win to ignore (like "CLOS").
Whatever the case, starting small, and demonstrating "no ill effects"
in a small community is a better way to push forward.  For lexical
scoping, "starting small" meant pointing to Algol and enough people
bought in at the start of CL that we took the "big risk".  For CLOS,
"starting small" meant a portable implementation that people could
try; ditto for the CL condition system.  (Base 10 was just achieved by
political coup, with no more demonstration of workableness than "all
human history to date", but I'm proud to say I supported it all along.
:-)

My obstinance here is more of the form "wrong community to try to convince
first" and "here are some issues to think about" than "this is why you
will fail".  I'm only trying to say "start small" and "think about these
things".  Beyond that, I'm willing to sit back and wait and watch.

Imagine, for instance if the CL language standard mandated that at
certain levels of optimisation, bounds checking should not be done for
arrayss.  Is this reasonable?  No, it's obviously absurd, since it
makes all sorts of hairy requirements on implementations.  Worse than
this, it's absurd for implementations which manage to transform
something like this:

  (dotimes (i 10000)
    (setf (aref a i) 0))

into

  (progn
    (when (< (length a) 10000)
      (error ...))
    (dotimes (i 1000)
      (setf (system::unchecked-aref a i) 0)))

Such an implementation has already essentially removed the
bounds-checking overhead: should it be required to remove the
remaining check, and if so why? (perhaps to make C programmers more
comfortable by introducing subtle and obscure vulnerabilities for no
benefit).

It's also absurd for implementations which run on hardware that
supports bounds checking.

Finally, it's absurd because attempting to overcome problems like the
above *in the standard* would result in a vast document which
attempted to cover all sorts of obscure possible implementation
quirks.  Even if such a standard could be written, it would very
likely become obsolete as new implementation techniques were
discovered.

So a standard should not mandate this.  At best it should hint that it
would be a good idea, but language like that has little place in a
standard!.

But this is obviously completely different than saying that
implementations should not remove bounds checks at certain levels of
optimisation.  Of course they should be allowed to do that, and good
implementations probably will (and, I hope, some will do this raising
of checks out of loops to get the best of both worlds).

One thing a standard *could* do is to mandate that something
*equivalent* to bounds checking *must* be done at certain levels of
optimisation.  This is much easier to assert in a standard because it
merely has to say that `an error should be signaled if ...' in the
terminology of the Cl standard (I don't, off the top of my head, know
if the CL standard does in fact mandate this) .

The same, I think is true for tail-call elimination.  For the standard
to mandate that it must happen in a way that has meaning would require
an enormous amount of hair in the standard, some small amount of which
Erik has demonstrated. This hair would probably become obsolete in a
rather short time.  Far better for the standard to remain silent on
the issue and leave it up to implementors.

Sorry, I was arguing that this is *not* the semantics of Common Lisp.

KMP> It is specifically left to the market to sort out ...

With all due respect, Kent, you overuse this argument.  Pushing it to
its logical extreme, we have no use for a carefully drafted Common
Lisp definition, because the market will choose right anyhow.

When I want something outside the standard implemented in my CL
system, I either go and speak with my vendor, or implement it myself.
Notwithstanding that possibility, I am grateful for the existence of a
rigorously drafted definition of Common Lisp that allows me to write
reliable programs reasonably independently of the implementation.  Alas,
the said definition does not include guarantees about tail-call
elimination are not included in that definition; I would like to see
such guarantees included in a future version.  (Whether my vendor does
actually perform tail-call elimination or not is a completely distinct
argument).

At the same time, I am conscious of the fact that the CL community
(including myself) is not ready right now for a new revision of the
standard.  This doesn't preclude people from making notes about what
users would want to see in a future version; tail-call elimination is
high on my list of desired futures.

(Another thing I would like to see are stronger guarantees about what
conditions are signalled in exceptional situations; again, I do not
have problems with my vendor here, as I can always test my implemen-
tation's behaviour, but with the standard itself.)

Hmm, interesting idea.  Seems Forth-like to me, though, rather than
Lisp-like.  I'm not sure whether I like it, but perhaps I've simply
had too much exposure to ML.

Note, by the way, that including such a function in CL naively would
break certain structural invariants.  Changing your proposed syntax
slightly, the following

  (declaim (special *foo*))

  (defun break-the-system ()
    (let ((*foo* 47))
      (tail-call #'identity nil)))

would cause the virtual machine to push a dynamic environment and jump
into IDENTITY without leaving the dynamic stack unwinding code in the
continuation.  A similar example could be built with UNWIND-PROTECT.

Of course, such cases can be detected statically (i.e. at compile
time) and could be forbidden, but requiring that an implementation
perform such checks imposes a similar burden to requiring automatic
tail-call elimination.

No, Erik, that would not be reasonable.  It would merely replace space
used on the call stack to space used on the dynamic bindings stack or
the unwind stack.

I think the correct approach here would be to ask the users who
actually do require tail-call elimination for their programming style
to specify a minimal set of conditions under which tail-call
elimination should be performed.  In my personal case, I need the
following:

 - tail-call elimination between source files and (less often) between
   packages;
 - tail-call elimination within and to a CLOS generic function;
 - eliminating a tail FUNCALL to a funarg, at least one with a null
   lexical environment.

The obvious case -- tail call to self -- is not interesting, as I
usually find it more natural to replace it with one of the standard
iteration constructs.

Regards,

                                        Juliusz

I, on the other hand, want to use CL (which I happen to find more
comfortable than Scheme) and still require that tail-call elimination
should be performed in some circumstances.

Tail-call elimination, by the way, is not specific to Scheme, and is
mandated for example by ML.  (Both major dialects, I believe.)

EN>   So you think tail-call optimization is _only_ for recursion?
EN>   What _is_ this?

It is only with recursion that tail-call elimination makes a
qualitative (as opposed to merely quantitative) difference.

Without recursion, tail-call elimination can provide you with a
(possibly large) *constant* space saving.  While this may be worth
having, it is not something that belongs in a language definition.

With recursion (possibly indirect), tail-call elimination will, under
the right circumstances, convert unbounded space usage into constant
space.  With some programming techniques, this means turning a
guaranteed crash into a reliable program.

Regards,

I see.  It's actually useless to try to write reliable programs,
because my machine might very well run out of batteries.

  Since the argument _only_ makes sense within the framework of a standard,
  I fail to see how "logical" it could possibly be to drop the context and
  still believe you have the same argument.  

| Alas, the said definition does not include guarantees about tail-call
| elimination are not included in that definition; I would like to see such
| guarantees included in a future version.

  You will not see such guarantees in any future version.  If you want
  this, there is Scheme.  So much of the rest of what you argue for are
  signs of a Scheme freak that I also fail to see why you do not just work
  happily within the Scheme community instead of unhappily in the Common
  Lisp community?  What does it do for you to whine about tail-calls and
  never get it?

| (Another thing I would like to see are stronger guarantees about what
| conditions are signalled in exceptional situations; again, I do not have
| problems with my vendor here, as I can always test my implementation's
| behaviour, but with the standard itself.)

  What do these stronger guarantees actually entail?  Is it the ability to
  sue vendors if they decide to claim to purport to conform to a voluntary
  standard?  If so, nobody will volunteer to conform to it.  We already
  have vendors who think it is OK to claim to purport to something they
  also display a strong disdain for when push comes to shove, and which
  they undermine when they see an opportunity to do so, and the only thing
  we can do with them is expose their destructiveness and applaud their
  constructiveness, but the whole attitude towards the standard is that it
  does not matter if you violate parts of it in ways that you cannot choose
  to ignore.  In my view, there should have been viable ways to solve these
  "incompatibility" problems so they could co-exist with standard ways, or
  even be _within_ the standard, but this is indeed the path less travelled.

Two major points follow.  If you find yourself hopelessly mired in the
first, skip to the second (there are separator markers) so you don't
lose it.

- - - - -

My personal technical position has generally been to add a great deal
more to the language than got in.  In CLHS, I've published the cleanup
issues that got accepted; you should see the heaps of things I have
proposed that did NOT get in, and you should hear the arguments
against them.

My position on this issue has been that tail recursion should be
something people could enable on a lexical basis with declarations.  I
don't plan to use it, but I understand that others might.  As long as
it's not invasive, I'm all for it.

The truth is that I spent a lot of time taking ideas from the Scheme
community and trying to get them into CL to narrow the gratuitous
distance between the two languages.  But that hasn't been seen as
a priority.

The reasons for rejection vary. But two of the most common were "there's
no current practice" (i.e., let a vendor implement it and we'll see how
successful it was in practice) or "this might be painful for some vendor
in a way we don't understand" (i.e., let a vendor implement it and prove
to us that it doesn't get us in trouble).  The other most common reasons
for rejecting otherwise well-formed proposals were "it's too late, we
feature-froze in 1988" or "gee, the spec is getting kind of big, do we
really need this?" or "if you require that, it will cost me a lot of money
to implement, can't we leave it to vendors to decide if it's a priority?".

I try to fairly represent the reasons for various things getting
rejected, to the best of my recollection.  Steve Haflich and I were
there for most of it.  Barry was there for quite a bit of it, though
not quite as much.  Barry and Steve and I are pretty quick to jump on
each other if there's a mis-remembered element, and I think that keeps
the discussion pretty fair and as true to the record as one can be.

But the fact is that it was a major theme of the design to leave
vendors a fair amount of autonomy, especially in the transition from
CLTL to ANSI CL both because the first book had tried to overdefine
some things it shouldn't have, and also because we were already doing
a fair amount of new design with the condition system, clos, etc. and
we didn't want to take more risks than we had to.  One of Larry
Masinter's invaluable contributions to the standard was the creation
of the cleanup form and the requirement to talk about current
practice, among other things.  So the absence of current practice was
conspicuous both for the possibility that it might not be something
any customer was clamoring for (as judged by "enough to make a vendor
fear it was influencing sales", not as judged by "number of newsgroup
posts") and/or it might not be something well enough understood to know
the exact right way to do it or whether it had any ill effects.

This issue WAS discussed.  This issue may or may not have had a
specific proposal--I'm busy bringing my personal notes back online
from backup tape and I may soon be able to answer such questions more
reliably.  But this issue had no action taken for exactly the reason I
cited.

So the frequency with which I use that argument is really irrelevant.
It's like asking why things fall when you drop them and being annoyed
taht "gravity" is used too often to explain the answer.  Truth is truth.

- - - - -

Then again, maybe your real gripe is that "leave it to vendors" is too
vendor-centric an answer.  But to understand why THAT happened, you have
only to note that users were (and are) entitled to be (X3)J13 members,
but over the period of the design, nearly all user members decided it
wasn't worth the economic bother (that is, they decided to stop attending
and trust who was left to vote in their favor).  That left only vendors.
And vendors act in their own interests, so of COURSE the votes were
vendor-centric.  When you voluntarily give up your right to vote, you
deserve what you get.

Even as a vendor employee, and at some risk of annoying my employers,
I regarded this as a war between users and vendors and was angry at
the users for dropping out and apparently expecting vendors to vote
all the right ways.  I spoke out at public forums about this,
blatantly berating users at ALU meetings saying "why don't you stand
up to these vendors and operate as a block to get what you want?",
"why don't you show up and outvote these vendors?".  Honestly, I was
surprised I was not directly fired for my remarks, which in some
companies would have constituted insubordination and would have
merited immediate dismissal.  It is to the credit of both Symbolics
and Harlequin that they quietly permitted me this degree of public
disagreement with their own policies without being angry at me. But
there was a limit to what I could do, and the users didn't exactly
start streaming into meetings.  It was all I could do to get the CLIM
folks to show up at one meeting to argue for compiler optimizers when
vendors were of a mind not to include those in the language because
they each already had their own way of doing things and were not
motivated to impose a burden on themselves to add yet another facility
just for the purpose of satisfying people who didn't care to represent
their own concerns.

We went from about 60 meeting attendees (about 15-20 organizations plus some
alternates, more users than vendors) at the outset to about 8-10 meeting
attendees at the end, representing about 5 or 6 organizations, almost all
vendors.  I think Kim Barrett and Sandra Loosemore, key contributors to the
process, might have been individual members.  That change affected things.

I don't think characterizing that as an "over used argument" is even close to
accurate.  It is a position Kent has stated was used in making design decisions
and as such can only be stated and explained, it is not an argument to be
debunked or proven.  We are free to disagree and/or discuss the merits of such
a position, but you can't refute it logically.

It seems to me to be a very reasonable and productive approach.

As an onlooker to this thread, unfamiliar with the theory of tail call
elimination, I have yet to see any convincing efforts to:
a. define what it is *exactly* pro-tail callers are proposing is added to the
standard.
b. answer any of the specific questions/objections/complications from those
opposed.

Not being familiar with Scheme outside of discussions in this forum, I am
blissfully unaware of the signs that give away a "Scheme Freak" and in my
ignorance I still find their arguments educational.  So there is at least some
good that comes out of "unhappy" lispers "whining" away.....

Coby