there's something that's had me curious for some time now. Given who
wrote the relevant CLtL2 chapter, it seems like a question directed
particularly at Kent Pitman, but of course any feedback is welcome.

So, I'm interested in knowing the history and background behind the CL
conditions and restarts system. I know it drew directly from the dialects
it was standardising, in particular, I seem to rememeber reading another
post that mentioned Symbolics had a particularly comprehensive collection
of standard conditions. However, I'd like to know where it came from to
Symbolics. Was Lisp the first to have conditions? Was it influenced by
something else? Was it very much ahead of other languages by having a
conditions system?

Even more interesting are restarts. Whereas most serious languages today
offer exceptions, ones to include a built-in support for restarting
computation afterwards are exceptionally rare; in fact, I haven't
encountered any that wouldn't be Lisps (and Dylan is a Lisp for purpose
of this classification. From what I understand, Smalltalk also has
restarts, but it's arguably in the Lisp family as well). Casual googling
around doesn't suggest I missed any very prominent languages, either.
There's a considerable mental gap between having a system that supports
non-local transfer control in the form of exceptions, and a system that
includes a formalised protocol for recovering and restarting computation,
as evidenced by the lack of languages to offer such a facility.

I'm still amazed by how woefully incomplete an exception system without
structured means of recovery is, yet how I never felt that prior to
learning CL. Therefore I'm really interested in learning the origins of
the concept. I suspect the strong tradition of interactive development
and built-in debugger naturally leads to the development of a complete
error handling system, whereas in environments where successful and
failed execution alike leads to process termination, one might never
consciously realise such a need. But that's just my guess, which might,
or might not have historical evidence to support it. I hope the village
elders will enlighten me in this regard :)

  http://gigamonkeys.com/book/beyond-exception-handling-conditions-and-...

has examples for when HANDLER-CASE won't cut it and you'll need
HANDLER-BIND instead.

--

Lisp is not dead, it just smells funny.

HANDLER-BIND, on the other hand, does not unwind the stack before
invoking your handlers. So if the signalling site has established any
restarts, you can invoke them.

Months or even years later, some of them called me back and said kind
of sheepishly, "uh, we tried that thing and gee, it seems to work. can
we use it?"  And then gradually one started to see marketing
literature at conferences saying they implemented Kent Pitman's
condition system, which was kind of flattering, but pointed to the
fact that it didn't have a catchy name.  

(But at least my case was better than what happened to David Gray at
TI, where his streams proposal came to be called Gray Streams and then
people on the other side of the pond started re-spelling it Grey
because they thought that it was a color ... or at least a colour.)

Anyway, the pollination of an idea from one language to another is
a tricky business that doesn't happen easily.

Just look at the similar difficulty we've had getting pretty printing
into the world in general.  Dick Waters wrote papers back in the early
80's showing that the techniques we use in CL are perfectly usable for
pretty printing code in block structured languages like Ada and PL/1.
Yet neither those languages nor their successors picked up on the option
to add such a facility to their programmatic repertoire, perhaps because
they were already too busy not picking up on the idea of having a
homoiconic language...

Several of the guys who designed the Lisp Machine had previously been
using that system, which amounted to a "PL/1 machine" (not in quite
the sense that a LispM was a "Lisp Machine", but certainly in the
sense that the operating system was written in a high level language,
PL/1, and there was a unifying execution model on the stack which was
best described by PL/1).

The restart facility was not so linguistic as practical. I think it used
some sort of 'on signal' construct (my PL/1 was never advanced, though I
used it in school, and is quite rusty now, so at this point accept that
this is all blurry and was second-hand to start with).  

And remember, too, that the world was in a kind of proto-state where the
ideas were coming fast and furious ahead of platforms to implement them on.
So they had the abstractions, but they needed a place to deploy those ideas
in a new context. So they lept to the Lisp Machine, and started to
use that platform as a forum to evolve new ideas.

Dave Moon (David A. Moon in the literature) had done Maclisp for PL/1
and was later a chief architect of the LispM system.

Bernie Greenberg was one of the most prominent Multicians, and
implemented Emacs for Multics in Lisp (no, not GNU Lisp, nor GNU
Emacs.  This was long before those days. I think he used Multics
Maclisp.)  Bernie later did the LMFS file system, among other things,
for Genera.

Others are enumerated in the Revision-18 paper mentioned above.

Anyway, if you follow the people, not the languages, you get a better
sense of the flow of things.  These people hopped from one implementation
and system to another, carrying ideas with them.

Likewise you can see that UNWIND-PROTECT is present in things like ITS
TECO [as the ..N q-register, which was executed upon unwind], but
again the unifying characteristic is the community at MIT in which
this arose.  I'm not sure who created ..N, maybe Steele, but certainly
Steele was present, and was present for UNWIND-PROTECT in CL, and later
for "finally" in Java.

Actually, what was strange and frustrating was that Steele was so
involved in Java and didn't put restarts into it, when he surely must
have known of them... and I think they mesh so nicely with
continuations, etc. that it's a shame.  

For myself, I used Lisp Machines, and liked them, but there were not
many of them and many people couldn't use them. I was amused and
saddened by how many of their ideas were regarded as "needing special
hardware", and I made it a personal mission to show that many LispM
ideas could be rehosted on stock hardware (i.e., off-the-shelf, or
non-special-purpose hardware) just fine.  That was how my ZBABYL
mail reader came about for TECO-based Emacs, and it's why I took an
interest in getting the Zetalisp condition system into CL.  (Early CL
didn't even have ERRSET or IGNORE-ERRORS, if memory serves me, so any
program that ever signaled ERROR under CLTL was dead in the water as
far as guarantees of portability.  But the idea of adding just that
one operator as a way of "fixing" the problem was horrifying to me,
so I wanted as much of the LispM New Error System  as I could get.)

And, incidentally, without a practice of defining the recovery points.
While we got restarts into CL, we were unable to get many specific
restarts in.  And that's an aggravation.  But again it was lack of
experience with the system, not to mention considerable political
disagreement over the nature of the inheritance model (I've spoken on
this in other forums, but ask me if the relevance of this in this
context is not clear and I'll expand), that interfered with getting
some specific conditions and many specific restarts added.

But my theory was that if we could get at least a few in, people would
come to understand the impoverished nature and would naturally want
more.  I have a meta-theory of design that says that if you don't
offer a feature, no one will send bug reports, but if you offer a stub
of a feature, everyone will complain wildly for extensions to it.  So
the real hurdle, IMO, is to get the proverbial "foot in the door".
Having ABORT, CONTINUE, USE-VALUE and STORE-VALUE and having them
defined in only a few places is very weak... but it illustrates an
idea that now people know they need fleshed out.  So it was enough.

I draw your attention to
 http://www.nhplace.com/kent/Papers/Condition-Handling-2001.html#footnote
where I remarked on this phenomenon from a similar but slightly different
angle.  

I think the homoiconicity reference above is also relevant, since the
usefulness of the debugger is also helped by the ability to use a familiar
langauge to operate in it.  Visual Studio finally has ways of doing this
in clumsy ways with Watch and Immediate panes, but for years this was
really hard.  And even now it's marginal and unpleasant by comparison to
Lisp in my opinion--though some might prefer it (at least it's finally to
the point where there can be a debate on the matter).

I think the CL error handling facility is extremely useful and
I always see it as a good sign, if a Common Lisp implementation
not just provides the functionality, but actually uses it.
It helps me. Especially if the development environment
uses it (system facility, REPL, ...) and there are
many useful restarts provided.

If you should ban features that can be dangerous when used by amateurs,
then they should ban C all together.

/andreas

Cheers,
Maciej

http://www.cawtech.demon.co.uk/lisp/ghost-benefits.html

and have submitted it to reddit

http://programming.reddit.com/info/61kgl/comments/

The idea that a feature can be valuable beyond the number of
times it is used is of interest beyond the confines of
c.l.l. I wonder what redditors will make of it?

Do features you do not use confer benefits?
===========================================
A baroque language
------------------
Common Lisp has some abstruse features that are rarely
used. DEFINE-METHOD-COMBINATION allows the programmer to
create his own method combinations. The condition system has
RESTART-BIND in addition to RESTART-CASE.

This leads to the criticism that Common Lisp is a baroque
language with too many features. Part of the logic of the
critique is that programmers gain no benefit from the
features that they do not use. This is too glib. Large
programs need planning, which introduces subtle
complications.  

Why plan?
---------
Why not plunge right in and get on with coding? At the heart
of planning computer programs lies the idea of a contrast
between some bits of code that are routine and other bits
that are tricky. Coding the routine bits always goes
smoothly. Coding the tricky bits sometimes ends in tears;
the code cannot be written as originally designed and the
work done on the tricky bit and on many routine parts of the
program feeding in to it must be scrapped.

When we plan, we lightly sketch in the routine parts of the
program, concentrating our efforts on anticipating the
problems in the tricky bits. The benefit we obtain from
planning is a reduction in the costs imposed upon us by the
the discovery of show-stopping problems in the tricky
bits. We lose the work we did implementing the tricky
bit. That is unavoidable. We lose the work we did on the
routine parts of the program feeding into the tricky bit,
but we only sketched those parts, so the loss is greatly
reduced compared to plunging into coding without looking
ahead.  

The effectiveness of planning
-----------------------------
The benefit of planning flows from the routine parts of the
code being mere sketches. The more that we can classify as
routine, the more effective the the planning process will be
at eliminating wasted effort.

When we plan we make two kinds of mistake.

   1. Sometimes we think that a part of the program is
      tricky when it is actually routine. This causes us to
      spend effort filling in the details earlier than we
      should. If these details survive into the final
      program, the mistake costs us nothing. If this
      detailed worked is discarded later in the planning
      process we pay a price for the mistake. Had we seen
      that the work was routine we would only have sketched
      and only discarded a sketch.

   2. Sometimes we think that a part of the program is
      routine when it is actually tricky. When we come to
      code it we may succeed, in which case our mistake has
      cost us nothing, or we may encounter a show-stopping
      problem and fail. Now we must discard much detailed
      work before trying again. This kind of mistake can be
      very expensive, emerging late in the project and
      impacting much detailed work

Calling a tricky part routine is potentially very
expensive. This creates a painful tension. When we are
conservative and call a routine part tricky, just to be on
the safe side, we are reducing the effectiveness of the
planning process. On the other hand, when we strain to say
that a part of the program is routine, hoping to maximize
the benefit from planning, we had better be right.

Language features and planning
------------------------------
Now that we are clear about how planning delivers its
benefits we can look at how this interacts with the features
present in or absent from our programming language.

For example, we may be called upon to decide whether a
mildly ambitious plan, based around an unusual method
combination is routine or tricky. We need to think ahead. A
bug may show us that the method combination we had planned
on is not in fact quite suitable. If the project is coded in
a language with custom method combinations we may be able to
anticipate that the code is routine. If problems arise, we
can code our way out of trouble. If the project is coded in
a language with a fixed set of method combinations we may
call the code tricky and feel obliged to nail down the
details early on.

The interest arises from the fact that planning needs to be
fairly conservative. As we explained earlier the costs of
the two kinds of error are different. We only call code
routine if we have some notion of Plan B, which we have in
reserve if things don't go smoothly. We are only justified
in calling it Plan B if Plan A usually works. Sometimes Plan
B is to use the advanced version of a language feature. In
this case we gain an advantage, in the planning stage, from
a feature that we don't actually get round to using.

That seems paradoxical. What are the ingredients that make
this happen? There seem to be two.

   1. The feature has to be never used in the ordinary sense
      that actually Fred used it a couple of months ago, but
      that doesn't count because it was a special case. If
      it is genuinely never used it can hardly be counted on
      as Plan B and doesn't help at the planning stage.

   2. It seems most likely to happen with features that
      offer a more general version of a commonly used
      built-in feature or some way of brute forcing past
      rare problems.

For example, Common Lisp has macros PUSH, POP, INCF for
modifying places. You can define your own with
DEFINE-MODIFY-MACRO. The interest is that you can plan with
confidence on the basis of using DEFINE-MODIFY-MACRO.

DEFINE-MODIFY-MACRO has limitations. As Graham explains on
page 169 of ANSI Common Lisp neither PUSH nor POP can be
defined as modify-macros. The reason that you can plan with
confidence is that you have DEFINE-SETF-EXPANDER available
as Plan B. You might never use DEFINE-SETF-EXPANDER, but you
benefit from it anyway because you can plan on the basis
that code that is going to use DEFINE-MODIFY-MACRO is
routine and leave the details until other parts of your plan
are firmed up.  

Ghost benefits are real!
------------------------
Software projects require planning. Planning gains in
effectiveness the more often you can declare "This bit of
code is routine, we can fill in the details later." The
right kind of advanced programming language feature helps
with this, allowing you to plan with confidence. In this way
programmers benefit from features that they don't use.

"Why do we plan before implementing? The big danger in plunging right
into a project is the possibility that we will paint ourselves into a
corner. If we had a more flexible language, could this worry be
lessened? We do, and it is. The flexibility of Lisp has spawned a whole
new style of programming. In Lisp, you can do much of your planning as
you write the program.

Why wait for hindsight? As Montaigne found, nothing clarifies your
ideas like trying to write them down. Once you're freed from the worry
that you'll paint yourself into a corner, you can take full advantage
of this possibility."

Take a laugh point, if you collect those.

/Andreas

<blockquote cite="http://www.research.att.com/~bs/bs_faq2.html#resume">
Why can't I resume after catching an exception?

In other words, why doesn't C++ provide a primitive for returning to the
point from which an exception was thrown and continuing execution from
there?

Basically, someone resuming from an exception handler can never be sure
that the code after the point of throw was written to deal with the
excecution just continuing as if nothing had happened. An exception
handler cannot know how much context to "get right" before resuming. To
get such code right, the writer of the throw and the writer of the catch
need intimate knowledge of each others code and context. This creates a
complicated mutual dependency that wherever it has been allowed has led
to serious maintenance problems.

I seriously considered the possibility of allowing resumption when I
designed the C++ exception handling mechanism and this issue was
discussed in quite some detail during standardization. See the exception
handling chapter of The Design and Evolution of C++.

If you want to check to see if you can fix a problem before throwing an
exception, call a function that checks and then throws only if the
problem cannot be dealt with locally. A new_handler is an example of this.
</blockquote>

If I write

 { f(x); g(x); }

in a traditional language, you could get all alarmist and insist that
f must never return  because there's no guarantee that g will do the
right thing .. perhaps it doesn't expect f to return.

I think the great insight of the New Error System (on the Lisp Machine)
was that there was nothing special about returning from an error that
distinguishes it from any other kind of return.

Part of the documentation of a function is that it either is or is not
expected to return.  So error does not return and signal does, and people
choose which they want to call based on their willingness to fall through.
But certainly the creation of a restart point is a proof that there's
a willingness to return, so I just don't see why that's an issue at all.

Another possible way of restating what Stroustrup is saying, and I don't
mean to put words in his mouth--I'm just speculating trying to figure
out what he meant--is that "people shouldn't program dynamically twisty
combinations of function calls and closures".  That would again be
ridiculous, so I hope he's not saying that.  If you look at the sample
code for the condition system you'll be struck by how it's nothing more
than a bunch of very straightforward function calls and macros.  So the
question is how that could be dangerous without, basically, all function
calls and/or macros being dangerous.  I just don't get it.

I have for a long time described restarts as being just
continuations+reflection.  They happen at a point in a program when
the choice point is found but you don't know which continuation to
invoke.  But surely there's nothing wrong with invoking a continuation
any more than returning or any more than invoking a delegate.

And it would work fine with strong typing, too, since under each
protocol there is a specific set of arguments that things take.

What he may be referring to is that C++ made a mess of the unwind-protect
operation, which you have to kludge up (as boost does) through destructors
on stack-allocated objects.  That may mean implicitly that people who were
not maintaining decent stack discipline were rudely exposed by injecting
restarts too easily into code that was not prepared for that.  But I don't
think it's fair to lay the blame on restarts for that, since the real blame
if that's the case goes to the language for having created a non-modular
case.

I'd liken it to the practice I've seen at some point in the past of
vendors saying "we poll for interrupts, and only in certain specific
and predictable places, so it's ok to not write without-interrupts
if you're in what wants to be a critical section because you're implicitly
in one just because the implementation promises".  That's just broken,
since the implementation should make you write the without-interrupts
(or without-preemption or whatever) and then it should optimize cases
where it knows that the rule allows it to be removed.  Doing anything else
means that if someone goes to change the compiler, all code will break.
And yet, it's not fair to say "that means it's dangerous to poll in other
places than were originally thought of".  What's dangerous is, instead,
to tell people they can rely on contingent truths [as per Kripke].

It is completely ridiculous to claim that any materially useful
protocol can be built on his workaround suggestion:
 | If you want to check to see if you can fix a problem before
 | throwing an exception, call a function that checks and then
 | throws only if the problem cannot be dealt with locally.
since the WHOLE POINT of the error system is not to add functionality
but to add protocol.  And protocol is not about calling your own functions,
it's about calling those supplied by someone else, someone who didn't
conspire with you in the design, but rather plugged into a framework
on the promise there would be someone at the "other end". By not providing
a way to register something that could do this check, and by not providing
a way to search the registered checks, he has forced each user into doing
precisely what each user is not empowered to do: to write their own
condition system.

When I wrote the sample implementation [1] of Revision 18 of the CL
condition system ages ago there was no part I couldn't write in terms
of the actions I wanted to happen.. the only part I couldn't write was
"making everyone call my functions".  That part has to be given by the
system.  And if it is not, that's the end of the discussion.

There may also have been an issue for C++ that passing arguments over the
control return was hard.  That was before delegates, and it is such a
pain in the neck to use parameterized functions that I can easily imagine
someone not wanting to bother (though I assume Stroustrup could have
managed it).  And macro abstraction in C++ is a pain, which could have
contributed, too.  So maybe again he was really complaining about how
darned cumbersome and inflexible C++ is syntactically.  I'd agree with
that if he wanted to say that.  

C# could have, and should have, done better.  (I'm not a big fan of
static languages, but I do use them, and among that space of
languages, I really like the design of C# as a local optimum in a
design space I wish I didn't have to spend much time in).  It's a bit
disappointing that C# didn't get this right because it did make some
good decisions and really thoughtful decisions on a number of other
things.  Maybe it's because the CLR has no support for it.  Ditto with
the JVM and Java.  Alas.

But nothing that any of these languages couldn't just suddenly fix if
they get a mind to.

The problem is that whenever there is an exception, there are generally
multiple ways to get out of the exceptional situation and resume. In a
dynamic language, this is pretty straightforward: You present the
different possibilities to the programmer / user and let them decide
what to do. (This could, for example, be presented in user-friendly
dialogs.)

In a static language, you would have to rely on automagically picking
the 'correct' restart - and I guess that's where the mental model of the
average static thinker doesn't cope anymore and they just bail out.

Pascal

The ones that are due to program bugs are hardware exceptions like bus
error or segmentation violation, which are caused by improper use of
pointers.

See http://java.sun.com/j2se/1.5.0/docs/api/java/lang/Exception.html and
http://java.sun.com/j2se/1.5.0/docs/api/java/lang/RuntimeException.html

BTW, my favorite Java exception is this:
http://java.sun.com/j2se/1.5.0/docs/api/java/lang/UnsupportedOperatio...

;)

Pascal

Pascal

1. Conditions and restarts are just functions + some syntactic sugar
around    them to aid with common cases.

2. There is an associated *protocol*. The language "support" for restarts
is actually a tiny addition that allows expressing the protocol in a way
that would be impossible to add in user code. Everything else is just a
small matter of programming. In particular, the only "magic" primitive is
SIGNAL; ERROR and friends are _not_.

The problems arise when you take restarts for more than they are and
associate some magical properties with them (as I did). Consider this:
the standard says a restart, as established by RESTART-BIND, can either
transfer control non-locally, or return. Therefore, you could try to
write the following:

(defun save-data ()
  nil)

(defun can-error ()
  (restart-bind ((ignore (lambda ()
                           nil) ; This should continue after ERROR and DO-MORE-STUFF
                   :report-function (lambda (s) (write "Ignore error and continue anyway"
                                                       :stream s))))
    (unless (save-data)
      (error "Could not save data")
      (do-more-stuff))))

(defun try-and-continue ()
    (handler-bind ((error #'(lambda (c)
                              (invoke-restart 'ignore))))
      (can-error)))

I was initially very confused when it invoked the debugger, and choosing
IGNORE there only produced the message "Restart returned NIL". Doing away
with that confusion takes two steps:

1. Understanding that signalling functions are not magic. The fact that
(error 'foo) breaks into the debugger does not come from the condition,
but from the fact that ERROR is essentially defined as[1]:

(defun error (datum)
  (signal datum)
  (invoke-debugger datum))

2. Because the debugger break is just a function call, and invoking a
restart is also a function call, and the restart itself is no magic, but
a function, all it can do is whatever a function can do. The only odd
thing about restarts is that their dynamic environment is adjusted
slightly at the time they are run. Therefore, if a restarts returns a
value, what will happen is what always happens, ie. its caller will
receive that value. And indeed, the debugger receives it. What *won't*
happen is a magic return to the point where ERROR was called, because
that's not a part of protocol specified by ERROR.

I believe Stroustrup's confusion stems from thinking along the above
code's lines, which gives rise to the fear that once you introduce
restarts, the callers will somehow be able to override each and every
exception and force continuing anyway.

Cheers,
Maciej