When in unwind-protect both the protected-form and the cleanup-form produce an error or just throw, Common Lisp (e.g. GCL) gives preference to the cleanup-form and not the protected-form.
Is there some rationale behind this behaviour? Why isn't the protected-form preferred?
On 2009-01-15 15:18:27 -0500, ulr...@mips.complang.tuwien.ac.at (Ulrich Neumerkel) said:
> When in unwind-protect both the protected-form and the > cleanup-form produce an error or just throw, Common Lisp > (e.g. GCL) gives preference to the cleanup-form and not > the protected-form.
> Is there some rationale behind this behaviour? Why isn't > the protected-form preferred?
> Error: protected-form error > While executing: CCL::CHEAP-EVAL-IN-ENVIRONMENT, in process Listener(5). > Type :POP to abort, :R for a list of available restarts. > Type :? for other options. 1 > :pop > Error: cleanup-form error > While executing: CCL::CHEAP-EVAL-IN-ENVIRONMENT, in process Listener(5). > Type :POP to abort, :R for a list of available restarts. > Type :? for other options.
1 > :pop
I see similar behavior in ecl, sbcl and lispworks.
ulr...@mips.complang.tuwien.ac.at (Ulrich Neumerkel) writes: > When in unwind-protect both the protected-form and the > cleanup-form produce an error or just throw, Common Lisp > (e.g. GCL) gives preference to the cleanup-form and not > the protected-form.
> Is there some rationale behind this behaviour? Why isn't > the protected-form preferred?
The preference is given to the cleanup-forms by all the conformant implementations because it is specified so.
Events 2 and 3 are actually performed interleaved, in the order corresponding to the reverse order in which they were established. The effect of this is that the cleanup clauses of an unwind-protect see the same dynamic bindings of variables and catch tags as were visible when the unwind-protect was entered.
See also the various cll discussions on the subject:
======================================================================== CLISP 2.47 (2008-10-23) (built on galatea.local [192.168.7.7])
Evaluation of (CATCH :RESULT (UNWIND-PROTECT (THROW :RESULT 1) (THROW :RESULT 2))) produced nothing on *STANDARD-OUTPUT* produced nothing on *ERROR-OUTPUT* produced no error produced the following values: --> 2
Evaluation of (CATCH :RESULT (UNWIND-PROTECT (THROW :RESULT 1) (THROW :RESULT 2))) produced nothing on *STANDARD-OUTPUT* produced nothing on *ERROR-OUTPUT* produced no error produced the following values: --> 2
Evaluation of (CATCH :RESULT (UNWIND-PROTECT (THROW :RESULT 1) (THROW :RESULT 2))) produced nothing on *STANDARD-OUTPUT* produced nothing on *ERROR-OUTPUT* produced no error produced the following values: --> 2
======================================================================== International Allegro CL Free Express Edition 8.1 [Mac OS X (Intel)] (Jun 29, 2008 13:34)
Evaluation of (CATCH :RESULT (UNWIND-PROTECT (THROW :RESULT 1) (THROW :RESULT 2))) produced nothing on *STANDARD-OUTPUT* produced nothing on *ERROR-OUTPUT* produced no error produced the following values: --> 2
On 2009-01-15, Ulrich Neumerkel <ulr...@mips.complang.tuwien.ac.at> wrote:
> When in unwind-protect both the protected-form and the > cleanup-form produce an error or just throw, Common Lisp > (e.g. GCL) gives preference to the cleanup-form and not
You can hardly do that what GCL does is Common Lisp behavior.
> the protected-form.
What do you mean by preference to the protected form?
They don't execute concurrently. You mean what if the cleanup form is being executed because of a nonlocal exit in the protected form, and the cleanup form decides to also perform a nonlocal exit?
Firstly, let's discuss the error (or, generally, condition) case. Signaling a condition is not a control transfer. The search for a handler takes place without terminating any forms. A handler may choose to perform a nonlocal exit (for instance by invoking a restart).
Non-local exits may be performed by INVOKE-RESTART, GO, RETURN-FROM and THROW.
> Is there some rationale behind this behaviour? Why isn't > the protected-form preferred?
It makes a lot of sense to abort the original control transfer if a cleanup form invokes another one! What would it mean to continue the original control transfer? That would require control transfer forms in cleanup bodies to have no effect. What should be behavior of the following be:
Let's suppose that CONDITION is true. And let's suppose that the protected form is preferred; i.e. the control tranfer initiated by the protected form (GO END-A) is not aborted by the execution of (GO END-B). So what does that mean?
Let's consider two possible designs:
Design 1:
(GO END-B) is simply ignored, and control passes into (PRINT 2).
Design 2:
that (GO END-B) causes the dynamic control transfer which is currently in progress to simply continue, so that (PRINT 2) (PRINT 3) are not performed, but control simply passes to the label END-A?
I can see serious problems with both of these designs. Those problems stem from the fact that (GO END-B) does not in fact go to the destination END-B.
The behavior is wrestled away from the (GO END-B) form by a ``spooky action at a distance'' so to speak, leading to surprising behaviors.
If the programmer wants the behavior of Design 1 (control transfer is ignored), he can simply not write any control transfer form into the cleanup body.
Any advantage of Design 2 could be achieved by instead giving the preference to the control transfer escaping from the cleanup form, but providing a special form called, say (RESUME) which could be used in a cleanup form to resume the original control transfer, similar to the ``throw;'' construct in C++ for re-throwing the current exception. I.e. the programmer would write (if condition (resume)) which would mean, if the condition is true, then dont' evaluate any more cleanup forms here, just continue with the control transfer. But note that you can already express this idea like this:
(unless condition ... remaining cleanup forms)
I.e. if the condition is true, nothing more is done in the cleanup handler and so control implicitly returns. You can also use a block, and so (RETURN) or (RETURN-FROM) will give the functionality:
(Note that the (RETURN) is a dynamic control transfer, but it does not abort the dynamic control transfer from the protected form. Why? Because its scope is nested within that of the cleanup block; it does not escape. The new control transfer only ``overtakes'' the outer one if it escapes from the cleanup block.)
So with the current design, you have all the flexibility and predictable behavior.
Some relevant sections in the CLHS are
5.3 The Data and Control Flow Dictionary (UNWIND-PROTECT)
``If a non-local exit occurs during execution of cleanup-forms, no special action is taken.'')
5.2 Transfer of Control to an Exit Point.
5.3 The Data and Control Flow Dictionary (THROW)
Here is an example in which a cleanup form invokes another throw:
``The consequences of the following are undefined because the catch of b is passed over by the first throw, hence portable programs must assume that its dynamic extent is terminated. The binding of the catch tag is not yet disestablished and therefore it is the target of the second throw.
(catch 'a (catch 'b (unwind-protect (throw 'a 1) (throw 'b 2))))
''
I.e. during the processing of (THROW 'A 1), the exit point (CATCH 'A ...) can be identified and its dynamic extent can be terminated before the cleanup for the UNWIND-PROTECT are invoked. This is because of the ordering of steps 1 and 2 in described in 5.2. The binding for the catch tag B is still visible, but that exit point is no longer available. Our TAGBODY example doesn't have this problem because the exit point for all the GO forms is established by the same TAGBODY, which remains active, but the example illustrates that there are potential pitfalls when trying to ``hijack'' an existing non-local return.
Depending on the exit point of the non-local return, the one you want to jump from out of the cleanup for may be no longer available, and undefined behavior results.
So you see, if a cleanup form performs a non-local exist, the exit point must not be one that intervened between the original non-local exit and /its/ exit point! It must either be the same one as the target of the in-progress control transfer that is being ``hijacked'', or it must be one which encloses that one. So the new control transfer must either be nested within the cleanup form entirely, or else if it escapes, it must go at least as far as the original. The takeover of the original escape plan can only extend that escape plan; it cannot cancel the escape plan! If you invoke a (THROW 'X ..) to a (CATCH X ...) somewhere, the CATCH is being terminated. If an intervening UNWIND-PROTECT tries to prevent the CATCH from terminating, the behavior is undefined. An intervening UNWIND-PROTECT can either initiate another (THROW 'X ...) to the same catch, or a control transfer to something outside. Either way, the CATCH will be terminated.
So in a very real sense, there is a preference given to the original control transfer: it's distance cannot be rolled back.
>> When in unwind-protect both the protected-form and the >> cleanup-form produce an error or just throw, Common Lisp >> (e.g. GCL) gives preference to the cleanup-form and not >> the protected-form.
>> Is there some rationale behind this behaviour? Why isn't >> the protected-form preferred?
> The preference is given to the cleanup-forms by all the conformant > implementations because it is specified so.
> Events 2 and 3 are actually performed interleaved, in the order > corresponding to the reverse order in which they were > established. The effect of this is that the cleanup clauses of an > unwind-protect see the same dynamic bindings of variables and > catch tags as were visible when the unwind-protect was entered.
This text is actually only tangentially relevant to the issue.
Even though cleanup forms in the unwind protect see the same bindings for places that catch dynamic control, those places may have already terminated their extent as a result of Event 1.
The binding being visible does not define what happens if you actually use the binding!
I quoted again the undefined-behavior example from the description of THROW:
(catch 'a (catch 'b (unwind-protect (throw 'a 1) (throw 'b 2))))
The THROW of B has the inner catch as destination, but that exit point is not available because it was torn down during step 1 of the processing of the THROW of A (it is an intervening exit point in the pat of that transfer).
The upshot is that cleanup clauses can re-initiate a new control transfer, but that transfer must escape at least as far as the original one being hijacked. (To restate the concluding remark from my other article).
Here you avoid the problem because the :RESULT catch is not only visible, but its extent is still intact. It is the exit point for the original throw, and not an /intervening/ exit point that would be torn down.
So the cleanup clause can re-initiate a new control transfer to this same exit point, which takes place of the old control transfer.
The cleanup clause cannot hijack the transfer such that the original exit point fails to be reached, as happens in the undefined behavior example above.
The THROW A guarantees that (if no undefined behavior chicanery occurs) CATCH A will be reached. Once (THROW 'A 1) is initiated, one of the following three must happen:
1. CATCH A terminates and returns value 1
2. CATCH A terminates and returns some other value. (An intervening clean-up form has hijacked the jump with another one to the same catch, using a different return value).
3. CATCH A terminates by way of control passing through it. (An intervening clean-up form has hijacked the control transfer, replacing it with an even greater leap to some enclosing exit point, causing the CATCH A exit point to be disestablished).
So in fact Lisp doesn't give complete precedence to the new transfer. The new transfer has a great deal of control over what happens, but it must not try to shorten the original control transfer.
>On 2009-01-15, Ulrich Neumerkel <ulr...@mips.complang.tuwien.ac.at> wrote: >> When in unwind-protect both the protected-form and the >> cleanup-form produce an error or just throw, Common Lisp >> (e.g. GCL) gives preference to the cleanup-form and not
>You can hardly do that what GCL does is Common Lisp behavior.
>> the protected-form.
>What do you mean by preference to the protected form?
>They don't execute concurrently. You mean what if the cleanup form is being >executed because of a nonlocal exit in the protected form, and the cleanup form >decides to also perform a nonlocal exit?
Yes. But in that situation (offline, no interaction) consider only errors like file-error etc.. I should have been more explicit. Do not consider restarts or other complex handler operations. The protected-form performs the actual work, wherease cleanup-form will remove some of the generated temporary data.
Now an error is signaled in the protected-form and (part of) the state for cleanup-form is corrupted (related to the signalled error). The cleanup-form thus reports its own seemingly unrelated error.
In this manner the actual reason that caused the trouble first is now overshadowed by the minor error.
In a programming situation the debuggerhook break - going into the first signaled error might be a good reaction. But offline, with some toplevel that briefly logs the error and recovers, priorities are differnt. In the examples you gave, the control flows are much more anticipated. That is not in the case in this situation - here errors are caused e.g. by an temporally overflowing file system.
> >On 2009-01-15, Ulrich Neumerkel <ulr...@mips.complang.tuwien.ac.at> wrote: > >> When in unwind-protect both the protected-form and the > >> cleanup-form produce an error or just throw, Common Lisp > >> (e.g. GCL) gives preference to the cleanup-form and not
> >You can hardly do that what GCL does is Common Lisp behavior.
> >> the protected-form.
> >What do you mean by preference to the protected form?
> >They don't execute concurrently. You mean what if the cleanup form is being > >executed because of a nonlocal exit in the protected form, and the cleanup form > >decides to also perform a nonlocal exit?
> Yes. But in that situation (offline, no interaction) consider only > errors like file-error etc.. I should have been more explicit. > Do not consider restarts or other complex handler operations. The > protected-form performs the actual work, wherease cleanup-form will > remove some of the generated temporary data.
> Now an error is signaled in the protected-form and (part of) the state > for cleanup-form is corrupted (related to the signalled error). > The cleanup-form thus reports its own seemingly unrelated error.
What happens if the protected form signals an error? Does it get out of the UNWIND-PROTECT and runs the clean up form? NO. Some available handler gets called in the error context. If the handler then wants to, say, abort and get out of the protect form, the clean-up forms will be executed. If there is an error, again a handler will be called - this time in the context of the clean-up forms.
It is similar to some code that logs the error and just aborts (uses the next abort restart).
It prints first an e1 condition for the protected form and then a e2 condition for the clean-up form.
If you observe something else, you should post a code example. Remember what you observe depends on the handlers and restarts that are available and what they are doing. In a Lisp IDE some handlers and restarts are usually available by default.
> In this manner the actual reason that caused the trouble first > is now overshadowed by the minor error.
> In a programming situation the debuggerhook break - going into the > first signaled error might be a good reaction. But offline, with > some toplevel that briefly logs the error and recovers, priorities > are differnt. In the examples you gave, the control flows are much > more anticipated. That is not in the case in this situation - > here errors are caused e.g. by an temporally overflowing file system.
>>On 2009-01-15, Ulrich Neumerkel <ulr...@mips.complang.tuwien.ac.at> wrote: >>> When in unwind-protect both the protected-form and the >>> cleanup-form produce an error or just throw, Common Lisp >>> (e.g. GCL) gives preference to the cleanup-form and not
>>You can hardly do that what GCL does is Common Lisp behavior.
>>> the protected-form.
>>What do you mean by preference to the protected form?
>>They don't execute concurrently. You mean what if the cleanup form is being >>executed because of a nonlocal exit in the protected form, and the cleanup form >>decides to also perform a nonlocal exit?
> Yes. But in that situation (offline, no interaction) consider only > errors like file-error etc.. I should have been more explicit. > Do not consider restarts or other complex handler operations. The > protected-form performs the actual work, wherease cleanup-form will > remove some of the generated temporary data.
> Now an error is signaled in the protected-form and (part of) the state > for cleanup-form is corrupted (related to the signalled error).
You mean the error is signaled and then a non-local exit takes place which leaves the data in a bad state?
> The cleanup-form thus reports its own seemingly unrelated error.
Cleanup form says, I cannot clean up because the state is fubar.
> In this manner the actual reason that caused the trouble first > is now overshadowed by the minor error.
The error no longer exists at this point. It was already handled. Only the last part of that is happening now: the transfer of control to a restart.
So the situation is: there was some error, and the protocol for that error found a suitable way to continue via a non-local exit to a restart. Only now, the cleanup handler discovers that it's not possible to clean up after the original error.
> In a programming situation the debuggerhook break - going into the > first signaled error might be a good reaction. But offline, with
By offline you seem to mean in the deployment field.
> some toplevel that briefly logs the error and recovers, priorities > are differnt. In the examples you gave, the control flows are much > more anticipated. That is not in the case in this situation - > here errors are caused e.g. by an temporally overflowing file system.
I'd say that, firstly, you have to code the module such that the cleanup forms can always execute. Don't allow corrupt, irrecoverable state! Arguably, a violated invariant is a programming bug. Though there can be things in the external environment that are irrecoverable.
The error handling protocol is broken in this case because the software thinks it can handle the situation and recover, but in fact there is a possiblity that the state may be irrecoverable.
You could design a better protocol to anticipate this situation, maybe along these lines.
Since the cleanup form suspects that the state may be bad, don't put all of the responsibility into the cleanup form. In addition to the cleanup form, you could set up an error handler (which catches the same kind of error). The error handler will actually decline to handle the error by returning, so that the condition looks for another handler. But before returning handler can record information about the error (stash it in a hidden local variable). When the cleanup form later executes, and is not able to clean up, it can look at the stashed error information. Then it has better information based on which to do something. It can signal a stronger form of that error, a variant which says ``this error occured, and state recovery is not possible''. The higher layer of software can distinguish this and react to it differently.
If you can specify how the software should respond to all the possible situations, you can craft the error recovery protocol to match the specification.
ulr...@mips.complang.tuwien.ac.at (Ulrich Neumerkel) writes: > Now an error is signaled in the protected-form and (part of) the state > for cleanup-form is corrupted (related to the signalled error). > The cleanup-form thus reports its own seemingly unrelated error.
No. The original error has already been handled by this point.
Lisp's condition system is much more powerful and sensible than the C++/Java/Python/etc. try/catch exception handling system. Let's examine that system for a bit before I explain why Lisp is different. If you're very unlucky, I'll also explain why Java's exceptions are quite so godawful slow. (I'll pick on Java here because it's an easy target. Err..., no, because it's fairly well known and representative of a number of languages with similar exception handling facilities -- it's better known than Python and it's much simpler than C++. But it is also an easy target -- and GLS should have known better.)
So: an error occurs in our Java program: an exception is constructed (if necessary) and thrown. The runtime system takes over and looks at the call stack. It unwinds the stack until it finds an active `try' block. It looks to see whether the `try' block has attached a `catch' clause which matches the exception, or a `finally' clause. If the latter, it executes the stuff in the `finally' block, and continues the search[1]. If the former, it executes the matching handler. In both of these cases, the `catch' or `finally' code is executed in the same dynamic context as the immediately surrounding stuff -- any functions called within the `try' are dead and gone. They can't come back.
The most important point to notice here is that, in a `finally' block, the exception hasn't yet been caught. When the `finally' code finishes, we'll resume the search for a matching handler.
Right. Lisp is different. (Take that out of context and use it as a slogan.) Lisp, at least in this regard, is /better/. When a condition is signalled (ooh, new words -- think `exception is thrown' if it makes you feel happier, though the concepts are subtly different), the SIGNAL (or ERROR, or whatever) function looks at a table of active handlers, and picks the most recent handler which matches, and invokes it. There and then. No unwinding, nothing. Not even a hint of UNWIND-PROTECT.
The handler can do all sorts of things now. It can return, for instance, in which case the system looks for another handler; if we run out of handlers, the program might continue merrily on its away, blithely assuming that everything has been fixed. Or the handler might make a nonlocal jump -- GO, RETURN, THROW, or INVOKE-RESTART -- in which case we start unwinding stack frames and doing UNWIND-PROTECT stuff. But the critical difference here is that the condition handlers have /already/ had their chance to do something about the condition, and we're just finding a safe place to continue execution.
(HANDLER-CASE does do unwinding before it invokes your handler; but if you cared, you'd use HANDLER-BIND instead, right?)
So, why do Java's exceptions suck? It's because Java exception objects (java.lang.Throwable things) carry a stack backtrace with them, which exception handlers might look at and do things with. But by the time the exception handler's been called, some of the stack frames have already been discarded. Therefore the runtime must reify the stack in advance (or at least build the backtrace while it's unwinding the stack).
Lisp doesn't need to do this. If you register a condition handler using HANDLER-BIND, it gets called in the context of the whatever signalled the condition -- so the stack is still there for poking, prodding, fiddling with or printing. Or debugging, of course.
It's a crying shame that other languages haven't adopted Lisp's clear separation between handling conditions and resuming execution.
[1] Java discards its exception if a `finally' block `ends abruptly' -- e.g., returns, or throws an exception. This really will lose exceptions. Sorry: Java blows goats.
> In a programming situation the debuggerhook break - going into the > first signaled error might be a good reaction. But offline, with > some toplevel that briefly logs the error and recovers, priorities > are differnt.
But that's not a problem. The toplevel HANDLER-BINDs its handler; the condition is signalled, the handler logs its message, and then THROWs or INVOKE-RESTARTs from some safe place. If some UNWIND-PROTECT form in the way finds itself in a bad state and signals a condition, well, that gets handled too. Nothing is lost; everything is good.
On Fri, 16 Jan 2009 02:11:34 +0000, <m...@distorted.org.uk> wrote:
> Right. Lisp is different. (Take that out of context and use it as a > slogan.) Lisp, at least in this regard, is /better/. When a condition > is signalled (ooh, new words -- think `exception is thrown' if it makes > you feel happier, though the concepts are subtly different), the SIGNAL > (or ERROR, or whatever) function looks at a table of active handlers, > and picks the most recent handler which matches, and invokes it. There > and then. No unwinding, nothing. Not even a hint of UNWIND-PROTECT.
> The handler can do all sorts of things now.
Cool stuff, thanks for explanation. This is why asdf can recompile upon encountering stale fasls.
--- Lisp is different - I wonder if I have enough lifetimes to learn it
ulr...@mips.complang.tuwien.ac.at (Ulrich Neumerkel) writes: > Yes. But in that situation (offline, no interaction) consider only > errors like file-error etc.. I should have been more explicit. > Do not consider restarts or other complex handler operations. The > protected-form performs the actual work, wherease cleanup-form will > remove some of the generated temporary data.
> Now an error is signaled in the protected-form and (part of) the state > for cleanup-form is corrupted (related to the signalled error). > The cleanup-form thus reports its own seemingly unrelated error.
> In this manner the actual reason that caused the trouble first > is now overshadowed by the minor error.
> In a programming situation the debuggerhook break - going into the > first signaled error might be a good reaction. But offline, with > some toplevel that briefly logs the error and recovers, priorities > are differnt. In the examples you gave, the control flows are much > more anticipated. That is not in the case in this situation - > here errors are caused e.g. by an temporally overflowing file system.
Use UNWIND-PROTECT-CASE:
(unwind-protect-case () (protected-form) (:normal (format t "PROTECTED-FORM executed normally.~%")) (:abort (format t "PROTECTED-FORM aborted preemptively.~%")) (:always (format t "Either case.~%")))
On Jan 15, 6:11 pm, Mark Wooding <m...@distorted.org.uk> wrote:
> ulr...@mips.complang.tuwien.ac.at (Ulrich Neumerkel) writes: > > Now an error is signaled in the protected-form and (part of) the state > > for cleanup-form is corrupted (related to the signalled error). > > The cleanup-form thus reports its own seemingly unrelated error.
> No. The original error has already been handled by this point.
> Lisp's condition system is much more powerful and sensible than the > C++/Java/Python/etc. try/catch exception handling system. Let's examine > that system for a bit before I explain why Lisp is different. If you're > very unlucky, I'll also explain why Java's exceptions are quite so > godawful slow. (I'll pick on Java here because it's an easy target. > Err..., no, because it's fairly well known and representative of a > number of languages with similar exception handling facilities -- it's > better known than Python and it's much simpler than C++. But it is also > an easy target -- and GLS should have known better.)
First off, thanks for a good post. However, although I agree that Java is an easy target ( :-D ), I don't think it is entirely fair to Guy Steele to say that he "should have known better". First off, Guy was/is not the only person involved in the Java language design choices. Second, I believe Guy has been very forthright in his discussions on the design of Java. Design decisions were made, and there were ramifications for the choices they made. As Guy has said, they were specifically targetting C++ users when designing Java, and they made design decisions that matched that goal. They were very pragmatic about it -- the decisions were intended to match that goal as best they could at the time those decisions were made, and with the information they had at the time.
Guy is an incredibly intelligent person, and I believe him to be a brilliant language designer. Honestly, he comes across as one of the most humble brilliant people I've ever had the pleasure of listening to. In this day and age, that quality gives him extra credibility in my eyes.
While I don't care to program in Java (or C++) if I can avoid it (and yes I have done a considerable amount of work in both languages), I don't believe that the design choices made for the language were the result of Guy not "knowing better". On the contrary, I think Guy *did* know for the most part what they were getting into with the choices they made. Those choices were aimed at a specific goal (well, a set of goals, with one of the primary goals being to attract C++ developers), and I think to a large extent they succeeded in achieving their goals.
In all fairness, I learned Java before learning Lisp, and before that, I had done a fair amount of programming in C++. Java did seem better than C++ in some ways, but I wasn't satisfied with it. My exposure to C++ and Java both made me long for something better, and that is the road that led me to Lisp. So in some sense, I am at least partial evidence that Java has had some success in bringing C++ developers to Lisp (even if it is only because it was better, but not near "better enough").
Mark Wooding wrote: > ulr...@mips.complang.tuwien.ac.at (Ulrich Neumerkel) writes:
> > Now an error is signaled in the protected-form and (part of) the state > > for cleanup-form is corrupted (related to the signalled error). > > The cleanup-form thus reports its own seemingly unrelated error. > Lisp's condition system is much more powerful and sensible than the > C++/Java/Python/etc. try/catch exception handling system.
Be careful what you bundle together there! C++ doesn't have a try/finally exception handling system. Moreover, C++ on Windows usually supports structured exception handling (SEH), which does not use the mechanism you outline.
SEH scans the stack twice. The first time, it calls handlers in reverse order of registration (handlers are registered on function entry) asking them if they handle this particular exception. Any handler can return a condition code to tell the OS to essentially ignore the exception and continue with the next instruction (a bit like VB 'On Error Resume Next'); perhaps the handler repaired the error condition.
It's only after it has found a handler that indicates that it can handle the given exception that the OS goes back and actually unwinds the stack. That process involves calling all the handlers a second time, only this time passing them a flag indicating that unwinding is actually taking place. That's when 'finally' blocks get run. The OS keeps on going until it reaches the handler that indicated it could handle the exception.
That's how things work in Win32. The situation in Win64 is slightly different, as rather than FS:[0] containing a pointer to the head of the exception chain, program-counter-based lookup tables are used instead. However, the two-phase exception dispatch is still used.
> It's a crying shame that other languages haven't adopted Lisp's clear > separation between handling conditions and resuming execution.
Visual Basic does have a 'On Error Resume Next', and the CLR uses an SEH-like mechanism that supports exception filters (it calls them "filter handlers"), which exposes the possibility of resumption to such languages that need it.
> Right. Lisp is different. (Take that out of context and use it as a > slogan.) Lisp, at least in this regard, is /better/. When a condition > is signalled (ooh, new words -- think `exception is thrown' if it makes > you feel happier, though the concepts are subtly different), the SIGNAL > (or ERROR, or whatever) function looks at a table of active handlers, > and picks the most recent handler which matches, and invokes it. There > and then. No unwinding, nothing. Not even a hint of UNWIND-PROTECT.
Except that almost always stack-frames searching is used to find the active handler, Smalltalk exception handling is the same as Lisp', in particular:
> The handler can do all sorts of things now. It can return, for > instance, in which case the system looks for another handler; if we run > out of handlers, the program might continue merrily on its away, > blithely assuming that everything has been fixed. Or the handler might > make a nonlocal jump -- GO, RETURN, THROW, or INVOKE-RESTART -- in which > case we start unwinding stack frames and doing UNWIND-PROTECT stuff. > But the critical difference here is that the condition handlers have > /already/ had their chance to do something about the condition, and > we're just finding a safe place to continue execution.
Regarding this:
> So, why do Java's exceptions suck? It's because Java exception objects > (java.lang.Throwable things) carry a stack backtrace with them, which > exception handlers might look at and do things with. But by the time > the exception handler's been called, some of the stack frames have > already been discarded. Therefore the runtime must reify the stack in > advance (or at least build the backtrace while it's unwinding the > stack).
This is a matter of storing only program counters in an array. It does not cost very much, only this:
> Lisp doesn't need to do this. If you register a condition handler using > HANDLER-BIND, it gets called in the context of the whatever signalled > the condition -- so the stack is still there for poking, prodding, > fiddling with or printing. Or debugging, of course.
Barry Kelly <barry.j.ke...@gmail.com> writes: > Be careful what you bundle together there! C++ doesn't have a > try/finally exception handling system.
True. C++ programmers usually use destructors for the same purpose. I don't think that matters much for the purposes of this discussion, though the correction is appreciated.
> Moreover, C++ on Windows usually supports structured exception > handling (SEH), which does not use the mechanism you outline.
The mechanism I described matches the specification in the C++ standard pretty well. If an implementation doesn't work that way, then it's broken. Implementation extensions are another matter, but `Windows C++' isn't the same as `C++'.
(OK, so technically the extent to which stack unwinding is interleaved with searching for a handler is implementation defined. The difference is detectable, for example, by examining the dynamic context in a `terminate' handler -- C++98 15.3#9 [except.handle]. But the important thing is that the stack is unwound prior to invoking an exception handler or destructor, and that happens the same way in either case.)
[snip interesting description of Windows SEH, which looks like it can emulate Lisp conditions.]
>> It's a crying shame that other languages haven't adopted Lisp's clear >> separation between handling conditions and resuming execution.
> Visual Basic does have a 'On Error Resume Next', and the CLR uses an > SEH-like mechanism that supports exception filters (it calls them > "filter handlers"), which exposes the possibility of resumption to > such languages that need it.
Also interesting. Thanks for your article!
I note that the C# language doesn't expose this richer exception- handling semantics to programmers, despite the language's general objective to provide an interface to the whole of the .NET platform.
(And just as a largely irrelevant aside, could Microsoft have chosen a less searchable name for their platform?)
>> Be careful what you bundle together there! C++ doesn't have a >> try/finally exception handling system.
> True. C++ programmers usually use destructors for the same purpose. I > don't think that matters much for the purposes of this discussion, > though the correction is appreciated.
>> Moreover, C++ on Windows usually supports structured exception >> handling (SEH), which does not use the mechanism you outline.
> The mechanism I described matches the specification in the C++ standard > pretty well. If an implementation doesn't work that way, then it's > broken. Implementation extensions are another matter, but `Windows C++' > isn't the same as `C++'.
> (OK, so technically the extent to which stack unwinding is interleaved > with searching for a handler is implementation defined. The difference > is detectable, for example, by examining the dynamic context in a > `terminate' handler -- C++98 15.3#9 [except.handle]. But the important > thing is that the stack is unwound prior to invoking an exception > handler or destructor, and that happens the same way in either case.)
> [snip interesting description of Windows SEH, which looks like it can > emulate Lisp conditions.]
Note that WIndows SEH is language independent, and not strictly a feature of just Microsoft C and C++. It's part of what you might call the Windows ABI.
It's analogous to POSIX signal handlers. A signal can be delivered to a process even if its executable image wasn't derived from C source code.
The __try/__except/__finally syntax is just the glue to the underlying platform.
Windows excetpions are used for hardware things too, like memory access violations.
They are more sanely designed than C++ or Java exception handling. Windows excpetion handling deos the smart thing: it saves the machine state and searches the stack for a handler without unwinding.
Each exception handling frame may supply a filter expression: a piece of code that is called during the search. This piece of code can return an indication to the the search that it should continue searching, similarly to how a Lisp condition handler can simply return to indicate that it's declining the condition. The filter expression can return two other indications: one is that it wants to handle the exception, which means that a non-local transfer will take place to the rest of the handling code in the local frame. This resembles HANDLER-CASE. The third possibility is that the code which triggered the exception can be restarted. So for instance the filter expression could handle a page fault and restart the faulting instruction.
Of course, Microsoft could not possibly have invented this. It came from DEC: exception handling on VMS and in Digital Unix. (Pardon me, OpenVMS and Tru64). For instance search for a document called ``OpenVMS Calling Standard'', chapter 6.
Paolo Bonzini <paolo.bonz...@gmail.com> writes: > This is a matter of storing only program counters in an array. It > does not cost very much, only this:
[snip]
It costs enough to make Java exceptions a poor mechanism for non-local control transfer. Which is a shame, because there isn't another one.
<m...@distorted.org.uk> wrote: >Barry Kelly <barry.j.ke...@gmail.com> writes:
>> Be careful what you bundle together there! C++ doesn't have a >> try/finally exception handling system.
>True. C++ programmers usually use destructors for the same purpose. I >don't think that matters much for the purposes of this discussion, >though the correction is appreciated.
Yes, some programmers try to use destructors, but destructor semantics are different from try-finally (which C++ does not have). The preferred idiom for emulating try-finally is to use a conditional block after try-catch.
It's actually pretty easy to implement a decent try-finally in C++ using preprocessor macros. It isn't perfect because there are too many ways to implement an exception (dynamically, on the stack, object or primitive type, etc.) and the exception may need to be preserved outside the catch block to be thrown again. But it's not hard if you can pick an implementation and standardize your code base.
>> Moreover, C++ on Windows usually supports structured exception >> handling (SEH), which does not use the mechanism you outline.
>The mechanism I described matches the specification in the C++ standard >pretty well. If an implementation doesn't work that way, then it's >broken. Implementation extensions are another matter, but `Windows C++' >isn't the same as `C++'.
Windows SEH is completely independent of C++. It is part of the Windows API and can be used from any language that can make system calls.
>I note that the C# language doesn't expose this richer exception- >handling semantics to programmers, despite the language's general >objective to provide an interface to the whole of the .NET platform.
.NET (like JVM) is severely lacking in control knobs. A number of important things like threading, mutual exclusion, load file control, etc. have only half-assed, lowest denominator implementations and make your life miserable if you require behavior different from what is provided.
That said, SEH is accessible from all .NET languages - but it doesn't play particularly nicely with any of them.
Ulrich Neumerkel wrote: > When in unwind-protect both the protected-form and the > cleanup-form produce an error or just throw, Common Lisp > (e.g. GCL) gives preference to the cleanup-form and not > the protected-form.
> Is there some rationale behind this behaviour? Why isn't > the protected-form preferred?
(Sorry, I posted this to the wrong subject line...)
By "cleanup handler" I mean the rest of the subforms of unwind-protect after the first one.
The following is mistaken:
I quoted again the undefined-behavior example from the description of THROW:
(catch 'a (catch 'b (unwind-protect (throw 'a 1) (throw 'b 2))))
The THROW of B has the inner catch as destination, but that exit point is not available because it was torn down during step 1 of the processing of the THROW of A (it is an intervening exit point in the pat of that transfer).
(throw 'a 1) enters the Lisp runtime's stack unwinding code. Before it reaches the catch of 'a, it sees the presence of the unwind-protect. So it executes the cleanup handler, namely (throw 'b 2). The catch for 'b then returns. Although 'a was thrown, it is never caught.
Regarding this comment:
[1] Java discards its exception if a `finally' block `ends abruptly' -- e.g., returns, or throws an exception. This really will lose exceptions. Sorry: Java blows goats.
Java and Common Lisp behave the same way. If a condition is signaled with "error" inside the dynamic extent of the cleanup handler, and is not handled within that dynamic scope, then control will be thrown through the unwind-protect and the form of the unwind-protect never returns.
Is this a good thing or a bad thing? Well, it's not easy to say. Suppose a Lisp function F really does contain an unwind-protect handler that could, under some circumstance, signal a condition. Every function should have a well-defined "contract" saying what it promises to do, including under what conditions it will signal conditions (and say what class and anything specific about the condition instance). In functions like F, you have to be very careful to get this right.
At ITA, I introduced a macro called safe-unwind-protect, which makes sure that the above never happens, by wrapping a handler-case for t (everything) around the cleanup handler. We use it more often than plain unwind-protect.
By the way, Guy Steele absolutely understood what he was doing. This whole issue was confusing back in the early Lisp machine days, and I'm pretty sure that any attempt for a cleanup handler to throw went into the debugger, and you had to try manually to figure out what to do. It was, as you'd expect, confusing.
Ulrich Neumerkel wrote: > When in unwind-protect both the protected-form and the > cleanup-form produce an error or just throw, Common Lisp > (e.g. GCL) gives preference to the cleanup-form and not > the protected-form.
> Is there some rationale behind this behaviour? Why isn't > the protected-form preferred?
(Sorry, I posted this to the wrong subject line...)
By "cleanup handler" I mean the rest of the subforms of unwind-protect after the first one.
The following is mistaken:
I quoted again the undefined-behavior example from the description of THROW:
(catch 'a (catch 'b (unwind-protect (throw 'a 1) (throw 'b 2))))
The THROW of B has the inner catch as destination, but that exit point is not available because it was torn down during step 1 of the processing of the THROW of A (it is an intervening exit point in the pat of that transfer).
(throw 'a 1) enters the Lisp runtime's stack unwinding code. Before it reaches the catch of 'a, it sees the presence of the unwind-protect. So it executes the cleanup handler, namely (throw 'b 2). The catch for 'b then returns. Although 'a was thrown, it is never caught.
Regarding this comment:
[1] Java discards its exception if a `finally' block `ends abruptly' -- e.g., returns, or throws an exception. This really will lose exceptions. Sorry: Java blows goats.
Java and Common Lisp behave the same way. If a condition is signaled with "error" inside the dynamic extent of the cleanup handler, and is not handled within that dynamic scope, then control will be thrown through the unwind-protect and the form of the unwind-protect never returns.
Is this a good thing or a bad thing? Well, it's not easy to say. Suppose a Lisp function F really does contain an unwind-protect handler that could, under some circumstance, signal a condition. Every function should have a well-defined "contract" saying what it promises to do, including under what conditions it will signal conditions (and say what class and anything specific about the condition instance). In functions like F, you have to be very careful to get this right.
At ITA, I introduced a macro called safe-unwind-protect, which makes sure that the above never happens, by wrapping a handler-case for t (everything) around the cleanup handler. We use it more often than plain unwind-protect.
By the way, Guy Steele absolutely understood what he was doing. This whole issue was confusing back in the early Lisp machine days, and I'm pretty sure that any attempt for a cleanup handler to throw went into the debugger, and you had to try manually to figure out what to do. It was, as you'd expect, confusing.
> Ulrich Neumerkel wrote: >> When in unwind-protect both the protected-form and the >> cleanup-form produce an error or just throw, Common Lisp >> (e.g. GCL) gives preference to the cleanup-form and not >> the protected-form.
>> Is there some rationale behind this behaviour? Why isn't >> the protected-form preferred?
> (Sorry, I posted this to the wrong subject line...)
> By "cleanup handler" I mean the rest of the subforms of unwind-protect > after the first one.
> The following is mistaken:
The following can't be mistaken because it's from ANSI Lisp. :)
> Regarding this comment:
> [1] Java discards its exception if a `finally' block `ends > abruptly' -- > e.g., returns, or throws an exception. This really will lose > exceptions. Sorry: Java blows goats.
Only goats?
> Java and Common Lisp behave the same way.
No they don't, because as others have pointed out repeatedly in this thread, the procesing of a signaled Lisp condition doesn't unwind the stack.
Java unwinds the stack while searching for a handler; Lisp does not.
This is a very big difference.
> If a condition is signaled > with "error" inside the dynamic extent of the cleanup handler, and is > not handled within that dynamic scope, then control will be thrown > through the unwind-protect and the form of the unwind-protect never > returns.
The signaling of a condition has absolutely no interaction with unwind-protect, because no unwinding is taking place.
Unwinding takes place when an error is handled, not during the search for a handler.
Handling an error means that a matching handler is found, invoked, and instead of returning (to continue the search) that handler terminates not by returning, but by invoking a dynamic, non-local control transfer somewhere. That's how a handler expresses ``I am taking this error, thank you''.
It is at this point that your unwind-protect activates --- /if/ the control transfer jumps far enough to pass through your dynamic contour. At this point, the condition is no longer being signaled.
The error handler might well /not/ make a control transfer that jumps far enough. For instance, the place where the error is signaled (below you) may provide a nearby set of restarts, and the handler (whose binding is established above you, but being invoked without unwinding through you) may find one of these restarts that are below you and invoke it.
> At ITA, I introduced a macro called safe-unwind-protect, which makes > sure that the above never happens, by wrapping a handler-case for t > (everything) around the cleanup handler. We use it more often > than plain unwind-protect.
Ouch, that macro has a behavior that is not very nice!
A handler-case that catches everything intervenes and potentially wrecks all error-handling protocols passing through it. You've seized an error that your code perhaps knows nothing about, and decided to handle it by making a dynamic exit from the handler to your function (implicitly within HANDLER-CASE, since that's what HANDLER-CASE does).
Even if you now re-signal that same condition, so that the search for a handler continues, you've done a chunk of unwinding, which may have wrecked the possibility of restarting!
In general a function has no business intercepting errors this way; you must allow the error search to procede through your dynamic contour and find a proper handler, without interjecting with any unwinding.
The action you want to intercept is the final dynamic control transfer when the restart is invoked.
You can also intercept the search for the error handler---without triggering unwinding.
Use HANDLER-BIND instead of HANDLER-CASE to register a handling function. Use it to handle T (everything). When the error bubbles up, use the handler to make a note of some information related to the error. Then simply return from the handler to decline the error, and cause the search to continue.
Later when your UNWIND-PROTECT cleanup activates, you can make some decisions based on the fact that the error search had passed through earlier.
That's just a rough idea; it would probably have to be refined based on experimenting with actual situations. It's hard to forsee everything. One question is what to do with false alarms: the error search passed your way, but the ensuing non-local transfer did not pass through you, but went to a restart down in the lower-level module, which merrily continued.
You're trying to detect whether there is a bad situation in some other error handling protocol, without participating properly in that protocol, and that is probably not going to work.
I.e. you want to detect that some problem occured in some other error handling that causes your cleanup handlers not to be applicable, and turn that situation into an error. For that to work, you need to have some way of knowing that there was a problem, please don't try to recover. You need to pay attention to that indication even if there is no dynamic control transfer out of your code. (What if the module is irrecoverable, but instead of signaling an error, it eturns an error code?)
On 2009-01-19, Daniel Weinreb <d...@alum.mit.edu> wrote:
> By "cleanup handler" I mean the rest of the subforms of unwind-protect > after the first one.
> The following is mistaken:
> I quoted again the undefined-behavior example from the description > of THROW:
> (catch 'a > (catch 'b > (unwind-protect (throw 'a 1) > (throw 'b 2))))
I've revisited this again, and this time read the X3J13 cleanup issue (exit-extent:minimal) which hinges on the very problem discussed here. I had read it before, but it makes a heck of a lot more sense this time around.
I no longer agree with the standardized behavior, which has consequences like the above throw being undefined.
In fact the reason it was left undefined is to have made it possible for this minimal extent proposal to pass, and what made the proposal passable is that it was easy for implementors.
Those already having the minimal semantics (the catch B is torn down before the cleanup and unwinding takes place, and so is not available) did not have to change. Those having other semantics also didn't have to change, thanks to the magic of undefined behavior!
This is simply a case of the committee codifying broken, inelegant behavior in a move that makes is practically tantamount to disallowing control transfers out of unwind-protect cleanups.
I accept the sane argument presented near the bottom, against the MINIMAL proposal.
I could present my own version of the argument like this: the extent of exit points should be intimately tied to the general idea of dynamic extent of forms. Therefore exit points should not be destroyed in advance, before the unwinding of dynamic scopes takes place.
If we have a (let ....) for a special variable nested within a (catch ...), then that catch tag must not be torn down until the special variable binding is torn down.
Take this example:
(catch 'a (catch 'b (let ((*x* 42)) ;; special defvar (unwind-protect (throw 'a 1) (throw 'b *x*)))))
In (throw 'b *x*), the use of *x* is valid, but not the use of catch tag B is not, even though catch tag B has is in a broader dynamic contour which surrounds that of *X*! This just seems wrong. It requires a notion of exit point extent that is dissociated from dynamic extent (how many flavors of dynamic extent do we need?). So in fact the control transfer initiated by (throw 'a 1) is simply allowed to march up the activation chain and blow off exit points that stand between it, and the target exit point, while the constructs that established those points remain in place. (No unwinding is taking place yet, just ``pre-unwinding annihilation of exits''). This is a dirty behavior, like an explicit free function for deallocating memory, when you already have commited to a more semantically clean model of managed memory.
The excuse for this no longer holds. The behavior may have been voted in 11-5 in 1989, but now it's 2009.
Kaz Kylheku <kkylh...@gmail.com> writes: > I no longer agree with the standardized behavior, which has consequences like > the above throw being undefined.
Finally!
In practise, I don't know of any implementation which wouldn't DTRT. It'd be broken (not by conformance, but by quality.)
Kaz Kylheku <kkylh...@gmail.com> writes: > On 2009-01-19, Daniel Weinreb <d...@alum.mit.edu> wrote: >> By "cleanup handler" I mean the rest of the subforms of unwind-protect >> after the first one.
>> The following is mistaken:
>> I quoted again the undefined-behavior example from the description >> of THROW:
>> (catch 'a >> (catch 'b >> (unwind-protect (throw 'a 1) >> (throw 'b 2))))
> I've revisited this again, and this time read the X3J13 cleanup issue > (exit-extent:minimal) which hinges on the very problem discussed here. I had > read it before, but it makes a heck of a lot more sense this time around.
> I no longer agree with the standardized behavior, which has consequences like > the above throw being undefined.
> In fact the reason it was left undefined is to have made it possible for this > minimal extent proposal to pass, and what made the proposal passable is that it > was easy for implementors. > [...] > The excuse for this no longer holds. The behavior may have been voted in > 11-5 in 1989, but now it's 2009.
It's probably worth a CDR to define a common, defined, behavior.