Damien Kick <dki...@email.mot.com> writes: > > Joe Marshall <j...@ccs.neu.edu> writes:
> > > There are ways to fix UNWIND-PROTECT so that it *is* `iron-clad'. The > > > easiest trick is to disable interrupts:
> > > (defmacro iron-clad-unwind-protect [...])
> But wansn't that, i.e. disabling interrupts, what I tried to do in the > WITH-FOO-ACL form, albeit in a different way than Joe's > IRON-CLAD-UNWIND-PROTECT? Why did it not seem to work correctly?
You turned off multitasking, but control-c was still being checked.
> If I understand Joe's IRON-CLAD-UNWIND-PROTECT correctly, it is > attempting to disable interrupts during the entire evaluation of the > protected form and that this is the problem that Duanne sees with it. > Am I correct in this reading?
No, I was disabling interrupts during the entire *cleanup* form, not the protected form. The funky way in which I was doing it (binding to turn them off, rebinding to re-enable) ensures that the unwinder turns off interrupts when it tries to throw past the frame.
Of course, there may be issues with turning off interrupts during the cleanup, and there are solutions to these as well. The solutions are very hairy, however.
Joe Marshall <j...@ccs.neu.edu> writes: > Duane Rettig <du...@franz.com> writes:
> > There are just two issues left, which I will explain after I > > ask a question:
> > The language you use in describing the 'breakage' of UNWIND-PROTECT, > > plus the dichototomy you use in those CLs which support ashynchronous > > interrupts and those which don't (I know of CLs which do not support > > multitasking, but are there any which don't support asynch interrupts?) > > leads me to believe that you consider it nonconformant (or at least > > "wrong") for a CL to allow interrupts to allow a protected form in an > > UNWIND-PROTECT to terminate early. Is that the case?
> No. The protected form should be interruptable, even asynchronously.
OK, then there was only one issue left, which was the name of the macro (I explained the issue to Damien in a previous posting).
* Joe Marshall | Let me clarify that, though. The cleanup forms do not necessarily | need to run in a `without-interrupts' context. The ideal case would | be that multitasking continues as usual, but asynchronous interrupts | to the particular stack group (thread) that is cleaning up would be | deferred until the cleanup is complete. I'm making the assumption | that as the author of the cleanup code, I can ensure that it is | error-free and completes in a timely manner.
Some of this discussion appears to exhibit some assumptions about what kind of code should be permissible in the cleanup forms of an `unwind-protect´ form, from the implication that not everything would be equally acceptable. Perhaps they should be made explicit?
-- Erik Naggum, Oslo, Norway
Act from reason, and failure makes you rethink and study harder. Act from faith, and failure makes you blame someone and push harder.
Erik Naggum <e...@naggum.no> writes: > * Joe Marshall > | Let me clarify that, though. The cleanup forms do not necessarily > | need to run in a `without-interrupts' context. The ideal case would > | be that multitasking continues as usual, but asynchronous interrupts > | to the particular stack group (thread) that is cleaning up would be > | deferred until the cleanup is complete. I'm making the assumption > | that as the author of the cleanup code, I can ensure that it is > | error-free and completes in a timely manner.
> Some of this discussion appears to exhibit some assumptions about > what kind of code should be permissible in the cleanup forms of an > `unwind-protect´ form, from the implication that not everything > would be equally acceptable. Perhaps they should be made explicit?
The primary assumption is that the cleanup code will in fact clean up if it is allowed to run. The cleanup code would be allowed to throw, return-from, signal errors, etc. But whatever it does, it should be run free of interference from user interrupts or ill-behaving code in other processes so that if it is properly written and debugged, there is no way whatsoever to prevent it from cleaning up.
However, if the cleanup code is run in some sort of protected context, for example, where interrupts are off, it ought to be aware that some actions may deadlock. Here's an example:
(defun example () (let ((foo nil)) (unwind-protect (progn (setq foo (acquire-foo ...)) (perform-computation foo)) (when foo (release-foo foo) (format t "~&Released a foo."))))) ;; ****
The line with the asterisks looks innocuous enough, but it could be a problem if UNWIND-PROTECT disabled multitasking. If *standard-output* were bound to a window object, it might have to acquire a lock, but with multitasking disabled, it could deadlock.
But this is a general issue with multitasking and not specific to UNWIND-PROTECT.
> > > > There are ways to fix UNWIND-PROTECT so that it *is* `iron-clad'. The > > > > easiest trick is to disable interrupts:
> > > > (defmacro iron-clad-unwind-protect [...])
> > But wansn't that, i.e. disabling interrupts, what I tried to do in > > the WITH-FOO-ACL form, albeit in a different way than Joe's > > IRON-CLAD-UNWIND-PROTECT? Why did it not seem to work correctly?
> You turned off multitasking, but control-c was still being checked.
Joe, thanks again for helping me with my question as much as you have.
It would seem that your IRON-CLAD-UNWIND-PROTECT* is turning off multi-tasking but still allowing for C-c to be checked, too. When I use the latest version of WITH-FOO that uses IRON-CLAD-UNWIND-PROTECT*, I still see an "acquire foo" message without a corresponding "release foo" message when it is interrupted during evaluation.
PG(26): (in-package :pg-user) #<The PLAYGROUND-USER package> PG-USER(27): (with-foo (x 69) (format t "use foo ~A~%" (foo-id-number x))) acquire foo 69 use foo 69 release foo 69 NIL PG-USER(28): (setq *time-to-sleep* 1000) 1000 PG-USER(29): (with-foo (x 69) (format t "use foo ~A~%" (foo-id-number x))) acquire foo 69 Break: Interrupting the process of the currently selected IDE Listener (Listener 1), which is doing an evaluation. PG-USER(30):
I'm still assuming that my ACQUIRE-FOO code with the sleep in the middle of it is an acceptable simulation of having ACQUIRE-FOO complete successfully but the interrupt occurring before the SETQ can capture the result. Is this a good assumption?
If I was experimenting with some code and needed to interrupt it, the evalution would still leak a FOO. How can one write a WITH-FOO so that the result would be as follows?
PG-USER(29): (with-foo (x 69) (format t "use foo ~A~%" (foo-id-number x))) acquire foo 69 Break: Interrupting the process of the currently selected IDE Listener (Listener 1), which is doing an evaluation. release foo 69 PG-USER(30):
Would it not require that IRON-CLAD-UNWIND-PROTECT* disable all kinds of interrupts during evaluation of its setup-form? Is this possible with ACL? I realize that this might lead to other problems if the setup-form is ill-behaved; i.e. it might be analgous to allowing a UNIX process to ignore SIGKILL and, therefore, become completely unstoppable. Of course, if all interrupts that might be raised from the top level are disabled, one should still be able to kill the entire Lisp engine from outside of itself. Would one have to make use of finalization, assuming that it would work in this case, to close all of the possible holes? If one would have to resort to using finalization, would this not take the WITH-FOO idiom down a peg or two?
Damien Kick <dki...@email.mot.com> writes: > It would seem that your IRON-CLAD-UNWIND-PROTECT* is turning off > multi-tasking but still allowing for C-c to be checked, too. When I > use the latest version of WITH-FOO that uses > IRON-CLAD-UNWIND-PROTECT*, I still see an "acquire foo" message > without a corresponding "release foo" message when it is interrupted > during evaluation.
Hmmm. I was pretty sure that EXCL::*WITHOUT-INTERRUPTS* really turned them off. It is possible that something is turning them back on, though.
Could you try this:
(defun acquire-foo (id-number) (let ((value (make-foo :id-number id-number))) (format t "Before acquiring interrupts are ~:[en~;dis~]abled.~%" excl::*without-interrupts*) (format t "acquire foo ~A~%" (foo-id-number value)) (sleep *time-to-sleep*) (format t "After acquiring interrupts are ~:[en~;dis~]abled.~%" excl::*without-interrupts*) value))
> I'm still assuming that my ACQUIRE-FOO code with the sleep in the > middle of it is an acceptable simulation of having ACQUIRE-FOO > complete successfully but the interrupt occurring before the SETQ can > capture the result. Is this a good assumption?
I'm not sure. The call to sleep might be the problem.
> If I was experimenting with some code and needed to interrupt it, the > evalution would still leak a FOO. How can one write a WITH-FOO so > that the result would be as follows?
> PG-USER(29): (with-foo (x 69) > (format t "use foo ~A~%" (foo-id-number x))) > acquire foo 69 > Break: Interrupting the process of the currently selected IDE Listener > (Listener 1), which is doing an evaluation. > release foo 69 > PG-USER(30):
> Would it not require that IRON-CLAD-UNWIND-PROTECT* disable all kinds > of interrupts during evaluation of its setup-form? Is this possible > with ACL?
It is possible with ACL (I've done it), but there are some subtleties and no doubt I've missed one. Unfortunately, I don't read usenet on a machine that has ACL on it, so I can't debug it easily.
> I realize that this might lead to other problems if the > setup-form is ill-behaved; i.e. it might be analgous to allowing a > UNIX process to ignore SIGKILL and, therefore, become completely > unstoppable. Of course, if all interrupts that might be raised from > the top level are disabled, one should still be able to kill the > entire Lisp engine from outside of itself.
Yes. You could write a Lisp program that is completely non-responsive (the OS could always kill it, though).
> Would one have to make use of finalization, assuming that it would > work in this case, to close all of the possible holes?
Joe Marshall <j...@ccs.neu.edu> writes: > Damien Kick <dki...@email.mot.com> writes:
> > It would seem that your IRON-CLAD-UNWIND-PROTECT* is turning off > > multi-tasking but still allowing for C-c to be checked, too. When I > > use the latest version of WITH-FOO that uses > > IRON-CLAD-UNWIND-PROTECT*, I still see an "acquire foo" message > > without a corresponding "release foo" message when it is interrupted > > during evaluation.
> Hmmm. I was pretty sure that EXCL::*WITHOUT-INTERRUPTS* really turned > them off. It is possible that something is turning them back on, > though.
> Could you try this:
> (defun acquire-foo (id-number) > (let ((value (make-foo :id-number id-number))) > (format t "Before acquiring interrupts are ~:[en~;dis~]abled.~%" excl::*without-interrupts*) > (format t "acquire foo ~A~%" (foo-id-number value)) > (sleep *time-to-sleep*) > (format t "After acquiring interrupts are ~:[en~;dis~]abled.~%" excl::*without-interrupts*) > value))
PG-USER(22): (with-foo (x 69) (format t "use foo ~A~%" (foo-id-number x))) Before acquiring foo 69, interrupts are disabled. acquire foo 69 After acquiring foo 69, interrupts are disabled. use foo 69 release foo 69 NIL PG-USER(23): (setq pg::*time-to-sleep* 1000) 1000 PG-USER(24): (with-foo (x 69) (format t "use foo ~A~%" (foo-id-number x))) Before acquiring foo 69, interrupts are disabled. acquire foo 69 Break: Interrupting the process of the currently selected IDE Listener (Listener 1), which is doing an evaluation. PG-USER(25):
Please excuse the following list of newbie questions...
Why is directly setting values for EXCL::*WITHOUT-INTERRUPTS* better than using MP:PROCESS-ALLOW-SCHEDULE and either EXCL:WITHOUT-INTERRUPTS (aka MP::WITHOUT-INTERRUPTS) or MP:WITHOUT-SCHEDULING ? My first version of WITH-FOO-ACL attempted to use these functions from the MULTIPROCESSING and EXCL packages (I'd have look back to remember exactly which ones I tried) to disable and then re-enable interrupts and, when they did not do the trick, I assumed that Joe was using EXCL::*WITHOUT-INTERRUPTS* because it would be better. However, using EXCEL::*WITHOUT-INTERRUPTS* directly does not seem to work any better than my original WITH-FOO-ACL. Do these functions indeed do the same thing as accessing EXCL::*WITHOUT-INTERRUPTS* directly? It certainly seems to be the case for EXCL:WITHOUT-INTERRUPTS from the following expansion.
Again, I can't make any sense of the results of using DISASSEMBLE on PROCESS-ALLOW-SCHEDULE but the description in the documentation <http://www.franz.com/support/documentation/6.2/doc/pages/operators/mp...> makes me think that it just might be morally equivalent to setting EXCL::*WITHOUT-INTERRUPTS* to NIL, though I am a bit confused by the wording used therein. Are the functions equivalent? If they are, would they not be preferred, for stylistic reasons, because they are external symbols and, therefore, meant to be used outside of their packages? And why is WITHOUT-INTERRUPTS not external to the MULTIPROCESSING package? Is it because one should be using MP:WITHOUT-SCHEDULING and it will decide on whether or not it can use EXCL:WITHOUT-INTERRUPTS based on the threading model being used?
> > I'm still assuming that my ACQUIRE-FOO code with the sleep in the > > middle of it is an acceptable simulation of having ACQUIRE-FOO > > complete successfully but the interrupt occurring before the SETQ > > can capture the result. Is this a good assumption? > I'm not sure. The call to sleep might be the problem.
<shrug> I can't make any sense of "(disassemble 'sleep)". I see a SYS::C_INTERRUPT in the comments somewhere. The name certainly implies something is happening with interrupts. Perhaps interrupts are re-enabled so that they can be used in the implementation of MP:PROCESS-SLEEP (as the ACL documentation <http://www.franz.com/support/documentation/6.2/doc/pages/operators/mp...> tells us the CL:SLEEP is changed to use MP:PROCESS-SLEEP under the hood). Totally guessing.
Joe Marshall <j...@ccs.neu.edu> writes: > Damien Kick <dki...@email.mot.com> writes:
> > It would seem that your IRON-CLAD-UNWIND-PROTECT* is turning off > > multi-tasking but still allowing for C-c to be checked, too. When I > > use the latest version of WITH-FOO that uses > > IRON-CLAD-UNWIND-PROTECT*, I still see an "acquire foo" message > > without a corresponding "release foo" message when it is interrupted > > during evaluation.
> Hmmm. I was pretty sure that EXCL::*WITHOUT-INTERRUPTS* really turned > them off. It is possible that something is turning them back on, > though.
> Could you try this:
> (defun acquire-foo (id-number) > (let ((value (make-foo :id-number id-number))) > (format t "Before acquiring interrupts are ~:[en~;dis~]abled.~%" excl::*without-interrupts*) > (format t "acquire foo ~A~%" (foo-id-number value)) > (sleep *time-to-sleep*) > (format t "After acquiring interrupts are ~:[en~;dis~]abled.~%" excl::*without-interrupts*) > value))
PG-USER(22): (with-foo (x 69) (format t "use foo ~A~%" (foo-id-number x))) Before acquiring foo 69, interrupts are disabled. acquire foo 69 After acquiring foo 69, interrupts are disabled. use foo 69 release foo 69 NIL PG-USER(23): (setq pg::*time-to-sleep* 1000) 1000 PG-USER(24): (with-foo (x 69) (format t "use foo ~A~%" (foo-id-number x))) Before acquiring foo 69, interrupts are disabled. acquire foo 69 Break: Interrupting the process of the currently selected IDE Listener (Listener 1), which is doing an evaluation. PG-USER(25):
I just got home and started to experiment a little. I'm going to take a wild guess that you are using the Allegro IDE for development. (Some guess. It says IDE in your output.)
At CII, we were using the Emacs interface.
Since the IDE is a windowing system, it probably has a `message pump', and I bet that the message pump runs in a separate thread even if interrupts are disabled. Probably the control-c is caught in the message pump below the level of Lisp, so without-interrupts is not having an effect.
If you are not wedded to the IDE, try it in a non-IDE lisp.
If you are wedded to the IDE, we'll have to poke around a lot more to find out how the control-c is delivered to the appropriate lisp thread.
> > > It would seem that your IRON-CLAD-UNWIND-PROTECT* is turning off > > > multi-tasking but still allowing for C-c to be checked, too. When I > > > use the latest version of WITH-FOO that uses > > > IRON-CLAD-UNWIND-PROTECT*, I still see an "acquire foo" message > > > without a corresponding "release foo" message when it is interrupted > > > during evaluation.
> > Hmmm. I was pretty sure that EXCL::*WITHOUT-INTERRUPTS* really turned > > them off. It is possible that something is turning them back on, > > though.
> > Could you try this:
> > (defun acquire-foo (id-number) > > (let ((value (make-foo :id-number id-number))) > > (format t "Before acquiring interrupts are ~:[en~;dis~]abled.~%" excl::*without-interrupts*) > > (format t "acquire foo ~A~%" (foo-id-number value)) > > (sleep *time-to-sleep*) > > (format t "After acquiring interrupts are ~:[en~;dis~]abled.~%" excl::*without-interrupts*) > > value))
> PG-USER(22): (with-foo (x 69) (format t "use foo ~A~%" (foo-id-number x))) > Before acquiring foo 69, interrupts are disabled. > acquire foo 69 > After acquiring foo 69, interrupts are disabled. > use foo 69 > release foo 69 > NIL > PG-USER(23): (setq pg::*time-to-sleep* 1000) > 1000 > PG-USER(24): (with-foo (x 69) (format t "use foo ~A~%" (foo-id-number x))) > Before acquiring foo 69, interrupts are disabled. > acquire foo 69 > Break: Interrupting the process of the currently selected IDE Listener > (Listener 1), which is doing an evaluation. > PG-USER(25):
When SLEEP is called, the current process is put on a queue to be awoken at a later time. Then control is transferred to the process scheduler. This, of course, transfers control to the next waiting process, or does some `idle' processing. Part of that `idle' processing is handling control-c.
So rather than using SLEEP to test if there is a race condition, you should write your own `time wasting' routine. Here's one:
(defparameter *fibarg* 35)
(defun waste-time (seconds) (labels ((fib (n) (if (< n 2) n (+ (fib (1- n)) (fib (- n 2)))))) (dotimes (i seconds) (fib *fibarg*))))
(adjust *fibarg* to suit your taste, and make sure you compile waste-time)
Using this instead of SLEEP should show that the IRON-CLAD-UNWIND-PROTECT* is working.
> > > It would seem that your IRON-CLAD-UNWIND-PROTECT* is turning off > > > multi-tasking but still allowing for C-c to be checked, too. When I > > > use the latest version of WITH-FOO that uses > > > IRON-CLAD-UNWIND-PROTECT*, I still see an "acquire foo" message > > > without a corresponding "release foo" message when it is interrupted > > > during evaluation.
> > Hmmm. I was pretty sure that EXCL::*WITHOUT-INTERRUPTS* really turned > > them off. It is possible that something is turning them back on, > > though.
> > Could you try this:
> > (defun acquire-foo (id-number) > > (let ((value (make-foo :id-number id-number))) > > (format t "Before acquiring interrupts are ~:[en~;dis~]abled.~%" excl::*without-interrupts*) > > (format t "acquire foo ~A~%" (foo-id-number value)) > > (sleep *time-to-sleep*) > > (format t "After acquiring interrupts are ~:[en~;dis~]abled.~%" excl::*without-interrupts*) > > value))
> PG-USER(22): (with-foo (x 69) (format t "use foo ~A~%" (foo-id-number x))) > Before acquiring foo 69, interrupts are disabled. > acquire foo 69 > After acquiring foo 69, interrupts are disabled. > use foo 69 > release foo 69 > NIL > PG-USER(23): (setq pg::*time-to-sleep* 1000) > 1000 > PG-USER(24): (with-foo (x 69) (format t "use foo ~A~%" (foo-id-number x))) > Before acquiring foo 69, interrupts are disabled. > acquire foo 69 > Break: Interrupting the process of the currently selected IDE Listener > (Listener 1), which is doing an evaluation. > PG-USER(25):
"Joe Marshall" <prunesqual...@attbi.com> writes: > Nope, I was wrong. The problem is in SLEEP.
<nod>
> When SLEEP is called, [... what sleep does ...].
Is this documented in a place available to a user of the trial edition so that when someone tells me to RTFM in the future I at least know where to find TFM? I couldn't find that level of detail in the description of PROCESS-SLEEP <http://www.franz.com/support/documentation/6.2/doc/pages/operators/mp...>.
> So rather than using SLEEP to test if there is a race condition, you > should write your own `time wasting' routine. Here's one:
<nod> Indeed, this was the case. Again, thank you for being so persistent with your help.
> > When SLEEP is called, [... what sleep does ...].
> Is this documented in a place available to a user of the trial edition > so that when someone tells me to RTFM in the future I at least know > where to find TFM? I couldn't find that level of detail in the > description of PROCESS-SLEEP > <http://www.franz.com/support/documentation/6.2/doc/pages/operators/mp...>.
I'm not sure. I disassembled the code for sleep to find out what it was *really* up to.
> <nod> Indeed, this was the case. Again, thank you for being so > persistent with your help.
