[Sbcl-devel] RFC: exiting SBCL vs returning from a thread vs aborting a thread
Nikodemus Siivola
QUIT, however, is not without issues. It occupies an uncomfortable
niche between processes and threads, and doesn't actually do what it
says on the tin unless you call it from the main thread.
Additionally, non-main threads have TERMINATE-THREAD restart, but no
ABORT restart -- but the latter is required by ANSI -- and there is no
"restart invocation function" for TERMINATE-THREAD which makes doing
(make-thread (lambda ()
(handler-bind ((serious-condition #'please-abort-this-thread))
...)))
way more tedious then it should be.
So, I intend to
1. rename the TERMINATE-THREAD restart ABORT -- so
#'please-abort-this-thread above can be just #'abort.
2. deprecate QUIT gently and slowly and introduce the following new operators:
function SB-EXT:EXIT &key code abort
Terminates the process, causing SBCL to exit with CODE. CODE
defaults to 0 when ABORT is false, and 1 when it is true.
When ABORT is false (the default), current thread is first unwound,
*EXIT-HOOKS* are run, and standard output streams are flushed before
SBCL calls exit(2) -- at which point atexit(3) functions will run.
When ABORT is true, SBCL exits immediately by calling _exit(2).
Recursive calls to EXIT cause EXIT to behave as it ABORT was true.
Consequences are unspecified if serious conditions occur during EXIT
excepting errors from *EXIT-HOOKS*, which cause warnings and stop
execution of the hook that signaled, but otherwise allow the exit
process to continue normally.
See also: SB-THREAD:RETURN-FROM-THREAD, SB-THREAD:ABORT-THREAD.
macro SB-THREAD:RETURN-FROM-THREAD values-form &key allow-exit
Unwinds from and terminates the current thread, with values from
VALUES-FORM as the results visible to JOIN-THREAD.
If current thread is the main thread of the process (see
MAIN-THREAD-P), signals an error unless ALLOW-EXIT is true, as
terminating the main thread would terminate the entire process. If
ALLOW-EXIT is true, returning from the main thread is equivalent to
calling SB-EXT:EXIT with :CODE 0 and :ABORT NIL.
See also: ABORT-THREAD and SB-EXT:EXIT.
macro SB-THREAD:ABORT-THREAD &key allow-exit
Unwinds from and terminates the current thread abnormally, causing
JOIN-THREAD on current thread to signal an error unless a
default-value is provided.
If current thread is the main thread of the process (see
MAIN-THREAD-P), signals an error unless ALLOW-EXIT is true, as
terminating the main thread would terminate the entire process. If
ALLOW-EXIT is true, aborting the main thread is equivalent to calling
SB-EXT:EXIT code 1 and :ABORT NIL.
Invoking the initial ABORT restart estabilished by MAKE-THREAD is
equivalent to calling ABORT-THREAD in other than main threads.
However, whereas ABORT restart may be rebound, ABORT-THREAD always
unwinds the entire thread. (Behaviour of the initial ABORT restart for
main thread depends on the :TOPLEVEL argument to
SB-EXT:SAVE-LISP-AND-DIE.)
restart SB-THREAD:ABORT-THREAD
Bound by all threads except the main thread of the process. Invoking
this restarts is equivalent to calling ABORT-THREAD.
function SB-THREAD:MAIN-THREAD-P &optional thread
True if THREAD, defaulting to current thread, is the main thread of the
process.
Full patch attached, and here:
https://github.com/nikodemus/SBCL/commit/83158869d1a3cf586f383c89a3c01b41c31921c9
Does anyone see issues with this? Random comments follow.
I'm mostly really interested in EXIT, but since currently QUIT unwinds
the current thread for non-main-threads, that functionality needed
covering as well and RETURN-FROM-THREAD seems like a logical parallel
to ABORT-THREAD. Given the functions above, the new
backwards-compatible definition for QUIT becomes:
(defun quit (&key recklessly-p (unix-status 0))
#!+sb-doc
"Deprecated. See: SB-EXT:EXIT, SB-THREAD:RETURN-FROM-THREAD,
SB-THREAD:ABORT-THREAD."
(if (or recklessly-p (sb!thread:main-thread-p))
(exit :code unix-status :abort recklessly-p)
(sb!thread:abort-thread))
(critically-unreachable "after trying to die in QUIT"))
RETURN-FROM-THREAD is the bit that's trivial to leave out if it is
deemed unnecessary or undesirable, but IMO it seems like a useful
thing to have.
TERMINATE-THREAD -> ABORT restart renaming is also easy to leave out,
but in all honesty I've been contemplating it for a while now, and
this seems as good a time as any.
&key ALLOW-EXIT is obviously not needed, but most of the time
ABORT-THREAD/RETURN-FROM-THREAD exiting the entire process would be
unintentional for me at least, so making it explicit seems like a nice
thing to do. (The other option would be to make returning from the
main thread wait for all running threads to return, but that's much a
deeper change, and seem liable to cause surprises down the road.) In
the event of finally making the main thread a housekeeping thread some
day, having an explicit ALLOW-EXIT seems even more pertinent for user
code that for one reason or another runs there.
In its current incarnation EXIT also terminates the current session
for non-ABORT case as an undocumented side-effect of unwinding the
current thread for backwards compatibility with QUIT, but if I have my
way sessions will go the way of the dodo sooner than later -- at which
point EXIT will stop terminating other threads, whereas QUIT will
start terminating all running threads before calling exit(2).
Cheers,
-- Nikodemus
Nikodemus Siivola
Par for course, I send the draft without all the fixes. So:
> Additionally, non-main threads have TERMINATE-THREAD restart, but no
> ABORT restart -- but the latter is required by ANSI -- and there is no
s/required/recommended/
Specifically: "Implementors are encouraged to make sure that there is
always a restart named abort around any user code so that user code
can call abort at any time and expect something reasonable to happen;
exactly what the reasonable thing is may vary somewhat."
> restart SB-THREAD:ABORT-THREAD
>
> Bound by all threads except the main thread of the process. Invoking
> this restarts is equivalent to calling ABORT-THREAD.
Ignore this bit. Not part of the proposal at all.
Cheers,
-- Nikodemus
------------------------------------------------------------------------------
Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex
infrastructure or vast IT resources to deliver seamless, secure access to
virtual desktops. With this all-in-one solution, easily deploy virtual
desktops for less than the cost of PCs and save 60% on VDI infrastructure
costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox
_______________________________________________
Sbcl-devel mailing list
Sbcl-...@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/sbcl-devel
Jianshi Huang
<niko...@random-state.net> wrote:
>
> function SB-EXT:EXIT &key code abort
>
> Terminates the process, causing SBCL to exit with CODE. CODE
> defaults to 0 when ABORT is false, and 1 when it is true.
>
> When ABORT is false (the default), current thread is first unwound,
> *EXIT-HOOKS* are run, and standard output streams are flushed before
> SBCL calls exit(2) -- at which point atexit(3) functions will run.
>
> When ABORT is true, SBCL exits immediately by calling _exit(2).
>
In AllegroCL, ABORT is called NO-UNWIND, which I think is more informative.
Is there any case that I don't want to unwind but still want
*EXIT-HOOKS* be called?
If we want to prohibit *EXIT-HOOKS* we can setf it explicitly.
Or how about this:
function SB-EXT:EXIT &key code abort no-unwind no-exit-hooks
and abort means :no-unwind t :no-exit-hooks t
> Recursive calls to EXIT cause EXIT to behave as it ABORT was true.
>
Do you mean another EXIT called from the *EXIT-HOOKS*? Why not just
binding *EXIT-HOOKS* to nil. Am I missing something?
> RETURN-FROM-THREAD is the bit that's trivial to leave out if it is
> deemed unnecessary or undesirable, but IMO it seems like a useful
> thing to have.
>
I think the convenience it brings is worthwhile.
Cheers,
--
黄 澗石 (Jianshi Huang)
http://huangjs.net/
Gábor Melis
> On Sat, Dec 31, 2011 at 10:31 PM, Nikodemus Siivola
> <niko...@random-state.net> wrote:
>>
>> function SB-EXT:EXIT &key code abort
>>
>> Terminates the process, causing SBCL to exit with CODE. CODE
>> defaults to 0 when ABORT is false, and 1 when it is true.
>>
>> When ABORT is false (the default), current thread is first unwound,
>> *EXIT-HOOKS* are run, and standard output streams are flushed before
>> SBCL calls exit(2) -- at which point atexit(3) functions will run.
What about the other threads? They should be [attempted to be] brought
down as well.
>> When ABORT is true, SBCL exits immediately by calling _exit(2).
>>
>
> In AllegroCL, ABORT is called NO-UNWIND, which I think is more
> informative.
As always there is a tradeoff between generality and specificity. ABORT
has the advantage that if the need arises more cleanups can be made
conditional on it. NO-UNWIND provides more information about the current
state of affairs.
> Is there any case that I don't want to unwind but still want
> *EXIT-HOOKS* be called?
> If we want to prohibit *EXIT-HOOKS* we can setf it explicitly.
>
> Or how about this:
>
> function SB-EXT:EXIT &key code abort no-unwind no-exit-hooks
>
> and abort means :no-unwind t :no-exit-hooks t
A superficial ouch: :NO-UNWIND makes me sad, much prefer a non-nil
default as in (UNWINDP T).
Jianshi Huang
> What about the other threads? They should be [attempted to be] brought
> down as well.
>
I can imagine possible deadlock or race issues if termination order
isn't what is expected. But it's a convenience feature if you don't
want to fiddle with *EXIT-HOOKS*.
I suggest we make it an option. m(><)m
>> Or how about this:
>>
>> function SB-EXT:EXIT &key code abort no-unwind no-exit-hooks
>>
>> and abort means :no-unwind t :no-exit-hooks t
>
> A superficial ouch: :NO-UNWIND makes me sad, much prefer a non-nil
> default as in (UNWINDP T).
Yeah...
Cheers,
--
黄 澗石 (Jianshi Huang)
http://huangjs.net/
------------------------------------------------------------------------------
Gábor Melis
> 2012/1/6 Gábor Melis <me...@retes.hu>:
>
>> What about the other threads? They should be [attempted to be] brought
>> down as well.
>>
>
> I can imagine possible deadlock or race issues if termination order
> isn't what is expected. But it's a convenience feature if you don't
> want to fiddle with *EXIT-HOOKS*.
Yes, it's somewhat safer to unwind from the calling thread.
But there is little point in unwinding from one of the threads and
basically aborting the rest.
> I suggest we make it an option. m(><)m
I can't imagine wanting to this behavior. It may be lack of imagination.
Nikodemus Siivola
https://github.com/nikodemus/SBCL/commit/ed4e592fc0f79f95c2e5deef05c8dce946b141a6
Essential change to previous version: EXIT :ABORT NIL will call
TERMINATE-THREAD on all other threads, and then wait for :TIMEOUT
seconds for those threads to actually finish (defaulting to 60.)
Re. recursive exit -> :abort t. This seemed the simplest way to ensure
things remain sane. It would be possible to provide more complex
handling, but I fear it would be tricky to keep documentation and
implementation in synch, and I'm reluctant to get too clever in
absence of use-cases.
Same applies to error handling during exit. There is actually a
mechanism for it, but I left it undocumented in the interest of
simplicity -- if it turns out people need it, we can see if the
current mechanism covers their uses cases, and then document it.
Cheers,
-- Nikodemus
Nikodemus Siivola
> Essential change to previous version: EXIT :ABORT NIL will call
> TERMINATE-THREAD on all other threads, and then wait for :TIMEOUT
> seconds for those threads to actually finish (defaulting to 60.)
Absent cries of horror, I've pushed this.
Cheers,
-- Nikodemus
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
Jianshi Huang
<niko...@random-state.net> wrote:
> 2012/4/26 Nikodemus Siivola <niko...@random-state.net>:
>
>> Essential change to previous version: EXIT :ABORT NIL will call
>> TERMINATE-THREAD on all other threads, and then wait for :TIMEOUT
>> seconds for those threads to actually finish (defaulting to 60.)
>
> Absent cries of horror, I've pushed this.
>
Some feedback for further discussion:
*exit-hooks* are invoked before calling %exit-other-threads, which
seems controversial to me. It means I can't use it for to free
resources or to guarantee all buffers are flushed since other threads
might still operating on them.
I'd suggest to terminate threads first then call the *exit-hooks*.
It would be nicer if we have *thread-exit-hooks* for each thread.
Cheers,
--
黄 澗石 (Jianshi Huang)
http://huangjs.net/
------------------------------------------------------------------------------
Nikodemus Siivola
> Some feedback for further discussion:
>
> *exit-hooks* are invoked before calling %exit-other-threads, which
> seems controversial to me. It means I can't use it for to free
> resources or to guarantee all buffers are flushed since other threads
> might still operating on them.
The reason *EXIT-HOOKS* are called first now is threefold:
(1) make it possible to use them to guarantee an orderly shut-down,
which for me includes non-asynch termination of other threads.
(2) since it is possible that we cannot cleanly terminate a thread
(eg. because it's stuck inside a WITHOUT-INTERRUPTS.), there is now
way to guarantee that we will reach a state where all threads are
running before *EXIT-HOOKS* are called.
(3) calling them early means that it's sane (well, almost) to still
spawn new threads from exit hooks. If we did that after thread
termination, we could not allow it and call the shutdown orderly.
While I don't expect exit hooks spawning threads to be a regular
occurrence, I can still image it happening by both design and harmless
accident.
Still, your point and use-case are both perfectly valid. Maybe we need
a second series of exit hooks as well. Name suggestions?
*AFTER-EXIT-HOOKS* ?
*FINAL-HOOKS* ?
Or maybe have *EXIT-HOOKS* act as you suggest, and add
*BEFORE-EXIT-HOOKS* ?
which could even be called before from the current thread. I think I
like this option the best -- if there's a hook which is called after
unwinding everything, it doesn't matter if the initial hook is called
before.
Thoughts?
> It would be nicer if we have *thread-exit-hooks* for each thread.
Can you clarify this a bit?
As a thread-local binding? Can do that, but don't quite see the point
-- how would it be different from sticking an UNWIND-PROTECT clause to
do the same?
Or do you mean as a global variable, called per thread? I guess that
makes sense, but don't see an immediate application.
Cheers,
-- Nikodemus
Nikodemus Siivola
*EXIT-HOOKS* kept in the place corresponding to where they've been so
far, for backwards compatibility.
Cheers,
-- nikodemus
Jianshi Huang
<niko...@random-state.net> wrote:
> On 30 April 2012 03:14, Jianshi Huang <jiansh...@gmail.com> wrote:
>> Some feedback for further discussion:
>>
>> *exit-hooks* are invoked before calling %exit-other-threads, which
>> seems controversial to me. It means I can't use it for to free
>> resources or to guarantee all buffers are flushed since other threads
>> might still operating on them.
>
> The reason *EXIT-HOOKS* are called first now is threefold:
>
> (1) make it possible to use them to guarantee an orderly shut-down,
> which for me includes non-asynch termination of other threads.
>
> (2) since it is possible that we cannot cleanly terminate a thread
> (eg. because it's stuck inside a WITHOUT-INTERRUPTS.), there is now
> way to guarantee that we will reach a state where all threads are
> running before *EXIT-HOOKS* are called.
>
Ah, I see. Good points. But I think orderly shutdown cannot be done
properly if it's not called from the main thread. Am I missing
anything? What would you suggest it should be done?
> (3) calling them early means that it's sane (well, almost) to still
> spawn new threads from exit hooks. If we did that after thread
> termination, we could not allow it and call the shutdown orderly.
> While I don't expect exit hooks spawning threads to be a regular
> occurrence, I can still image it happening by both design and harmless
> accident.
>
> Still, your point and use-case are both perfectly valid. Maybe we need
> a second series of exit hooks as well. Name suggestions?
>
> *AFTER-EXIT-HOOKS* ?
>
> *FINAL-HOOKS* ?
>
> Or maybe have *EXIT-HOOKS* act as you suggest, and add
>
> *BEFORE-EXIT-HOOKS* ?
>
> which could even be called before from the current thread. I think I
> like this option the best -- if there's a hook which is called after
> unwinding everything, it doesn't matter if the initial hook is called
> before.
>
> Thoughts?
>
*exit- and *final-, *before- and *after- both seem good to me.
>> It would be nicer if we have *thread-exit-hooks* for each thread.
>
> Can you clarify this a bit?
>
> As a thread-local binding? Can do that, but don't quite see the point
> -- how would it be different from sticking an UNWIND-PROTECT clause to
> do the same?
>
> Or do you mean as a global variable, called per thread? I guess that
> makes sense, but don't see an immediate application.
>
Yes, a global binding but also re-bound per thread. Unwind-protect is
ok, but having bindings means I can inherit same exit functions in new
threads (just rebind it, or use global binding). For convenience and
also the API seems more complete (process-scope hooks and thread-scope
hooks :)
Cheers,
--
黄 澗石 (Jianshi Huang)
http://huangjs.net/
------------------------------------------------------------------------------
Jianshi Huang
>
>>> It would be nicer if we have *thread-exit-hooks* for each thread.
>>
>> Can you clarify this a bit?
>>
>> As a thread-local binding? Can do that, but don't quite see the point
>> -- how would it be different from sticking an UNWIND-PROTECT clause to
>> do the same?
>>
>> Or do you mean as a global variable, called per thread? I guess that
>> makes sense, but don't see an immediate application.
>>
>
> Yes, a global binding but also re-bound per thread. Unwind-protect is
> ok, but having bindings means I can inherit same exit functions in new
> threads (just rebind it, or use global binding). For convenience and
> also the API seems more complete (process-scope hooks and thread-scope
> hooks :)
More clarification, I think the right configuration for the exit hook should be:
1) A global binding so that all threads can follow the same termination protocol
2) A thread local binding, so that I can do local changes to the exit hook
3) Threads spawned can be configured to inherit the thread local
bindings, but the default settings should not do that
Overkill? Any idea what level of convenience can we provide? What's
your idea of the implementation?
Nikodemus Siivola
> Overkill? Any idea what level of convenience can we provide? What's
> your idea of the implementation?
Dunno. Haven't thought about the implementation yet, really. I think
I'd like to sort out thread local variable definitions generally first
-- as in Faré's patch, or as discussed way-back-when with Gabor &al.
The question of overkill also applies to *BEFORE/AFTER- hooks.
Managing global resources after all threads have shut down is a valid
use-case for *AFTER-, I think. Yet in absence of *AFTER- anyone
wanting to do that has to use plain *EXIT-HOOKS* instead, which means
the same exit hook has to be responsible for shutting down threads.
For /applications/ this means people cannot skirt their responsibility
to shut down threads so easily -- but for /libraries/ it means that
things do not compose properly.
*BEFORE- hook seems mostly unnecessary, really. Only use-case I can
imagine for it would be trying to debug "who the hell is calling
exit?", but there are other --and better-- tools for that. At the same
time, having only *EXIT-HOOKS* and *AFTER-EXIT-HOOKS* seems mildly
confusing, not to mention lopsided.
The more I think of this, the more I think having both is /right/,
because one is the place libraries spawning threads can push their own
hooks to shut them down, and the other is the place where libraries
having eg. open streams can close them after they're no longer used.
Yet at the same time I'm somewhat skeptical.
The current portability meme seems to mean these hooks would just go
unused, and as long as we're talking about applications and not
libraries, a single function in *EXIT-HOOKS* should be enough to deal
with everything: first shut down threads, then clean up global stuff.
The EXIT stuff mostly happened because I wanted to add EXIT to
Madeira, and only then realized how shoddy QUIT was. Portably,
MADEIRA:EXIT can just about promise to call a bunch of hooks and
exit(3), but even unwinding the current thread is probably more than
it can promise. :/
Cheers,
-- Nikodemus