> On Tue, Feb 21, 2012 at 10:24 AM, Graham Dumpleton > <graham.dumple...@gmail.com> wrote:
>> ...
>> > But I don't think you can guarantee that everything is still up in >> > memory by >> > the time atexit gets called, >> > so you can't really call cleanup code there.
>> The only thing which is done prior to atexit callbacks being called is >> waiting on threads which weren't marked as daemonised.
> which can lead to completely lock the shutdown if a lib or the program has a > thread with a loop that waits for a condition.
In mod_wsgi at least there are fail safes such that background C threads will force kill the process if such a lockup occurs on shutdown.
> which it is not the case with signals, since you get a chance to properly > stop everything beforehand.
Yes and no. For a signal handler to even be able to be triggered, there must be Python code executing in the main thread that originally created the main interpreter.
In an embedded system such as mod_wsgi, the main thread is never used to handle requests and actually runs in C code blocked waiting for an internal notification that process is being shutdown.
>> what do you mean by bypassing its destruction ?
>> Non catchable signal from within process or from a distinct monitoring >> process. >> One of this things I pointed out is being missed. >> That is, a WSGI adapter may be running on top of another layer of >> abstraction, such as FASTCGI for example, where the lower layer isn't >> going to have any callback mechanism of its own to even notify the >> WSGI layer to trigger registered cleanup callbacks. >> This is why the only mechanism one can universally rely on is the >> Python interpreters own atexit mechanism.
> I see.. but what I don't understand is the following: when the whole stack > is shut down, the python process is being killed by *someone*.
> And that someone, as far as I understand, is also able to send requests to > the WSGI application.
> So what makes it impossible to send a shutdown signal prior to killing the > process ?
Is not impossible and in mod_wsgi at least a signal is used to initiate shutdown, this coming either from itself in some cases, or from Apache parent process in others. The signal handler then uses a socketpair pipe to wake up the blocked main thread to begin shutdown steps. Either way it is handled at C code level because can't rely on Python level signal handlers to actually run.
To further complicate things, in a process with multiple sub interpreters where would the Python signal handler even run. There is no main thread running waiting to exit. It also can't just cause the main Python interpreter to be exited. Simply exiting a main thread even if it did exist wouldn't allow you to cleanup sub interpreters.
In short, embedded systems are going to be quite different to what you are used to with pure WSGI servers. It is because it is doing all this to ensure that reliable shutdown can occur that mod_wsgi ignores all Python signal handler registrations by default.
That all said, technically mod_wsgi could on reception of its signal, and if there was a registry of known WSGI applications, it could tell them the process is being shutdown. Presence of sub interpreters makes that a lot of fun, but would be doable.
Right now without such a registry of applications with enter/exit methods as being discussed in this thread, the only way in mod_wsgi is to rely on atexit.
On 21 February 2012 21:07, Simon Sapin <simon.sa...@exyr.org> wrote:
> Le 21/02/2012 10:31, Graham Dumpleton a écrit :
>> You do realise you are just reinventing context managers?
>> With this 'application' do requests.
> Indeed. I didn’t want to go too far from the initial "shutdown function" > proposal, but actual context managers would be better.
FWIW, I have been playing with context managers in other ways to solve per request resource cleanups issues as well. I will cover some of what I have been doing with that in my State of WSGI 2 talk at PyCon web summit.
I sort of wish this whole discussion could perhaps wait until the web summit where after my talk I can perhaps discuss with interested parties all the stuff I have been playing with around improving WSGI rather than taking shots at little bits now when there is a lot more to consider than just this.
> On Tue, Feb 21, 2012 at 10:24 AM, Graham Dumpleton <graham.dumple...@gmail.com> wrote: > ... > > But I don't think you can guarantee that everything is still up in memory by > > the time atexit gets called, > > so you can't really call cleanup code there. > The only thing which is done prior to atexit callbacks being called is > waiting on threads which weren't marked as daemonised.
> which can lead to completely lock the shutdown if a lib or the program has a > thread with a loop that waits for a condition.which it is not the case with > signals, since you get a chance to properly stop everything beforehand.
That's a buggy lib or program. This has nothing to do with WSGI really. The snippet Graham showed is run at any interpreter shutdown, even when you simply run "python" in your shell.
On Tue, Feb 21, 2012 at 1:43 PM, Antoine Pitrou <solip...@pitrou.net> wrote: > Tarek Ziadé <ziade.tarek@...> writes:
> > On Tue, Feb 21, 2012 at 10:24 AM, Graham Dumpleton > <graham.dumple...@gmail.com> wrote: > > ... > > > But I don't think you can guarantee that everything is still up in > memory by > > > the time atexit gets called, > > > so you can't really call cleanup code there. > > The only thing which is done prior to atexit callbacks being called is > > waiting on threads which weren't marked as daemonised.
> > which can lead to completely lock the shutdown if a lib or the program > has a > > thread with a loop that waits for a condition.which it is not the case > with > > signals, since you get a chance to properly stop everything beforehand.
> That's a buggy lib or program. This has nothing to do with WSGI really.
No, that has to do with : please let me clean my program before you try to kill it because I can't use signals :)
> The > snippet Graham showed is run at any interpreter shutdown, even when you > simply > run "python" in your shell.
> There are two typos but the effect remains the same since you are locked > before you reach those lines: > - atexit call worker.stop() instead > of worker.join() > - in worker.stop(), it calls worker.join() instead of > worker.join(self)
I find your example convincing. I think that's worth fixing in Python (e.g. by offering an atexit() method on Thread objects). Perhaps you can open a bug?
On Tue, Feb 21, 2012 at 3:02 PM, Antoine Pitrou <solip...@pitrou.net> wrote: > ... > I find your example convincing. I think that's worth fixing in Python > (e.g. by > offering an atexit() method on Thread objects). Perhaps you can open a bug?
Sure yeah -- Notice that I think it's still a good idea to provide the shutdown function in WSGI, because it gives the full control to the web server on the ordering of events.
On Tue, Feb 21, 2012 at 3:06 PM, Tarek Ziadé <ziade.ta...@gmail.com> wrote: > On Tue, Feb 21, 2012 at 3:02 PM, Antoine Pitrou <solip...@pitrou.net>wrote:
>> ... >> I find your example convincing. I think that's worth fixing in Python >> (e.g. by >> offering an atexit() method on Thread objects). Perhaps you can open a >> bug?
> Sure yeah -- Notice that I think it's still a good idea to provide the > shutdown function in WSGI, because it gives the full control to the web > server on the ordering of events.
If you want to be able to control a thread like that from an atexit callback, you need to create the thread as daemonised. Ie. setDaemon(True) call on thread.
By default a thread will actually inherit the daemon flag from the parent. For a command line Python where thread created from main thread it will not be daemonised and thus why the thread will be waited upon on shutdown prior to atexit being called.
If you ran the same code in mod_wsgi, my memory is that the thread will actually inherit as being daemonised because request handler in mod_wsgi, from which import is trigger, are notionally daemonised.
Thus the code should work in mod_wsgi. Even so, to be portable, if wanting to manipulate thread from atexit, make it daemonised.
> On Tue, Feb 21, 2012 at 1:43 PM, Antoine Pitrou <solip...@pitrou.net> wrote:
>> Tarek Ziadé <ziade.tarek@...> writes:
>> > On Tue, Feb 21, 2012 at 10:24 AM, Graham Dumpleton >> <graham.dumple...@gmail.com> wrote: >> > ... >> > > But I don't think you can guarantee that everything is still up in >> > > memory by >> > > the time atexit gets called, >> > > so you can't really call cleanup code there. >> > The only thing which is done prior to atexit callbacks being called is >> > waiting on threads which weren't marked as daemonised.
>> > which can lead to completely lock the shutdown if a lib or the program >> > has a >> > thread with a loop that waits for a condition.which it is not the case >> > with >> > signals, since you get a chance to properly stop everything beforehand.
>> That's a buggy lib or program. This has nothing to do with WSGI really.
> No, that has to do with : please let me clean my program before you try to > kill it because I can't use signals :)
>> The >> snippet Graham showed is run at any interpreter shutdown, even when you >> simply >> run "python" in your shell.
On Wed, 2012-02-22 at 09:06 +1100, Graham Dumpleton wrote: > If you want to be able to control a thread like that from an atexit > callback, you need to create the thread as daemonised. Ie. > setDaemon(True) call on thread.
> By default a thread will actually inherit the daemon flag from the > parent. For a command line Python where thread created from main > thread it will not be daemonised and thus why the thread will be > waited upon on shutdown prior to atexit being called.
> If you ran the same code in mod_wsgi, my memory is that the thread > will actually inherit as being daemonised because request handler in > mod_wsgi, from which import is trigger, are notionally daemonised.
> Thus the code should work in mod_wsgi. Even so, to be portable, if > wanting to manipulate thread from atexit, make it daemonised.
I've read all the messages in this thread and the traffic on the bug entry at http://bugs.python.org/issue14073 but I'm still not sure what to tell people who want to invoke code "at shutdown".
Do we tell them to use atexit? If so, are we saying that atexit is sufficient for all user-defined shutdown code that needs to run save for code that needs to stop threads?
Is it sufficient to define "shutdown" as "when the process associated with the application exits"? It still seems to not necessarily be directly correlated.
