Upstart pre-stop and post-stop semantics

[Chris Masone]

Sending this so that it is archived for all time, because I spent a bunch of time fiddling to figure this out and don't want to lose the info :-)

In upstart, one can define the start and stop triggers for one job based on the "starting", "started", "stopping" and "stopped" status of other jobs.  Once can also define pre-stop and post-stop scripts.  It is not necessarily clear what happens when a job is stopped versus having its process crash/exit.  It is even less clear which order things happen when job A has pre/post stop scripts, stops on "starting" job B, and job C stops on "stopping" job A.

I spent some time figuring out a couple cases here.

First, let's imagine there is a UI job that has pre/post stop scripts.

If the UI job is stopped, the following happens:

1) pre-stop script

2) jobs that start/stop on "stopping UI" will run, and we'll wait for them to get to started/stopped

3) the PID that upstart is tracking is sent the TERM signal

4) post-stop

5) jobs that start/stop on "stopped UI" will run, and if we stopped on starting/stopping another job, that job begins starting/stopping

If the UI process crashes/exits:

1) jobs that start/stop on "stopping UI" will run, and we'll wait for them to get to started/stopped

2) jobs that start/stop on "stopped UI" will run, and we'll wait for them to get to started/stopped.

3) post-stop

If the UI process is restarted:

1) pre-stop script

Let's assume now that UI stops on starting the "reboot" job, and that the "windowmanager" job stops on stopping UI.

if the reboot job is started:

1) UI pre-stop

2) windowmanager does pre-stop, gets a TERM, and post-stop, then it's stopped

3) the UI process is sent the TERM signal

4) UI post-stop

5) the reboot process can now begin stopping

If the UI process crashes/exits:

1) windowmanager does pre-stop, gets a TERM, and post-stop, then it's stopped

2) UI post-stop

3) we may loop back around and re-start UI now, if it's set to respawn