Why does ShutdownHookThread 'setDaemon true'?

[Mark Feeney]

Greetings.

I asked this same question on SO (http://stackoverflow.com/questions/7768877/why-does-shutdownhookthread-setdaemon-true), but probably should have asked here first.

In scala.sys.ShutdownHookThread -- a little helper for adding shutdown hooks -- the new shutdown hook thread is explicitly set daemon=true.  I don't believe it's causing an issue, but I am curious about why this was done since it seems backwards, imo. i.e. ideally you want your shutdown hook threads to run in their entirety before the jvm exits.

Thanks,

Mark.

[Paul Phillips]

On Tue, Oct 18, 2011 at 7:25 AM, Mark Feeney <mark....@gmail.com> wrote:
> In scala.sys.ShutdownHookThread -- a little helper for adding shutdown hooks
> -- the new shutdown hook thread is explicitly set daemon=true.  I don't
> believe it's causing an issue, but I am curious about why this was done
> since it seems backwards, imo. i.e. ideally you want your shutdown hook
> threads to run in their entirety before the jvm exits.

This is why it errs in that direction; whether it is necessary or
desirable given a fuller understanding of the system, I can't say
offhand.

commit 6c1815706fc00e32eb1a1a07ace9df4c801ce76c
Author: extempore <extempore@5e8d7ff9-d8ef-0310-90f0-a4852d11357a>
Date: Sun Aug 15 15:45:02 2010 +0000

Since r22186 scripts wait for all non-daemon threads to exit
before the script will exit (not doing so was the cause of several
other bugs involving early script termination) but this means we
must be careful not to introduce unnecessary non-daemon threads
in the script infrastructure. Two such sources (now fixed) were
the shutdown hook which deletes temporary files and the stream
consumers used by Process. Closes #3678, review by harrah.

[Paul Phillips]

I see upon reflection that the code which waits for existing threads
to complete should filter out not only daemon threads but shutdown
hook threads specifically, and the shutdown hook threads should be
non-daemon. Does that sound correct?

[Mark Feeney]

I think conceptually shutdown hook threads should be non-daemon.  Ad-hoc testing suggests daemon status doesn't actually matter though, which is interesting:

mark@mark-pc1:~/tmp$ cat shutdown.scala 

import scala.sys.ShutdownHookThread

object Shutdown {

  @volatile var keepRunning = true

  def main(args: Array[String]) {

    println("starting main")

    ShutdownHookThread { 

      println("in shutdown hook")

      keepRunning = false // cause main to exit shortly

      Thread.sleep(10000) // wait a long time

      println("done shutdown hook")

    }

    while (keepRunning) {

      print(".")

      Thread.sleep(100)

    }

    println("exiting main")

  }

}

mark@mark-pc1:~/tmp$ scala shutdown.scala 

starting main

.............^Cin shutdown hook

exiting main

*** wait 10 s happens here ***

done shutdown hook

My hook here is daemon=true, yet the JVM waits for it to run to completion before exiting (else 'done shutdown hook' wouldn't have been printed).

I'm not an authority on this stuff, but my thinking at the moment is:

- shutdown hooks should be non-daemon (to avoid confusion)

- you should filter only daemon threads when waiting for threads in the scripting code

- explicitly waiting for shutdown hook threads, daemon or not, is probably unnecessary since the test above seems to  suggest that the JVM will wait for them anyway.

Mark.

[Paul Phillips]

On Tue, Oct 18, 2011 at 8:54 AM, Mark Feeney <mark....@gmail.com> wrote:
> - you should filter only daemon threads when waiting for threads in the
> scripting code
> - explicitly waiting for shutdown hook threads, daemon or not, is probably
> unnecessary since the test above seems to  suggest that the JVM will wait
> for them anyway.

I think the motivating issue wasn't clear to you. I have to wait for
all threads which I can't exclude. Nothing is explicitly waiting for
shutdown threads, it's waiting for all threads which might be
user-initiated so it doesn't call exit with the program still running.
The risk posed by the shutdown threads is deadlock: they're waiting
for shutdown, I'm waiting for them.

I have another idea though: rather than waiting for all non-daemon
threads, I can exit when there are no running non-daemon threads
(because shutdown hook threads won't have been started.)

[Mark Feeney]

On Tue, Oct 18, 2011 at 12:35 PM, Paul Phillips <pa...@improving.org> wrote:
> I think the motivating issue wasn't clear to you.  I have to wait for
> all threads which I can't exclude.  Nothing is explicitly waiting for
> shutdown threads, it's waiting for all threads which might be
> user-initiated so it doesn't call exit with the program still running.
>  The risk posed by the shutdown threads is deadlock: they're waiting
> for shutdown, I'm waiting for them.

Yep, I missed the point. Thanks for clarifying.

> I have another idea though: rather than waiting for all non-daemon
> threads, I can exit when there are no running non-daemon threads
> (because shutdown hook threads won't have been started.)

The idea sounds good, but it's also exactly what the JVM already does:
initiate shutdown when all non-daemon threads have exited. This makes
me wonder if your code could itself be a shutdown hook? Probably I'm
missing something again.

Do you have a pointer to the scripting code in question? I may be
able to contribute something more intelligent with some additional
context.

Thanks,

Mark.

[Paul Phillips]

On Tue, Oct 18, 2011 at 11:57 AM, Mark Feeney <mark....@gmail.com> wrote:
> The idea sounds good, but it's also exactly what the JVM already does:
> initiate shutdown when all non-daemon threads have exited.  This makes
> me wonder if your code could itself be a shutdown hook?  Probably I'm
> missing something again.

The only way to return an exit code is to call System.exit. Would
have been nice if "public static void main" had set off some alarm
bells somewhere in the early '90s, but so it goes. So those are the
requirements: you have to call System.exit, but only after everything
which should complete has completed.

https://github.com/scala/scala/blob/master/src/compiler/scala/tools/nsc/ScriptRunner.scala

There are a bunch of other files if you really want to get into it,
but there's a starting point.

[Mark Feeney]

> The only way to return an exit code is to call System.exit.

Ah, that's a nasty problem. I see now why you've had to go down this
path. I like your util.waitingForThreads() function.

> Would have been nice if "public static void main" had set off some alarm
> bells somewhere in the early '90s, but so it goes.  So those are the
> requirements: you have to call System.exit, but only after everything
> which should complete has completed.
>
> https://github.com/scala/scala/blob/master/src/compiler/scala/tools/nsc/ScriptRunner.scala
>
> There are a bunch of other files if you really want to get into it,
> but there's a starting point.

I think waiting for non-daemon, live threads like you've done
(https://github.com/scala/scala/commit/71e733b887a1db34cc9fb461f561c6e7223f88f8#L0R36)
is a good improvement. Thanks for taking the time to look at this and
explain the issues to me.

Mark.