Automatic shutdown of agents
ataggart
old, unresolved discussion [2] and ticket [3]...
My assumption is that agent thread pool threads are non-daemon to
prevent currently executing actions from terminating prematurely when
the system exits. If correct, then simply making them daemon threads
and adding a shutdown hook that calls ExecutorService.shutdown() and
ExecutorService.awaitTermination() should maintain the current
behaviour without the need for application and library writers to ever
concern themselves with calling shutdown-agents.
I've tested such a change [4] locally and it seems to work well.
[1] http://groups.google.com/group/clojure/browse_thread/thread/085fce6a3ae89329#
[2] http://groups.google.com/group/clojure-dev/browse_thread/thread/a837f14a87aca78a
[3] http://dev.clojure.org/jira/browse/CLJ-124
[4] https://gist.github.com/1067690
Kevin Downey
non-deamon threads you hang on the shutdown hook?
> --
> You received this message because you are subscribed to the Google Groups "Clojure Dev" group.
> To post to this group, send email to cloju...@googlegroups.com.
> To unsubscribe from this group, send email to clojure-dev...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/clojure-dev?hl=en.
>
>
--
And what is good, Phaedrus,
And what is not good—
Need we ask anyone to tell us these things?
Phil Hagelberg
> doesn't that just move the issue around? instead of hanging on the
> non-deamon threads you hang on the shutdown hook?
I believe the shutdown hook would finish once all the agents and futures
finished their execution. IIRC the agent threadpool doesn't allow an
exit until 60 seconds or so after its last use.
-Phil
ataggart
once the VM begins its shutdown sequence, and they do not prevent the
VM from exiting (since that would be silly).
See the javadoc for Runtime.addShutdownHook():
http://download.oracle.com/javase/6/docs/api/java/lang/Runtime.html#addShutdownHook(java.lang.Thread)
On Jul 6, 9:56 am, Kevin Downey <redc...@gmail.com> wrote:
> doesn't that just move the issue around? instead of hanging on the
> non-deamon threads you hang on the shutdown hook?
>
>
>
>
>
>
>
>
>
> On Wed, Jul 6, 2011 at 9:42 AM, ataggart <alexclojuregr...@gmail.com> wrote:
> > Triggered by a question on the main board [1], and continuing from an
> > old, unresolved discussion [2] and ticket [3]...
>
> > My assumption is that agent thread pool threads are non-daemon to
> > prevent currently executing actions from terminating prematurely when
> > the system exits. If correct, then simply making them daemon threads
> > and adding a shutdown hook that calls ExecutorService.shutdown() and
> > ExecutorService.awaitTermination() should maintain the current
> > behaviour without the need for application and library writers to ever
> > concern themselves with calling shutdown-agents.
>
> > I've tested such a change [4] locally and it seems to work well.
>
> > [2]http://groups.google.com/group/clojure-dev/browse_thread/thread/a837f...
> > [4]https://gist.github.com/1067690
>
> > --
> > You received this message because you are subscribed to the Google Groups "Clojure Dev" group.
> > To post to this group, send email to cloju...@googlegroups.com.
> > To unsubscribe from this group, send email to clojure-dev...@googlegroups.com.
ataggart
and would shutdown the pool so no new tasks could be queued, and then
(if we want to maintain current behaviour) block until all currently
executing and pending tasks complete.
I don't see anything about a 60 second delay. Currently, calling
shutdown-agents eventually calls shutdown() on the pools, which
prevents new tasks from being submitted, and the non-deamon threads in
the pool are terminated once the last task is complete.
On Jul 6, 10:57 am, Phil Hagelberg <p...@hagelb.org> wrote:
Alex Miller
used by the agents. The pool used by "send-off" is an expandable
thread pool with the default 1-minute keep-alive. After 1 minute, all
threads in the pool will die off and there are then no non-daemon
threads preventing execution. The "send" pool is fixed-size and those
threads (2+# cores) will stick around forever, preventing shutdown.
Adding a shutdown hook will have no effect on the above behavior as
shutdown hooks are only run when a JVM is terminated, and non-daemon
threads prevent a JVM from terminating.
Personally I favor making these pools use daemon threads. The
contributed the patch (applied in 1.3) that uses thread pools with
custom names in 1.3 and it would be trivial in that thread factory to
call thread.setDaemon(true) around Agent line 61. Some people have
objected to this as one pattern is to have the main thread kick off a
bunch of work, then exit allowing the work to complete. There are
many simple ways to fix this problem (awaiting on the agent, starting
a keep-alive thread, etc). I discussed this in more depth here:
http://tech.puredanger.com/2010/06/08/clojure-agent-thread-pools/
Alex
ataggart
solution I outlined above.
Alex Miller
In general, shutdown hooks can be tricky and are to be avoided if
possible. They create strange garbage collection scenarios, weird
problems with finalization (which runs after shutdown hooks), and
problems with other shutdown hooks (as they all run concurrently). I
believe the introduction of shutdown hooks are also restricted under
some security policies.
Is there a strong reason for using one? If you wish to wait for the
threads to complete, your program may be able to do that via awaiting
on the agent (depending on who called what). You can also call
Agent.shutdown() (not sure if there is a Clojure wrapper for this)
which will wait for pending jobs to finish. Or if you want to kill
things more urgently, you can call Agent.soloExecutor.shutdownNow()
and Agent.pooledExecutor.shutdownNow().
In other words, there are hooks available to solve these problems now,
and introducing shutdown hooks may introduce new and even stranger
problems, so it seems not worth the added potential complexity to me.
On Wed, Jul 6, 2011 at 4:22 PM, ataggart <alexcloj...@gmail.com> wrote:
> On Jul 6, 12:56 pm, Alex Miller <a...@puredanger.com> wrote:
>> The 60 second delay is due to the configuration of the thread pools
>> used by the agents.
>
> Ah, I see now, it's the behaviour as specified by
> Executors.newCachedThreadPool(), and not any clojure-specific
> configuration.
>
>> Adding a shutdown hook will have no effect on the above behavior as
>> shutdown hooks are only run when a JVM is terminated, and non-daemon
>> threads prevent a JVM from terminating.
>
> True, which is why I also noted the pools should use daemon threads.
ataggart
> used by the agents.
Executors.newCachedThreadPool(), and not any clojure-specific
configuration.
> shutdown hooks are only run when a JVM is terminated, and non-daemon
> threads prevent a JVM from terminating.
ataggart
board) that the onus for calling shutdown-agents is unclear.
Libraries that happen to use send-off or pmap will cause application
termination to be delayed for one minute; libraries that happen to use
send will cause applications to never terminate. Defensively adding
(shutdown-agents) at the end of every application's shutdown process,
or worse rolling ones own shutdown hook, is an inelegant solution.
Simply marking the threads as daemons would obviate the need to ever
call shutdown-agents. On this I think we agree.
I presume Rich had a reason for not doing that in the first place.
Further, I presume the point was to avoid agent threads from
terminating in the middle of execution (which would be bad news if
you're in the middle of writing to a file post-STM-transaction).
The shutdown hook is an attempt to preserve the assumed desire to not
terminate agent threads in the middle of their execution, while
gaining the benefits of no longer needing to explicitly clean up the
pools that clojure implicitly creates.
As for the concerns over shutdown hooks themselves, any tool can be
misused (see: eval). Does _this_ solution have those hypothetical
problems?
----
That said, there may be another solution if the concern over
terminating agent threads only applies to IO: have the fixed-pool used
by send contain daemon threads, leaving the one-minute-ttl-pool used
by send-off and pmap unchanged.
Doing that would mean no need for a shutdown hook, at the cost of
killing cpu-bound agent ops, and leaving a worst-case one minute
termination delay.
Phil Hagelberg
> Personally I favor making these pools use daemon threads. The
> contributed the patch (applied in 1.3) that uses thread pools with
> custom names in 1.3 and it would be trivial in that thread factory to
> call thread.setDaemon(true) around Agent line 61.
Strongly seconded; we have had a few subtle errors creep into Leiningen
plugins due to having to work around the shortcomings of the current
thread pool behaviour.
-Phil