Scheduler shutdown does not cancel scheduled actions but seems to flush them

1,795 views
Skip to first unread message

ido

unread,
Mar 11, 2012, 8:24:09 AM3/11/12
to Akka User List
I was expecting Scheduler.shutdown to cancel all pending actions also.
But it seems as if the next one is still executed and actually
immediately after shutdown. I think I would have to go thought all
actors or scedules and cancel them myself before calling shutdown.
Is this on purpose?

thanks,
ido


val system = ActorSystem("test")
import java.util.concurrent.TimeUnit._
system.scheduler.schedule( Duration(10, SECONDS), Duration(50,
SECONDS), new Runnable(){
override def run(){
println("still running delay")
}
}
)
system.scheduler.schedule( Duration(0, MILLISECONDS), Duration(10,
SECONDS), new Runnable(){
override def run(){
println("still running immediate")
}
}
)
Thread.sleep(100)
system.shutdown()
println("shutdown")

still running immediate
shutdown
still running delay
still running immediate

Patrik Nordwall

unread,
Mar 11, 2012, 8:46:17 AM3/11/12
to akka...@googlegroups.com
I can confirm that all scheduled tasks are run when shutdown. I think it was needed for a clean shutdown of the ActorSystem (dispatchers), but I agree that it can be rather strange to run something that was scheduled some (long) time in the future.

/Patrik

> --
> You received this message because you are subscribed to the Google Groups "Akka User List" group.
> To post to this group, send email to akka...@googlegroups.com.
> To unsubscribe from this group, send email to akka-user+...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/akka-user?hl=en.
>

rkuhn

unread,
Mar 11, 2012, 9:26:46 AM3/11/12
to akka...@googlegroups.com
If memory serves the specific reason was that dispatchers schedule their shutdown, and if we lose that task we’ll leak thread pools. In general, the current semantics were the only one I found which does not (immediately) drive me into madness:

upon shutdown:
- it must execute those vital tasks which were already queued
- it must stop accepting new tasks, otherwise there would be ample opportunity for endless loops

Hence you must cancel those tasks which shall not be run in case of system shutdown, best opportunity is the postStop() hook of an actor which has the necessary “big picture”.

Regards,

Roland

> To unsubscribe from this group, send email to akka-user+unsubscribe@googlegroups.com.

ido

unread,
Mar 11, 2012, 10:00:28 AM3/11/12
to Akka User List
Is it possible that even with cancel I get something executed?
Interestingly the first task which was never executed
is not executed upon shutdown.

I need to shut this down because its a database task and at some point
I have to close the connection without anything
trying to access it again.

best,
ido

def testShutdown(){
val system = ActorSystem("test")
val cancels = ListBuffer[Cancellable]()
import java.util.concurrent.TimeUnit._
cancels += system.scheduler.schedule( Duration(10, SECONDS),
Duration(5, SECONDS), new Runnable(){
override def run(){
System.err.println("still running delay")
}
}
)
cancels += system.scheduler.schedule( Duration(0, MILLISECONDS),
Duration(1, SECONDS), new Runnable(){
override def run(){
System.err.println("still running immediate")
}
}
)
Thread.sleep(1000 * 2)
cancels.foreach(_.cancel)
Thread.sleep(1000 * 2)
system.shutdown()

System.err.println("shutdown")
}



On Mar 11, 2:26 pm, rkuhn <goo...@rkuhn.info> wrote:
> If memory serves the specific reason was that dispatchers schedule their
> shutdown, and if we lose that task we’ll leak thread pools. In general, the
> current semantics were the only one I found which does not (immediately)
> drive me into madness:
>
> upon shutdown:
> - it must execute those vital tasks which were already queued
> - it must stop accepting new tasks, otherwise there would be ample
> opportunity for endless loops
>
> Hence you must cancel those tasks which shall not be run in case of system
> shutdown, best opportunity is the postStop() hook of an actor which has the
> necessary “big picture”.
>
> Regards,
>
> Roland
>
> Am Sonntag, 11. März 2012 13:46:17 UTC+1 schrieb Patrik Nordwall:
>
>
>
>
>
>
>
>
>
> > I can confirm that all scheduled tasks are run when shutdown. I think it
> > was needed for a clean shutdown of the ActorSystem (dispatchers), but I
> > agree that it can be rather strange to run something that was scheduled
> > some (long) time in the future.
>
> > /Patrik
>
> > akka-user+...@googlegroups.com.

Roland Kuhn

unread,
Mar 11, 2012, 2:29:56 PM3/11/12
to akka...@googlegroups.com

On Mar 11, 2012, at 15:00 , ido wrote:

> Is it possible that even with cancel I get something executed?

Well, obviously there is a race between cancellation and running: if it has started running, you cannot cancel it anymore.

> Interestingly the first task which was never executed
> is not executed upon shutdown.
>

You mean in the code below? You canceled it, hence it will not be run.

> I need to shut this down because its a database task and at some point
> I have to close the connection without anything
> trying to access it again.
>

The code below prints

scala> testShutdown
still running immediate
still running immediate
shutdown

and if I remove the cancels.foreach() and the following sleep():

scala> testShutdown
still running immediate
still running immediate
shutdown

scala> still running immediate
still running delay

Here you can see that the tasks are executed exactly once after the shutdown is initiated. Keep in mind that the scheduler is shutdown only after all actors in the ActorSystem have stopped (only four in this case), not necessarily before system.shutdown() returns. You can use system.awaitTermination() if you need to be sure.

Regards,

Roland

Roland Kuhn
Typesafe – The software stack for applications that scale.
twitter: @rolandkuhn


jxtps

unread,
Nov 14, 2014, 5:35:08 PM11/14/14
to akka...@googlegroups.com
I have some tasks that must *not* run on shutdown, but do still need to be scheduled. What's a good way to detect upon task execution that the task was called on shutdown, as opposed to during normal operation?

AkkaSystem.isTerminated doesn't seem to be it - is there anything that can give me an AkkaSystem.isTerminating?

jxtps

unread,
Nov 14, 2014, 5:58:22 PM11/14/14
to akka...@googlegroups.com

object AkkaShutdownMonitor {
  @volatile var isTerminating = false

  Akka.system.registerOnTermination {
    isTerminating = true
  }
}

Appears to work (i.e. gating inside the task on this isTerminating appears to solve it in dev mode in my Play 2 app).

I realize that the shutdown behavior is documented at the top of the Scheduler trait, but it would be great to see a comment about it on the actual schedule* methods - this lead to some most unfortunate issues in production.

Roland Kuhn

unread,
Dec 9, 2014, 9:13:23 AM12/9/14
to akka-user
Sorry for the delay, this fell through the cracks: I have created a ticket for tracking this, and I agree that it can be surprising (and wrong).

Regards,

Roland

--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+...@googlegroups.com.

To post to this group, send email to akka...@googlegroups.com.



Dr. Roland Kuhn
Akka Tech Lead
Typesafe – Reactive apps on the JVM.
twitter: @rolandkuhn


Reply all
Reply to author
Forward
0 new messages