Can't shutdown scheduler when all thread pool workers are busy

[Nathan Baulch]

I'm unable to shutdown the scheduler if all thread pool workers are busy, even when setting the interruptJobsOnShutdownWithWait flag and making sure all my jobs are interruptible.

If I attach a debugger and pause execution I see the main thread waiting on a QuartzScheduler_QuartzSchedulerThread join (QuartzSchedulerThread.cs line 182), which in turn is blocking for an available worker thread (SimpleThreadPool.cs line 329).

I see that the QuartzSchedulerThread loop exits when the halted flag is set, but this doesn't work if the loop is already stuck waiting for an available worker thread (line 280).

The following sample code reliably reproduces the problem:

    public class TestJob : IInterruptableJob
    {
        private readonly EventWaitHandle _handle = new ManualResetEvent(false);
        public void Execute(IJobExecutionContext context)
        {
            _handle.WaitOne();
        }
        public void Interrupt()
        {
            _handle.Set();
        }
    }

    var scheduler = new StdSchedulerFactory(new NameValueCollection
        {
            {"quartz.threadPool.threadCount", "1"},
            {"quartz.scheduler.interruptJobsOnShutdownWithWait", "true"}
        }).GetScheduler();
    scheduler.AddJob(new JobDetailImpl("TestJob", typeof (TestJob)) {Durable = true}, true);
    scheduler.Start();
    scheduler.TriggerJob(new JobKey("TestJob"));
    Thread.Sleep(1000);
    scheduler.Shutdown(true);
    Console.WriteLine("finish");
    Console.Read();

It seems like the BlockForAvailableThreads method needs to return zero after a 500ms timeout so that the halted flag can be checked again in QuartzSchedulerThread. Alternatively a Halt method could be added to the thread pool which breaks the BlockForAvailableThreads loop. I'd be happy to send a pull request with either of these ideas implemented if desired.

[Sascha Scholz]

Same as https://jira.terracotta.org/jira/browse/QTZ-493?

[Nathan Baulch]

Yep, looks the same to me. If they fix the issue then I'll look into porting it across, though I'm not sure how much of this code is still relevant in Quartz.net 3.0 with the move towards TPL/async.