Scheduler leaking thread pools?

[Robert Ferris]

Hi All,

I'm working on a project using Lift 2.5. We're having issues with Tomcat not shutting down cleanly when being sent a shutdown signal. After a bit of digging, everything seems to point at net.liftweb.util.Schedule as the culprit; specifically, the behavior where it creates a new ThreadPoolExecutor if the old one was shut down.

We've tried Lift 2.6.2, and the problem remains (which makes sense, since the code in question appears to be the same). I compiled a copy of 2.6.2 with some logging added whenever a new pool is created in Schedule, and it is indeed creating a new one at shutdown time. It appears that what's happening is that Schedule.shutdown is called, while LiftSession.shutDownAllSessions is running asynchronously someplace. Shutting down a session uses Schedule to send a shutdown message to any comet actors associated with that session, causing a new pool to be spun up (since the old is already shut down), and now there's nothing hanging around to shut that pool down. The threadpool is effectively leaked and will hold up a graceful shutdown of Tomcat, since they're not daemon threads. It'll also leak in the case that the webapp is stopped and restarted without restarting Tomcat.

Is this an issue anyone else has observed? Are there any known workarounds or bugfix releases addressing the issue?

[Antonio Salazar Cardozo]

I feel like I've heard of this issue before, but I thought it was fixed…

Here: https://github.com/lift/framework/issues/1001

That fix has been in since 2.4.1, but it seems to only apply to LAPinger,

while Schedule uses its own thread-pool that doesn't have the

setContextClassloader call to prevent whatever this situation seems to

be.

I think we'd be willing to accept a PR that adds this to Schedule.

Thanks,

Antonio

[Robert Ferris]

Would this actually fix the issue? It seems like it will make Tomcat not complain about the leaked thread pools, but they would still be leaked. This would become apparent if you reload the app several times in the container, or try to shut down Tomcat gracefully (the scheduler threads don't appear to be daemon threads, so they'll keep the JVM alive unless you explicitly shut it down).

[Antonio Salazar Cardozo]

I'm honestly not sure, I haven't looked into the issue super-deeply—just remembered

I'd seen something about thread pools on shutdown.

Either way looks like something that merits a bug.

Thanks,

Antonio

[Antonio Salazar Cardozo]

Robert, is there any chance you might be able to make this change and try

it in Tomcat and see if it just fixes the complaint, or actually prevents the thread

pool from being leaked? That would be super-useful and make the life of any

committer who wants to pick the problem up a lot easier. I don't believe most

of us use Tomcat, so working on this has a pretty high barrier to entry.

Thanks,

Antonio