Shutdown and restart Lift with OSGi

[Ken Wenzel]

Hello,

we are using Lift 2.6 in an OSGi environment where multiple bundles make up a modular Lift application.
Unfortunately, Lift is not really designed for this use case since it uses a lot of global variables like LiftRules, Schedule and others.
Hence, we need to restart the whole bundle that packages all Lift libraries (in our case "net.liftweb") and all dependent application bundles
if one application bundle is hot deployed.
This is accomplished by calling

LiftFilter.terminate

on the registered filter instance and by registering a new instance after "net.liftweb" has been reloaded by the OSGi framework.

So far, so good, this works in principle, but after a short delay an exception is thrown:

10:36:52.679 [pool-14-thread-1] ERROR net.liftweb.actor.ActorLogger - Actor threw an exception
java.lang.NoClassDefFoundError: scala/runtime/BooleanRef
    at net.liftweb.actor.SpecializedLiftActor$class.net$liftweb$actor$SpecializedLiftActor$$proc2(LiftActor.scala:272) [lift-actor_2.11-2.6-M4.jar:2.6-M4]
    at net.liftweb.actor.SpecializedLiftActor$$anonfun$net$liftweb$actor$SpecializedLiftActor$$processMailbox$1.apply$mcV$sp(LiftActor.scala:227) [lift-actor_2.11-2.6-M4.jar:2.6-M4]
    at net.liftweb.actor.SpecializedLiftActor$$anonfun$net$liftweb$actor$SpecializedLiftActor$$processMailbox$1.apply(LiftActor.scala:227) [lift-actor_2.11-2.6-M4.jar:2.6-M4]
    at net.liftweb.actor.SpecializedLiftActor$$anonfun$net$liftweb$actor$SpecializedLiftActor$$processMailbox$1.apply(LiftActor.scala:227) [lift-actor_2.11-2.6-M4.jar:2.6-M4]
    at net.liftweb.actor.SpecializedLiftActor$class.around(LiftActor.scala:241) [lift-actor_2.11-2.6-M4.jar:2.6-M4]
    at net.liftweb.http.SessionMaster$.around(LiftSession.scala:211) [lift-webkit_2.11-2.6-M4.jar:2.6-M4]
    at net.liftweb.actor.SpecializedLiftActor$class.net$liftweb$actor$SpecializedLiftActor$$processMailbox(LiftActor.scala:226) [lift-actor_2.11-2.6-M4.jar:2.6-M4]
    at net.liftweb.actor.SpecializedLiftActor$$anonfun$2$$anonfun$apply$mcV$sp$1.apply$mcV$sp(LiftActor.scala:190) [lift-actor_2.11-2.6-M4.jar:2.6-M4]
    at net.liftweb.actor.LAScheduler$$anonfun$9$$anon$2$$anon$3.run(LiftActor.scala:76) [lift-actor_2.11-2.6-M4.jar:2.6-M4]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_79]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_79]
    at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79]
Caused by: java.lang.ClassNotFoundException: scala.runtime.BooleanRef cannot be found by net.liftweb_2.6.0.qualifier
    at org.eclipse.osgi.internal.loader.BundleLoader.findClassInternal(BundleLoader.java:439) ~[na:na]
    at org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:352) ~[na:na]
    at org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:344) ~[na:na]
    at org.eclipse.osgi.internal.loader.ModuleClassLoader.loadClass(ModuleClassLoader.java:160) ~[na:na]
    at java.lang.ClassLoader.loadClass(ClassLoader.java:358) ~[na:1.7.0_79]
    ... 12 common frames omitted

The problem is, that SessionMaster.reaction is executed since

Schedule.shutdown

uses

ScheduledExecutorService.shutdown

to stop the underlying scheduler service.
The documentation for this method http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ExecutorService.html#shutdown%28%29
says:

Initiates an orderly shutdown in which previously submitted tasks are executed, but no new tasks will be accepted. Invocation has no additional effect if already shut down.

I think this means that previously scheduled tasks may be executed after the executor service has been shutdown.

Maybe it would help if the associated thread factory

private object TF extends ThreadFactory {
  val threadFactory = Executors.defaultThreadFactory()
  def newThread(r: Runnable) : Thread = {
    val d: Thread = threadFactory.newThread(r)
    d setName "Lift Scheduler"
    d setDaemon true

    if (ThreadPoolRules.nullContextClassLoader) {
      d setContextClassLoader null
    }
    d
  }
}

receives a shutdown method and that it simply returns "null" instead of a thread object after it has been shutdown
to prevent the execution of scheduled tasks after Schedule.shutdown has been called.

What are your suggestions?

Thank you and best regards,
Ken

[Ken Wenzel]

Hello,

I found a workaround for my problem. I simply use reflection to really shutdown the executor service of Lift's Schedule object in the following way:

      // access private schedule service field by reflection
      val serviceField = Schedule.getClass.getDeclaredFields.filter { f => f.getName == "service" || f.getName.endsWith("$$service") }.headOption
      val service = serviceField.map { f =>
        f.setAccessible(true)
        f.get(Schedule).asInstanceOf[ExecutorService]
      }

      // destroy lift servlet
      terminate

      // ensure that schedule service is really canceled
      // Lift calls only shutdown that does not cancel already enqueued tasks
      service.foreach(_.shutdownNow)

Best regards,
Ken