How to deal with multiple versions of Akka in same JVM?

[Bryn Keller]

Hi Folks,

I have two 3rd party libraries I need to use in my application. Both use Akka, but they use different versions (2.1 and 2.3, I think). Since I need to use these versions together in the same JVM (I don't need 2.1 and 2.3 to talk to each other, just share a VM), I started looking at loading these libraries using separate classloaders. I ran into this exception, which (just from looking at the stack trace) makes me think that Akka is trying to load classes in an unusual way. ReflectiveDynamicAccess is the name that makes me wonder this. 

Whatever the reason, Akka can't find classes that are in the same jar with the classes that had to be loaded in order to even get this far. Does anyone have suggestions about better ways to deal with this situation? Is anyone using OSGi for this kind of thing? Please let me know if the question isn't clear or if I'm overlooking something simple.

Thanks,

Bryn

Exception in thread "main" java.lang.ClassNotFoundException: akka.remote.RemoteActorRefProvider

        at java.net.URLClassLoader$1.run(Unknown Source)

        at java.net.URLClassLoader$1.run(Unknown Source)

        at java.security.AccessController.doPrivileged(Native Method)

        at java.net.URLClassLoader.findClass(Unknown Source)

        at java.lang.ClassLoader.loadClass(Unknown Source)

        at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)

        at java.lang.ClassLoader.loadClass(Unknown Source)

        at java.lang.Class.forName0(Native Method)

        at java.lang.Class.forName(Unknown Source)

        at akka.actor.ReflectiveDynamicAccess$$anonfun$getClassFor$1.apply(DynamicAccess.scala:67)

        at akka.actor.ReflectiveDynamicAccess$$anonfun$getClassFor$1.apply(DynamicAccess.scala:66)

        at scala.util.Try$.apply(Try.scala:161)

        at akka.actor.ReflectiveDynamicAccess.getClassFor(DynamicAccess.scala:66)

        at akka.actor.ReflectiveDynamicAccess.createInstanceFor(DynamicAccess.scala:84)

        at akka.actor.ActorSystemImpl.liftedTree1$1(ActorSystem.scala:584)

        at akka.actor.ActorSystemImpl.<init>(ActorSystem.scala:577)

        at akka.actor.ActorSystem$.apply(ActorSystem.scala:141)

        at akka.actor.ActorSystem$.apply(ActorSystem.scala:118)

        at org.apache.spark.util.AkkaUtils$.createActorSystem(AkkaUtils.scala:96)

[√iktor Ҡlang]

Hi Bryn,

you aren't telling us how you are running this, so it becomes impossible (my crystal ball is always in for repairs) to divine why the classloader you supply to your ActorSystem creation does not have access to the akka-remote jar.

Could you share that?

--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+...@googlegroups.com.
To post to this group, send email to akka...@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

--

Cheers,

√

[Bryn Keller]

Hi Viktor,

Sure, here's some more information, please let me know if there are other things that would help. The code looks basically like this:

object Boot extends App {

  val sparkLoader = {

    val sparkJar = Directory.Current.get / "lib" / "spark-client.jar"

    val loader = new URLClassLoader(Array(sparkJar.toURL), getClass.getClassLoader)

    loader

  }

  val thing = {

      val class_ = sparkLoader.loadClass("org.my.ClassName")

      val instance = class_.newInstance()

      instance.asInstanceOf[MyInterface]

    }

}

So Boot lives in one jar with all its dependencies, call it boot.jar. I run it with java -jar boot.jar. In the lib folder of the current directory is spark-client.jar, which includes org.my.Classname as well as Spark and all its dependencies. Both boot.jar and spark-client.jar were prepared using the sbt assembly plugin. org.my.ClassName starts a SparkContext in its constructor, which is what leads to the Akka exception.

Thanks,

Bryn

[√iktor Ҡlang]

You might want to explore:

a) check what class loader Spark passes to the ActorSystem

b) setting the current Threads ContextClassLoader to be the appropriate one around the thing-block

c) try to load the class Akka reports as missing from the ClassLoader you're using

(You have entered the world of class loaders, a world of advice: Here be dragons)

[Bryn Keller]

Thanks, those sound like good advice (including the dragons)! I'll try those.