Classloading Problem with Init Services

[Stratos Pavlakis]

Hi all,

I 'd like your input on a rather strange issue we encountered when we tried our app on a new environment.

Environment
-------------
Oracle JDK 1.7.0_10
JRuby 1.7.2
JRuby/Rack  1.1.12
Rails 3.2.6
Trinidad 1.4.4
Trinidad Init Services 1.2.2

In few words the problem comes down to:

"NameError (ClassX not found in packages ....  cannot load Java class ClassX)" aka a ClassNotFoundException.

The problem appears only when the application is started through jsvc (init services) and the user performs an action that requires this class to be instantiated.
Works fine with "rails start".

The jar containing ClassX is located in lib/java and a Rails initializer (jar_loader.rb) requires it like:

require 'java'
Dir[Rails.root.join("lib/java/**/*.jar")].each {|f| require f}

All jars in lib/java are also loaded in the webapp classloader as its the default setting in trinidad:

trinidad.yml
   java_lib: "lib/java"

So we thought ok... we're fools, we try to load twice the same jars into the classpath.
Normally this shouldn't be a problem but anyway we  the following actions:

No jar_loader.rb. Let only trinidad load our classes in tomcat's webapp classloader from lib/java. FAILED (Class not found again)

Point java_lib to and empty folder (java_lib: "lib/empty"). WORKED!

Hmmm.... what else to do.... we took a heap dump of the jvm both when started with jsvc and when started through rails start.

These are the classloader hierarchies of the class in question:

JSVC(FAILS):  org.jruby.util.JRubyClassLoader#47->org.jruby.util.JRubyClassLoader#46->sun.misc.Launcher$AppClassLoader#1->sun.misc.Launcher$ExtClassLoader#1->null

RAILS S (WORKS): org.jruby.util.JRubyClassLoader#145->sun.misc.Launcher$AppClassLoader#1->sun.misc.Launcher$ExtClassLoader#1->null

On rails s the heap dump must be taken after performing the action that uses the specific class otherwise it cannot be found in the PermGen at all. (Have no idea why)

Any ideas or directions are much appreciated!

[kares]

 Hi Stratos, this seems rather interesting ... the additional JRubyCL is there since Trinidad is scripted in JRuby itself.
But it should work rather transparently in terms of Java classes.

Since you only mention startin with `rails s` I assume the error is the same when started with `trinidad` (without jsvc) right ?
Are the packaged .jar files proprietary or would it be possible to have a sample setup where this is reproduced ?
Cause it might be related to how class-es (from the .jar) and their dependencies are loaded when the JVM looks for .class files.

K.

[Stratos Pavlakis]

Hey Kares,

How are you ?

Well, no by 'rails s' I meant rails s trinidad. The comparison is between using jsvc or not. Always on Trinidad. That's why the extra class loader seemed strange to me. I hope I find the time to dig further in.

Thx

[kares]

 Hey Stratos, great thanks ...

I meant `rails s trinidad` as well but by `trinidad` I meant Trinidad only (it's different) - JSVC starts Trinidad similar to how you would do using simply `trinidad` (no rails s).
Thus I was wondering to rule out CL issues caused by JSVC if it happens when you start Trinidad this way (but without JSVC) ... it probably will ... I suspect it is a ClassLoader issue. Maybe if you look into what do your classes in the .jar file depend on (what imports outside of the .jar and standard JDK classes are there) ... and analyze how those are going to get loaded using the CL hierarchy.

Hope you resolve soon, K.