Using JPPF in Clojure - Class Loading Issues

[Gunnar Völkel]

JPPF is a Java framework to perform distributed execution of computation jobs.
In my experiment to use JPPF (http://www.jppf.org) in Clojure I noticed a class loading problem.
A JPPFTask implemenation created via 'proxy could not be loaded by the JPPF framework.
As a result I got the following ClassNotFoundException: "Could not load class 'clj_jppf_example.core.proxy$org.jppf.server.protocol.JPPFTask$0'".
When AOT-compiling the corresponding namespace there is no problem.
But AOT-compilation seems to be a strong restriction for distributed computation in pure Clojure projects.

I have an example project for demonstration purposes on github: https://github.com/guv/clj-jppf-example

Laurent from JPPF told me that the problem is a missing cache for the byte[] representation of dynamically generated classes.
He suggested a change to Clojure's DynamicClassLoader that is adding an in-memory cache.
Most likely, an in-memory cache is not suitable in general and that change should be extended to an on-disk file cache in a temporary directory.

I hope we can discuss and realize a solution to use JPPF in Clojure without the need for AOT compilation.

Sincerely,
Gunnar

[Gunnar Völkel]

My first try to get JPPF to work from REPL is changing the ContextClassLoader to an implementation derived from clojure.lang.DynamicClassLoader which caches the class bytes on definition.
That did not work so far. Sometimes (.setContextClassLoader (Thread/currentThread) cacheClassLoader) does not even seem to work - since (.getContextClassLoader (Thread/currentThread)) still returns a DynamicClassLoader afterwards.

In case caching dynamically created classes will not be included in any future Clojure version, I want to be able to intercept class definition for the caching.
The best scenario would be, if Clojure had a compiler option that creates class files for dynamically created classes like *compile-files* in Clojure's compiler does when using (compile 'my-ns). *compile-files* seems not usable for that case in REPL by simply binding it to true.