Clojure can't import some Java classes

[Zach Oakes]

I recently learned that merely importing a Java class in Clojure causes static initializers to be run. Sometimes, this causes compilation errors, because they are written with the assumption that they will only be run during runtime.

I ran into this just now while trying to make a simple Clojure game with LibGDX. After simply importing its Timer class, I began getting compilation errors. The stack trace shows it is due to a static initializer attempting to instantiate the class!

I also ran into this recently while trying to use RoboVM. My question is, do I have any options? I haven't found many discussions about this here or elsewhere. This surprises me, because it seems like something more people should be running into.

[Zach Oakes]

I should add, I am aware I can bring in a class dynamically with Class/forName, and that is what I ended up doing for the Timer class. However, this is not always practical, and sometimes is simply not an option if aot-compilation is required.

[Wujek Srujek]

So you are saying compilation is trying to instantiate class and run static initializers? This seems very backward, are you sure?

--
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Colin Fleming]

Yup, it's true. I suffer from this as well. When I'm compiling Cursive normal compilation fails because a bunch of the IntelliJ classes assume the IntelliJ platform is running and barf if it's not. I have an awful hack which is to run the compilation within their test framework which sets up a mock platform, but it's really ugly. I'd appreciate a clever workaround to this too. The other thing I've considered is to create a set of API classes to compile against which would be the standard classes with the static initialisers stripped out with ASM or something. In fact you could strip out everything but the signatures.

[Zach Oakes]

Compiling against mock classes is the other solution I've tried as well. I'm not sure how the real classes would take over once the program is run, and also it's a quite laborious and brittle solution since updates to the real classes would break the mock classes. Definitely interested in ideas others may have.

[Colin Fleming]

In my case I'm AOTing, so compiling against one set of classes then running against another isn't difficult. If you're compiling from source it's tougher, but aren't you basically compiling at runtime at that point? Couldn't you work around it changing your startup order (i.e. loading LibGDX with some Java and then requiring your Clojure)?

[Zach Oakes]

I'm just pulling LibGDX from maven and trying to call its classes via Clojure. I don't doubt that mock classes could work, but I think the effort in writing and maintaining them would be more than it's worth.

[Alex Miller]

I'd love to see a stack trace when that happens (could even be triggered by dumping stack in your static initializer if nothing else).

[Zach Oakes]

Here's the error I get when I import LibGDX's Timer class:

http://pastebin.com/q7wys8yi

[Colin Fleming]

I've also had a lot of problems with def'ed forms. For example, if I do this:

(def no-spacing (Spacing/createSpacing 0 0 0 false 0))

Which looks like this:

  public static Spacing createSpacing(int minSpaces,
                                      int maxSpaces,
                                      int minLineFeeds,
                                      boolean keepLineBreaks,
                                      int keepBlankLines) {
    return myFactory.createSpacing(minSpaces, maxSpaces, minLineFeeds, keepLineBreaks, keepBlankLines);
  }

myFactory is injected at some earlier point:

  static void setFactory(SpacingFactory factory) {
    myFactory = factory;
  }

But during compilation that obviously hasn't happened and I get an NPE during compilation. It's a little annoying that there really seems to be no good way to create this value at namespace load time if I'm AOT compiling. I can obviously defer it using defn with memoize or something, but that seems clunky for what doesn't seem like that crazy a use case.

[Aaron Cohen]

So, is this a correct summary of the problem?

"Importing a class in clojure causes static initializers to run. This differs from Java where static initializers don't run until the first 'initialization' (either the invocation of a constructor, or invocation of a 'main' method) of a class. In some cases, this makes it impossible to create the environment that java classes expect to be present before they are initialized."

[Aaron Cohen]

I'm not sure if this is related to the original problem.

 How is the setFactory method expected to be invoked, through some sort of dependency injection framework?

[Alex Miller]

That's how I read things. Importing a class causes the class to be loaded (via Class.forName()) so that it can be stored in the namespace's mappings. My impression from the commented out older code is that some prior incarnation of the code used class names instead of Class objects in some prior incarnation which would avoid the loading. Seems like importing could potentially defer loading till use (and loading could still be forced via Class.forName() if needed).

[Alex Miller]

Agreed - merely importing this class shouldn't be an issue (which is the issue at hand). 

If you need to configure Spacing, you should do that prior to using it in your def, but that should work fine. If that needs to happen at runtime, then you shouldn't do this in a def. 

[Colin Fleming]

Right, my Spacing problem is not related to the static initializer problem (which I also suffer from) but it is another example of (what seems to me) a fairly standard pattern that I can't use in Clojure if I AOT and which doesn't have an obvious workaround.

[Robin Heggelund Hansen]

Is this something that is fixable?

[Colin Fleming]

I've just spent some time today looking at the compiler code, and unfortunately I think the answer is "no". When a symbol is imported, Clojure currently instantiates the Class object using Class.forName() and stores that in the namespace's mapping. At the point the Class is instantiated, static initializers are run. So the only way to avoid that is not instantiate the Class and store something else in the mapping.

Alex's suggestion above to store the string representing the class name and load the class on demand might work for REPL style development but won't work for AOT compilation since reflection is used to find fields, methods etc on the class during compilation. The only solution that I can see that would work for AOT would be to store some sort of class wrapper object which reads the class bytecode to get that information without instantiating the class.

However both of these suggestions break a fairly fundamental assumption - that importing a class creates a mapping from its name to a Class object. I have no idea what sort of code might be out there making that assumption, but it's probably fair to assume there could be a lot of it. At some point I might look into creating a fork of Clojure I just use to AOT compile.

On 12 December 2013 03:41, Robin Heggelund Hansen <skinn...@gmail.com> wrote:

Is this something that is fixable?

--

[Colin Fleming]

In case anyone is interested in a workaround for this, I managed to "fix" my compilation by stubbing out the problematic classes and putting the stubs ahead of the real classes in the classpath when I compile. Where those stubs return other objects that are required during static initialisation, I create those classes with Mockito. I feel dirty all over but it does work.

I'm also tinkering with a fork of Clojure in the background that uses ASM Types instead of Class objects. It's still a long way from done but it's looking like that might provide a solution for compilation, at least. I'll report back if I ever get it working.

[Zach Oakes]

Yeah I tried this with RoboVM, but there are so many classes that needed to be stubbed that it turned into an endless rabbit hole, so I gave up. It may be a good solution for those who just have one or two problematic classes, though.

[Aaron Cohen]

I have the sneaking suspicion that this may be as simple as changing 

https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/Compiler.java#L697

to:

final static Method forNameMethod = Method.getMethod("Class classForNameNonLoading(String)");

and making "classForNameNonLoading" public at https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/RT.java#L2074

static public Class classForNameNonLoading(String name)

I should try it myself, but may not get to it this weekend.

[Colin Fleming]

Wow, that works! You just saved me an extraordinary amount of pain - thank you!

I had to make one further small change, to invoke the method on RT instead of Class, but that was it.

[Zach Oakes]

Definitely exciting to hear that it works. Is this something you could propose as a patch on the Clojure JIRA?

[Colin Fleming]

Absolutely, although with this fix execution doesn't work properly (for example, the tests fail because when you import a gen-class'ed class its namespace isn't loaded). However it would be good to be able to toggle it via a system property or something for compilation purposes. It's possible that that example is an incorrect use of such a class, but given that it's in the core Clojure test suite it's reasonable to assume that this would be a fairly breaking change for other code as well.

I'll mail clojure-dev and see what everyone thinks.