Complete lack of helpful compilation errors with :gen-class

[James Elliott]

I don't know if this is an issue with the compiler in general, with Leiningen more specifically, or something I am doing wrong in trying to use them, but I have wasted many hours over the last couple weeks while working on afterglow-max due to the fact that when I am compiling a Clojure source file which extends a Java class, any little mistake I make while coding gives me the following opaque output when I try to compile, rather than the helpful message telling me where the problem occurred that I am used to:

lein -U uberjar

Compiling afterglow.max.core

Compiling afterglow.max.Cue

Compiling afterglow.max.Eval

Compiling afterglow.max.init

Compiling afterglow.max.Metro

Exception: java.lang.UnsatisfiedLinkError thrown from the UncaughtExceptionHandler in thread "main"

Compilation failed: Subprocess failed

Uberjar aborting because jar failed: Compilation failed: Subprocess failed

Error encountered performing task 'uberjar' with profile(s): 'base,system,user,provided,dev,update'

Uberjar aborting because jar failed: Compilation failed: Subprocess failed

Trying to just do a lein compile seems to silently do nothing at all when it is in this state. In this case it was because I had called a function which was not defined until later in the file. Normally super easy to figure out from the compiler error messages, of course!

So I end up having to tediously try commenting out sections of my code until I can get it compiling again and then narrow down the problem. I had developed a style of making a tiny addition then rebuilding the uberjar to avoid this problem (and this is made worse by the fact that the compilation process creates futures at some point, due to one of the libraries involved, so compilation delays for an extra sixty seconds each time, waiting for Clojure's agent thread pool to shut itself down! I can't believe that *still* does not use daemon threads)... But this time I was writing the final object, which was very similar to others I had already written, and forged through until the end until trying to compile, then had to spend hours figuring out my simple stupid error.

Has anyone else seen a problem like this? Don't get me wrong, I still think Clojure is the best thing going, but it could be better without this problem. Is there anything I can do to avoid it, beyond what I already was?

[Mars0i]

I don't have anything helpful to say, but:  I've often gotten a useful stacktrace from compile-time errors using 'lein compile' with :gen-class.  Not always.  Sometimes I have to use the guess-and-comment-out method.  So I think that whatever's happening is not just an issue with compilation of gen-class.  Maybe an issue involving libraries?  This page is consistent with that hypothesis.

[James Elliott]

Indeed, I visited the page you cite while trying to pin down this problem. However, I am fairly certain that in my case it is a weird side effect, not a root cause: I get that message when compiling my Clojure file if I try to call a function which is defined later in the file. If I move the function definition up before the point where I was calling it, everything compiles fine, and no linkage error. No library changes in between the two tests. And this only happens when my Clojure file is subclassing a Java object. Very, very strange...

[Alex Miller]

If you can create a small, reproducible test case (that does not require Leiningen), please file a ticket.

[James Elliott]

Wow, that is a challenge, Alex! I would have no idea how to even compile a project without Leiningen. If I ever am not using 200% of my free time working on these projects, I may try to research that. I am afraid it may be some time, though. I was just hoping this was a known issue with know solutions.

Thanks,

  -James

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to a topic in the Google Groups "Clojure" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/clojure/yONboCAt-UA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to clojure+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Mars0i]

On Monday, August 3, 2015 at 9:27:47 AM UTC-5, James Elliott wrote:

Indeed, I visited the page you cite while trying to pin down this problem. However, I am fairly certain that in my case it is a weird side effect, not a root cause: I get that message when compiling my Clojure file if I try to call a function which is defined later in the file. If I move the function definition up before the point where I was calling it, everything compiles fine, and no linkage error. No library

Ah--if that's the problem, it sounds familiar.  Try doing 'lein clean' before 'lein compile' and see whether you get the same error.  That's assuming you have namespaces/classes listed after :aot in project.clj.  (If not, then I don't think that 'lein compile' is actually doing anything.)

You probably know this, but: When using Java classes, whether defined by gen-class or one of the other four ways to define classes (defrecord, deftype, proxy, reify), Clojure is very persnickety about compilation order.   A class has to be compiled before you can compile something that explicitly references it.  (No cyclic dependencies between explicit references to classes.)
 

[Mars0i]

On Monday, August 3, 2015 at 9:47:10 AM UTC-5, James Elliott wrote:

Wow, that is a challenge, Alex! I would have no idea how to even compile a project without Leiningen. If I ever am not using 200%

I think you can do this with compile and then java -cp ... .  I've never tried it.
 

[Alex Miller]

You can just use the clojure.core/compile function to compile. There is a simple example at http://clojure.org/compilation.  

I have not seen or experienced a linkage error like this before and I'm not aware of any ticket like this. Generally a linkage error indicates you are seeing the wrong version of a class loaded by a different classloader.

[James Elliott]

Yay, this is the kind of insight I was hoping for! ^_^

On Monday, August 3, 2015 at 9:49:17 AM UTC-5, Mars0i wrote:

Ah--if that's the problem, it sounds familiar.  Try doing 'lein clean' before 'lein compile' and see whether you get the same error.  That's assuming you have namespaces/classes listed after :aot in project.clj.  (If not, then I don't think that 'lein compile' is actually doing anything.)

Excellent point, I completely forgot about that (something about spending fifty or so hours in the previous three days cranking out code). The only time compilation would actually happen, given my project configuration, would be in creating the überjar, because that is the only profile which has :aot turned on. So I should have not been surprised that my "lein compile" did nothing.

 

You probably know this, but: When using Java classes, whether defined by gen-class or one of the other four ways to define classes (defrecord, deftype, proxy, reify), Clojure is very persnickety about compilation order.   A class has to be compiled before you can compile something that explicitly references it.  (No cyclic dependencies between explicit references to classes.)

I had not thought in precisely those terms before, but I am definitely used to struggling with cyclic dependencies and the Clojure compiler, so I am fairly confident that I did not have this issue.

I think, though, that between what you just reminded me about when compilation (doesn't) happen, and what Alex is suggesting about running the compiler directly, I likely have a workaround. When I get in this state, I will try running the compiler directly on just the suspect Clojure file and see what it says.

I now strongly suspect the problem is that somewhere in the bowels of Leiningen, this output is being discarded.

Thanks,

  -James

[James Elliott]

Thanks, Alex, I will give this a try the next time I run into this. Given a little distance from the problem, and the insights that you and Mars0i have been sharing, I have a clearer picture of what was going on at compile time. I suspect the problem is something along these lines: Leiningen invokes the compiler on the file that is supposed to create my class, and it fails because of my incorrect ordering of function invocation and definition. For reasons unknown, but which if I can pin down better I will open as an issue with Leiningen, the output from this failed compilation is lost. Then when something tries tor refer to the class which was supposed to have been compiled and fails, the linkage error happens.

But now that you have pointed me at a way to just invoke the compiler directly on my problem source file, I bet I will be able to see the compilation errors when I do that, which will enable me to fix them without trial and error. 

Indeed, I just tested that by commenting out one of the functions in my source, confirming that Leiningen gives me the cryptic linkage error, and tried a manual compile from the REPL, and was given this completely helpful compilation error:

 

afterglow.max.core=> (compile 'afterglow.max.Metro)

CompilerException java.lang.RuntimeException: Unable to resolve symbol: set-bar in this context, compiling:(afterglow/max/Metro.clj:121:7) 

So the next time I screw up, I will be able to figure out how.

Thanks, everyone! Perhaps I should open a Leiningen issue about this now.

Cheers,

  -James