Leiningen, AOT compilation, and classloaders

[Tianxiang Xiong]

I recently ran into an issue with AOT compilation that I'd like to understand better. A minimal working example (MWE) can be found here. 

Basically, using `:aot :all` in `project.clj` with `lein trampoline test` results in different classloaders for class `Foo`.  

Classloader for lein_trampoline_aot.core.Foo:  #object[clojure.lang.DynamicClassLoader 0x4982cc36 clojure.lang.DynamicClassLoader@4982cc36]
Classloader for (->Foo):  #object[sun.misc.Launcher$AppClassLoader 0x55f96302 sun.misc.Launcher$AppClassLoader@55f96302]

While the same classloaders are used with `lein test`:

Class loader for lein_trampoline_aot.core.Foo:  #object[sun.misc.Launcher$AppClassLoader 0x55f96302 sun.misc.Launcher$AppClassLoader@55f96302]
Classloader for (->Foo):  #object[sun.misc.Launcher$AppClassLoader 0x55f96302 sun.misc.Launcher$AppClassLoader@55f96302]

When is one classloader used instead of another, and why do `lein trampoline test` and `lein test` behave differently with `:aot :all`?

[Mike Rodriguez]

I haven't been able to get to the bottom of this as of yet.  Primarily the problem is I need to investigate how `lein trampoline` works compared to without it, from an implementation perspective.

I'll note that `lein test` does do a :reload option to `require` when running tests.  Typically forced reloads of namespaces doesn't mix well with AOT compilation, due to classloader issues like you are reporting here.

However, I'm still missing key parts to understanding this.  I want to look a bit closer at it, but just responding here in case these thoughts are meaningful to anyone else that is looking at it.

[Tianxiang Xiong]

Thanks for investigating--any luck?

[Kevin Downey]

On 03/07/2017 09:07 PM, 'Tianxiang Xiong' via Clojure wrote:
> I recently ran into an issue with AOT compilation that I'd like to
> understand better. A minimal working example (MWE) can be found here

> <https://github.com/xiongtx/lein-trampoline-aot>.

>
> Basically, using `:aot :all` in `project.clj` with `lein trampoline
> test` results in different classloaders for class `Foo`.
>
> |
> Classloaderforlein_trampoline_aot.core.Foo: #object[clojure.lang.DynamicClassLoader
> 0x4982cc36 clojure.lang.DynamicClassLoader@4982cc36]

> Classloaderfor(->Foo): #object[sun.misc.Launcher$AppClassLoader

> 0x55f96302 sun.misc.Launcher$AppClassLoader@55f96302]
> |
>
> While the same classloaders are used with `lein test`:
>
> |
> Classloader
> forlein_trampoline_aot.core.Foo: #object[sun.misc.Launcher$AppClassLoader 0x55f96302
> sun.misc.Launcher$AppClassLoader@55f96302]

> Classloaderfor(->Foo): #object[sun.misc.Launcher$AppClassLoader

> 0x55f96302 sun.misc.Launcher$AppClassLoader@55f96302]
> |
>
> When is one classloader used instead of another, and why do `lein
> trampoline test` and `lein test` behave differently with `:aot :all`?

When you aot compile you generate a class file on disk that is visible
to the system classloader. When the reference to the Foo class is
compiled, it is compiled as a call something like
clojure.lang.RT.classForName("Foo"); RT.classForName uses different
class loaders depend on this that an the other, and in this case it is
using the system classloader (the AppClassLoader) when using lein
trampoline. When you loaded your clojure code it ran through the
compiler which generated bytecode which is loaded via clojure's
DynamicClassLoader.


The way RT.classForName determines which classloader to use is something
like: if there is a DynamicClassloader available from the compiler, use
it, otherwise use the system classloader.

Once you finish loading code the DynamicClassloader the compiler uses
is, uh, for lack of a better word, popped, so it isn't available to
RT.classForName.

So given that and the error we can deduce that `lein trampoline` runs in
to phases a code loading phase, and then an execution phase. Which is a
perfectly reasonable thing to do and easy to do with the command line
options to clojure.main.

I suspect adding an import of the Foo class would work around this
issue, but I strongly recommend you consider not aot compiling instead.
AOT compilation is a source of weird behavior and just isn't worth it.

The #1 reason people aot compile is because they don't know how else to
start their clojure programs, so I have a little write up here
https://github.com/hiredman/clojure-site/blob/df56aef005d5d867213a51c2d3bbec5a86b0acad/content/guides/running_a_clojure_program.adoc



> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+u...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to clojure+u...@googlegroups.com
> <mailto:clojure+u...@googlegroups.com>.
> For more options, visit https://groups.google.com/d/optout.


--
And what is good, Phaedrus,
And what is not good—
Need we ask anyone to tell us these things?