How does clojure class reloading work?
Brent Millare
case there are people in here that are not on stack.
http://stackoverflow.com/questions/7471316/how-does-clojure-class-reloading-work
I will sync the good answers)
I've been reading code and documentation to try to understand how
class reloading works in clojure. According to many websites, such as
http://tutorials.jenkov.com/java-reflection/dynamic-class-loading-reloading.html
, whenever you load a class essentially you obtain the bytecode (via
any data mechanism), convert the bytecode into an instance of class
Class (via defineClass), and then resolve (link) the class via
resolveClass. (Does defineClass implicitly call resolveClass?). Any
given classloader is only allowed to link a class once. If it attempts
to link an existing class, it does nothing. This creates a problem
since you cannot link a newly instantiated class, therefore you have
to create a new instance of a classloader everytime you reload a
class.
Going back to clojure, I tried examining the paths to load classes.
In clojure, you can define new classes in multiple ways depending on
what you want:
Anonymous Class: reify proxy
Named Class: deftype defrecord (which uses deftype under the hood) gen-
class
Ultimately, those codes point to clojure/src/jvm/clojure/lang/
DynamicClassLoader.java
where DynamicClassLoader/defineClass creates an instance with super's
defineClass and then caches the instance. When you want to retrieve
the class, clojure load with a call to forName which calls the
classloader and DynamicClassLoader/findClass, which first looks in the
cache before delegating to the super class (which is contrary to the
way most normal classloaders work, where they delegate first, than try
it themselves.) *********The important point of confusion is the
following: forName is documented to link the class before it returns
but this would imply you can not reload a class from the existing
DynamicClassLoader and instead need to create a new
DynamicClassLoader, however I don't see this in the code.******** I
understand that proxy and reify define anonymous classes, so their
names are different thus can be treated as if its a different class.
However, for the named classes, this breaks down. In real clojure
code, you can have references to the old version of the classes and
references to the new version of the classes simultaneously, but
attempts to create new class instances will be of the new version.
Please explain how clojure is able to reload classes without creating
new instances of DynamicClassLoader, if I can understand the mechanism
to reload classes, I would like to extend this reloading functionality
to java's .class files I may create using javac.
Notes: This question refers to class RELOADING, not simply dynamic
loading. Reloading means that I have already interned a class but want
to intern a new updated version of that instance.
Brent Millare
An update to this question. While Chouser gave a good explanation
about the details behind proxy, reify, and gen-class, I feel that the
explanation behind deftype is incomplete. It's not clear how clojure
is able to reload deftype defined classes. Calling deftype eventually
leads to a call to clojure.lang.DynamicClassLoader/defineClass.
Calling deftype again leads to another call to defineClass, but doing
this manually results in a Linkage Error. What is happening underneath
here that allows clojure to do this with deftypes?
-Brent
On Sep 19, 9:07 am, Brent Millare <brent.mill...@gmail.com> wrote:
> (Note: I've copied my question from stackoverflow to get more looks in
>
> I've been reading code and documentation to try to understand how
Kevin Downey
a var clojure.lang.Compiler/LOADER and pushes and pops class loaders
from there.
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com
> Note that posts from new members are moderated - please be patient with your first post.
> To unsubscribe from this group, send email to
> clojure+u...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
--
And what is good, Phaedrus,
And what is not good—
Need we ask anyone to tell us these things?
Brent Millare
So when you say pushing and popping, you are painting the picture that
there is some basal loader, and every time a new class is compiled, a
new dynamic classloader is created and used to load the new class,
then the class is stored by some context reference, instances of that
class can be created. Then, we want to load a new class, and we do the
same thing, create a new classloader, set the context reference to
this new classloader, create new class instances. So now we lost the
reference to the old classloader, but objects of the old class still
exist, and will continue until they stop getting pointed to.
So two main points to confirm with you:
-A new Dynamic Classloader is created during each deftyp call
-the old classloader is lost
This would imply that the new classloader, would have to re-"find" the
classes the old classloader found from before. How does the pushing/
popping mechanism work so that it only applies to the deftypes?
Best,
Brent
Kevin Downey
> Hi Kevin,
>
> So when you say pushing and popping, you are painting the picture that
> there is some basal loader, and every time a new class is compiled, a
> new dynamic classloader is created and used to load the new class,
> then the class is stored by some context reference, instances of that
> class can be created. Then, we want to load a new class, and we do the
> same thing, create a new classloader, set the context reference to
> this new classloader, create new class instances. So now we lost the
> reference to the old classloader, but objects of the old class still
> exist, and will continue until they stop getting pointed to.
>
> So two main points to confirm with you:
>
> -A new Dynamic Classloader is created during each deftyp call
> -the old classloader is lost
dunno, I generally don't keep this stuff loaded in my brain, I would
suggest looking at the eval static method in Compiler.java, it has a
boolean parameter for evaling with a new class loader or not, that is
about as far down the path as I recall. What it does, how it does it
it, when it does it, dunno.
Chouser
> most likely the compiler is creating a new DynamicClassLoader, it uses
> a var clojure.lang.Compiler/LOADER and pushes and pops class loaders
> from there.
You nailed it.
It looks like each top-level form currently gets its own fresh
DynamicClassLoader.
(do
(deftype C [])
(def x1 (C.))
(deftype C [a])
(def x2 (C. 42)))
(.getClassLoader (class x1))
;=> #<DynamicClassLoader clojure.lang.DynamicClassLoader@3e5b38d7>
(.getClassLoader (class x2))
;=> #<DynamicClassLoader clojure.lang.DynamicClassLoader@727f3b8a>
Note the numbers after the @s are different -- each got its own
classloader. This is not exclusive to deftype of course:
(do (def a #()) (def b #()))
(.getClassLoader (class a))
;=> #<DynamicClassLoader clojure.lang.DynamicClassLoader@638bd7f1>
(.getClassLoader (class b))
;=> #<DynamicClassLoader clojure.lang.DynamicClassLoader@581de498>
--Chouser
Brent Millare
files.
(defn reload-class [classname]
(.defineClass (clojure.lang.DynamicClassLoader.)
classname
(to-byte-array (io/file "/home/user/dj/usr/src/scratch/src/scratch/
hello.class"))
nil))
(def x (reload-class "scratch.hello"))
(def xo (.newInstance x))
(.hi xo) ;;=> "hello world"
;; modify hello.java and compile with javac
(def y (reload-class "scratch.hello"))
(def yo (.newInstance y))
(.hi yo) ;;=> "hillo wirld"
-Brent
On Sep 22, 5:19 pm, Chouser <chou...@gmail.com> wrote:
Phil Hagelberg
> It looks like each top-level form currently gets its own fresh
> DynamicClassLoader.
>
>
> (do (def a #()) (def b #()))
If you'll pardon a nit-pick, this example is somewhat misleading since
do forms are special-cased by the compiler so that each form they
contain is treated as a top-level form.
Witness:
(do (def a #()) (def b #()))
(= (.getClassLoader (class a)) (.getClassLoader (class b)))
;; => false
;; we can avoid the compiler's special-casing
(when true (def a #()) (def b #()))
(= (.getClassLoader (class a)) (.getClassLoader (class b)))
;; => true
The point you are getting at is correct, but it's better shown without do.
-Phil
Alan Malloy
(let [] ...), which is in fact what some built-in constructs like
deftype expand to.
Chouser
> On Thu, Sep 22, 2011 at 2:19 PM, Chouser <cho...@gmail.com> wrote:
>> It looks like each top-level form currently gets its own fresh
>> DynamicClassLoader.
>>
>> [...]
>>
>> (do (def a #()) (def b #()))
>
> If you'll pardon a nit-pick, this example is somewhat misleading since
> do forms are special-cased by the compiler so that each form they
> contain is treated as a top-level form.
Yes, you're right. I shouldn't have used that example without
explaining that. Thanks for doing it for me. :-)
--Chouser