Issue request: RT.load's "don't load if already loaded" mechanism breaks ":reload-all"

16 views
Skip to first unread message

Stephen C. Gilardi

unread,
Feb 6, 2009, 8:28:08 AM2/6/09
to clo...@googlegroups.com
"require" and "use" support a ":reload-all" flag that is intended to
cause the specified libs to be reloaded along with all libs on which
they directly or indirectly depend. This is implemented by temporarily
binding a "loaded-libs" var to the empty set and then loading the
specified libs.

AOT compilation added another "already loaded" mechanism to
clojure.lang.RT.load() which is currently not sensitive to a "reload-
all" being in progress and breaks its operation in the following case:

A, B, and C are libs
A depends on B. (via :require in its ns form)
B depends on C. (via :require in its ns form)
B has been compiled (B.class is on classpath)

At the repl I "require" A which loads A, B, and C (either from class
files or clj files)
I modify C.clj
At the repl I "require" A with the :reload-all flag, intending to
pick up the changes to C
C is not reloaded because RT.load() skips loading B: B.class exists,
is already loaded, and B.clj hasn't changed since it was compiled.

I'd like to add an issue for this to the Clojure project and supply a
patch to fix it. My current patch creates a "*reload-all*" var with a
root binding of nil and code to bind it to true when the current
thread has a :reload-all call pending. When *reload-all* is true,
RT.load() will (re)load all libs from their ".clj" files even if
they're already loaded.

--Steve

Stephen C. Gilardi

unread,
Feb 6, 2009, 8:33:25 AM2/6/09
to clo...@googlegroups.com

On Feb 6, 2009, at 8:28 AM, Stephen C. Gilardi wrote:

> When *reload-all* is true, RT.load() will (re)load all libs from
> their ".clj" files even if they're already loaded.

To clarify:

When *reload-all* is true, RT.load will (re)load any lib it is asked
to load from the lib's ".clj" file even if the lib is already loaded.

--Steve

Laurent PETIT

unread,
Feb 6, 2009, 8:45:11 AM2/6/09
to clo...@googlegroups.com
Hello,

Does it also mean that the following use case will work with *reload-all* :

- x.y.z is a lib that is made of two files : x/y/z.clj, and x/y/z1.clj , and z.clj loads z1.clj
- x.y.z is compiled
- z1.clj is modified
- x.y.z is compiled => even if x/y/z.clj nor x/y/z__init.class are in sync, the changes in z1.clj will be reloaded, because z1.clj will be forced to be reloaded from source file

?

Is my understanding correct ?

If so, ++1 for this !

Regards,

--
Laurent

2009/2/6 Stephen C. Gilardi <sque...@mac.com>

Stephen C. Gilardi

unread,
Feb 11, 2009, 8:01:58 AM2/11/09
to clo...@googlegroups.com

On Feb 6, 2009, at 8:45 AM, Laurent PETIT wrote:

> Hello,
>
> Does it also mean that the following use case will work with *reload-
> all* :
>
> - x.y.z is a lib that is made of two files : x/y/z.clj, and x/y/
> z1.clj , and z.clj loads z1.clj
> - x.y.z is compiled
> - z1.clj is modified
> - x.y.z is compiled => even if x/y/z.clj nor x/y/z__init.class are
> in sync, the changes in z1.clj will be reloaded, because z1.clj will
> be forced to be reloaded from source file
>
> ?
>
> Is my understanding correct ?

Yes, I tested it and for that case, the following will recompile x.y.z1:

(binding [*reload-all* true]
(compile 'x.y.z))

With *reload-all* in place, one could also provide clojure.core/
compile-all which calls "load-all" internally rather than "load-one".
This would recompile everything on which the given lib (directly or
indirectly) depends without regard to mod-dates. (And because *loaded-
libs* is still at work, each lib would only be compiled once.)

> If so, ++1 for this !

I'm glad the proposed change will help with the situation you asked
about. The :reload-all mechanism is currently broken in Clojure and I
would like to fix it. I'll be happy to include a clojure.core/compile-
all in that patch if it's welcome. (I think it would be useful.)

--Steve

> 2009/2/6 Stephen C. Gilardi <sque...@mac.com>

>
> On Feb 6, 2009, at 8:28 AM, Stephen C. Gilardi wrote:
>
> When *reload-all* is true, RT.load() will (re)load all libs from
> their ".clj" files even if they're already loaded.
>
> To clarify:
>
> When *reload-all* is true, RT.load will (re)load any lib it is asked
> to load from the lib's ".clj" file even if the lib is already loaded.
>
> --Steve
>
>
>

> --~--~---------~--~----~------------~-------~--~----~
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com
> To unsubscribe from this group, send email to clojure+u...@googlegroups.com
> For more options, visit this group at http://groups.google.com/group/clojure?hl=en
> -~----------~----~----~----~------~----~------~--~---
>

Stephen C. Gilardi

unread,
Feb 13, 2009, 9:04:46 AM2/13/09
to clo...@googlegroups.com
Rich,

May I please enter an issue to track the defect that require/use's ":reload-all" flag is not working properly in Clojure.

--Steve

On Feb 11, 2009, at 8:01 AM, Stephen C. Gilardi wrote:


On Feb 6, 2009, at 8:45 AM, Laurent PETIT wrote:

Hello,

Does it also mean that the following use case will work with *reload-all* :

- x.y.z is a lib that is made of two files : x/y/z.clj, and x/y/z1.clj , and z.clj loads z1.clj
- x.y.z is compiled
- z1.clj is modified
- x.y.z is compiled => even if x/y/z.clj nor x/y/z__init.class are in sync, the changes in z1.clj will be reloaded, because z1.clj will be forced to be reloaded from source file

?

Is my understanding correct ?

Yes, I tested it and for that case, the following will recompile x.y.z1:

(binding [*reload-all* true]
 (compile 'x.y.z))

With *reload-all* in place, one could also provide clojure.core/compile-all which calls "load-all" internally rather than "load-one". This would recompile everything on which the given lib (directly or indirectly) depends without regard to mod-dates. (And because *loaded-libs* is still at work, each lib would only be compiled once.)


If so, ++1 for this !

I'm glad the proposed change will help with the situation you asked about. The :reload-all mechanism is currently broken in Clojure and I would like to fix it. I'll be happy to include a clojure.core/compile-all in that patch if it's welcome. (I think it would be useful.)

Rich Hickey

unread,
Feb 13, 2009, 10:12:37 AM2/13/09
to Clojure


On Feb 13, 9:04 am, "Stephen C. Gilardi" <squee...@mac.com> wrote:
> Rich,
>
> May I please enter an issue to track the defect that require/use's
> ":reload-all" flag is not working properly in Clojure.
>

How does this interact with:

http://code.google.com/p/clojure/issues/detail?id=3

Rich

Stephen C. Gilardi

unread,
Feb 13, 2009, 11:13:11 AM2/13/09
to clo...@googlegroups.com

I don't know enough to answer properly. Perhaps we can make it moot by
fixing issue 3.

I have ideas for things to try, but to test them I need to reproduce
the problem that caused you to remove the throw on circular load from
clojure.core/load.

Here's what I tried to reproduce that problem:

- restored the "throw on circular load" lines to clojure.core/load

- compiled all of Clojure and contrib successfully

- Compiled/loaded/ran this successfully from Clojure and by using
hello as a java main:

(ns hello
(:gen-class))

(defn hello [] (prn "hi"))

(defn -main []
(hello))

I thought that the default of :load-impl-ns to true would trigger the
problem, but it didn't.

Could you please give me an example of code that triggers the
undesired throw on circular load?

--Steve

Stephen C. Gilardi

unread,
Mar 20, 2009, 12:39:15 PM3/20/09
to clo...@googlegroups.com
Rich,

Can we please track this as an issue.

--Steve

Rich Hickey

unread,
Mar 20, 2009, 1:33:03 PM3/20/09
to clo...@googlegroups.com

On Mar 20, 2009, at 12:39 PM, Stephen C. Gilardi wrote:

> Rich,
>
> Can we please track this as an issue.
>

Yes, sure.

Rich

Reply all
Reply to author
Forward
0 new messages