So in my experiments with using clojure / contrib w/ the "new" branch,
I've noticed a pattern of binary incompatibility. Jars pushed to
clojars, maven repos and other places that are are unnecessarily
AOTed, don't work with clojure "new". I noticed the same thing going
from clojure 1.0 to clojure 1.1.0-SNAP.
It doesn't really seem to matter if you include the src "clj" files
along with the class files in your jar. As soon as clojure hits a
classfile with an older AOTed namespace, it blows up. This isn't a
criticism of clojure. Clojure needs it's freedom to innovate.
...but clojure, by default, JIT compile clj files. I would just like
to start a conversation about using AOT sparingly if you publish your
clojure project as a library. Ask yourself "why am I AOTing this
namespace? Do I think JIT compiled dynamic clojure is dumb? Is this a
gen-class I need outside clojure in some fancy XML config file? Is
there some classloader issue that makes it necessary to compile AOT ?"
If the answer is NO, then please don't AOT the code.
I went through the pain of 3 days of on/off twiddling with libraries
so our project could use clojure "new". The hair-pulling details
where in all the code that was AOT when there was no reason to do
this. 99% of time there were no gen-class or anything else to require
it. People just like compiling stuff [ I get the feeling they think it
has some sort of ricer benefit to AOT compile ]. Most clj files work
great with all versions of clojure if you just jar the clj files and
leave class generation for runtime.
Don't get me wrong, there are places where a single namespace or two
are necessarily AOTed for interop or for classloader issues. But just
carte blanche AOT makes it harder on other people using your library -
to the point of trying to remove your library from their dependencies
(trust me - been there).
Code that _could_ run perfectly on clojure 1.0, 1.1.0-SNAP and "new"
are restricted to one version when AOTed and 99% for no reason.
Maybe Rich can comment on where I am missing it. But I think this is
my stance.
> So in my experiments with using clojure / contrib w/ the "new" branch,
> I've noticed a pattern of binary incompatibility. Jars pushed to
> clojars, maven repos and other places that are are unnecessarily
> AOTed, don't work with clojure "new". I noticed the same thing going
> from clojure 1.0 to clojure 1.1.0-SNAP.
> It doesn't really seem to matter if you include the src "clj" files
> along with the class files in your jar. As soon as clojure hits a
> classfile with an older AOTed namespace, it blows up. This isn't a
> criticism of clojure. Clojure needs it's freedom to innovate.
> ...but clojure, by default, JIT compile clj files. I would just like
> to start a conversation about using AOT sparingly if you publish your
> clojure project as a library. Ask yourself "why am I AOTing this
> namespace? Do I think JIT compiled dynamic clojure is dumb? Is this a
> gen-class I need outside clojure in some fancy XML config file? Is
> there some classloader issue that makes it necessary to compile AOT ?"
> If the answer is NO, then please don't AOT the code.
> I went through the pain of 3 days of on/off twiddling with libraries
> so our project could use clojure "new". The hair-pulling details
> where in all the code that was AOT when there was no reason to do
> this. 99% of time there were no gen-class or anything else to require
> it. People just like compiling stuff [ I get the feeling they think it
> has some sort of ricer benefit to AOT compile ]. Most clj files work
> great with all versions of clojure if you just jar the clj files and
> leave class generation for runtime.
> Don't get me wrong, there are places where a single namespace or two
> are necessarily AOTed for interop or for classloader issues. But just
> carte blanche AOT makes it harder on other people using your library -
> to the point of trying to remove your library from their dependencies
> (trust me - been there).
> Code that _could_ run perfectly on clojure 1.0, 1.1.0-SNAP and "new"
> are restricted to one version when AOTed and 99% for no reason.
> Maybe Rich can comment on where I am missing it. But I think this is
> my stance.
> -Tim
I've never really used many clojure libs at the same time or even any
very big ones so maybe I'm mistaken here, but does not AOT compilation
help startup time?
I know everyone is all internetty and webby these days but I kinda
like desktop apps myself and those need to start and stop a lot more
than server apps, which makes such things important.
Also from a developer point of view working with server software it's
nice if you can restart things quickly when the system has gotten into
a bad state because you carelessly deployed something broken on your
dev machine.
But if the time difference is not noticeable, then I certainly agree
with you for open source projects.
On Mon, Dec 14, 2009 at 07:26, mac <markus.gustavs...@gmail.com> wrote:
> On Dec 13, 11:44 pm, dysinger <t...@dysinger.net> wrote:
>> So in my experiments with using clojure / contrib w/ the "new" branch,
>> I've noticed a pattern of binary incompatibility. Jars pushed to
>> clojars, maven repos and other places that are are unnecessarily
>> AOTed, don't work with clojure "new". I noticed the same thing going
>> from clojure 1.0 to clojure 1.1.0-SNAP.
>> It doesn't really seem to matter if you include the src "clj" files
>> along with the class files in your jar. As soon as clojure hits a
>> classfile with an older AOTed namespace, it blows up. This isn't a
>> criticism of clojure. Clojure needs it's freedom to innovate.
>> ...but clojure, by default, JIT compile clj files. I would just like
>> to start a conversation about using AOT sparingly if you publish your
>> clojure project as a library. Ask yourself "why am I AOTing this
>> namespace? Do I think JIT compiled dynamic clojure is dumb? Is this a
>> gen-class I need outside clojure in some fancy XML config file? Is
>> there some classloader issue that makes it necessary to compile AOT ?"
>> If the answer is NO, then please don't AOT the code.
>> I went through the pain of 3 days of on/off twiddling with libraries
>> so our project could use clojure "new". The hair-pulling details
>> where in all the code that was AOT when there was no reason to do
>> this. 99% of time there were no gen-class or anything else to require
>> it. People just like compiling stuff [ I get the feeling they think it
>> has some sort of ricer benefit to AOT compile ]. Most clj files work
>> great with all versions of clojure if you just jar the clj files and
>> leave class generation for runtime.
>> Don't get me wrong, there are places where a single namespace or two
>> are necessarily AOTed for interop or for classloader issues. But just
>> carte blanche AOT makes it harder on other people using your library -
>> to the point of trying to remove your library from their dependencies
>> (trust me - been there).
>> Code that _could_ run perfectly on clojure 1.0, 1.1.0-SNAP and "new"
>> are restricted to one version when AOTed and 99% for no reason.
>> Maybe Rich can comment on where I am missing it. But I think this is
>> my stance.
>> -Tim
> I've never really used many clojure libs at the same time or even any
> very big ones so maybe I'm mistaken here, but does not AOT compilation
> help startup time?
Yes, it does help considerably with startup time. I got into the habit
of AOT compiling my stuff because my primary development machine has a
1.6GHz Atom and starting up from "slim" builds was too slow for my
taste.
That said, I think Tim's point is well taken. I should back away from
AOT compiling again and see if the performance is still as much of a
problem as I remember.
This issue has also been brought up in connection with leiningen,
which currently AOT-compiles everything as part of its normal build.
My impression is that it would be smarter to be selective about what
gets AOT compiled, and what doesn't.
> I know everyone is all internetty and webby these days but I kinda
> like desktop apps myself and those need to start and stop a lot more
> than server apps, which makes such things important.
> Also from a developer point of view working with server software it's
> nice if you can restart things quickly when the system has gotten into
> a bad state because you carelessly deployed something broken on your
> dev machine.
> But if the time difference is not noticeable, then I certainly agree
> with you for open source projects.
I like to suggest the following policy: publish libraries as jarred-
up .clj sources and AOT them locally when building an application.
Leiningen could sure help here and everen cache different compiled
versions dependening on the clojure version.
IIRC this is what Debian does with elisp packages.
But from what I (currently) know, AOTing jars should be harmless.
So maybe the problem is that there's a bug in the new branch, and this bug
needs to be corrected ?
> So in my experiments with using clojure / contrib w/ the "new" branch,
> I've noticed a pattern of binary incompatibility. Jars pushed to
> clojars, maven repos and other places that are are unnecessarily
> AOTed, don't work with clojure "new". I noticed the same thing going
> from clojure 1.0 to clojure 1.1.0-SNAP.
> It doesn't really seem to matter if you include the src "clj" files
> along with the class files in your jar. As soon as clojure hits a
> classfile with an older AOTed namespace, it blows up. This isn't a
> criticism of clojure. Clojure needs it's freedom to innovate.
> ...but clojure, by default, JIT compile clj files. I would just like
> to start a conversation about using AOT sparingly if you publish your
> clojure project as a library. Ask yourself "why am I AOTing this
> namespace? Do I think JIT compiled dynamic clojure is dumb? Is this a
> gen-class I need outside clojure in some fancy XML config file? Is
> there some classloader issue that makes it necessary to compile AOT ?"
> If the answer is NO, then please don't AOT the code.
> I went through the pain of 3 days of on/off twiddling with libraries
> so our project could use clojure "new". The hair-pulling details
> where in all the code that was AOT when there was no reason to do
> this. 99% of time there were no gen-class or anything else to require
> it. People just like compiling stuff [ I get the feeling they think it
> has some sort of ricer benefit to AOT compile ]. Most clj files work
> great with all versions of clojure if you just jar the clj files and
> leave class generation for runtime.
> Don't get me wrong, there are places where a single namespace or two
> are necessarily AOTed for interop or for classloader issues. But just
> carte blanche AOT makes it harder on other people using your library -
> to the point of trying to remove your library from their dependencies
> (trust me - been there).
> Code that _could_ run perfectly on clojure 1.0, 1.1.0-SNAP and "new"
> are restricted to one version when AOTed and 99% for no reason.
> Maybe Rich can comment on where I am missing it. But I think this is
> my stance.
> -Tim
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscribe@googlegroups.com<clojure%2Bunsubscribe@googlegroups.com >
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
There are certainly binary incompatibility issues between the
different versions/branches -- that'll settle out as the core matures,
and IIUC, especially once c-in-c becomes a reality.
However, only providing libraries as source (non-AOT-compiled) jars
whenever possible only shifts the problem around. Interop
requirements are often known of ahead of time (ha!), but many
"classloader issues" are totally in the hands of whomever is deploying
the end application, their java security policy, etc. In those
contexts, having only clojure source libraries available in maven
repos is an active hurdle to usage perhaps as irritating as having to
unwind "unnecessary" AOT compilation. Of course, people with a strict
AOT-compilation requirement can do the AOT-compilation on their own,
but I for one do not want to see Gentoo-esque norms emerge in the
Clojure community.
A more general solution would be to encourage library authors to
produce and deploy both source *and* binary, AOT-compiled artifacts,
and leave the choice to the library's users. I set this up on
Saturday for clutch (a clojure couchdb library and view server) in the
process of mavenizing the project:
I think offering both types of artifacts is a much more user-friendly
approach, is *very* easy to do (assuming maven usage), and resolves
all of the issues in play here (modulo the clojure.lang binary
signature incompatibilities that exist, but those should resolve
themselves in due course).
Cheers,
- Chas
On Dec 13, 5:44 pm, dysinger <t...@dysinger.net> wrote:
> So in my experiments with using clojure / contrib w/ the "new" branch,
> I've noticed a pattern of binary incompatibility. Jars pushed to
> clojars, maven repos and other places that are are unnecessarily
> AOTed, don't work with clojure "new". I noticed the same thing going
> from clojure 1.0 to clojure 1.1.0-SNAP.
> It doesn't really seem to matter if you include the src "clj" files
> along with the class files in your jar. As soon as clojure hits a
> classfile with an older AOTed namespace, it blows up. This isn't a
> criticism of clojure. Clojure needs it's freedom to innovate.
> ...but clojure, by default, JIT compile clj files. I would just like
> to start a conversation about using AOT sparingly if you publish your
> clojure project as a library. Ask yourself "why am I AOTing this
> namespace? Do I think JIT compiled dynamic clojure is dumb? Is this a
> gen-class I need outside clojure in some fancy XML config file? Is
> there some classloader issue that makes it necessary to compile AOT ?"
> If the answer is NO, then please don't AOT the code.
> I went through the pain of 3 days of on/off twiddling with libraries
> so our project could use clojure "new". The hair-pulling details
> where in all the code that was AOT when there was no reason to do
> this. 99% of time there were no gen-class or anything else to require
> it. People just like compiling stuff [ I get the feeling they think it
> has some sort of ricer benefit to AOT compile ]. Most clj files work
> great with all versions of clojure if you just jar the clj files and
> leave class generation for runtime.
> Don't get me wrong, there are places where a single namespace or two
> are necessarily AOTed for interop or for classloader issues. But just
> carte blanche AOT makes it harder on other people using your library -
> to the point of trying to remove your library from their dependencies
> (trust me - been there).
> Code that _could_ run perfectly on clojure 1.0, 1.1.0-SNAP and "new"
> are restricted to one version when AOTed and 99% for no reason.
> Maybe Rich can comment on where I am missing it. But I think this is
> my stance.
On Mon, Dec 14, 2009 at 11:31, Chas Emerick <cemer...@snowtide.com> wrote:
> There are certainly binary incompatibility issues between the
> different versions/branches -- that'll settle out as the core matures,
> and IIUC, especially once c-in-c becomes a reality.
> However, only providing libraries as source (non-AOT-compiled) jars
> whenever possible only shifts the problem around. Interop
> requirements are often known of ahead of time (ha!), but many
> "classloader issues" are totally in the hands of whomever is deploying
> the end application, their java security policy, etc. In those
> contexts, having only clojure source libraries available in maven
> repos is an active hurdle to usage perhaps as irritating as having to
> unwind "unnecessary" AOT compilation. Of course, people with a strict
> AOT-compilation requirement can do the AOT-compilation on their own,
> but I for one do not want to see Gentoo-esque norms emerge in the
> Clojure community.
> A more general solution would be to encourage library authors to
> produce and deploy both source *and* binary, AOT-compiled artifacts,
> and leave the choice to the library's users. I set this up on
> Saturday for clutch (a clojure couchdb library and view server) in the
> process of mavenizing the project:
Hmmm... but in the Clojure world there's a third variant, isn't there?
i.e. something like 'slim' which contains AOT only where required:
namely in the case of genclass or such.
> I think offering both types of artifacts is a much more user-friendly
> approach, is *very* easy to do (assuming maven usage), and resolves
> all of the issues in play here (modulo the clojure.lang binary
> signature incompatibilities that exist, but those should resolve
> themselves in due course).
> Cheers,
> - Chas
> On Dec 13, 5:44 pm, dysinger <t...@dysinger.net> wrote:
>> So in my experiments with using clojure / contrib w/ the "new" branch,
>> I've noticed a pattern of binary incompatibility. Jars pushed to
>> clojars, maven repos and other places that are are unnecessarily
>> AOTed, don't work with clojure "new". I noticed the same thing going
>> from clojure 1.0 to clojure 1.1.0-SNAP.
>> It doesn't really seem to matter if you include the src "clj" files
>> along with the class files in your jar. As soon as clojure hits a
>> classfile with an older AOTed namespace, it blows up. This isn't a
>> criticism of clojure. Clojure needs it's freedom to innovate.
>> ...but clojure, by default, JIT compile clj files. I would just like
>> to start a conversation about using AOT sparingly if you publish your
>> clojure project as a library. Ask yourself "why am I AOTing this
>> namespace? Do I think JIT compiled dynamic clojure is dumb? Is this a
>> gen-class I need outside clojure in some fancy XML config file? Is
>> there some classloader issue that makes it necessary to compile AOT ?"
>> If the answer is NO, then please don't AOT the code.
>> I went through the pain of 3 days of on/off twiddling with libraries
>> so our project could use clojure "new". The hair-pulling details
>> where in all the code that was AOT when there was no reason to do
>> this. 99% of time there were no gen-class or anything else to require
>> it. People just like compiling stuff [ I get the feeling they think it
>> has some sort of ricer benefit to AOT compile ]. Most clj files work
>> great with all versions of clojure if you just jar the clj files and
>> leave class generation for runtime.
>> Don't get me wrong, there are places where a single namespace or two
>> are necessarily AOTed for interop or for classloader issues. But just
>> carte blanche AOT makes it harder on other people using your library -
>> to the point of trying to remove your library from their dependencies
>> (trust me - been there).
>> Code that _could_ run perfectly on clojure 1.0, 1.1.0-SNAP and "new"
>> are restricted to one version when AOTed and 99% for no reason.
>> Maybe Rich can comment on where I am missing it. But I think this is
>> my stance.
>> -Tim
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscribe@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> But from what I (currently) know, AOTing jars should be harmless. > So maybe the problem is that there's a bug in the new branch, and this bug needs to be corrected ?
No. The inner workings of Clojure might change. AOT compiled code doesn't know these changes. I had this with some change to defmulti a while ago. However the source is always recompiled when it's loaded. So it always see the change and all is fine. (Of course only, as long as the change doesn't involve a public API.)
> > But from what I (currently) know, AOTing jars should be harmless.
> > So maybe the problem is that there's a bug in the new branch, and this
> bug needs to be corrected ?
> No. The inner workings of Clojure might change. AOT compiled code doesn't
> know these changes. I had this with some change to defmulti a while ago.
> However the source is always recompiled when it's loaded. So it always see
> the change and all is fine. (Of course only, as long as the change doesn't
> involve a public API.)
So i's a kind of break of compatibility with AOT compiled code. That should
be noted in future official clojure releases, indeed, since it requires
every library to be recompiled.
> Hmmm... but in the Clojure world there's a third variant, isn't there?
> i.e. something like 'slim' which contains AOT only where required:
> namely in the case of genclass or such.
Yes, that's true. This is something that lein and clojure-maven- plugin et al. will have to consider going forward.
B Smith-Mannschott <bsmith.o...@gmail.com> writes:
> This issue has also been brought up in connection with leiningen,
> which currently AOT-compiles everything as part of its normal build.
> My impression is that it would be smarter to be selective about what
> gets AOT compiled, and what doesn't.
Yep, this will be changed in the next release of leiningen. The default
build will only AOT namespaces that actually need it to
function. Naturally you will still be able to build jars that AOT
everything, but you will have to specifically ask for that behaviour.
On Tue, Dec 15, 2009 at 7:05 AM, Phil Hagelberg <p...@hagelb.org> wrote:
> Yep, this will be changed in the next release of leiningen. The default
> build will only AOT namespaces that actually need it to
> function. Naturally you will still be able to build jars that AOT
> everything, but you will have to specifically ask for that behaviour.
I also have an almost working polyglot maven3 clojure module working
as well (reads and writes maven poms in clojure using a syntax
initially borrowed from leiningen but extended for the rest of mavens
bits and pieces - trying to sort out the syntax I want to use is
annoyingly hard!).
The way I was thinking of doing it was configuring up a default
assembly plugin instance to build the slim jar, a clojure:slimjar goal
on the clojure-maven-plugin ( probably the better place to put it ).
