Changes for AOT compilation
395 views
Skip to first unread message
Rich Hickey
Nov 13, 2008, 1:43:38 PM11/13/08
to Clojure
We're coming around the other side of the few breaking changes I
wanted to get done before release 1.0.
The changes are:
New regex format:
http://groups.google.com/group/clojure/msg/eddd7f0d292da683
Uniform binding syntax using vectors:
http://groups.google.com/group/clojure/browse_frm/thread/55213f2eb203ca39#
And ahead-of-time (AOT) compilation (SVN 1094+)
There is now an AOT compiler that can compile .clj files to .class
files on disk. This can be used to save startup time, deliver
applications without source, interop with bytecode-based tools etc.
The main structural change related to AOT is that the mapping between
a Clojure namespace and a Java package has changed. Where it was ns
maps to package:
my.fancy.lib ==> my/fancy/lib/lib.clj
it is now ns maps to class:
my.fancy.lib ==> my/fancy/lib.clj, when compiled, my/fancy/lib.class
Just lift your files up a directory to accommodate this change. Note
that this implies that all namespaces should have at least 2 segments,
and following the Java package name guidelines, e.g.
com.mydomain.mylib, is recommended.
Accordingly, the clojure ns has been renamed clojure.core. Any
explicitly qualified references will need to be changed. boot.clj has
been renamed core.clj.
The only other source-level change here was a small one to load - you
no longer should supply the .clj extension, in keeping with load's new
ability to load from .class files.
The latter is the only functional difference AOT brings to existing
code - load will load from a .class file if it is newer than the .clj.
Both files need to be in the classpath, using a directory structure
corresponding the the Java package as per above. Once compiled, .clj
files need not be present at all if .class files are supplied.
The unit of compilation is the lib. To compile a lib, simple say:
(compile 'my.fancy.lib)
The source must be in the classpath, in my/fancy/lib.clj Compilation
output will go under the path designated by *compile-path*, which
defaults to the "classes" directory under cwd. Output will go to
*compile-path*/my/fancy/lib.class, and the subdirectories will be
created if needed. *compile-path* is bound in the repl and can be set!
or bound with binding. The *compile-path* directory must also be in
the classpath.
Compilation occurs under a same-world model, i.e. compiling a file has
the effect of loading it as well. As each file/fn is compiled, it is
loaded using the standard class lookup (thus the *compile-path* in
classpath requirement above). This is so that definitions can see and
utilize the definitions and macros that preceded them. While
compiling, a *compile-files* flag is set. This will cause nested load/
require/uses to compile as well, if needed. If your file has some def
initializers you don't want to run at compile-time, you can
conditionalize them like this:
(def foo (when-not *compile-files* (my-runtime-init)))
in which case foo will be nil at compile time.
A .class file will be produced for every file loaded. This file
defines a class called my.fancy.lib, which will have a static void load
() method (this name is likely to change). Calling this method has the
same effect as loading the original .clj would.
Namespaces defined in multiple files are supported. Simply (load "lib-
helper-file") for any helper files at the bottom of your ns-
defining .clj file. Note that classes will be created for these files
as well, even though they are not namespaces, and are not intended for
standalone use
In addition to the class file per .clj file, there will be a .class
file for every function. These will be named my/fancy/lib
$myfun__xxxx.class. They too are not intended for standalone use.
There will be many of these, so please use jar files for distribution.
The clojure.contrib and tools folks are working on getting in sync
with these changes. For minimal disruption, please stick with the SVN
rev they support (say, 1088), or lend a hand in testing their patches.
Still to come is a folding of genclass functionality into AOT, making
for a much smoother process.
Those of you who've used Clojure for a while know such breaking
changes are few and far between. AOT compilation is an important, non-
trivial, and often requested feature. Please be patient during the
transition.
As always, feedback and bug reports welcome.
Thanks,
Rich
wanted to get done before release 1.0.
The changes are:
New regex format:
http://groups.google.com/group/clojure/msg/eddd7f0d292da683
Uniform binding syntax using vectors:
http://groups.google.com/group/clojure/browse_frm/thread/55213f2eb203ca39#
And ahead-of-time (AOT) compilation (SVN 1094+)
There is now an AOT compiler that can compile .clj files to .class
files on disk. This can be used to save startup time, deliver
applications without source, interop with bytecode-based tools etc.
The main structural change related to AOT is that the mapping between
a Clojure namespace and a Java package has changed. Where it was ns
maps to package:
my.fancy.lib ==> my/fancy/lib/lib.clj
it is now ns maps to class:
my.fancy.lib ==> my/fancy/lib.clj, when compiled, my/fancy/lib.class
Just lift your files up a directory to accommodate this change. Note
that this implies that all namespaces should have at least 2 segments,
and following the Java package name guidelines, e.g.
com.mydomain.mylib, is recommended.
Accordingly, the clojure ns has been renamed clojure.core. Any
explicitly qualified references will need to be changed. boot.clj has
been renamed core.clj.
The only other source-level change here was a small one to load - you
no longer should supply the .clj extension, in keeping with load's new
ability to load from .class files.
The latter is the only functional difference AOT brings to existing
code - load will load from a .class file if it is newer than the .clj.
Both files need to be in the classpath, using a directory structure
corresponding the the Java package as per above. Once compiled, .clj
files need not be present at all if .class files are supplied.
The unit of compilation is the lib. To compile a lib, simple say:
(compile 'my.fancy.lib)
The source must be in the classpath, in my/fancy/lib.clj Compilation
output will go under the path designated by *compile-path*, which
defaults to the "classes" directory under cwd. Output will go to
*compile-path*/my/fancy/lib.class, and the subdirectories will be
created if needed. *compile-path* is bound in the repl and can be set!
or bound with binding. The *compile-path* directory must also be in
the classpath.
Compilation occurs under a same-world model, i.e. compiling a file has
the effect of loading it as well. As each file/fn is compiled, it is
loaded using the standard class lookup (thus the *compile-path* in
classpath requirement above). This is so that definitions can see and
utilize the definitions and macros that preceded them. While
compiling, a *compile-files* flag is set. This will cause nested load/
require/uses to compile as well, if needed. If your file has some def
initializers you don't want to run at compile-time, you can
conditionalize them like this:
(def foo (when-not *compile-files* (my-runtime-init)))
in which case foo will be nil at compile time.
A .class file will be produced for every file loaded. This file
defines a class called my.fancy.lib, which will have a static void load
() method (this name is likely to change). Calling this method has the
same effect as loading the original .clj would.
Namespaces defined in multiple files are supported. Simply (load "lib-
helper-file") for any helper files at the bottom of your ns-
defining .clj file. Note that classes will be created for these files
as well, even though they are not namespaces, and are not intended for
standalone use
In addition to the class file per .clj file, there will be a .class
file for every function. These will be named my/fancy/lib
$myfun__xxxx.class. They too are not intended for standalone use.
There will be many of these, so please use jar files for distribution.
The clojure.contrib and tools folks are working on getting in sync
with these changes. For minimal disruption, please stick with the SVN
rev they support (say, 1088), or lend a hand in testing their patches.
Still to come is a folding of genclass functionality into AOT, making
for a much smoother process.
Those of you who've used Clojure for a while know such breaking
changes are few and far between. AOT compilation is an important, non-
trivial, and often requested feature. Please be patient during the
transition.
As always, feedback and bug reports welcome.
Thanks,
Rich
Stephen C. Gilardi
Nov 13, 2008, 2:21:10 PM11/13/08
to clo...@googlegroups.com
On Nov 13, 2008, at 1:43 PM, Rich Hickey wrote:
The clojure.contrib and tools folks are working on getting in sync
with these changes. For minimal disruption, please stick with the SVN
rev they support (say, 1088), or lend a hand in testing their patches.
I've updated my contribs and others that hadn't yet been updated to minimal compatibility with the AOT compilation change. Minimal compatibility means that they've been updated so they reside at the proper path (pulled up from their individual directories) and that they load into Clojure. Please test any that you use and report any problems.
For the contrib files I updated, I did an "svn cp" so Subversion history should be preserved over the transition. I also left the old versions at the old locations (appropriate for Clojure SVN 1088 and below) in place for a transitional period. I plan to delete my contribs at their old locations in a couple of days. I recommend other contrib authors do the same. Folks who want to have clojure-contrib as it was for Clojure SVN 1088 should use clojure-contrib SVN 235.
The new changes to Clojure look great! I look forward to exploring them.
--Steve
Wilkes Joiner
Nov 13, 2008, 2:32:05 PM11/13/08
to Clojure
On Nov 13, 12:43 pm, Rich Hickey <richhic...@gmail.com> wrote:
> We're coming around the other side of the few breaking changes I
> wanted to get done before release 1.0.
>
Is there a roadmap published anywhere?
> We're coming around the other side of the few breaking changes I
> wanted to get done before release 1.0.
>
Thanks,
Wilkes
Stuart Sierra
Nov 13, 2008, 5:06:29 PM11/13/08
to Clojure
On Nov 13, 1:43 pm, Rich Hickey <richhic...@gmail.com> wrote:
> We're coming around the other side of the few breaking changes I
> wanted to get done before release 1.0.
> New regex format:
> We're coming around the other side of the few breaking changes I
> wanted to get done before release 1.0.
> Uniform binding syntax using vectors:
> And ahead-of-time (AOT) compilation (SVN 1094+)
Wow, cool! I haven't had a lot of time to follow the list lately, but
I'm excited by these developments.
-Stuart
walterc
Nov 14, 2008, 2:22:13 AM11/14/08
to Clojure
does this mean that writing android application in clojure is
possible? if not, what else needs to be done to make it happen?
On Nov 14, 2:43 am, Rich Hickey <richhic...@gmail.com> wrote:
> We're coming around the other side of the few breaking changes I
> wanted to get done before release 1.0.
>
> The changes are:
>
> New regex format:
>
> http://groups.google.com/group/clojure/msg/eddd7f0d292da683
>
> Uniform binding syntax using vectors:
>
> http://groups.google.com/group/clojure/browse_frm/thread/55213f2eb203...
possible? if not, what else needs to be done to make it happen?
On Nov 14, 2:43 am, Rich Hickey <richhic...@gmail.com> wrote:
> We're coming around the other side of the few breaking changes I
> wanted to get done before release 1.0.
>
> The changes are:
>
> New regex format:
>
> http://groups.google.com/group/clojure/msg/eddd7f0d292da683
>
> Uniform binding syntax using vectors:
>
Parth Malwankar
Nov 14, 2008, 4:21:00 AM11/14/08
to Clojure
On Nov 13, 11:43 pm, Rich Hickey <richhic...@gmail.com> wrote:
> We're coming around the other side of the few breaking changes I
> wanted to get done before release 1.0.
>
> The changes are:
>
> New regex format:
>
> http://groups.google.com/group/clojure/msg/eddd7f0d292da683
>
> Uniform binding syntax using vectors:
>
>
> And ahead-of-time (AOT) compilation (SVN 1094+)
Thanks very much Rich. All these changes are very exciting.
> And ahead-of-time (AOT) compilation (SVN 1094+)
I look forward to using them.
Does this mean that at some point clojure.jar would
contain precompiled class files for clojure code?
I suppose that would mean faster startup of the
repl.
Parth
Zak Wilson
Nov 14, 2008, 4:51:22 AM11/14/08
to Clojure
It's currently possible to compile the files and rebuild the jar. It
does result in a faster startup time.
There doesn't seem to be a way to generate precompiled class files
with main methods at this point. Is there any reason such
functionality can't be added? Right now, if I want an executable
precompiled jar, I have to make a wrapper with gen-and-save-class or
Java.
does result in a faster startup time.
There doesn't seem to be a way to generate precompiled class files
with main methods at this point. Is there any reason such
functionality can't be added? Right now, if I want an executable
precompiled jar, I have to make a wrapper with gen-and-save-class or
Java.
Rich Hickey
Nov 14, 2008, 7:29:30 AM11/14/08
to Clojure
On Nov 14, 4:21 am, Parth Malwankar <parth.malwan...@gmail.com> wrote:
> On Nov 13, 11:43 pm, Rich Hickey <richhic...@gmail.com> wrote:
>
> > We're coming around the other side of the few breaking changes I
> > wanted to get done before release 1.0.
>
> > The changes are:
>
> > New regex format:
>
> >http://groups.google.com/group/clojure/msg/eddd7f0d292da683
>
> > Uniform binding syntax using vectors:
>
> >http://groups.google.com/group/clojure/browse_frm/thread/55213f2eb203...
>
> > And ahead-of-time (AOT) compilation (SVN 1094+)
>
> Thanks very much Rich. All these changes are very exciting.
> I look forward to using them.
>
> Does this mean that at some point clojure.jar would
> contain precompiled class files for clojure code?
> I suppose that would mean faster startup of the
> repl.
>
Patch welcome.
Rich
Rich Hickey
Nov 14, 2008, 7:30:33 AM11/14/08
to Clojure
Rich
MikeM
Nov 14, 2008, 9:53:21 AM11/14/08
to Clojure
> Just lift your files up a directory to accommodate this change. Note
> that this implies that all namespaces should have at least 2 segments,
> and following the Java package name guidelines, e.g.
> com.mydomain.mylib, is recommended.
>
this remain a one-segment namespace or be changed to two segments at
some point?
islon
Nov 14, 2008, 11:30:51 AM11/14/08
to Clojure
These are great changes!!
I see a bright future ahead.
Thanks Rich.
I see a bright future ahead.
Thanks Rich.
Stuart Sierra
Nov 14, 2008, 12:28:33 PM11/14/08
to Clojure
On Nov 14, 4:51 am, Zak Wilson <zak.wil...@gmail.com> wrote:
> It's currently possible to compile the files and rebuild the jar. It
> does result in a faster startup time.
I can confirm -- precompiling the core classes cuts startup time in
> does result in a faster startup time.
half:
WITHOUT precompiling:
real 4.489
user 6.612
sys 0.441
WITH precompiling
real 1.934
user 2.361
sys 0.236
(Averaged over ten trials.)
java version "1.6.0_05"
Intel Mac, Darwin Kernel Version 9.5.0
-S
Reply all
Reply to author
Forward
0 new messages