Clojure deployment questions w.r.t. jars, clojure source files, compiled class files

[Dave Tenny]

So if I sample some clojure jars in my local maven .m2 directory, most of the jar files have only clojure code, and a project.clj.

If I look at the org.clojure project there, it has many java class files, what appear to be corresponding clojure source files, and *no* project.clj.

With only a little experiencing on some single AOT compiled modules, I wondered 

1) When and under what circumstances projects are compiled in  source versus .class form?

2) Why there is no project.clj in the org.clojure jar file?

3) When the clojure 'compile' function comes into play in your typical clojure project deployments? (vs. :aot targets or other leiningen deployment techniques).

Any pointers or enlightenment on the above questions appreciated.

I read the clojure "compilation" page, but I haven't quite put together what that's telling me about *how* to comjpile a file and gen-class a class,

versus why I see all these clojure .m2 jar files with no classes whatsoever.

Thanks,

Dave

[Gary Trakhman]

AOT in my experience is a little dicey and complicated.

On Tue, Jan 7, 2014 at 7:26 PM, Dave Tenny <dave....@gmail.com> wrote:

So if I sample some clojure jars in my local maven .m2 directory, most of the jar files have only clojure code, and a project.clj.

If I look at the org.clojure project there, it has many java class files, what appear to be corresponding clojure source files, and *no* project.clj.

With only a little experiencing on some single AOT compiled modules, I wondered 

1) When and under what circumstances projects are compiled in  source versus .class form?

In general, you always want to distribute libs as maven artifacts with source included and no AOT compilation.  Adding in compiled classes means also adding a dependency on whatever clojure version was used to compile those classes.  The effects of this lead to often confounding errors.

As an extra caveat, AOT currently does not make it easy to separate that project's class files from every other deps generated class files, so any project artifact will have extra classfiles in it.  My company has grappled with separating these about five different ways in order to not distribute source, and there is a ticket to address it, but it looks like it'll persist.

 

2) Why there is no project.clj in the org.clojure jar file?

Clojure itself isn't built with leiningen, hence no project.clj.

 

3) When the clojure 'compile' function comes into play in your typical clojure project deployments? (vs. :aot targets or other leiningen deployment techniques).

Any pointers or enlightenment on the above questions appreciated.

I read the clojure "compilation" page, but I haven't quite put together what that's telling me about *how* to comjpile a file and gen-class a class,

versus why I see all these clojure .m2 jar files with no classes whatsoever.

Thanks,

Dave

--
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Sean Corfield]

On Tue, Jan 7, 2014 at 4:26 PM, Dave Tenny <dave....@gmail.com> wrote:
> 1) When and under what circumstances projects are compiled in source versus
> .class form?

Most Clojure projects ship in source form (and are therefore compiled
to bytecode on demand as they are loaded).

> 2) Why there is no project.clj in the org.clojure jar file?

It's built with Maven. As are most of the Clojure contrib libraries
too, although some are now starting to sport project.clj files to make
local development easier. In the last round of development, I added
Leiningen support to clojure.java.jdbc to make it easier to "jack in"
with Emacs and test the code. It still uses Maven for primary testing
(on build.clojure.org) and packaging - and Clojure plus its contrib
libraries are hosted on Maven Central (where they are retrieved
primarily by Leiningen into other Clojure projects).

> 3) When the clojure 'compile' function comes into play in your typical
> clojure project deployments? (vs. :aot targets or other leiningen deployment
> techniques).

At World Singles, we AOT compile very little of our code. We only AOT
namespaces that generate Java-compatible classes for situations where
we must be natively callable from Java (e.g., we have a log4j appender
written in Clojure). Within that AOT-compiled code, we require
namespaces and resolve symbols dynamically (at runtime) to bind the
Java-called code to the rest of our code base.

Part of the reason is for the flexibility that source deployments
provide: the ability to REPL into a live, running process and reload
code from updated source files without needing to "stop the world",
for example.

If you're relatively new to Clojure, I'd recommend completely ignoring
the whole "compilation" thing unless you specifically need to generate
natively callable code for Java to Clojure interop.

In case anyone is interested, our pattern for bridging from the
AOT-compiled namespace to the rest of the code base tends to look like
this:

(def the-symbol
(delay
(do
(require 'the.namespace)
(resolve (symbol "the.namespace/the-symbol")))))

and then:

... (@the-symbol arg1 arg2) ...

Our AOT-compiled layer is deliberately minimal and serves only to
provide the Java-callable API so the rest of our code can be developed
and tested in our normal interactive, incremental way.
--
Sean A Corfield -- (904) 302-SEAN
An Architect's View -- http://corfield.org/
World Singles, LLC. -- http://worldsingles.com/

"Perfection is the enemy of the good."
-- Gustave Flaubert, French realist novelist (1821-1880)

[Softaddicts]

Just to throw a wrench in this thread :)))

a) We AOT our stuff to avoid shipping our sourde code to customer sites
(:aot [regex...]), this requirement is the basis for what follows.

b) We filter classes using lein to remove from our artifacts classes
from other libs so only our AOT code is shipped in our jar files
(:jar-exclusions [regex...]). This prevents mixing compiled classes
of third party in our own jars which I agree can be pretty confusing
(from which jar comes this compiled class ? Euuuhhhhh....)

c) We are strict on version control and obviously will not ship
Clojure code compiled with different versions of Clojure, this is
a recipe for disaster when there are feature changes in Clojure.
We follow Orson's advice here, "We will sell no wine before its time".
We skipped some Clojure releases in the past and we moved to 1.5.1
by mid 2013.

d) Most of the source code shipped comes from the open source
libs we build upon. We can deploy site patches as source code and
get them applied at app startup time when we need quick fixes in prod
but they are not part of the build (otherwise they would not be called
patches :)

e) We also have Clojure data/code stored as part of the configuration and
loaded on demand, it's been mixing fine with our AOT compiled
aritfacts.

e) We use an integrated nrepl console embedded in our apps and
attach to it. We can investigate live in prod, apply patches live
albeit with some limitations but as far as I can see AOT is
not bitting us here. Restrictions have more to do with the app life cycle
which we are improving along the way.

f) We obvisously do integration tests using AOT compiled artifacts while dev
testing is not to avoid time travels to stone age (edit compile restart hell)

In our design most of the stuff is done in our supervisor process and each
supervisor auto configures itself differently using plugins according to the
local configuration and it has to run 24/7. I think we kept most of the
dynamic benefits while meeting the goal in a).

We have been using AOT for 4 years now. It's not at as hard as people
think it is but you need a minimal understanding of the underlying
machinery and if you come from a non Java background I agree
100%, it's not easy to find a path between the sand traps to get you
to putt in the hole from dev to prod.

End of the wrench throwing :)))

Luc P.

--
Softaddicts<lprefo...@softaddicts.ca> sent by ibisMail from my ipad!

[Softaddicts]

To complement my previous post and in relation to this one,
we use (if-not *compile-file* ...) to wrap expressions that cannot be
AOT compiied (like loading configuration or connecting to external
resources which you do not want to do at compile time but at run time only:)

This confused a lot of people in the past trying to use AOT.
For many the compilation phase and code execution are perceived
as a single step. When working in the REPL you do not need to be aware of
this. With AOT it's important to be aware of the difference when writing code
in dev.

Luc P.

[Dave Tenny]

Excellent answers and I usually find a use for extra wrenches.

I'm still confused about when anybody actually calls the (compile) function, any more tips here?  Or is it being done for me by leiningen?

You received this message because you are subscribed to a topic in the Google Groups "Clojure" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/clojure/sU0r00i7Ls0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to clojure+u...@googlegroups.com.

[Softaddicts]

Look at the compile fn at the bottom:

https://github.com/technomancy/leiningen/blob/master/src/leiningen/compile.clj

you will find your answer :)

Luc P.