Currently implemented keys

[Lau Jensen]

Greetings,

A good place to start, might be to try and agree upon the types of the
arguments to keys already present in all 3 build systems. This
discussion was initiated a short while ago between myself, Justin
(cake) and Phil (lein), the history is below, shared here for the
purpose of continue the debating in the open.

I would also like to check with the Maven crew, if you guys are
interested in supporting a project.clj instead of (or with) a pom.clj
file?

--- Starting with Justins last mail (replying inline to Phil) ---

Hey Guys,

This sounds like a good start. As I said to Lau, I think it is
important to keep the spec minimal and allow each build tool to have
custom keys and even custom options within keys that are in the spec.
e.g. Cake will have additional dependency options associated with Ivy.

The purpose of the spec should be to document the features that are
supported by all build tools, and to make it clear to a project
maintainer when they are going outside the spec and thus specializing
their project for a specific build tool. Perhaps a :format key with a
version (like :min-lein-version) would be useful so project
maintainers can specify which tool and version they intend to be used
for maximum feature support.

I've added a few additional comments below:

>> >> A few other minor notes:
>> >>
>> >> 0) Repositories should be a map rather than a vector of vectors.
>> >> Technically it could be any seq of two-value seqs, but it makes more
>> >> sense to show it as a map than anything else.
> > Makes sense to use a map if in all cases, what we need is a key/val thing.
> > Justin what are your thoughts on making cake follow leins lead on this?
> >
Currently, Cake will accept repositories as a map or a vector of
vectors, though there is no way to guarantee search order unless you
use a vector of vectors. Clojure does use an ArrayMap if the map has
less than 8 keys, so in practice, this probably won't be an issue, but
the threshold is an implementation detail that may change, so it seems
incorrect to rely on it. Cake doesn't support advanced repository opts
like Lein, but it shouldn't be too hard to add if that becomes part of
the spec.

>> >> 1) I'm not sure of the point of the single-string syntax for
>> >> :dependencies. It doesn't save any keystrokes over the symbol/string
>> >> combination and introduces parsing ambiguities.
> > Agreed. It was a note added because of how I was told PMaven consumes
> > the project.clj - So until we could ask them to switch to Symbols it needs to
> > be there for compatability reasons.
I have been thinking a bit about the cost of permitting unquoted
symbols in defproject values. Syntax like this makes it difficult to
use the power of clojure inside of defproject because values must be
quoted. It might be a good idea overall to move away from this type of
thing. Phil, what are your thoughts on this? The syntax certainly
looks nicer with unquoted symbols, but is it worth the tradeoff?

In cake, we currently use symbols for:

:dependencies, :dev-dependencies, :main, :aot/:namespaces,
and :tasks

In all these cases the symbol represents a namespace or a project
name. Are there additional uses in Leiningen? I suspect there may be
some plugins that have symbols in their values.

We *could* deprecate all symbols, or we could keep existing uses, but
document which keys have symbols in their values, so tools can keep
from quoting everything else. Or should the default behavior to be to
quote values and have a list of keys that permit dynamic values?


>> >> 2) In Leiningen, :namespaces and :aot are aliases of each other, the
>> >> latter being preferred. You can say :namespaces :all or :aot :all and
>> >> it will have the same effect. Having :aot be a separate switch as in
>> >> the linked spec seems unnecessary since setting it to :all makes
>> >> :namespaces a no-op.
> >
> > Justin, any reason to keep these as 2 separate keys in Cake?
I'm fine with using just :aot. We only support both currently for Lein
compatibility.

>> >> 4) Exclusions should be mentioned as part of the :dependencies syntax.
>> >> Classifiers and types may also be desired; not sure.
> >
> > Right - Justin there is no difference between lein/cake in how they handle exclusions right?
Right, Cake currently supports :exclusions and :classifier. But Luke
Renn's Ivy branch (http://github.com/lrenn/cake/wiki/Ivy) supports
additional ivy-specific options (:conf and :transitive false).


>> >> 5) Password-protected :repositories are important.
> >
> > Is there a uniform syntax for these yet, if not, can you (phil/justin) list yours?
Cake doesn't support this yet, but I'm happy to adopt Phil's syntax.
We use :username and :password in Cake's :deploy key, so this is
inline with that. Phil, can repositories be accessed with ssh keys
too? If so, Cake uses :identity and :passphrase for these in :deploy.

>> >> 6) :source-path, :test-path, :resources-path, :library-path, and
>> >> :jar-dir are probably worth mentioning.
> >
> > Justin, does Cake align with Lein on these keys?
Cake does not currently support altering these locations primarily
because the project JVM is launched from the Ruby client script and I
didn't want to parse project.clj in that code. I plan to change the
code so that the project JVM is launched from Clojure as soon as I get
a free day to do it, so we will use the same keys as Lein when that
happens.

One other key that may be worth adding is :java-source-path. Currently
Cake uses 'src/clj' and 'src/jvm' for :source-path and :source-java-
path respectively if 'src/clj' exists and just 'src/' otherwise. Since
Lein is incorporating the lein-javac plugin, perhaps we can both adopt
this standard?

Cake's current defaults for the other keys are:
:source-path "src/" or "src/clj/"
:test-path "test/"
:resources-path "resources/"
:library-path "lib/"
:jar-dir "build/jar/" ;; assuming this does what i think it does,
which is copy files from this location into the generated jar, we also
use 'build/uberjar/' and 'build/war/'

>> >> 7) :test-resources-path may be worth mentioning, but I'm considering
>> >> renaming it to :dev-resources-path and supporting the original name
>> >> only for backwards-compatibility. Open to suggestions on that.
> >
> > Im wondering if this has a representation in Maven as well. If it doesn't, we don't
> > need to put this in the Spec. However, I still think it would be a great idea to
> > extend the spec to cover cake/lein specific keys and to keep these in sync between
> > both build-tools. That will save a lot of suffering in the future.
I prefer :dev-resources-path. This currently defaults to "dev/" for
Cake.

That's all I can think of for now.

Cheers,
Justin

[Lau Jensen]

And here is the Original from Phil (small overlap, sorry)

----

On Fri, Oct 29, 2010 at 5:04 AM, Lau B. Jensen
<lau.j...@bestinclass.dk> wrote:
> > At lot of interesting discussions flowed from Conj Labs. Among other things
> > we talked about the various build systems now available to Clojurians and their differences.
Interestingly enough this also came up at the Conj; Rich asked about
it after my talk.

Just to be precise, what this discussion defines is the semantics of
the defproject macro; polyglot Maven actually uses pom.clj rather than
project.clj. Minor point, but worth clarifying.

> > Please share your thoughts
I'm not 100% sure about this, but it seems that Maven is in a unique
position among Clojure build tools in that the semantics of its
project model are fixed. As far as I know there's very little chance
that something Clojure-specific from our discussions would be adapted
to Maven's overall project model. This would seem to imply that if
something cannot be represented in pom.xml it can't really be
considered standard. This would mean that :jar-files, :war-files, and
possibly :omit-source should be dropped from the spec. Also I'm not
sure if the semantics of :dev-dependencies is a superset of Maven's
notion of test-scoped dependencies or not; someone more familiar with
Maven would have to make that call.

A few other minor notes:

0) Repositories should be a map rather than a vector of vectors.
Technically it could be any seq of two-value seqs, but it makes more
sense to show it as a map than anything else.

1) I'm not sure of the point of the single-string syntax for
:dependencies. It doesn't save any keystrokes over the symbol/string
combination and introduces parsing ambiguities.

2) In Leiningen, :namespaces and :aot are aliases of each other, the
latter being preferred. You can say :namespaces :all or :aot :all and
it will have the same effect. Having :aot be a separate switch as in
the linked spec seems unnecessary since setting it to :all makes
:namespaces a no-op.

3) The spec doesn't mention the group-id/artifact-id notation in
name-as-SYMBOL-OR-STRING.

4) Exclusions should be mentioned as part of the :dependencies syntax.
Classifiers and types may also be desired; not sure.

5) Password-protected :repositories are important.

6) :source-path, :test-path, :resources-path, :library-path, and
:jar-dir are probably worth mentioning.

7) :test-resources-path may be worth mentioning, but I'm considering
renaming it to :dev-resources-path and supporting the original name
only for backwards-compatibility. Open to suggestions on that.

I can patch the spec to reflect these, but I thought they should be
raised for discussion first.

Also, it might be wise to move this discussion to the Clojure list
soon.

thanks,
Phil

[Justin Balthrop]

I should clarify since I'm the one who removed the context when I
shouldn't have. This is me replying to Lau replying to Phil. So the
text quoted one level deep is Lau and two levels deep is Phil.

[Phil Hagelberg]

On Nov 1, 1:36 pm, Justin Balthrop <jus...@justinbalthrop.com> wrote:
> Currently, Cake will accept repositories as a map or a vector of
> vectors, though there is no way to guarantee search order unless you
> use a vector of vectors. Clojure does use an ArrayMap if the map has
> less than 8 keys, so in practice, this probably won't be an issue, but
> the threshold is an implementation detail that may change, so it seems
> incorrect to rely on it.

Right, if the ordering here is important then a map is the wrong
representation even though it's logically associative. Makes sense. Of
course, a map will still be accepted, but the canonical representation
should be a map.

> I have been thinking a bit about the cost of permitting unquoted
> symbols in defproject values. Syntax like this makes it difficult to
> use the power of clojure inside of defproject because values must be
> quoted. It might be a good idea overall to move away from this type of
> thing. Phil, what are your thoughts on this? The syntax certainly
> looks nicer with unquoted symbols, but is it worth the tradeoff?

Sure, you definitely want to be flexible. Leiningen now supports
arbitrary execution of forms inside defproject by unquoting them with
~. I'd much rather have the extra syntax for the edge case and have
the common symbol case remain unquoted. I'm not sure if the unquote
notation belongs in the spec, but the fact that it's not hard to embed
arbitrary computation in a defproject makes me less inclined to
consider adding quotes everywhere.

> Cake doesn't support this yet, but I'm happy to adopt Phil's syntax.
> We use :username and :password in Cake's :deploy key, so this is
> inline with that. Phil, can repositories be accessed with ssh keys
> too? If so, Cake uses :identity and :passphrase for these in :deploy.

Only :username and :password are currently supported. If Maven or
Aether makes it easy to add SSH support I could look into it, but I've
never heard of that being used in the wild. (Maybe I just haven't been
paying attention?) Here's the format Leiningen supports:

:repositories [["java.net" "http://download.java.net/maven/2"]
["private" {:url "http://private.repo"
:username "milgrim"
:password "locative.1"}]]

> One other key that may be worth adding is :java-source-path. Currently
> Cake uses 'src/clj' and 'src/jvm' for :source-path and :source-java-
> path respectively if 'src/clj' exists and just 'src/' otherwise. Since
> Lein is incorporating the lein-javac plugin, perhaps we can both adopt
> this standard?

Adding :java-source-path to the spec sounds fine. I don't think
putting defaults for these in the spec makes sense; Maven already
disagrees strongly with the de-facto standards in the Clojure
community and it's unlikely we'll get consensus there.

> :jar-dir "build/jar/" ;; assuming this does what i think it does,
> which is copy files from this location into the generated jar, we also
> use 'build/uberjar/' and 'build/war/'

No, in Leiningen :jar-dir is a directory for generated jars to be
placed, much like "target/" in Maven.

> I prefer :dev-resources-path. This currently defaults to "dev/" for Cake.

Then it's agreed. I will have the next version of Leiningen issue a
deprecation notice for :test-resources-path and :namespaces.

-Phil

[Justin Balthrop]

On Nov 2, 2010, at 9:42 PM, Phil Hagelberg wrote:

> On Nov 1, 1:36 pm, Justin Balthrop <jus...@justinbalthrop.com> wrote:
>> Currently, Cake will accept repositories as a map or a vector of
>> vectors, though there is no way to guarantee search order unless you
>> use a vector of vectors. Clojure does use an ArrayMap if the map has
>> less than 8 keys, so in practice, this probably won't be an issue, but
>> the threshold is an implementation detail that may change, so it seems
>> incorrect to rely on it.
>
> Right, if the ordering here is important then a map is the wrong
> representation even though it's logically associative. Makes sense. Of
> course, a map will still be accepted, but the canonical representation
> should be a map.

so a vector of vectors it is

>> I have been thinking a bit about the cost of permitting unquoted
>> symbols in defproject values. Syntax like this makes it difficult to
>> use the power of clojure inside of defproject because values must be
>> quoted. It might be a good idea overall to move away from this type of
>> thing. Phil, what are your thoughts on this? The syntax certainly
>> looks nicer with unquoted symbols, but is it worth the tradeoff?
>
> Sure, you definitely want to be flexible. Leiningen now supports
> arbitrary execution of forms inside defproject by unquoting them with
> ~. I'd much rather have the extra syntax for the edge case and have
> the common symbol case remain unquoted. I'm not sure if the unquote
> notation belongs in the spec, but the fact that it's not hard to embed
> arbitrary computation in a defproject makes me less inclined to
> consider adding quotes everywhere.

this should work in cake too. i'll try doing this in places where i need dynamic evaluation and see how far it gets me.

>> One other key that may be worth adding is :java-source-path. Currently
>> Cake uses 'src/clj' and 'src/jvm' for :source-path and :source-java-
>> path respectively if 'src/clj' exists and just 'src/' otherwise. Since
>> Lein is incorporating the lein-javac plugin, perhaps we can both adopt
>> this standard?
>
> Adding :java-source-path to the spec sounds fine. I don't think
> putting defaults for these in the spec makes sense; Maven already
> disagrees strongly with the de-facto standards in the Clojure
> community and it's unlikely we'll get consensus there.

i'm not sure how useful a spec is without defaults. i should be able to use either of the three build tools for a simple project. perhaps the default could be the cake/lein locations and the maven defaults could be used if the aforementioned :format key is set to :maven?

i guess maven would have to throw an exception if you change any of these paths if they are not interested in letting you change your directory structure.

>> :jar-dir "build/jar/" ;; assuming this does what i think it does,
>> which is copy files from this location into the generated jar, we also
>> use 'build/uberjar/' and 'build/war/'
>
> No, in Leiningen :jar-dir is a directory for generated jars to be
> placed, much like "target/" in Maven.

got it. what do you think of renaming it to :target-path to be more in line with the other keys?

-Justin