Minimizing the number of model implementations
Chas Emerick
quickly, but I wanted to raise an important meta issue.
AFAIK, it is relatively straightforward to add new serializations for
the POM in Polyglot Maven; IIRC it was Antony Blakey that added the
lein-compatible de/serializer to the Clojure pmaven project earlier
this year.
Assuming the cake and lein leads/communities can settle on a
serialization of project.clj and the semantics associated with each
key, etc, etc...it would seem to be a huge loss for everyone if
multiple implementations of that model were maintained.
Why not ensure that pmaven can consume/produce the serialization that
you agree upon? Then, assuming cake and lein used pmaven (for the
model at the very least), *all* build tooling for Clojure will be
working off of the same model, and you guys wouldn't have to maintain
your own (surely slightly different) implementations of it. Even
better, as tools add support for maven 3 / pmaven (such as hudson,
bamboo, IDE plugins, etc), lein and cake projects will be
indistinguishable to them. Finally, you could add commands to your
respective frontends to drive various pmaven/maven-provided
functionality, such as generalized deployment, using existing maven
plugins, etc.
Perhaps this is already implicitly the plan, but I wanted to throw
this out there to either confirm that, or if not, suggest it.
Cheers,
- Chas
Antony Blakey
On 12/11/2010, at 11:35 PM, Chas Emerick wrote:
> AFAIK, it is relatively straightforward to add new serializations for
> the POM in Polyglot Maven; IIRC it was Antony Blakey that added the
> lein-compatible de/serializer to the Clojure pmaven project earlier
> this year.
The architecture of clojure + pmaven consists of a mediated reflective exposure of the pmaven model classes that produces a reader (i.e. macros that define lots of functions and macros), coupled with a lein compatible macro that uses that canonical form.
> Why not ensure that pmaven can consume/produce the serialization that
> you agree upon? Then, assuming cake and lein used pmaven (for the
> model at the very least), *all* build tooling for Clojure will be
> working off of the same model, and you guys wouldn't have to maintain
> your own (surely slightly different) implementations of it. Even
> better, as tools add support for maven 3 / pmaven (such as hudson,
> bamboo, IDE plugins, etc), lein and cake projects will be
> indistinguishable to them. Finally, you could add commands to your
> respective frontends to drive various pmaven/maven-provided
> functionality, such as generalized deployment, using existing maven
> plugins, etc.
I did clojure pmaven integration into Netbeans. The primary problem is that the Maven IDE integrations have the XML syntax deeply embedded. Netbeans Maven does it's own parsing, and in the end the best integration I could do without replicating all the effort put into the Maven plugin was to watch the pom.clj/project.clj files (either lein compatible or canonical), and recreate the XML file to trigger the normal update process.
IDE integration of a non-XML format is a significant amount of work for very little return given the (IMHO NIH) community impulse to use clojure-native build infrastructure. Actually, the best approach IMO is the pom.xml editor in Eclipse which does away with text.
> Perhaps this is already implicitly the plan, but I wanted to throw
> this out there to either confirm that, or if not, suggest it.
Another issue is that the tools triggered by the build process need to be integrated with pmaven as plugins if you want to use lein/cake seamlessly from the IDE. The IDE integrations assume things like directory layouts.
Antony Blakey
--------------------------
CTO, Linkuistics Pty Ltd
Ph: 0438 840 787
Human beings, who are almost unique in having the ability to learn from the experience of others, are also remarkable for their apparent disinclination to do so.
-- Douglas Adams
Chas Emerick
> On 12/11/2010, at 11:35 PM, Chas Emerick wrote:
>
> > Why not ensure that pmaven can consume/produce the serialization that
> > you agree upon? Then, assuming cake and lein used pmaven (for the
> > model at the very least), *all* build tooling for Clojure will be
> > working off of the same model, and you guys wouldn't have to maintain
> > your own (surely slightly different) implementations of it. Even
> > better, as tools add support for maven 3 / pmaven (such as hudson,
> > bamboo, IDE plugins, etc), lein and cake projects will be
> > indistinguishable to them. Finally, you could add commands to your
> > respective frontends to drive various pmaven/maven-provided
> > functionality, such as generalized deployment, using existing maven
> > plugins, etc.
>
> I did clojure pmaven integration into Netbeans. The primary problem is that the Maven IDE integrations have the XML syntax deeply embedded. Netbeans Maven does it's own parsing, and in the end the best integration I could do without replicating all the effort put into the Maven plugin was to watch the pom.clj/project.clj files (either lein compatible or canonical), and recreate the XML file to trigger the normal update process.
>
essentially the same, i.e. relying upon the XML serialization.
> IDE integration of a non-XML format is a significant amount of work for very little return given the (IMHO NIH) community impulse to use clojure-native build infrastructure. Actually, the best approach IMO is the pom.xml editor in Eclipse which does away with text.
>
than where it is now.
Presuming pmaven to be a strategic concern for Sonatype, they'll
likely ensure that Eclipse and hudson (at the very least) are fitted
with pmaven-aware support. Whether this means driving that support
from their implementation of the POM or automagically doing what you
did – writing out XML serializations from the authoritative clj/scala/
groovy/etc file – is entirely an implementation detail AFAICT.
Between now and then, it seems that the key is to make sure that
Clojure pmaven is suitable for use with cake and lein projects in the
wild. Of course, if the lein/cake leads/userbase don't see value in
the proposition at all (my current impression given irc
conversations), then I suppose all this is moot.
> > Perhaps this is already implicitly the plan, but I wanted to throw
> > this out there to either confirm that, or if not, suggest it.
>
> Another issue is that the tools triggered by the build process need to be integrated with pmaven as plugins if you want to use lein/cake seamlessly from the IDE. The IDE integrations assume things like directory layouts.
experience. I've successfully used maven projects that defined
nonstandard source, test source, and resources directories in Eclipse
and NetBeans, using their standard maven support?
Beyond that, I'm not familiar enough with either cake or lein plugins
(or their fairly different models, IIRC?) to be able to comment on
that. I wonder if exposing them as additional available goals in the
POM provided to pmaven by the clj deserializer would be possible?
- Chas
Antony Blakey
On 14/11/2010, at 8:22 AM, Chas Emerick wrote:
> Of course, if the lein/cake leads/userbase don't see value in
> the proposition at all (my current impression given irc
> conversations), then I suppose all this is moot.
I'm assuming that lack of interest. To be honest, I've given up on pmaven for clojure because:
a) so much work is required across three IDEs to make a clojure serialization first-class e.g. completion of version numbers in specs etc; and
b) the desire to have the *.clj file be an executable specification as opposed to a strictly declarative specification makes tool support even more difficult, and in particular makes it less likely that the existing models of coding assistance are applicable - it's no longer a serialization of the model, rather it is a program that builds a model either directly or as a side-effect of execution in some arbitrary runtime context; and
c) the community impulse to redevelop the wheel for build/dependency management without concern for building on the significant ecosystem that already exists is too depressing. I could speculate on why that has occurred, but IMHO it's a disaster for clojure adoption. Seamless integration into build/deployment/aggregation/versioning infrastructure is the *only* way Clojure can go viral. IMHO once again.
>> Another issue is that the tools triggered by the build process need to be integrated with pmaven as plugins if you want to use lein/cake seamlessly from the IDE. The IDE integrations assume things like directory layouts.
>
> Maybe I misunderstand what you're saying, but that's not at all my
> experience. I've successfully used maven projects that defined
> nonstandard source, test source, and resources directories in Eclipse
> and NetBeans, using their standard maven support?
It technically works in NB as far as execution is concerned, but the directories don't show up in the navigator properly. I had to mod the NB core to get src/main/clojure e.g. to show up as a source folder with packages rather than a simple folder with directories. That's a pretty basic piece of IDE integration IMO.
Antony Blakey
-------------
CTO, Linkuistics Pty Ltd
Ph: 0438 840 787
There are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies, and the other way is to make it so complicated that there are no obvious deficiencies.
-- C. A. R. Hoare
Justin Balthrop
here's my perspective. i don't see the value in replacing a short and simple macro with pmaven just to parse a defproject form that is easily handled by the Clojure reader. is there something i'm missing? are you suggesting something more than just using the maven code to parse defproject? in cake, the result of defproject is just a map. is there a good reason for it to be anything else?
On Nov 13, 2010, at 10:25 PM, Antony Blakey <antony...@gmail.com> wrote:
>
> On 14/11/2010, at 8:22 AM, Chas Emerick wrote:
>
>> Of course, if the lein/cake leads/userbase don't see value in
>> the proposition at all (my current impression given irc
>> conversations), then I suppose all this is moot.
>
> I'm assuming that lack of interest. To be honest, I've given up on pmaven for clojure because:
>
> a) so much work is required across three IDEs to make a clojure serialization first-class e.g. completion of version numbers in specs etc; and
>
> b) the desire to have the *.clj file be an executable specification as opposed to a strictly declarative specification makes tool support even more difficult, and in particular makes it less likely that the existing models of coding assistance are applicable - it's no longer a serialization of the model, rather it is a program that builds a model either directly or as a side-effect of execution in some arbitrary runtime context; and
this is clearly a difference between how maven does things and how cake/lein do things. but under the hood, maven is still written in code and it still allows plugins to be loaded that introduce their own code that affects the build. so it seems to me that the strictly declarative specification is really just a way to make it much more difficult for users to customize their build when they need to. this is against the cake philosophy and the lisp philosophy too, isn't it?
> c) the community impulse to redevelop the wheel for build/dependency management without concern for building on the significant ecosystem that already exists is too depressing. I could speculate on why that has occurred, but IMHO it's a disaster for clojure adoption. Seamless integration into build/deployment/aggregation/versioning infrastructure is the *only* way Clojure can go viral. IMHO once again.
just because both cake and lein believe users should be able to express their builds using Clojure does not mean we are reimplementing the wheel. both cake and lein are built on top of ant for building and maven for dependency management. very soon, cake users will have the option of using ivy instead of maven for the latter.
the availability of clojars for distributing jars and the ease of fetching dependencies is one big reason why clojure has gone viral, IMO. i still don't understand why it isn't enough to "meet at the dependencies". every build tool produces a jar file with a pom.xml and releases that to clojars or some other repository. why does it matter which build tool was used to create it? my IDE uses maven is not a good answer IMO. i use the command line to run my builds :)
:justin
Antony Blakey
> hi Chas and Antony,
>
> here's my perspective. i don't see the value in replacing a short and simple macro with pmaven just to parse a defproject form that is easily handled by the Clojure reader. is there something i'm missing? are you suggesting something more than just using the maven code to parse defproject? in cake, the result of defproject is just a map. is there a good reason for it to be anything else?
In short, yes.
>
> On Nov 13, 2010, at 10:25 PM, Antony Blakey <antony...@gmail.com> wrote:
>
>>
>> On 14/11/2010, at 8:22 AM, Chas Emerick wrote:
>>
>>> Of course, if the lein/cake leads/userbase don't see value in
>>> the proposition at all (my current impression given irc
>>> conversations), then I suppose all this is moot.
>>
>> I'm assuming that lack of interest. To be honest, I've given up on pmaven for clojure because:
>>
>> a) so much work is required across three IDEs to make a clojure serialization first-class e.g. completion of version numbers in specs etc; and
>>
>> b) the desire to have the *.clj file be an executable specification as opposed to a strictly declarative specification makes tool support even more difficult, and in particular makes it less likely that the existing models of coding assistance are applicable - it's no longer a serialization of the model, rather it is a program that builds a model either directly or as a side-effect of execution in some arbitrary runtime context; and
>
> this is clearly a difference between how maven does things and how cake/lein do things. but under the hood, maven is still written in code and it still allows plugins to be loaded that introduce their own code that affects the build. so it seems to me that the strictly declarative specification is really just a way to make it much more difficult for users to customize their build when they need to. this is against the cake philosophy and the lisp philosophy too, isn't it?
It's probably against the lisp philosophy, but that philosophy is predicated on a 100% lisp universe. And invoking the lisp philosophy isn't, per-se, a valid justification for, or proof of, anything. Having a declarative format is the key to interoperability with other languages/tools/ecosystems that have different underlying models, degrees of dynamism etc, because it defines an abstract model that is independent of the individual languages. Introducing the runtime semantics of one particular language quickly makes interop effectively impossible.
>> c) the community impulse to redevelop the wheel for build/dependency management without concern for building on the significant ecosystem that already exists is too depressing. I could speculate on why that has occurred, but IMHO it's a disaster for clojure adoption. Seamless integration into build/deployment/aggregation/versioning infrastructure is the *only* way Clojure can go viral. IMHO once again.
>
> just because both cake and lein believe users should be able to express their builds using Clojure does not mean we are reimplementing the wheel.
I disagree, for the reason given above. IMO it's reinvention for the sake of it. I understand that, for some people, the Joy of Clojure is in it's usefulness in building tools, especially tools whose look-and-feel reflects the language itself. Unfortunately IMO that impulse is contrary to the requirements of cross-language/system interop.
I like the CL model of everything-is-lisp, but my experience with Smalltalk (VW), CL, Haskell amongst others, is that success according to the metric of wider-usage and penetration (which is for sure not the only measure of success) is more dependent on the ability of a niche language/product/whatever to be used/consumed/called by the more popular languages and ecosystems, rather the other way around. I feel this is a fundamental truth that is not recognised by evangelists/strategists for niche tools and systems.
> both cake and lein are built on top of ant for building and maven for dependency management. very soon, cake users will have the option of using ivy instead of maven for the latter.
The fact that they are built on top of ant and maven might allow Clojure to take advantage of some existing code and/or infrastructure, but it doesn't allow existing code and/or infrastructure to take advantage of Clojure, which is the key to adoption for a niche language.
> the availability of clojars for distributing jars and the ease of fetching dependencies is one big reason why clojure has gone viral, IMO.
I haven't used Clojars for 6 months. Last time I tried to integrate it with Nexus in a mixed-language project it was just-too-hard.
> i still don't understand why it isn't enough to "meet at the dependencies". every build tool produces a jar file with a pom.xml and releases that to clojars or some other repository. why does it matter which build tool was used to create it? my IDE uses maven is not a good answer IMO. i use the command line to run my builds :)
It may or may not be enough. Truly I am not trying to convince you, because I'm sure I can't - I was just responding to Chas's desire to see cake and lein be wrappers around the maven model and give some background to what I had done and the conclusions I reached. Personally I think that the Clojure community needs to focus more on bidi interop at all levels rather than building some sweet tools that build a wall around the language and result eventually in cll v2.
Clojure is a beautiful thing, for sure, and by comparison XML Maven carries all the messiness of the significant problems it attempts to address, but I think that a lein or cake that does everything that Maven does, for multiple languages, with lots of integration, would be just as messy. If pmaven is pushed, along with IDE support that correctly abstracts away the serialization format, then the XML format can disappear. In the meantime XML has the virtue of being a form that offends all language lovers equally.
Cheers,
Antony Blakey
-------------
CTO, Linkuistics Pty Ltd
Ph: 0438 840 787
The intuitive mind is a sacred gift and the rational mind is a faithful servant. We have created a society that honours the servant and has forgotten the gift.
-- Albert Einstein
Phil Hagelberg
On Sat, Nov 13, 2010 at 11:52 PM, Justin Balthrop <jus...@justinbalthrop.com> wrote:
> here's my perspective. i don't see the value in replacing a short and simple macro with pmaven just to parse a defproject form that is easily handled by the Clojure reader. is there something i'm missing? are you suggesting something more than just using the maven code to parse defproject? in cake, the result of defproject is just a map. is there a good reason for it to be anything else?
As you may have guessed, this is basically where I stand. If the benefits Antony cites of the declarative format are really that compelling for a certain class of users, then maven is there for them, while the rest of us can revel in our dynamism.
cheers,
Phil
Chas Emerick
topic, but I'll add just a few thoughts in closing:
On Nov 14, 6:45 am, Antony Blakey <antony.bla...@gmail.com> wrote:
> On 14/11/2010, at 6:22 PM, Justin Balthrop wrote:
>
>
> > this is clearly a difference between how maven does things and how cake/lein do things. but under the hood, maven is still written in code and it still allows plugins to be loaded that introduce their own code that affects the build. so it seems to me that the strictly declarative specification is really just a way to make it much more difficult for users to customize their build when they need to. this is against the cake philosophy and the lisp philosophy too, isn't it?
>
specification. It's very hard (nigh impossible) to accurately assess
the origins of others' opinions, but I can only suspect that knocking
declarative builds (as imperfect and flawed as the current
implementations of those are) comes from an insufficient regard of
history, namely make and ant in this case. I've used them, abused
them, and thankfully forgotten much about them – but their approach is
not made better simply because it's been implemented in Clojure.
The invocation of "lisp philosophy" here is particularly worrisome for
me – for all the reasons that Antony gives, but perhaps even more
significantly, for what that implies culturally. Rich Hickey has
notably talked of the dangers and pains of being "on an island" from a
language/host perspective, and I think it's just as applicable to
tools.
Every application, large or small, that I've written in the past ten
years has had to interop in deep ways with a variety of platforms,
frameworks, and tools: web app containers, RCP frameworks (e.g.
Eclipse's and NetBeans'), Hudson and Bamboo, installer generators
(e.g. NSIS), a variety of compilers (e.g. javacc, cross-compilation
via ikvm, etc), and so on. Being able to use these components
effectively (and have some confidence that I'll be able to use others
to come in the future) is far more important to me than the
particulars and eccentricities of my build tooling.
Put another way, to quote myself (from
http://cemerick.com/2010/09/22/wherein-i-feel-the-pain-of-being-a-generalist/):
"…I much prefer tools and practices that can ramp from fairly simple
cases…up to the heights of build automation, automated functional
testing, and continuous deployment. One should not have to switch
horses at various stages of the growth of a project just to
accommodate changing tooling requirements."
> >> c) the community impulse to redevelop the wheel for build/dependency management without concern for building on the significant ecosystem that already exists is too depressing. I could speculate on why that has occurred, but IMHO it's a disaster for clojure adoption. Seamless integration into build/deployment/aggregation/versioning infrastructure is the *only* way Clojure can go viral. IMHO once again.
>
> > just because both cake and lein believe users should be able to express their builds using Clojure does not mean we are reimplementing the wheel.
are build tools that provide much the same functionality as e.g. maven
or ant, but with a Clojure project definition and Clojure-centric
extension facilities. The fact that maven, with the minor addition of
clojure-maven-plugin, is strictly speaking more capable than either
lein or cake should make it plain that the latter are
reimplementations.
Anyway, this state re: reimplementation would be perfectly fine by me
if they either (a) aimed to deeply solve the build/package/deploy
problem rather than skinning some of Java's existing build tooling,
(b) provided Clojure-quality bidirectional interop, or (c) weren't
seen as "Clojure's build tools".
The ugliest part of this is that the design and implementation details
of both efforts encourage (require?) a default stance of
reimplementation by users. A quick look at https://github.com/technomancy/leiningen/wiki/Plugins
confirms this: third-party plugins for building war files, hadoop
jobs, testing against alternate sets of dependencies, gwt apps, and
one (that I wasn't aware of until just now) that generates daemon
scripts for a project (instead of using any of the existing, well-
tested java service wrappers like jsvc!).
This culture of reimplementing things in Clojure with no tangible gain
in capability in exchange does no one any good. Lein and cake users
can't use existing build systems' plugins, and others cannot use lein
and cake plugins. Even leaving aside tool interop due to the model
question: this is an island.
> > the availability of clojars for distributing jars and the ease of fetching dependencies is one big reason why clojure has gone viral, IMO.
>
IMO; an experiment that was worth running, but I think the upshot is
clear at this point. Forgetting for now the issues with it from a
technical/governance perspective (mixed snapshots/releases, ssh-only
deployments, no record of provenance, etc), it's used as a scratch
space for people's personal projects, and it is common (recommended,
even, in the main Leiningen README!) for people to deploy others'
libraries, often slightly-patched. Having an easily-accessible maven
repo has made things simpler for some, but let's not lose sight of
fundamentals.
> > i still don't understand why it isn't enough to "meet at the dependencies". every build tool produces a jar file with a pom.xml and releases that to clojars or some other repository. why does it matter which build tool was used to create it? my IDE uses maven is not a good answer IMO. i use the command line to run my builds :)
above. If all a build tool needed to do was build a jar and stick it
into a repo, I'd still be using ant + ivy.
On Nov 15, 10:12 pm, Phil Hagelberg <p...@hagelb.org> wrote:
>
> As you may have guessed, this is basically where I stand. If the benefits
> Antony cites of the declarative format are really that compelling for a
> certain class of users, then maven is there for them, while the rest of us
> can revel in our dynamism.
projects that really take advantage of this "dynamic" model and show
what it can do? I've never seen any that go beyond enumerating
dependencies, and maybe adding some plugin-specific configuration.
Of course, the model is but the tip of the iceberg in this discussion,
though many of the other issues block on it, i.e. re: the
reimplementation problem especially w.r.t. plugins, and the question
of bidirectional interop as Antony described.
For some time, I've assumed that such issues were simply
implementation details, things for the future, etc. This thread has
thankfully given me an opportunity to see just how deep the divide is
between our respective worldviews / requirements in this domain.
Cheers,
- Chas
Alex Miller
From: Chas Emerick <ceme...@snowtide.com>
To: Project.clj - SPEC <projectc...@googlegroups.com>
Sent: Fri, November 19, 2010 8:50:56 AM
Subject: Re: Minimizing the number of model implementations
Justin Balthrop
declarative build specifications. For 95% of projects, you never have to write a single line of code
for your builds. You just declare your project with defproject and bam! You have a bunch of available
tasks (aka targets or goals).
In the context of build tools, "declarative" is just a fancy way of saying, "the people who made
this build tool thought of everything you might want to do and created an option for it, so you
don't have to tell the build tool how to accomplish something, you just need to declare properties
of your project and the build tool will figure it out." There are also plugins, which let people who
didn't make the build tool extend its functionality so that you can use it by declaring some more
properties of your project.
The difference between Maven and Cake/Lein is not here. Unlike Make and Ant, all three tools provide
a declarative way to specify your project rather than describing the steps of your build over and
over for each project. And all three tools support plugins. The real difference between Maven and
Cake/Lein is what happens when you want to do something the creators of the build tool (and the
plugin authors) didn't think of. In maven, your only option is to write a plugin yourself in Java,
which means diving into the gory details of the "imperfect and flawed" maven implementation
(which is 43,875 lines of code).
In Cake and Lein, it is significantly easier. In Cake, you can write a plugin if you want, which is
just a simple namespace that contains deftask calls, or if what you want to do is simple or
project-specific, you can put deftask calls in tasks.clj (or project.clj if you prefer a single
file). In Lein, extending tasks is only a tiny bit more complicated (add one namespace per task and
use hooks to extend existing tasks). This is why many people prefer Lein and Cake over maven. Because
they want to customize their build in ways the authors of Maven didn't think of, and they know from
either good sense (or experience) that they shouldn't spend their time diving around in the Maven
source.
But all hope is not lost. I'm interested in the idea of writing a cake-maven plugin that makes Maven
plugins available in Cake. I'm also curious about making it possible for Cake tasks to work in Maven
by writing a cake-tasks plugin for Maven. But we don't need to use the POM internally in Cake or run
Cake on the Maven platform for this to happen. Maybe we could all coalesce on a plugin protocol?
There would be some challenges in mapping Cake's task dependency-based model to Maven's goals and
lifecycle phases, and it would take someone experienced with the internals of Maven to make this
happen. But if we can get past this Blub mentality, then I think it could be done. Any volunteers?
:justin
Phil Hagelberg
<jus...@justinbalthrop.com> wrote:
> But all hope is not lost. I'm interested in the idea of writing a cake-maven plugin that makes Maven
> plugins available in Cake.
I've had this on my back-burner for months but haven't gotten around
to it since nobody asked about it until now. Once I get the next
release out I can take a closer look at it. I can't think of any
reason it would be particularly difficult.
-Phil
Lance Bradley
Mark Derricutt
thread which I've been silently reading along with ( as well as
discussing on IRC ).
My first through here is "what crack have you been smoking?" followed
by "no sir, I don't touch that stuff".
I'm not sure where you get the idea of having to dive into 43,875
lines of code in order to write a plugin comes from - just run "mvn
archetype:generate" then select the mojo archetype option - BANG! In
a few moments you have a maven project with A SINGLE CLASS that
implements a plugin. All you have to do then is add properties,
sprinkle some implementation logic, install and bobs your uncle mary's
husband. This is precisely how I started the clojure-maven-plugin.
From my perspective, working on a single plugin inside a much larger
ecosystem is a nicer place to be than reimplementing a full build
tool. When I started at my current gig, I sadly discovered we had a
custom in house build tool written in groovy, which wrapped ant and
was tightly focused around OSGi and a REPL.
For the first 4 months or so of the project, I was probably spending
about 70%/30% working on our product, and fixing/adding functionality
to the build tool so that it actually did what we needed. Ultimately
the lack of a release process/infrastructure was what drove me to
stand up and say ENOUGH, and forced the move to maven. At the time I
had little love for maven, but I'd grown to loath both groovy, and the
tool aptly named "SOD".
Similar to Alex's project, we have a mixture of Antlr parsers
generating code, plugins processing OSGi manifest data, javascript
testing, database migrations and all manner of things. If we had to
implement them all ourselves we'd never get our product out.
Moving on to the clojure side, when I first wrote the clojure pmaven
code I was interested to see what could be done when you break out of
the declarative only model that maven gives you, I had a basic
reader/writer implemented ( sure, my clojure skills sucked then, and
largely still do now but it worked ).
The first two main problems I came up against ( which still largely
drive my opinion that pmaven is good in theory, but a failure in
reality ):
1) Internally a maven plugins configuration is just an XML blob,
representing inside the model as a raw DOM element. Why the hell
would they do this when everything else is a nice clean object model?
For the simple, and horrible reason that a plugins configuration -
whilst usually a simple key/value, or key/value-collection pair - can
actually be arbitrary XML - and I've come across plugins that use both
attributes, and namespaces! Both which cannot be easily, or cleanly
represented in YAML, JSON, or clojure.
2) The moment you bring any form of dynamic "build" step to the model,
you loose any form of two-way round tripping of the source POM. The
main place I saw this being a problem was the maven-release-plugin,
which as part of its operation will modify the version string of your
POM ( 1.0-SNAPSHOT -> 1.0 -> 1.1-SNAPSHOT ), I've not checked HOW it
does this, ( string search and replace or DOM rewrites) but when you
move into a source disconnected state, operating solely on a model and
not the source representation ) then you could easily fall into the
state of loosing the round-trip ability.
To be honest - I never stuck to pmaven long enough to actually see if
this would be a problem or not. I decided to spend more effort on
concentrating on the maven plugin, moving on with the project I needed
it for, and getting out into production.
My understanding of the project itself was a) a experiment into seeing
what could be done with different languages on maven b) to aid and
drive the internal refactoring of maven to separate the model from the
source ( required for projects such as maven tycho ). From various
discussions with people, the main gripe about mavens XML use is its
verbosity due to only using elements, rather than a mix of
elements/attributes, which is a valid, but minor gripe ( there was one
pmaven branch/patch that let you use elements/attributes
interchangeable, which was nice but also didn't really go anywhere
beyond the experiment stage.
The whole talk of plugins is also where I see a unified/standard build
format falling down as well, unless you have 100% plugin parity
between cake/lein/pmaven-clj - I don't see it mattering one bit
whether your syntax matches.
I missed the whole beginnings of these conversations so I've not
actually seen a usecase for WHY we want the common format - is it so
that one can jump from cake, to lein, to pmaven, then back to cake on
a whim? So that I can write something using pmaven, and expect Phil
to build it using lein without error?
</rant>
Mark
--
"Great artists are extremely selfish and arrogant things" — Steven
Wilson, Porcupine Tree
Justin Balthrop
As Chas says, there was a bit too much hyperbole in that objection. But the point still stands that most people aren't going to go
to the lengths you and write a plugin just to customize their build.
The original point of a project.clj spec was so that someone could build a simple project using either Cake or Lein or pmaven. This
is the case even today for Cake and Lein without the spec, but the hope was to expand the number of projects this would work for.
The other, more difficult, and more interesting goal was the ability to use Maven plugins in Lein/Cake and possibly vice versa.
:justin
Phil Hagelberg
> I'm amazed that we're debating the value of a declarative
> specification. It's very hard (nigh impossible) to accurately assess
> the origins of others' opinions, but I can only suspect that knocking
> declarative builds (as imperfect and flawed as the current
> implementations of those are) comes from an insufficient regard of
> history, namely make and ant in this case. I've used them, abused
> them, and thankfully forgotten much about them – but their approach is
> not made better simply because it's been implemented in Clojure.
Hmmm--I suppose I should have been clearer. I'm in favour of
declarative builds, (obviously, all Clojure build tools are
declarative at heart), it's the 100% no-compromises declarative that
I'm not fond of. I'm using the term "dynamism" to refer to what Antony
seems to be railing against--the notion that you can have an escape
hatch and put executable code in your build configuration.
-Phil
Antony Blakey
I'm only 'railing' against dynamism in respect of interoperability, for what I hope are obvious reasons. For interop, you don't want to customise your build process. Similarly, in a larger shop, doing some custom or funky build step in an ad-hoc manner is a major no-no.
I get the impression that this argument comes down to those that want *bidi* interop and seamless build integration, either because they need it themselves, or believe it to be critical for viral uptake in larger shops, versus those looking at a solely clojure environment, that imports components from other languages, but otherwise has clojure as the driver.
All of this competition would be a good thing if not for the fact that Maven has already 'won' the argument in the wider community, and can build clojure, scala, java, groovy et al, in a standard way. So this is a battle for hearts and minds of the clojure community, occurring while it is relatively young. Actually, it's a replay of the aforementioned already-determined battle. (The aggressive terminology is standard, no personal aggression intended). The end result will be either a ghetto-ized community whose tooling and mindset prevents it from being taken up more widely, or a community that has compromised and uses the same tooling as everyone else, to make the kool-aid easier to drink.
Note that Ruby, being a non-JVM language (excl JRuby), doesn't have this issue, and is therefore succeeding along with it's own reflective toolchain. IMO Clojure doesn't have that luxury.
My own lack of motivation on pmaven and clojure is because I think that lein has already won this battle. I suspect that there is an essential quality of Lisp-like languages that encourages such an isolation-inducing world-view, in the same way that Smalltalk does. Probably it's the thrill of having such a sharp knife that makes you want to recarve everything from scratch, in non-standard sizes and idiosyncratic shapes that don't fit with the rest of the world.
BTW: I've delivered commercial software in Smalltalk and Clojure (and Scheme, amongst many other languages), but it's very hard to do because they are more consume-the-world than be-consumed-by-the-world languages. I hope that makes sense, and explains why bidi-interop in a commercial context is uppermost in my mind. I may have the wrong end of the stick on this.
Having said all that, I think this thread is now non-productive.
Antony Blakey
-------------
CTO, Linkuistics Pty Ltd
Ph: 0438 840 787
Nothing is really work unless you would rather be doing something else.
-- J. M. Barre
Laurent Petit
exposing world views.
Now, please, let's go to the metal.
(serious-content
"Me, as a clojure developer (not even considering whether I'm on an
island or not), would like not to have to learn the subtle differences
between lein, cake, pmaven, cljr ways of declaring the project
properties. So YES, this only single argument seems sufficient to me
as a driving force to have a reference." )
(rant-content
"And really, let's do some empathy and be in the body of e.g. a java
developer examining the clojure ecosystem:
\"Wee, they are a small community, but they have the luxury to waste
resources by having already 3 different working build tools (not that
diversity and Darwin's evolution theory aren't good, but still, it's a
small community with limited resources). And they also have the luxury
to each lock their users by having project definition varying in
subtle ways. Something like what Java EE vendors do with java EE
specs, but at a very tiny level.\". Let them extrapolate from this
what the ecosystem of clojure libraries could look like when things
are not as simple as just a map definition of one or two dozens of key/
value definitions ..., let them merge this thought with the new
knowledge they've learned: ")
(serious-content
"Me, as an IDE developer, would also like the above mentioned issue
solved, for evident reasons of not having to suffer the NIH syndrom 3
times, but rather just one :-P.")