Java builds (maven, ant), the java way is broken!

[phil.s...@gmail.com]

Dick and Carl have talked a lot about builds lately, prompting this
post...

I spent about a year doing nothing but Ruby and then ended up going
back to my old Java job. I learned a lot in the Ruby world, and one
of the big take-aways I had was that the way build systems work in the
Java world is completely broken.

In ruby there is this wonderful tool called rake (http://
rake.rubyforge.org). Rake is more than a build system, it's really a
system for anything you want automated.

You can leverage your code inside this system. So if you wrote a
class that cleaned a database as part of your main code base, you can
hook rake into it and run it something like: rake clean_db

Rake is just ruby code, you have the full power of Ruby within a
syntax a bit similar to Ant:

task :default => [:test]

task :test => [:clean_db] do
ruby "test/unittest.rb"
end

task :clean_db do
DBUtil.cleanDB
end

in this case the test target has a dependency on clean_db. And
clean_db is simply a ruby class that cleans all data from the
database.

In the ruby world, there are many Rake plugins that allow you to do
all sorts of automation. There are plugins that deploy/manage to
Amazon EC2. One called Capistrano that you run on your prod system
checks your source code out from source control, runs migration
scripts, backs up the old version, and starts up your new updated
system, all with one command. One of the most brilliant ideas in
Rails, database migrations is driven by Rake. So your deployment
might be as simple as "rake deploy". Your database migration is "rake
db:migrate", which will check your current version of the database and
apply and schema and/or database migrations needed to get up-to-date.

Since you can write Ruby code in your build file you can easily write
code that has conditionals, loops, string manipulations, etc. In
other words, you can easily put intelligence into your build! Doing
these simple tasks in Ant or Maven is nightmarish. Coding them sucks
and debugging them is even worse. XML is a terrible way to define
your build systems, it works ok for trivial cases, but as any
experienced developers know, trivial cases soon evolve - or devolve -
into complex cases. Don't believe me? Here's what the creator of Ant
(James Duncan Davidson)said back in 2003:
http://weblogs.java.net/blog/duncan/archive/open_source/index.html .
Or in 2004, he's more explicit:
http://web.archive.org/web/20041217023752/http://x180.net/Journal/2004/03/31.html
"If I knew then what I knew now, I would have tried using a real
scripting language, such as JavaScript via the Rhino component or
Python via JPython, with bindings to Java objects which implemented
the functionality expressed in todays tasks. Then, there would be a
first class way to express logic and we wouldn't be stuck with XML as
a format that is too bulky for the way that people really want to use
the tool. "

But 4 and half years later and 99% of the java world is still writing
their builds in XML.

Solution? Well there is Gant: http://gant.codehaus.org/. Gant is
simply a groovy hook into Ant. Write your builds in Groovy script and
leverage ant tasks. Where I work, we are going down this path. After
doing this for a while, I can't recommend it. It's better than xml,
but has it's own set of problems - mainly due to the nature of
Groovy. Since Groovy isn't truly interpreted, it's very hard to
leverage your existing code base. For example, if I want to call my
Java class "DBUtil.cleanDB()", this class must be compiled for the
groovy script that calls it to run. So you have to jump through
multiple stage builds, just as you would with Ant. Another issue I
have with Gant/Groovy are the exception stack straces are terrible.
There are dozens and dozens of useless lines in every stack dump,
which stem from how groovy is implemented. Here's an example in an
issue I submitted: http://jira.codehaus.org/browse/GROOVY-2944. I
have other issues, but these are the two big ones.

Better solution? Well you can always use Ruby to do Java builds using
a Ruby Gem called antwrap. Here's an example of a simple build I did
in JRuby: http://pastie.org/323131
It's interpreted so you can leverage your whole code base and write
clever code to do whatever you can imagine. Ruby has a much easier,
more powerful File API than Groovy or Java, which is huge for builds.

Another Ruby option, this is more maven-ish with a bunch of dependency
management features - Buildr http://incubator.apache.org/buildr/ .
This project scares me at first glance. I looked at it and got
confused pretty quickly. I actually discovered the AntWrap gem trying
to figure out Buildr and decided AntWrap was good enough for my
purposes.

The power of a scripting language is huge. I wish Sun would endorse
some sort of scripting(interpreted) language as the official java
scripting language. I say scripting instead of "dynamic" in this case
because I think being interpreted is very important, much more so than
duck typing or dynamic features. But I don't really see any scripting
language really making the big move on the JVM because Sun is
remaining agnostic. They still do all their builds with Ant (yuck).
They don't leverage a scripting language for any automation (like
maybe administrating glassfish?).

Perhaps JavaFx script could be leveraged and made into the language
I'm thinking about? The start is there... the syntax is quite a bit
nicer than Java. It's not interpreted, but it could be.

any thoughts on this? Am I the only one who sees this need (it seems
like it)...

[Marcelo Fukushima]

where i work, we use maven exclusively and even tough editing xml
files drives me crazy, we seldom look/edit them so im okay with that

--
[]'s
Marcelo Takeshi Fukushima

[Casper Bang]

Why the requirement it be interpreted? That seems to be just an
implementation detail... could you clarify what that has to do with
the DSL friendliness you appear to be calling for (which I agree with,
but also know a lot of the Java community would veto with a
"readability" argument)?

/Casper
.

On Nov 25, 3:37 am, "phil.swen...@gmail.com" <phil.swen...@gmail.com>
wrote:

> Or in 2004, he's more explicit:http://web.archive.org/web/20041217023752/http://x180.net/Journal/200...

[phil.s...@gmail.com]

On Nov 24, 9:00 pm, Casper Bang <casper.b...@gmail.com> wrote:
> Why the requirement it be interpreted?

it makes it difficult to leverage the proejct code. If a build is
compiled and it references existing code then the build won't compile
until the existing code is compiled. I ran into this exact scenario
with Gant. What I have to do is have two scripts, one that just
compiles the code with no references to project code (just a compile
task) and another build script that references the project code.

It's awkward and confusing....

[Bill Robertson]

How is rake (dependency checking + ruby) better than make (dependency
checking + shell commands)? Or is rake more than that? I guess it
might be more portable.

Did anybody watch the 50 in 50 talk? Were they comparing rake to JCL?



On Nov 25, 12:39 am, "phil.swen...@gmail.com" <phil.swen...@gmail.com>
wrote:

[Mark Derricutt]

Given this thread it'd be great if the posse could get Kelly O'Hair to
talk about JMake:

http://blogs.sun.com/kto/entry/jmake_source_repository_formerly_called

"This was formerly called the javamake utility from
http://www.experimentalstuff.com/Technologies/JavaMake/index.html.
This tool's functionality is analogous to the "smart checking" feature
of Borland JBuilder or the dependency analysis feature of IBM Jikes.
However, it is a command line tool that is not tied to any IDE, can be
used with any Java compiler, and can run as a task in the popular Ant
make system."

It'd be interesting to hear about, esp. with the ties to JBuilder.

--
"It is easier to optimize correct code than to correct optimized
code." -- Bill Harlan

[Peter Andersen]

Phil I see a problem of mixing your build code with the production
codebase,
if you separate them the clean_db build code can be compiled (or fully
build - as it own project) before
it is needed in the test part of the production build.

I know that you mostly compare rake and ant - but you also bring maven
into the picture.
Although ant and maven both use xml to specify the build, maven work
on a much higher abstractions level.
I just started using maven after years using ant - hopefully i do not
need to go back.

I agree with Dick - If it was more flexible it was easier to work
with.
If it was possible the disable lifecycles directly from the pom - a
disabling of the install phase would had helped Dick with his big war
files.
The deploy plugin has such a skip property - the install plugin has
not - direct liftcycle control will make it possible to override
plugin behavior.

It is ok that the build is defined in xml - as long as it is done at
the right abstraction level.
Scripting is ok - but it must not bring us back to defining all of the
"what to do" instead of "what to get" with standard layout,
conventions, and configuration.

/peter

On Nov 26, 4:08 am, "Mark Derricutt" <m...@talios.com> wrote:
> Given this thread it'd be great if the posse could get Kelly O'Hair to
> talk about JMake:
>
> http://blogs.sun.com/kto/entry/jmake_source_repository_formerly_called
>

> "This was formerly called the javamake utility fromhttp://www.experimentalstuff.com/Technologies/JavaMake/index.html.

[RogerV]

We've settled on Maven for managing new projects. However, we have an
extensive set of existing Ant-based projects.

The thing about Maven is that it's best for new green field projects
where you can shape your approach to the way Maven likes to do things.
That's the best way to have success with Maven.

However, for existing Ant projects, it can sometimes be a formidible
undertaking to adapt them to Maven's approach - and a time sink. So
instead, I'm taking a different tact. We'll use the Maven Ant task to
get Maven to manage our .jar library dependencies on behalf of our Ant
build.xml project files. We'll therefore leave the ant build files
pretty much as is - just modify them to use Maven pom.xml to get
classpath definition for .jars and some basic directory definitions
and such.

Here are two links for a small experimental project involving ANTLR
parser where I've melded Maven and Ant together in that manner:

http://vossnet.org/build.xml
http://vossnet.org/pom.xml

We're also using Artifactory repository at our internal network as
well. The declarations for that are dealt with in our $HOME/.m2/
settings.xml file (located in each developer's workstation - and build
server - local Maven repository folder).

We tried Ivy but gave it up. This approach is so far working out much
better.

The nice thing about have a pom.xml for all our projects (both new and
legacy) is that I can easily do this:

mvn eclipse:eclipse

to generate the .project and .classpath files for the Eclipse IDE.

We don't check these files into source control as we're a strictly IDE-
agnostic shop. Every project must build from the command line (or
Hudson plugin) via Ant or Maven (nothing pertaining to any IDE should
ever be checked in per association with any project). So it's nice
that it's very easy to get the IDE project files quickly and easily
generated on the fly when they're needed.

Course for pure Maven projects, one can then use the Maven plugins for
either Eclipse or NetBeans, which includes browsing and searching
repositories to associate dependencies when they're needed for the
first time.

On Nov 26, 3:29 am, Peter Andersen <pe...@boelskifte-andersen.dk>
wrote:

[RogerV]

The animus expressed toward .XML style syntax is something that tends
to resonate with me. I do tend to like declarative approaches - which
both Ant and Maven more or less take - vs overly imperative approaches
to describing builds. Hence I tend to resist the temptation to take a
full blown scripting language approach to describing project builds.

As to another area where XML inflicts cognitive pain - Spring
Framework applicationContext.xml files.

I do believe in separating bean initialization away from compiled Java
code into a strictly interpreted-at-runtime text file of some kind. I
like a few annotations for some things but I don't want to use them to
fully replace the semantic actions that go on in
applicationContext.xml files.

I want a better bean configuration/initialization script language -
but not really an imperative language. I don't want to script logic
there, I just want to declare the initialization actions and the bean
relationships.

So hence I've grabbed ANTLR and am devising a new configuration DSL to
supplant XML. I'm calling it jfig. The syntax of jfig looks like this:

applicationContext.jfig
===============================================

properties_include "classpath:application.properties";

org.apache.commons.dbcp.BasicDataSource dataSource {
@destroy-method = close;
driverClassName = "${jdbc.driverClassName}";
url = "${jdbc.url}";
username = "${jdbc.username}";
password = "${jdbc.password}";
defaultAutoCommit = true;
}

org.springframework.orm.ibatis.SqlMapClientFactoryBean sqlMapClient {
configLocation = "classpath:sqlmap-config.xml";
dataSource = $dataSource;
}

java.util.Properties props {
"James Ward" = "Adobe Flex evangelist";
"Stu Stern" = "Gorilla Logic - Flex Monkey test automation";
Dilbert = "character in popular comic strip of same title";
}

java.util.HashMap map {
this($props);
}

===============================================

The '@' prefixes Spring BeanDefintion attributes that can be set.

The '$' prefixes bean ID names when they're being referenced as a
dependency.

Obviously includes for Java .properties definitions are supported and
use the ${foo.bar} syntax for referencing property definitions in any
bean configuration.

Notice the 'this' keyword is used when doing constructor injection (as
opposed to setter injection). Constructor injection can still be
combined with setter injection too.

The java.util.Properties class is specially recognized so that it's
easy to initialize an instance with name/value pairs, and then that
can be used to initialize other beans that accept a Properties
argument.

A certain amount of type checking will be done during jfig config file
parsing. If a bean class doesn't exist on the classpath, or a property
is not found, or is not of a compatible type (same with constructor
args), then will fail on the spot, referencing the relevant lines in
the jfig file.

Very early days. I've got a working parser that processes this syntax
and instantiates these beans. I'm next going to rewrite the parse
actions to start invoking Spring Framework APIs for bean definition
and bean registration.

Later on I'll use ANTLR to creat an ActionScript runtime in order to
devise a mini dependency injection framework for Flex apps - using
this same jfig syntax. It'll hold the AST in memory as the so-called
"application context". To instantiate a requested bean will involve
traversing the AST to instantiate dependencies in a just-in-time
manner.

[Brent]

I highly recommend using Nexus maven repo manager. Pretty much
everyone else is moving toward it for maven as well.

http://nexus.sonatype.org/

Just take a look at the demo site. It's real easy to setup as well.
I used to use artifactory, but switched to this one.


--Brent
http://geekyryan.blogspot.com/

[Casper Bang]

Hi Brent,

Have you tried Archiva? And if so, what are the advantages of Nexus
over Archiva?

/Casper

On Nov 26, 9:03 pm, Brent <brent.r...@gmail.com> wrote:
> I highly recommend using Nexus maven repo manager.  Pretty much
> everyone else is moving toward it for maven as well.
>
> http://nexus.sonatype.org/
>
> Just take a look at the demo site.  It's real easy to setup as well.
> I used to use artifactory, but switched to this one.
>

> --Brenthttp://geekyryan.blogspot.com/

[Brent Ryan]

I tried Archiva a while back but it was too buggy and hard to install/configure at the time.  I'm pretty sure that the UI and features of nexus are far and above Archiva.  I believe that the Maven central repository uses nexus now as well.

--Brent

[sherod]

We use Nexus also. Works well.

[Marcelo Fukushima]

weve been using archiva for quite some time now, without any problems