CruiseControl vs Continuum vs Luntbuild vs Hudson - John Ferguson Smart
Muthu Ramadoss
environment?
A comparative review of four open source continuous integration tools
By John Ferguson Smart, JavaWorld.com
The term "continuous integration," commonly attributed to Martin
Fowler, who first wrote about the software development practice back in
2000 (see Resources for a link to the article), is one of the pillars
of modern programming techniques. CI involves automatically building
and testing an application at frequent intervals on a dedicated
integration server. Developers regularly commit small updates and are
notified rapidly if their changes cause the build to fail. This
practice is first and foremost a risk-reduction strategy, ensuring that
the application integrates correctly at all times and reducing the risk
of integration issues appearing late in the project. It has the
additional benefits of improving communication between team members and
reducing the time needed to prepare a release.
The most fundamental feature of any CI tool is the automatic build
cycle. The CI tool, either at a scheduled point in time or whenever any
changes are detected in the source code repository, fetches the source
code's latest version and attempts to build and test the project. If
the build fails, developers can be notified in any number of ways:
email, instant messaging, or through some other notification technique.
Many CI tools, both open source and commercial, are on the market, and
deciphering which one would best suit your particular environment can
prove difficult. Some CI tools also boast additional SDLC (software
development lifecycle) features, such as release management and build
artifact management, while others concentrate on core CI
functionalities. Some try to provide a wide range of notification
methods and support virtually every version control system under the
sun, whereas others concentrate on delivering a small, light,
easy-to-understand product. In this article, we will look at four of
the better-known and more interesting continuous integration tools in
the open source world and see how they measure up:
* Cruise Control
* Continuum
* Luntbuild
* Hudson
This list is obviously not exhaustive. There are many other CI tools
out there, both open source and commercial. Of course, each tool has
its particularities, and there is no one-size-fits-all solution. The
aim of this article is to provide a brief survey of some of the more
interesting tools available and to help readers decide which tool suits
their particular environment best.
CruiseControl: The original Java CI tool
CruiseControl is a mature and robust Java continuous integration tool
that enjoys a strong user base and a solid industry reputation. It
supports virtually any type of project, be it Ant, Maven, Maven 2,
make, or just a plain-old command line script, as well as a wide range
of SCM (source configuration management) tools. Its notification
techniques are second to none. And it also benefits from a large number
of third-party plug-ins.
Setting up a project using CruiseControl involves a bit more work than
with the other tools we look at in this article. CruiseControl is a
highly configurable tool, and all CruiseControl configuration is done
in a central XML configuration file. As of recent versions, there is a
graphical interface written in Swing that helps you manipulate this
file, but it is still a good idea to be able to configure this file
directly. Thus, configuring a CruiseControl project can be a somewhat
painful experience, with a fairly steep learning curve.
In addition to configuring your project in the XML configuration file,
you also need to set up a copy of your project manually in the
CruiseControl projects directory. As with all CI tools, SCM integration
plays a crucial role in the CruiseControl architecture. CruiseControl
supports a wide range of SCM products, including CVS, Subversion,
BitKeeper, IBM Rational ClearCase, MKS, Perforce, Serena PVCS, Borland
StarTeam, Visual SourceSafe, and more. However, CruiseControl simply
polls these SCM tools to detect code changes: getting the code out of
the code repository is largely your responsibility, and CruiseControl
expects your project to be able to do this by itself. For this reason,
developers using Ant often write a "wrapper" Ant build script
(typically called "cc-build.xml"), which updates the source code from
the repository and then calls the main build script to do the real
work. Maven users can use the SCM plug-in to avoid this issue.
Once your project is correctly configured, you can monitor
CruiseControl builds via the CruiseControl Website, which is actually a
separate application built on top of the CruiseControl core. The latest
version comes bundled with a Jetty server; this site is fairly easy to
deploy.
Figure 1. The CruiseControl Website. Click on thumbnail to view
full-sized image.
The Website is simple and to the point. The latest build status of each
project displays on the homepage. You can also view detailed build
results, unit test results, and some simple build metrics, such as the
percentage of successful builds over time.
One of CruiseControl's strengths is its sophisticated and reactive
build cycle. CruiseControl continuously monitors the SCM servers for
any updates and triggers a build whenever it detects any new changes to
the source code repository. This makes for a more reactive agile
development environment, as developers don't need to wait for scheduled
builds to learn about integration issues. Builds can also be scheduled
to be run at certain times or intervals.
Running your builds regularly is good, but it's even better to let
everyone know how they're going. Notification is another of
CruiseControl's strong points. Out of the box, CruiseControl supports
email, Jabber IM, RSS feeds, and even blogs. You can also perform
various tasks such as implementing FTP or SCP file transfers, invoking
an Ant target, running command line tasks, or even using an X10
interface to hook up a lava lamp, or any other X10-compatible device
you can put your hands on. X10 is an industry standard for controlling
electronic devices. CruiseControl has an X10 interface that lets you
publish build status information through a variety of electronic
devices. For example, a lava lamp may turn from green to red and bubble
frenetically when a build fails.
One common issue in larger projects is that of inter-project
dependencies. Basically, when you build a particular project, you may
also need to build other libraries to avoid integration issues. This is
not one of CruiseControl's strengths. In the current version,
CruiseControl offers only limited support for inter-project
dependencies (via the <buildstatus> plug-in). CruiseControl also offers
little in the way of build management in the broader sense: it is
difficult to define inter-project dependencies, for example.
Overall, CruiseControl is a powerful and flexible continuous
integration tool, albeit with a non-trivial learning curve. It is a
pure CI tool, with little in the way of inter-project dependency or
build artifact management. However, its flexibility and sophisticated
notification techniques can make it an excellent choice for experienced
continuous integration practitioners.
Continuum: The official Maven 2 CI tool
Continuum is a simple, intuitive, and lightweight CI tool developed by
the team who brought you Maven. It is easy to install and involves
little initial configuration. Continuum comes with its own Jetty Web
server and uses an Apache Derby embedded database, which makes the tool
pretty self-reliant.
Continuum handles Maven 1 and 2, Ant, and even plain-old shell scripts.
In fact, one of Continuum's selling points is its strong integration
with Maven 2. In Maven 2, you can define SCM configuration details and
list team members all within your project definition file (or POM,
project object model). If your Maven 2 project definition file contains
enough information, all you need to do is provide the POM file to add
your project to the Continuum server. If this information isn't
available in the POM file, you can always define it through the
Continuum Web interface.
Continuum also handles Maven 1 and Ant projects well: you just need to
supply the SCM repository details and the build script, and Continuum
will do the rest. This proves more convenient than CruiseControl, where
you need to write a special script to check out the latest source code
from the repository.
Continuum uses the Maven SCM library, which supports a reasonable and
growing number of SCM systems: Subversion, CVS, Borland StarTeam,
Perforce, and Bazaar, as well as partial support for IBM Rational
ClearCase and Visual Source Safe.
Continuum provides a Web-based administration tool that is simple and
fairly intuitive (see Figure 2). From here, you can add new projects,
monitor the status of your project builds, and examine the detailed
build results when a build fails. You can also browse the working copy
of the source code that Continuum is using for its builds. This is
particularly useful when you are trying to figure out why the build
that works on your machine fails on the Continuum server.
Figure 2. The Continuum Web administration console. Click on thumbnail
to view full-sized image.
Continuum builds are scheduled using cron-type expressions. Cron
expressions, well-known in Unix administration circles, provide a
succinct, if cryptic, way of defining a schedule for a repeated task.
For example, "0 0 * * * *" means "on the hour, every hour," and "0 0
6-18 ? * MON-FRI" means "on the hour, every hour from 6 a.m. to 6 p.m.,
Monday through Friday."
In the current version, however, Continuum does not actively query SCM
servers to detect updated source code, which results in a slightly less
reactive CI tool than one such as CruiseControl, which can do this.
Continuum provides the usual notification techniques: email and instant
messaging (Internet Relay Chat, Jabber, MSN). Email notifications
provide ample details of the changes made and the build results.
Currently no built-in support for RSS notifications is available,
though you can write your own fairly easily using the API. RSS
notification is useful for project managers or quality assurance people
who just need to monitor the build status and history, and don't need
to react quickly if a build fails.
Continuum is still a relatively young product, with a lot of room for
new and improved features. A potential issue for big projects or
programs is that, like CruiseControl, Continuum provides no simple way
of handling dependencies between projects.
Nevertheless, Continuum is easy to install and easy to use, and will
suffice in many situations. Its simplicity, and its intuitive Web
administration console, make it a good candidate for introducing
continuous integration into organizations that have had no previous
experience with the subject.
Luntbuild: A graphical build management tool
Luntbuild is another open source CI tool written in Java. It is
particularly easy to install and configure, and, like Continuum, all
server administration tasks are done via a Web-based administration
console. In fact, Luntbuild is designed to do more than just manage the
continuous integration process: it also lets you store and manage
generated artifacts, label and promote versions, and manage
dependencies between builds. It supports a wide range of version
control tools, and notifications can be diffused via email, IM, and
even on a blog site. Indeed, it is one of the most feature-rich of the
open source CI tools.
Luntbuild is produced and maintained by a company called PMEase.
Luntbuild also has a commercial cousin, called QuickBuild, which is
marketed by the same company.
Luntbuild comes with a real installer, which walks you through the
installation process and proposes a number of configuration options. At
the simplest level, you can use a fully self-contained embedded
database and Jetty Web server. Or you can configure Luntbuild to work
with an external database, and/or on an external Web server. You can
even set up your user accounts from an external LDAP (lightweight
directory access protocol) directory.
It is a powerful and flexible tool, and as such, the administration
console is relatively complex (see Figure 3). Precisely because of
numerous options that Luntbuild proposes, creating a new project is a
fairly involved task. For a typical project, you need to set up the
version control configuration, define user access rights and
notification methods, create one or several build scripts that can be
executed for this project, and set up schedules to define when these
scripts are to be executed.
Figure 3. Creating a project in Luntbuild. Click on thumbnail to view
full-sized image.
The notification methods are defined at a system level and can be
shared across all projects. Luntbuild supports a limited range of
notification methods, including email, MSN, Jabber, and some blogging
software. On the other hand, it supports a wide range of version
control products, including CVS, Subversion, Borland StarTeam, AccuRev,
IBM Rational ClearCase, and even Visual Source Safe. Since each CVS
product has its own particularities, each CVS adaptor has its own
specific configuration screen. The Subversion screen, for example,
recognizes the Subversion convention of storing the main development
trunk, separate branches, and revision tags in separate specific
directories.
One of the distinctive features of Luntbuild is the way it
distinguishes between configuring the builds and scheduling them.
Command line, Ant, Maven, and even Rake build scripts are supported.
Setting up the build tasks is intuitive and flexible, and you can set
up as many as you like for each project.
Once you've set up your build scripts, you define schedulers to
indicate when to run the scripts. By separating the schedulers from the
build scripts, you can easily write modular build scripts that can be
used and reused in different schedules. Schedules can only be, well,
scheduled. Luntbuild cannot actively poll the source code repository
for changes as both CruiseControl and Hudson (see below) can.
Figure 4. Monitoring project builds in Luntbuild. Click on thumbnail to
view full-sized image.
Luntbuild does provide good support for project dependency management
and build artifact management, and you can create different types of
dependencies between projects. For example, you can require that all
the dependent libraries of a project be built successfully before the
project itself can be built.
Other than an arguably uncool name, there is little to find fault with
Luntbuild. It is a solid, feature-rich tool with a clean user
interface, support for a wide range of SCM tools, and a good variety of
notification techniques. However, the scheduling SCM architecture used
by Luntbuild is, as I have already pointed out, less reactive than the
polling approach. Some of its complexity may nevertheless be overkill
for users with little experience in continuous integration.
Nevertheless, it is definitely a tool worth evaluating.
Hudson: The new kid in town
Hudson is a newcomer to the CI field and is a focused CI tool. In many
regards, it has considerably fewer features then the other tools we've
been looking at, concentrates more on Subversion and CVS-based
projects, and provides only a limited number of notification
techniques. But the features it does have are extremely well
thought-out, with some being quite innovative.
Hudson is easy to install, though you do need to deploy it onto your
own Java Web server (it is delivered as a deployable war file). Both
system configuration details and projects can be managed from a slick
Web console (see Figure 5). System-wide configuration is fairly simple:
you just need to configure the paths to your system JDKs, and to Maven
and Ant, as well as some configuration details for email notification.
Figure 5. The Hudson Web administration console. Click on thumbnail to
view full-sized image.
Its SCM support is limited to CVS and Subversion, which are,
admittedly, currently the two most popular version control tools on the
market. Its notification capabilities are also limited to the strict
minimum: only email and RSS notifications are supported out of the box.
It would be nice to see a few more notification techniques, such as
instant messaging support.
Like CruiseControl, in addition to cron-based scheduling, you can ask
Hudson to poll the SCM server for updates (you can even configure the
frequency of its polling). This is a powerful feature. This, combined
with email notification and the ability to display unit test results,
provides for a tight development/integration cycle.
Hudson boasts a particularly clear presentation of build results (see
Figure 6). In addition to the build status and a summary of the changes
made in the source code repository since the last build, the build
results page can also display downloadable artifacts generated by the
build. Hudson also provides excellent built-in support for JUnit test
reports: test results, as well as a graph of historical test data can
be viewed directly from within the build results page.
Figure 6. The Hudson build results page. Click on thumbnail to view
full-sized image.
There is also a nifty "fingerprint" feature. Hudson records a unique
"fingerprint" (actually, just the MD5 checksum) for each artifact
generated during a build. So if you find yourself with an artifact, and
you aren't sure which build generated it, Hudson can work it out for
you. This functionality can be useful when tracking down
incompatibilities between library versions.
Hudson provides some interesting support for inter-project
dependencies: you can specify that a given project wait until all other
projects are built, and you can also specify that other projects be
rebuilt after a project is successfully built.
The main issue developers will find with Hudson, other than its
relatively modest feature set, is probably its lack of flexibility. For
example, Hudson assumes you have your Ant build.xml file in your
project's root directory. Although this is a widely followed
convention, some Ant projects have their build scripts in a
subdirectory, which gives Hudson a bit of trouble. Maven projects are
handled more consistently, probably because of the strong emphasis on
following standards and conventions found in Maven projects.
Despite its modest feature set (which will probably improve in future
versions), Hudson offers some powerful and efficient core CI
functionalities, as well as some interesting innovations. It is
definitely a tool to keep on your CI radar!
Conclusion
Many CI tools are available out there, and there is no
one-size-fits-all solution. Different tools will suit different
environments and different people. At the end of the day, to choose the
right tool for you, you need to set up the tools and try them for
yourself. Hopefully I have given you some tips to help you investigate
further.
Author Bio
John Ferguson Smart has been involved in the IT industry since 1991,
and in Java EE development since 1999. His specialties are Java EE
architecture and development and IT project management, including
offshore project management. He has experience in open source Java
technologies; has worked on many large-scale Java EE projects for
government and business in both hemispheres, involving international
and offshore teams; and also writes technical articles in the Java EE
field. His technical blog can be found at
http://www.jroller.com/page/wakaleo.
link:
http://www.javaworld.com/cgi-bin/mailto/x_java.cgi
This story appeared on JavaWorld at
http://www.javaworld.com/javaworld/jw-11-2006/jw-1101-ci.html