Re: [rundeck] Make, Gradle, Maven, and Grails

[Greg Schueler]

Hi John, thanks for the offer.

We actually support Maven as well as the gradle-based system.  We use gradle to generate the maven poms for those subprojects, in order to keep dependencies and version numbers in synch (caveat: rundeckapp has a mostly hand-hewn pom).

We do want to maintain maven compatible builds, as some people haven't made the leap to gradle and need to build rundeck themselves.

When not built with maven, Gradle is used for the java components, but Grails is used directly for the 'rundeckapp'. Also, the documentation is built with a groovy script + Pandoc, and a few more recursive Makefiles. Rpm/deb packaging uses Makefiles. This is tied together in the top Makefile. Some of this unfortunately has evolved slower than other bits.

Ideally we would have just direct gradle and maven builds.

There are a few steps that might be tricky to convert:

* the main Grails build of rundeckapp: If we can build grails 1.3.7 easily with gradle, that's great. I think grails 2.+ might have gradle support, but I haven't had a chance to investigate. (Grails 2+ upgrade is on the roadmap, but not in the near future) 

* documentation build could probably be replaced with gradle directly to call the groovy wrapper over Pandoc (Note: the maven build *doesn't* currently support building the documentation)

* rpm/deb packaging: might be more natural to leave to use make? it also currently depends on the documentation build. 

So, it's a bit of a tangle, if you'd like to help clear out the old-growth forest and plant the new trees that'd be great!

We should also consider whether maintaining documentation and packaging build in the same repo makes sense.  Maybe it makes more sense to migrate those parts to separate repos/builds, and use several Jenkins jobs to do the builds.  People only interested in the war/java components wouldn't have to care about the rest.

And, since we're talking about builds, here are some other improvements that I'd like to make eventually as well: 

* build a rundeck-plugin-api jar in addition to rundeck-core, and publish all the artifacts to maven central

* change the launcher-jar build to actually just be an executable form of the war

* if possible, find (or build) a java/groovy-based replacement for pandoc

cheers,

Greg

On Wed, Sep 12, 2012 at 11:58 AM, John Burbridge <johnbu...@gmail.com> wrote:

Hello folks,

I'm working on setting up my dev environment to start making contributions to the Rundeck project and I noticed that Make is doing a lot of calls to Gradle which is currently used to build artifacts for each individual subproject. These subprojects in turn generate pom files that are wired up together in a Maven multi-module setup via a parent pom. Does that sound about right?

Unless I'm missing something, I believe the build could be simplified (and sped) up by leveraging Gradle across the board. The idea being that it could be setup as a multi-project build replacing Make, as well as managing dependencies, generating snapshots, packaging, downloading and installing components like Grails, etc.   Maven poms could still be generated to tie everything together in a multi-module setup, but given that STS has pretty good Gradle tooling, I see little benefit in supporting Maven.

I'd be glad to take a stab at implementing a pure Gradle build if others think this would be a good idea.

Thoughts?

JB

[John Burbridge]

Hi Greg,

Sounds good. I'll start with the low hanging fruit -- just to push the gardening metaphor. :)

On the tricky bits:

- Agreed, I have yet to find a good gradle-grails plugin for versions 1.3.x, though there are both plugins and wrappers that work well with grails 2. Instead of trying to support that, I was thinking of approaching it the same way Make does it now. Download and extract grails to build/local and issue a "grails war".  The benefit of doing this with Gradle would be to take advantage of tasks, dependencies on other sub projects (core really), outputs and caching.

- Documentation build - groovy wrapper around pandoc seems straight-foward.

- Rpm/deb packaging - I had missed that bit. Looking at it now, I agree that looks like a great fit for Make. Doable with Gradle but not sure what the immediate benefit would be of converting it.

On the other improvements:

* build a rundeck-plugin-api jar in addition to rundeck-core, and publish all the artifacts to maven central

Pretty straight-forward. We could setup a release task on Jenkins to publish snapshots and releases that have been blessed.

* change the launcher-jar build to actually just be an executable form of the war

Could you elaborate a little more on this one? Right now the launcher has the embedded jetty-server which has the grails war. Correct?

* if possible, find (or build) a java/groovy-based replacement for pandoc

I know close to zero about pandoc. I'm sure it's possible though. Hey, it's software - anything is possible! 

Cheers,

JB

[Greg Schueler]

On Wed, Sep 12, 2012 at 2:11 PM, John Burbridge <johnbu...@gmail.com> wrote:

Hi Greg,

Sounds good. I'll start with the low hanging fruit -- just to push the gardening metaphor. :)

On the tricky bits:

- Agreed, I have yet to find a good gradle-grails plugin for versions 1.3.x, though there are both plugins and wrappers that work well with grails 2. Instead of trying to support that, I was thinking of approaching it the same way Make does it now. Download and extract grails to build/local and issue a "grails war".  The benefit of doing this with Gradle would be to take advantage of tasks, dependencies on other sub projects (core really), outputs and caching.

Sounds good 

- Documentation build - groovy wrapper around pandoc seems straight-foward.

- Rpm/deb packaging - I had missed that bit. Looking at it now, I agree that looks like a great fit for Make. Doable with Gradle but not sure what the immediate benefit would be of converting it.

On the other improvements:

* build a rundeck-plugin-api jar in addition to rundeck-core, and publish all the artifacts to maven central

Pretty straight-forward. We could setup a release task on Jenkins to publish snapshots and releases that have been blessed.

* change the launcher-jar build to actually just be an executable form of the war

Could you elaborate a little more on this one? Right now the launcher has the embedded jetty-server which has the grails war. Correct?

 

Yes, the launcher jar contains the exploded war, template files, various other jars, and has a Main class that can both install and run the server. Installation sets up rundeck_base dir structure, extracts the contents to the right places, and expands templates into config files. Running the server basically just runs embedded jetty on the exploded war dir, although it makes use of some config input for port nums/SSL, etc.

I'd like to change this so that we instead use the war file built by grails and add in the other needed components. We'd modify the Manifest to set a Main class so that you can do "java -jar rundeck.war", in addition to still being able to deploy the war as-is into e.g. Tomcat.  (This is how Jenkins' war works I believe.)  I think it should be pretty easy to do, although this requires modifying some of the code that knows how to extract things.  In the end we would no longer publish a launcher-jar artifact, and just use the war.

If we were on grails 2.0+, we could look into using the war-exec grails plugin, which can build an executable war like this for us:  http://grails.org/plugin/war-exec

* if possible, find (or build) a java/groovy-based replacement for pandoc

I know close to zero about pandoc. I'm sure it's possible though. Hey, it's software - anything is possible! 

 

Yeah, this is a nice-to-have, but in using pandoc (haskell based) and writing the groovy wrapper for it, I was in great desire of a simple java lib that could just be aimed at a directory of Markdown files and produce HTML in a similar way to pandoc.  A (supposed) benefit of still using pandoc is it can produce other formats like pdf, epub, etc.  Pandoc also allows some useful metadata in the source files. I put some effort into adding support for a single-page HTML and single-page PDF build of the documentation, but this ended up requiring some newer version of pandoc which I could not for the life of me get installed on both my osx box and our linux build server. A java solution would just be much simpler.