Java Buildpack modularity

[Daniel Grippi]

Is there a reason why we haven't taken a more modular approach to our Java Buildpack?  Over half of Heroku's official Buildpacks are JVM languages:

Daniel

[Mike Youngstrom]

Do the Heroku buildpacks have a concept of "frameworks?  The current java build pack supports 6 containers but 9 frameworks.  Each framework works across all the containers.

So, to modularize the build pack you'd have to come up with a way to not duplicate the framework code in each container.

We add an additional 3 custom frameworks to our fork of the java buildpack.  I'd hate to have to maintain those across 6 different build pack forks.

There may be a reasonable way to do this.  But I think the current approach is a valid first attempt in that light.  My 2 cents.

Mike

To unsubscribe from this group and stop receiving emails from it, send an email to vcap-dev+u...@cloudfoundry.org.

[Ben Hale]

Is there a reason why we haven't taken a more modular approach to our Java Buildpack?  Over half of Heroku's official Buildpacks are JVM languages:

Mike actually beat me to the punch, but here's a more detailed explanation:

1.  Buildpacks cannot have dependencies

As things stand today, much of the code for any given component (e.g. Tomcat, Grails, etc.) is shared.  This includes bits like logging, dependency resolution, and detection.  The buildpack is written in Ruby and could potentially share code via gems, but there’s no point in the buildpack lifecycle where bundle cloud be called because there’s no guarantee that a buildpack is written in Ruby.  We toyed around with the stager detecting that a Gemfile.lock exists and calling bundle then but it was an edge case for an unproven buildpack, so it never went anywhere.  There's also the penalty to be paid every time a gem needs to be downloaded when an application staged.  All things considered, I think the earlier decision to stick to the no-dependencies rule has driven us to be very focused on a minimal buildpack.

2.  Only a single buildpack can be run

The Java Buildpack is designed with three orthogonal component types.  A single JRE is selected (OpenJDK by default, but we support Oracle, and the Liberty buildpack supports J9), a single container is selected, and any number of frameworks are selected.  Regardless of which JRE is selected, a container must be selected.  Regardless of what container is selected, any number of frameworks can be selected (e.g. you might want New Relic, custom JAVA_OPTS, and Spring Auto-reconfiguration all simultaneously). The current design essentially gives us “multi-buildpack” functionality, for JRE-based applications (that's why the component API looks so familiar...).  Way back in the beginning, trying to justify "multi-buildpack" functionality was difficult and thus wasn't ever incorporated into the stager.  As Mike points out, it has proven valuable and we might want to open discussions about it again.

In general, Heroku’s choice of containers as a dominant decomposition only works because their buildpacks aren’t as sophisticated as ours is.  They force users to explicitly configure much of the stuff we do (and want to do more of) automatically; I consider a user doing anything other than a `cf push` to be a failure on our part in most cases.  Even if we wanted to aim for a container-based decomposition as they do, the need to duplicate much of the of the functionality is hamstrung by #1.

-Ben Hale

Cloud Foundry Java Experience

[David Williams]

Good points, Mike and Ben.

From what I understand, there may be another reason for the various buildpacks, at least for the JVM ones. To answer Mike's question, Heroku buildpacks do have the concept of frameworks...in a way. But like Ben said, it's less sophisticated. Rather than saying "the Java buildpack supports 6 languages and 9 frameworks", they go the route of "this language-centric buildpack supports Scala and all frameworks we currently support for Scala (e.g. Play, Lift, etc.). I think one reason, from what I can see, why Heroku has this language separation is because they support both compiled, ready to deploy apps (e.g. .war files) as well as uncompiled .class files. If you want to push a Scala app uncompiled, it will detect this, download a version of sbt, compile your app, then set up the environment for the framework and release. They have probably handled buildpacks from a more modular approach to help keep the complexity of all of that down and more manageable.

I think in a world with CI, there's no need for that, and CF's approach to the Java buildpack makes it cleaner and easier to use.

My 1 cent (less valuable than Mike's 2). :)