Java Buildpack v2.0 Now Available

[Ben Hale]

One of the happy results of the recent v155 release is that version 2.0 of the Java Buildpack is now available on http://run.pivotal.io and in release stream.  The release includes the following changes:

• Output detection changes
  
• Play Framework 2.2 support
  
• Spring Insight support
  
• New Relic zero-touch support now works with user-provided services in addition to platform provided services
  
• AppDynamics support
  
• New abstractions and test support for building components
  
• Automatic download of MariaDB and PostgreSQL drivers
  

We've incremented the major version number for the buildpack because this release is backwards-incompatible with previous release for buildpack extenders.  We do not believe that we've changed the buildpack in ways that affect buildpack users.  We did not take the decision to break compatibility lightly, but after working with the community, we realized that the amount of repetitive code extenders were required to write just wasn't good enough.  Based on our experiences so far, we've seen a 50% reduction in both production and test code.  I highly recommend that people using forked buildpacks take the time to update to a 2.0 base.  If you have any trouble doing this, please use the list to let me know and I'm happy to work with you to figure out what's wrong.

-Ben Hale

Cloud Foundry Java Experience

[Completes #63749354]

[Mike Youngstrom]

Congrats Ben!  I'll updated my fork to 2.0 this month.

Mike

To unsubscribe from this group and stop receiving emails from it, send an email to vcap-dev+u...@cloudfoundry.org.

[Stephan Oudmaijer]

Hi Ben,

nice work, congrats. I will update our internal fork of the buildpack, thanks for the warning.

For Java developers, we noticed that the cf maven plugin does not work well with user provided services. I am working on a fix. We are also adding blue-green deployment to the maven plugin as a feature. Something that we need internally.

Regards, Stephan

[Mike Youngstrom]

Stephan,

+1 blue green support in the maven plugin.  I'm excited for that contribution.  We'll be using it. :)

Mike

To unsubscribe from this group and stop receiving emails from it, send an email to vcap-dev+u...@cloudfoundry.org.

[Scott Frederick]

Support for user-provided services was added to the Maven plugin a little while ago: https://github.com/cloudfoundry/cf-java-client/commit/0f37609308af90d5e66bcdc1f1e7fa11b452f149. Similar support has also been added to the Gradle plugin. I'm updating the READMEs for both plugins now and preparing a release. 

Blue/green deployment support was recently added to the Gradle plugin to support a few projects that I was made aware of: https://github.com/cloudfoundry/cf-java-client/commit/7d3740d7d419a02e163212cb134f7ecc07b9a0f9. It would be good if the Maven plugin support followed a similar design (goal names, configuration values etc) unless there is a desire for a different design that was implemented for Gradle. If your ideas are significantly different maybe we should have a chat and talk about it. 

Scott

[Scott Frederick]

The design of the Gradle blue/green deployment is hard to get from the code, and I haven't written the README updates for that yet. This document discusses the design as it is currently implemented: https://docs.google.com/a/gopivotal.com/document/d/1ZhEOQQcUoGKEVgFgHn6q-BX54TImTI2yE5M4QMJdYIE/edit#.

Scott

[Stephan Oudmaijer]

Hi Scott,

thanks for sharing your document.

I like your approach implemented in the gradle plugin. I will start working on adding simular goals to the maven plugin tomorrow.

What we would like to add is an automated way of pushing an app next to the version already running and switch to this version once all instances are running (not really blue green deployment IMO).

Regards, Stephan

[James Bayer]

since we have a “rename” command in the CLI and can do so without changing the GUID of the app, one of the things i had talked to scott about was instead of using the concept of blue/green, we could use an app suffix name convention. so when pushing a new app to the env in a zero-downtime way, you would have a concept of a production route, say, myapp.example.com. let’s take a look at the suffixes and examples in this scenario:

APPNAME-current - the live in production version mapped to the production route
APPNAME-new - the new version of the app that is intended to take the place of the production app
APPNAME-old - the previous production version of the app

i’m not attached to the suffix names in these examples, it could be next/previous instead of old/new or something else.

time0, the apps in a space look like this:
APPNAME-current - running and mapped to myapp.example.com
APPNAME-old - not running, still mapped to myapp.example.com

time1, the apps in a space look like this:
APPNAME-current - running and mapped to myapp.example.com
APPNAME-old - the previous production version of the app, not running, still mapped to myapp.example.com

APPNAME-new - pushed to CF with a route like myapp-next.example.com and is running at the end of this step to continue

time2, the apps in a space look like this:
APPNAME-current - running and mapped to myapp.example.com
APPNAME-old - the previous production version of the app, not running, still mapped to myapp.example.com

APPNAME-new - once running, this version now gets mapped to myapp.example.com

time3, the apps in a space look like this:
APPNAME-current - running and mapped to myapp.example.com
APPNAME-old - this version is deleted or renamed to something else

APPNAME-new - running and mapped to myapp.example.com

time4, the apps in a space look like this:
APPNAME-old - app is renamed, still running and mapped to myapp.example.com
APPNAME-new - running and mapped to myapp.example.com

time5, the apps in a space look like this:
APPNAME-old - app is stopped, and still mapped to myapp.example.com
APPNAME-current - app is renamed, running and mapped to myapp.example.com

you could even keep several copies of the older apps like:
APPNAME-old-1

APPNAME-old-2

APPNAME-old-3

what do you think of this approach therefore each deploy takes the same flow instead of swapping blue/green and having to remember which is which.

--

Thank you,

James Bayer

[mana]

Hey,

great job on the version 2 of this buildpack.
I wanted to test Spring Insight. Is there any document that describes how we can get Spring Insights working within PCF?

-- Matthias

[Ben Hale]

great job on the version 2 of this buildpack. 

I wanted to test Spring Insight. Is there any document that describes how we can get Spring Insights working within PCF?

At the moment, there is no hosted-version of Spring Insight available, but the team is working on it as fast as they can.  We're just ahead of the curve to help them in their development.  I'll get one of the Insight team to comment here with more information.

[cfr...@gopivotal.com]

We don't have a release of Insight that works with the support in the Buildpack. As Ben says we need the buildpack support to finish our work integrating CF and Insight. We expect to have it all working in the next couple of weeks but I don't know what the timeline is for our 2.0 release. In the mean time the only way to get something to play with is to find a commit in out source code that matches the support in the Buildpack, build the project and deploy the Insight CF-Dashboard project as an app on CF and then configure a user provided service. We have a Gateway application that will be used to create Insight services and Dave Tillman the project lead can tell you more about that. Insight as a single click service on CF is coming but not quite ready yet, stay tuned.

Chris Frost.

mana wrote:

Hey,

great job on the version 2 of this buildpack.
I wanted to test Spring Insight. Is there any document that describes how we can get Spring Insights working within PCF?

-- Matthias

[Mike Youngstrom]

I've finally finished upgrading our buildpack to v2.0.2.  It took several days worth of work for us to learn and understand the changes then make and test the result.  I have a few notes:

* I know you said that the v2 breakage was not taken lightly.  I just want to emphasize that it was a pain for us and we hope we won't have to do it again for a long while. :)

* For our unit tests we use the same test helpers you use in your tests.  One of the difficult parts of the upgrade is we had to re-write the tests for our extensions in addition to rewriting the actual logic.  Do you consider your test helpers a public api as well?  It would be nice if you did.

Thanks again for the buildpack.  Even having to rewrite occasionally it's better than maintaining our own. :)

Mike

On Fri, Feb 7, 2014 at 8:40 AM, Ben Hale <bh...@gopivotal.com> wrote:

To unsubscribe from this group and stop receiving emails from it, send an email to vcap-dev+u...@cloudfoundry.org.

[Mike Youngstrom]

One other note.

Our DEAs don't have access to the internet.  Instead of relying on the buildpack-cache for application binaries we are now syncing an internal repository with s3://download.pivotal.io.  We then maintain our own index.yml files to make visible versions that we wish to expose to our customers when we are ready to expose them.

Anyway, just thought I'd make you aware in case there is an issue with us doing that. :)

Mike

[Ben Hale]

* I know you said that the v2 breakage was not taken lightly.  I just want to emphasize that it was a pain for us and we hope we won't have to do it again for a long while. :)

Absolutely understood.  Now that the abstraction layer is in place, I wouldn't expect major changes for a long time.  You can trust that there will be additions for new functionality, but the base contract (what's passed to each module during initialization) shouldn't change in an incompatible way.  Note that the internals of a component (for example, I know you guys modify the Tomcat container) are still not public API.  In master, we've broken up the Tomcat component into something more modular and maintainable and that is likely to hurt you guys a bit (which sucks considering how much work you've done on the buildpack recently).  If you need some help updating your buildpack to cope with this, send me an email off-list.

 

* For our unit tests we use the same test helpers you use in your tests.  One of the difficult parts of the upgrade is we had to re-write the tests for our extensions in addition to rewriting the actual logic.  Do you consider your test helpers a public api as well?  It would be nice if you did.

I don't think we've ever specified that we do, but we'll take some time to evaluate what's there, harden it, and document it.  Once that's done we'll officially declare it as API.  https://www.pivotaltracker.com/story/show/67731100

[Ben Hale]

Our DEAs don't have access to the internet.  Instead of relying on the buildpack-cache for application binaries we are now syncing an internal repository with s3://download.pivotal.io.  We then maintain our own index.yml files to make visible versions that we wish to expose to our customers when we are ready to expose them.

Anyway, just thought I'd make you aware in case there is an issue with us doing that. :)

That's exactly what we expected on-prem customers to do actually.  In fact in addition to exposing the tool that we use to populate the cache, we also expose a tool that makes re-hosting the entire repository locally, easy to do.  It downloads the entire repository and updates all of the index files with a new host name.  If you guys start using this (instead of the tool you use now), let us know how it goes.  I'm not aware of anyone yet using it and we'd be happy for some feedback on improving it.

[Mike Youngstrom]

In master, we've broken up the Tomcat component into something more modular and maintainable and that is likely to hurt you guys a bit (which sucks considering how much work you've done on the buildpack recently).  If you need some help updating your buildpack to cope with this, send me an email off-list.

We actually maintain a separate container from tomcat now.  A few months ago you added some utility methods that made extending Tomcat less useful allowing us to create a separate container.  So, no worries with that.  Our buildpack consists of 1 custom container, 3 custom frameworks.  We also customize app-dynamics for some internal app and tier naming requirements.

 

I wasn't aware of the re-hosting tool.  I'll look into using that.

Thanks,

Mike

[Ben Hale]

We actually maintain a separate container from tomcat now.  A few months ago you added some utility methods that made extending Tomcat less useful allowing us to create a separate container.  So, no worries with that.  Our buildpack consists of 1 custom container, 3 custom frameworks.  We also customize app-dynamics for some internal app and tier naming requirements.

Understood.  Are the changes you're making to the AppDynamics framework generally useful?  Are there some requirements there that you'd like to see us take on?

[Mike Youngstrom]

We have 3 main customizations in app-dynamics:

1. We get app and tier names from another custom service (not generally useful)

2. App dynamics sends up all Environment variables.  This is a security risk for us sending VCAP_SERVICES to app dynamics.  So we have some custom hacks to not send those up.  Once app dynamics provides official support for filtering what environment variables get sent to app dynamics there is probably some potential for enhancement here.

3. We support a custom app-agent-config.xml file in "resources".  This is something that would probably be generally useful but it was such a small amount of work I haven't gotten around to it.

Mike

To unsubscribe from this group and stop receiving emails from it, send an email to vcap-dev+u...@cloudfoundry.org.

[Ben Hale]

2. App dynamics sends up all Environment variables.  This is a security risk for us sending VCAP_SERVICES to app dynamics.  So we have some custom hacks to not send those up.  Once app dynamics provides official support for filtering what environment variables get sent to app dynamics there is probably some potential for enhancement here.

Ah, interesting.  Do you know if they're actively working on this filtering?  If not, it's probably something that we can push on a bit.  Seems to be a pretty big security risk in a PaaS environment.

 

3. We support a custom app-agent-config.xml file in "resources".  This is something that would probably be generally useful but it was such a small amount of work I haven't gotten around to it.

I'll add this one to the backlog now.  If we get to it before you, we'll take care of it.  If you find the time for a PR before then, we'll happily take it.

[Mike Youngstrom]

Ah, interesting.  Do you know if they're actively working on this filtering?  If not, it's probably something that we can push on a bit.  Seems to be a pretty big security risk in a PaaS environment.

We have requested it a month or 2 ago.  It sounded like others had requested it and that they were planning to fix it.  However, we haven't checked on the status lately.

Mike

[Ben Hale]

We have requested it a month or 2 ago.  It sounded like others had requested it and that they were planning to fix it.  However, we haven't checked on the status lately.

Understood.  I'll ask them the next time we chat.  Thanks for the input.