Proposed Changes to `bosh create release`

[Karl Isenberg]

Currently `bosh create release` has some edge case behavior that avoids creating a new release if the fingerprint of the release is the same as any pre-existing release. This behavior may have been added as an optimization at some point, but it creates some confusing UX, like the possibility of 'creating' an old version, instead of incrementing.

To avoid this odd behavior, we'd like to change `bosh create release` to always create a new release, unless a specific manifest file is provided.

We're wondering if anyone is relying on the behavior where `bosh create release` doesn't create a new release because no code has changed.

-Karl Isenberg

Pivotal Engineer - BOSH Team

[Dr Nic Williams]

Karl, I'd only continue to expect that create release would do nothing if it's exactly the same as the last created release.

To unsubscribe from this group and stop receiving emails from it, send an email to bosh-users+...@cloudfoundry.org.

[Matthew Sykes]

Exactly - we shouldn't create a new release if it's the same as the most recently created release.

--
Matthew Sykes
matthe...@gmail.com

[Karl Isenberg]

Are you actually using this behavior? 

We considered that approach but decided it would probably be confusing now that you can create 'old' releases using --version. Given that, 'latest' doesn't always mean 'the last release created'. 

We're also thinking about adding a --name option that allows creating a new release with a new name, which adds some permutations around 'latest'.

-Karl

[Greg Oehmen]

Matthew & Dr.Nic:  Thanks for the feedback.  We are trying to get solid input here from the community of BOSH release creators.  I hope you (and others) can provide more details about your release creation use cases and what behavior you are depending on BOSH to provide.  etc.  For reference, this is all in the context of this story:  https://www.pivotaltracker.com/story/show/72987828

Thanks

Greg

Greg Oehmen

Cloud Foundry Product Manager - Bosh

Pivotal

[Aristoteles Neto]

I find myself using this mostly when modifying existing releases - eg. if we’re talking CF specifically, I need to patch the DEA but end up having new versions of login, UAA, loggregator, etc, without having any idea on what’s changed.

My expectation would be for packages/jobs to remain static if they’ve been previously created/published, regardless of a new release creation.

-- Neto

[Karl Isenberg]

@aristoteles

Packages & Jobs are indexed by fingerprint. So they will be re-used whenever they are the same, and are not ordered or sortable. 

Releases, on the other hand, are sortable and comparable such that there is a concept of 'latest' which is the highest version number. 

The current behavior is such that if you create a release that exactly matches any previous release, then a new release would not actually be created. This can break the standard automated workflow that uses 'latest' (`bosh create release && bosh upload release && bosh deploy`). So we want to fix this without increasing the complexity and harming UX.

If we make it so that create release always creates a new release, it will not change the behavior of package/job creation.

-Karl

[Aristoteles Neto]

I see, it looks like I misunderstood what you were referring to then, apologies.

From your last reply, I then gather that you’re referring to situations such as:

- A new release is created (0.1-dev).

- Modify package abc, to include file 123, create new release (0.2-dev)

- modify package def, to include file 456, create new release (0.3-dev)

- Modify package abc again and remove file 123, create new release - at this point it doesn’t create anything because the fingerprint of package abc matches that of 0.1-dev.

Is this what you’re referring to? If so, then yes, I find that quite annoying. This is particularly true because there isn’t a clean/reliable way of downgrading releases once deployments have been upgraded (think of schema updating releases).

-- Neto

[Karl Isenberg]

Neto,

Your example isn't quite right. In that case, _package_ abc would not be re-created, but _release_ 0.4-dev would be created, because it has a different release fingerprint than any previous release. 

The minimal example of the case I was describing is this:

1) create release => 0.1-dev

2) update package X

3) create release => 0.2-dev

4) revert #2

5) create release => 0.1-dev

We think that step #5 should auto-increment to 0.3-dev, but instead it sees your release fingerprint matches 0.1-dev and doesn't make a new release.

The catch is that that behavior uses the same logic that allows you to do this:

1) create release => 0.1-dev

2) update package X

3) create release => 0.2-dev

4) create release => 0.2-dev

So we want to know if anyone actually relies on that behavior. 

-Karl

[Alan Morán]

I think it makes sense, would this just changes the version number and that would be it? It would be nice if it shares some kind of message informing that an early revision will be set as the new one because I can see this as being a typical mistake in distributed teams where you might step over someone else version while using the same bosh and working on the same release.

Cheers,

- - - -

Alan Moran

 
Altoros — Cloud Foundry deployment, training and integration

http://bonzofenix.com/ 

[Karl Isenberg]

The early one doesn't get set as the new one so much as the new one would have the same content as the early one. 

Given that there seems to be some desire to know when this happens, but disagreement on when it's desirable, I'd like to update my proposal:

bosh create release would continue to compare new release fingerprints to old ones, but would now prompt: "The contents of version '0.3-dev' exactly matches the contents of version '0.1-dev'. Do you still want to create '0.3-dev'? yes/no" Non-interactive mode would then default to 'yes'. A new --skip-if-unchanged flag could be passed to skip the prompt as if 'no' was entered. 

Would that satisfy everyone?

It isn't exactly as simple as I was hoping for, but it would at least change the 'default' behavior to be the more commonly desired behavior. 

I'm open to alternate suggestions on flag name. I didn't want to use --skip-if-exists because that's used elsewhere to refer to matching name & version, not fingerprint. 

-Karl

[Greg Oehmen]

Karl:

I agree that this new proposal is not as simple but it does look like it solves the use cases I've heard here.  I hope others can give this some thought and respond.

Best

Greg

Greg Oehmen

Cloud Foundry Product Manager - Bosh

Pivotal

[Alan Morán]

It does work for us the updated proposal.

Cheers,

- - - -

Alan Moran

 
Altoros — Cloud Foundry deployment, training and integration

http://bonzofenix.com/ 

[Greg Oehmen]

Thanks Alan.  (By the way, you've been really vocal in this group the past few weeks.  Your effort here is really appreciated and you've done a great job, Alan!)

All: 

I've had the opportunity to have a lot of conversations and a lot of feedback in this area of behavior around creating BOSH releases.  Thank you- Aristotle, Dr.Nic, Matthew, Karl and Alan for your feedback specifically in this thread.  And many more folks via many other threads/channels of communication.  Based on the sum of all that input, the way we are going to proceed (barring some sort of all out revolt) is this:

If a user executes `bosh create release`, bosh will create a release.  It will not check to see if that exact release (any release with the same fingerprint) already exists.  It will not apply any other logic or prompt for user input.  It will dumbly do exactly what the user has instructed it to do.  The release will be incremented either according to the semi-semantic versioning that we've discussed in this forum or as indicated by the `--version` flag.

The current behavior of doing nothing when a potential new release matches a pre-existing release is flawed.  This case occurs a lot in development with the current behavior and is a poor experience: 

• release 1, 2, 3 are created 3 is current
• user tries to create release 4 but cannot because 4 would be the exact same as 1
• now the user has to use release 1 but does not want 1 to be 'latest' so the user actually adds whitespace to create uniqueness and now can create release 4

Also, in order to compare fingerprints between a potential new release and all preexisting release fingerprints, BOSH has to actually finish creating the release in order to generate the fingerprint so it can compare it to the others.  No time is saved, no performance gains are realized.  I don't see the benefit of halting at that point and declaring that the release already exists.  I have not been able to find anyone who can articulate the problem or challenge that having release 1 and release 4 with exact identical bits.  

Like always, your opinion matters and all feedback is appreciated.  You can follow the story here:  https://www.pivotaltracker.com/story/show/72987828

Best

Greg

Greg Oehmen

Cloud Foundry Product Manager - Bosh

Pivotal