2.0 - breaking out into multiple tracks

[Andrew Bayer]

We have differing visions of what "Jenkins 2.0" would actually mean, and those visions are to a certain extent mutually incompatible - getting 2.0 out in the timeframe Kohsuke has proposed wouldn't be possible if that requires not just the user experience work he has mentioned but also storage changes, a jenkins configuration API, etc... So I have my own proposal: break the work into a few tracks, each overseen by a shepherd, on different schedules. So the initial 2.0 release would be the work Kobsuke has put forward, but at the same time, another set of devs will be working on the initial storage changes, targeting 2.15 or something like that.

Since the Big Bang model with significant comparability breaking changes is not a viable option, these tracks will still need to be concerned with backwards compatibility for existing jobs, plugins and build history, as well as working with the tracks scheduled to be released earlier, but this will allow us to make bigger, longer-term plans than we have previously. Kohsuke's 2.0 proposal gives us the blueprint for this - define a scope, define a goal timeline, implement the changes, verify compatibility and stability, and then release. We can follow this model for significant work going forward, not just now. Having a designated person serving as a shepherd will give us someone who is focused on defining the scope and making sure the work gets done. Let's aim at having three tracks going at a time as a max - we need to focus the development efforts to be sure we complete them!

My initial proposal would look something like this:

- 2.0 track - As Kohsuke has laid out. Shepherded by Kohsuke. Release planned for February 2016 (I like to cushion schedules a bit to make sure they're not just aspirational but actually realistic - given the breadth of front end changes discussed, this may be an overly aggressive date).

- Pluggable storage/database backend - Scope to be determined, myself as shepherd, target release date of April 2016.

- ? For the third track, let's try to reach a consensus on what the next priority should be. It could be stability, it could be plugin cleanup (too many plugins!), it could be configurability and platform (I.e., config API, Java 8, etc).

Thoughts?

A.

[Vojtech Juranek]

Hi,

> another set of devs will be working
> on the initial storage changes, targeting 2.15 or something like that

in general I agree with your (I'd like to see changes in backend happen as
well), but I'm not sure about this one - you don't expect any API changes or
you propose to do backward incompatible changes in a minor version change? If
the former, IMHO it doesn't need to be bind to 2.0, just form group of
interested devs and make the change, if later it seems to me quite unfortunate
to do any incompatibility changes during minor version bump. IMHO lots of
people expect, if not state clearly otherwise, that any backward incompatible
changes happen only during major version bump.

With 2.0 bump discussion, IMHO it would be useful if we define what major
version bump exactly means for Jenkins (i.e. still very limited backward
incompatible changes; only some API changes - only plugin maintainers should
take care; backward incompatible changes can happen - every admin should take
care and look if some config migration is needed; etc) and also how often we
expect major version changes. If cca once a year (or e.g. once some major
feature is developed and tested), IMHO it's fine to wait with it for 3.0, if
we stick with 2.0 for another 10 years, than makes sense for me to postpone
2.0 and implement more changes.

Cheers
Vojta

[Andrew Bayer]

On Thu, Oct 8, 2015 at 11:12 AM, Vojtech Juranek <vjur...@redhat.com> wrote:

in general I agree with your (I'd like to see changes in backend happen as
well), but I'm not sure about this one - you don't expect any API changes or
you propose to do backward incompatible changes in a minor version change? If
the former, IMHO it doesn't need to be bind to 2.0, just form group of
interested devs and make the change, if later it seems to me quite unfortunate
to do any incompatibility changes during minor version bump. IMHO lots of
people expect, if not state clearly otherwise, that any backward incompatible
changes happen only during major version bump.

Yeah, the former - unless we as a community decide to change our philosophy on backwards incompatible changes, 2.0 or otherwise, we're pretty much stuck maintaining existing APIs forever. 

And no, it's not bound to 2.0 - I just think the confusion over what 2.0 would mean and the desire to get bigger changes in shows we need to work on longer-term changes deliberately, as a group. 2.0 (in Kohsuke's vision) is one set of thematically connected longer-term changes, pluggable storage backend is another, etc.

 

With 2.0 bump discussion, IMHO it would be useful if we define what major
version bump exactly means for Jenkins (i.e. still very limited backward
incompatible changes; only some API changes - only plugin maintainers should
take care; backward incompatible changes can happen -  every admin should take
care and look if some config migration is needed; etc) and also how often we
expect major version changes. If cca once a year (or e.g. once some major
feature is developed and tested), IMHO it's fine to wait with it for 3.0, if
we stick with 2.0 for another 10 years, than makes sense for me to postpone
2.0 and implement more changes.

Last night/morning (time zones!) at the office hours, I tossed out the idea of changing our numbering - to 2.0.x, 2.1.x, etc, to give us the opportunity to make more drastic/compatibility-related changes before a distant 3.0. Kohsuke was not enthused. =) He'd like to stay on the same model as 1.x for versioning.

A.

[Nigel Magnay]

We have differing visions of what "Jenkins 2.0" would actually mean, and those visions are to a certain extent mutually incompatible - getting 2.0 out in the timeframe Kohsuke has proposed wouldn't be possible if that requires not just the user experience work he has mentioned but also storage changes, a jenkins configuration API, etc... 

​I don't understand the rush to label something as 2.0 ? Why not extend the timeframe?

If the changes are so watered down - (no breaking changes, no database, no jdk upgrade​) I don't really see the point of incrementing the big number. Isn't it that just 'business as usual' ?

​I think a 2.0 should include all of the things you're identifying​. If lots of things are constantly put off for 'we'll do that, but in v.Next as it's a breaking change' surely it makes sense to actually do them, take the hit in one go?

Also - why can't 1.xxx continue in parallel to 2.x ? That gives it a longer time to cook, provide upgrade paths from 1.x. e.g: plugins could declare themselves 'ready for 1->2 upgrade', and the installation could inform the user 'you're good to go'. This is a fairly well-used pattern on other projects

 

[Oleg Nenashev]

+1 for breaking activities into threads, +1 to let Andrew lead the database backend activity; +100500 for moving all compat changes to the third track. Had the same proposal in the Jenkins 2.0 thread, but it has been lost in other comments. Compat track is a painful stuff, but I think that Jenkins needs it to avoid the architectural paralysis.

If the changes are so watered down - (no breaking changes, no database, no jdk upgrade​) I don't really see the point of incrementing the big number. Isn't it that just 'business as usual' ?

No reason from the technical side. BTW it should be a message to the community.

 Also - why can't 1.xxx continue in parallel to 2.x ?

If we introduce breaking changes, we will have to do it. If the scope of changes is limited, it is not a big deal. For a real "Jenkins 2.0" we need to have an internal fork for at least 6 month in order to let Jenkins stabilize a bit. I always vote for long-term release lines (1 year or above), which would allow large-scale installations to avoid making de-facto custom forks. This upgrade is an opportunity to change the approaches.

Best regards,
Oleg

четверг, 8 октября 2015 г., 13:45:43 UTC+3 пользователь nigelm написал:

[Stephen Connolly]

On 8 October 2015 at 11:45, Nigel Magnay <nigel....@gmail.com> wrote:
>
>> We have differing visions of what "Jenkins 2.0" would actually mean, and
>> those visions are to a certain extent mutually incompatible - getting 2.0
>> out in the timeframe Kohsuke has proposed wouldn't be possible if that
>> requires not just the user experience work he has mentioned but also storage
>> changes, a jenkins configuration API, etc...
>
>
>
> I don't understand the rush to label something as 2.0 ? Why not extend the
> timeframe?

Ok, I'll bite.

There are a number of conflicting things we need to balance.

* There are some bigger UI/UX refreshes that we want to get out to
users. A long standing complaint is that the Jenkins UI/UX is dated.
Moving to a 2.0 label corresponding to the visible UI changes helps
advertise the fact that the Jenkins UI/UX is being updated
* It is hard enough getting users to upgrade to LTS lines, when they
see a 2.0, there will be a bigger fear of upgrade breakages... in a
sense that is why we have not done a 2.0 yet... I believe that to be a
mistake. I think a better thing to do is to bump the major version
more regularly... so I would see 2.0 being the 2016 release, 3.0 being
the 2017 release, etc (though KK may feel differently). If users build
up the expectation that "yes it's a major bump, but normally they
don't break too much in a major bump... it's more like jumping 4 or 5
LTSes" then we can keep the users with us.
* We all have our pet crappy APIs that we want to kill off... the
allure of a major version bump is a siren luring us towards breaking
more stuff

So I see Jenkins 2.0 more as a new version numbering scheme...

We could call it Jenkins 2016 rather than 2.0... but that would set an
expectation that in 2017 we would roll out a 2017... given how hard we
find sticking to the LTS schedule I'd rather go with a major version
bump every 12-18 months.

Then we can set out a deprecation policy, say that we remove APIs that
have been deprecated for 2 or more major version numbers... perhaps we
can use some static analysis or other tooling to alert you if your
plugins are using deprecated APIs.

In my vision, thus, Jenkins 2.0 is about sending a message that the
project is changing how it views compatability with the past... it's
still important so we are not removing dead APIs yet, but we have to
start putting lines in the sand so that we can remove dead APIs in the
future.

>
> If the changes are so watered down - (no breaking changes, no database, no
> jdk upgrade) I don't really see the point of incrementing the big number.
> Isn't it that just 'business as usual' ?

It's about setting expectations for subsequent versions.

>
> I think a 2.0 should include all of the things you're identifying. If lots
> of things are constantly put off for 'we'll do that, but in v.Next as it's a
> breaking change' surely it makes sense to actually do them, take the hit in
> one go?
>
> Also - why can't 1.xxx continue in parallel to 2.x ? That gives it a longer
> time to cook, provide upgrade paths from 1.x. e.g: plugins could declare
> themselves 'ready for 1->2 upgrade', and the installation could inform the
> user 'you're good to go'. This is a fairly well-used pattern on other
> projects

Python 3 anyone?

>
>
>
>
>
>
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Jenkins Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jenkinsci-de...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jenkinsci-dev/CAPYP83QixUeFYp5UU1gekE1KwadQVV-cA9r4fjQg-1oQHy1eZQ%40mail.gmail.com.
>
> For more options, visit https://groups.google.com/d/optout.

[Nigel Magnay]

Ok, I'll bite.

There are a number of conflicting things we need to balance.

* There are some bigger UI/UX refreshes that we want to get out to
users. A long standing complaint is that the Jenkins UI/UX is dated.
Moving to a 2.0 label corresponding to the visible UI changes helps
advertise the fact that the Jenkins UI/UX is being updated 

* It is hard enough getting users to upgrade to LTS lines, when they
see a 2.0, there will be a bigger fear of upgrade breakages... in a
sense that is why we have not done a 2.0 yet... I believe that to be a
mistake. I think a better thing to do is to bump the major version
more regularly... so I would see 2.0 being the 2016 release, 3.0 being
the 2017 release, etc (though KK may feel differently). If users build
up the expectation that "yes it's a major bump, but normally they
don't break too much in a major bump... it's more like jumping 4 or 5
LTSes" then we can keep the users with us.

I don't follow why that's a bad thing though.

"Users" ​are trained - by basically the entire software industry and for better or worse - to feel that a 1.x -> 1.y upgrade they can consider 'easy', but a 1.x to a 2.0 should be considered 'harder', and at least to read the changelog before performing an upgrade. We even codified it as 'semantic versioning'.

If I understand your goal, it's to try to un-train that behaviour, so somehow users will learn that - for Jenkins - an v(x) -> v(x+1) isn't a 'hard' change.

​The problem I have with that is a) it's counter to expectations, and b) what do you do if you do want to signal a major bump with compatibility consequences?

* We all have our pet crappy APIs that we want to kill off... the
allure of a major version bump is a siren luring us towards breaking
more stuff

So I see Jenkins 2.0 more as a new version numbering scheme...

We could call it Jenkins 2016 rather than 2.0... but that would set an
expectation that in 2017 we would roll out a 2017... given how hard we
find sticking to the LTS schedule I'd rather go with a major version
bump every 12-18 months.

Then we can set out a deprecation policy, say that we remove APIs that
have been deprecated for 2 or more major version numbers... perhaps we
can use some static analysis or other tooling to alert you if your
plugins are using deprecated APIs. 

​Fine, but as a plugin writer it feels like death by a thousand tiny cuts. I'd rather a direct train from A to B, rather than stopping at all the minor stations in-between.

Part of the reason I think there is excitement about a '2.0 line' is the ​ability to break long-standing compatibility - to finally address some of the stuff that's perennially kicked into the long grass. I think there's a tension between a commercial desire to maintain high levels of backwards compatibility, and a developer desire to get on and fix stuff. Endlessly updating plugins as Jenkins painfully modernises inch-by-inch isn't much fun.

​

 

In my vision, thus, Jenkins 2.0 is about sending a message that the
project is changing how it views compatability with the past... it's
still important so we are not removing dead APIs yet, but we have to
start putting lines in the sand so that we can remove dead APIs in the
future.

​I'd rather 2.0 was the one where the dead APIs were removed.​ That matches my expectation of what a 'big number change' means.

 

>
> If the changes are so watered down - (no breaking changes, no database, no
> jdk upgrade) I don't really see the point of incrementing the big number.
> Isn't it that just 'business as usual' ?

It's about setting expectations for subsequent versions.

 

>
> I think a 2.0 should include all of the things you're identifying. If lots
> of things are constantly put off for 'we'll do that, but in v.Next as it's a
> breaking change' surely it makes sense to actually do them, take the hit in
> one go?
>
> Also - why can't 1.xxx continue in parallel to 2.x ? That gives it a longer
> time to cook, provide upgrade paths from 1.x. e.g: plugins could declare
> themselves 'ready for 1->2 upgrade', and the installation could inform the
> user 'you're good to go'. This is a fairly well-used pattern on other
> projects

Python 3 anyone?

​Linux 2.4 / 2.6? Ubuntu? We already manage an LTS line.

 

[Jesse Glick]

On Thu, Oct 8, 2015 at 3:00 AM, Andrew Bayer <andrew...@gmail.com> wrote:
> Pluggable storage/database backend - Scope to be determined, myself as
> shepherd, target release date of April 2016.

This seems a very optimistic schedule based on my experience of what
ought to have been far simpler modifications.

[Andrew Bayer]

Fair enough.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr2dMrOqZYs8m15sH5SJ2A%2BsB6nbQi%3DwwEmHqw0wWXKKgg%40mail.gmail.com.

[Baptiste Mathus]

Maybe the tick-tock model Oleg proposed could be adopted? And IIUC, that would also kind of match Stephen's vision about updating major version number more often.

This way:

* 2.0 could be a mostly be a user facing changes release, to be expected in early 2016.

* while work on 3.0 could be announced with potentially more technically disruptive changes, with an alpha to be expected around 2nd mid/end 2016 for example? And GA end 2016/early 2017?

(Obviously not pursuing 'breaking' changes per-se, goes wo saying I guess, but more to fix actual longstanding issues)

My 2 cents

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAPbPdObw2OXXH2wJ5sXN3%3DwG7e1TO6gcjzD_n97bL1qe-Cjeag%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Baptiste <Batmat> MATHUS - http://batmat.net
Sauvez un arbre,
Mangez un castor !

[Andrew Bayer]

That sounds reasonable to me. Given the likelihood that I misjudged how long the storage change would take, it's very plausible that any of the larger architectural improvements would take long enough to wait for late 2016.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANWgJS5jhAKuxJu%3Dz2tt2-xACkR98rvmZUsT2D8KNw8h6ecbaw%40mail.gmail.com.

[Kohsuke Kawaguchi]

2015-10-08 5:01 GMT-07:00 Nigel Magnay <nigel....@gmail.com>:

Ok, I'll bite.

There are a number of conflicting things we need to balance.

* There are some bigger UI/UX refreshes that we want to get out to
users. A long standing complaint is that the Jenkins UI/UX is dated.
Moving to a 2.0 label corresponding to the visible UI changes helps
advertise the fact that the Jenkins UI/UX is being updated 

* It is hard enough getting users to upgrade to LTS lines, when they
see a 2.0, there will be a bigger fear of upgrade breakages... in a
sense that is why we have not done a 2.0 yet... I believe that to be a
mistake. I think a better thing to do is to bump the major version
more regularly... so I would see 2.0 being the 2016 release, 3.0 being
the 2017 release, etc (though KK may feel differently). If users build
up the expectation that "yes it's a major bump, but normally they
don't break too much in a major bump... it's more like jumping 4 or 5
LTSes" then we can keep the users with us.

I don't follow why that's a bad thing though.

"Users" ​are trained - by basically the entire software industry and for better or worse - to feel that a 1.x -> 1.y upgrade they can consider 'easy', but a 1.x to a 2.0 should be considered 'harder', and at least to read the changelog before performing an upgrade. We even codified it as 'semantic versioning'.

If I understand your goal, it's to try to un-train that behaviour, so somehow users will learn that - for Jenkins - an v(x) -> v(x+1) isn't a 'hard' change.

​The problem I have with that is a) it's counter to expectations, and b) what do you do if you do want to signal a major bump with compatibility consequences?

Users are trained to expect that v(x) -> v(x+1) is a big change. Often as a result of that, a major upgrade is incompatible, but it's not necessarily so. 

Eclipse, Bamboo, and Windows all version in this way. Major number incrment is used to signal feature changes, not necessarily compatibility changes.

And Jenkins 2.0 is a big change for users --- you suddenly get a whole lot of new things you haven't used before, and it encourages & promotes different ways of using it.  I think it's well qualified to call it 2.0, and I don't think it's trying to untrain the general expectation.

And as Andrew is suggesting, let's then start collecting ideas for more developer-focused 3.0, which will likely take a longer time frame.

--

Kohsuke Kawaguchi

[Kanstantsin Shautsou]

I was on jenkins 2.0 meeting that took half of it presenting workflow. What is the problem to use workflow in 1.xx? I don't see that workflow requires breaking something in core. 

UI change yes. DB change - yes. 

And as Andrew is suggesting, let's then start collecting ideas for more developer-focused 3.0, which will likely take a longer time frame.

Then people will wait for 3.0 as UI is not a blocker for current usage (of course everybody wants new UI, but it not a really usage blocker), while performance and features - yes :) 

--

Kohsuke Kawaguchi

[Arnaud Héritier]

And Jenkins 2.0 is a big change for users --- you suddenly get a whole lot of new things you haven't used before, and it encourages & promotes different ways of using it.  I think it's well qualified to call it 2.0, and I don't think it's trying to untrain the general expectation.

I was on jenkins 2.0 meeting that took half of it presenting workflow. What is the problem to use workflow in 1.xx? I don't see that workflow requires breaking something in core. 

UI change yes. DB change - yes. 

I also had this feedback when I discussed about a 2.0 with few Jenkins users. 

Will it increase the stability and performances of Jenkins. 

For them these are clearly the most important improvements which may motivate them to take the time to upgrade to a 2.0

They’ll love to have a better UI but it seems that it’s not enough interesting compared to the cost to upgrade to a major version (they probably have more in ming the upgrade maven 1.0 -> 2.0 than 2.0 -> 3.0)

We clearly need to identify the cost/risk of the upgrade and discuss with users if the value provided is enough to justify it

Arnaud

[Jesse Glick]

On Fri, Oct 9, 2015 at 4:41 PM, Arnaud Héritier <aher...@gmail.com> wrote:
> Will it increase the stability and performances of Jenkins.

Not under current plans, no. IIUC the benefits would mostly be for new
users who would start with small installations anyway.

[Arnaud Héritier]

Yes it is also how I understood it. 2.0 will be focused on adoption / new users. 

We have to take care to not forget our existing users who are following us for 10 years and for these new users who will be in a near future the ones to ask for performances and stability improvements. 

Maybe like discussed in others threads it could be the target of 3.0 but we'll have to take care to communicate about it. 

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr0W%2BTZOvLgmrCYq%2B7RR7u1a%2BFvWmSHMO_7itPjfvDamaQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

-----

Arnaud Héritier

http://aheritier.net

Mail/GTalk: aheritier AT gmail DOT com

Twitter/Skype : aheritier