Jenkins 2.0 proposal

[Kohsuke Kawaguchi]

Jenkins is over 10 years old, and it came quite a long way. I still remember the first few plugins that I wrote by myself, and now we have close to 1100 plugins. What's started as a hobby project that had run under my desk today boasts more than 100K installations driving half a million-ish build machines.

We collectively came quite a long way. When I started Jenkins, having a server building & testing on every commit was a cutting-edge practice. So are automatically capturing changelogs and analysing test reports. But now, those are tablestakes, and the frontier of automation has moved further. Now we are talking about building pipelines & workflows, continuously deploying to servers, leveraging containers, and/or testing pull requests before they get merged. I'm going to call this much bigger entangled automation "Continuous Delivery", to contrast this with more classical automated build & test executions (aka "Continuous Integration") that we set out to do.

The other thing I'd like to point out is that the adoption of Jenkins continues to grow at the incredible pace of 30% year over year. That is, a lot of people are starting new with Jenkins, and they are looking for us to help them get Continuous Delivery done. Therefore, this is a good time to step back and think about whether we are addressing those current user demands.

For example, despite this advance during the last 10 years and 1000+ plugins we've created, messaging in our website hasn't changed much since the first version I wrote on java.net. It spends more space talking about JNLP and zero mention of Git, pipeline, or Docker.

The fresh installation of Jenkins is not much better. The CVS support is available out of the box, but Git isn't. All the cool stuff that the community has done and its collective best practices still need be learned by each and every new user. It's bit like we are making everyone assemble LEGO blocks. That's not doing enough justice to the 30K+ users that will be joining us in this year.

So I propose we do Jenkins 2.0 to fix this.

There are three important goals that I see in Jenkins 2.0.

1. We need to claim our rightful place in Continuous Delivery. We have lots of pieces that address these modern needs, but we are not communicating this very well.
   
   
2. We need to revisit out of the box experience, so that Jenkins itself speaks that story and makes it clear what we are aiming for. Our software needs to speak for itself and tell people where we are going, so that the community can better focus efforts on the important parts.
   
   
3. And we need to do this while keeping what makes Jenkins great in the first place, which are the ecosystem, the community, and the extensibility that recognizes that one size does not fit all and let people do what they want to do.
   

Incrementing the major version sends a clear message to people that we are moving forward. That's why I think 2.0 is appropriate for this effort.

Now, 2.0 can mean a lot of different things to a lot of people, so let me outline what I think we should do and we shouldn't do.

It's very important for me to make sure that my fellow Jenkins developers understand the motivation and the goal of this proposal, because that drives much of what we should and shouldn't do. So instead of deep-diving into technical parts, please take time to try to understand why I am proposing this.

We need to contain the scope. 2.0 has to have enough in it to justify the major version number increase, but it creates a period of pause and uncertainty, so it cannot keep dragging on for too long. 2.0 cannot be everything everyone ever wanted.

We cannot do massively disruptive 2.0, because it ends up splitting the community. If users perceive that the upgrade to 2.0 is risky, enough of them will stay behind with 1.x, plugin authors would want to continue supporting them, which makes 1.x more liveable, which makes the transition slower. I do not want to end up in Python2/3 situation, and nobody wins.

That means we cannot be really breaking plugins. We cannot do s/hudson/jenkins/g in the package names because doing that will break all the plugins. 2.0 does come with the expectation that it is more disruptive than usual 1.630 to 1.631 upgrade, so we have some "disruption budget", but we have to use it really wisely.

Simiarly, for me it is an absolute requirement that we keep people's $JENKINS_HOME functioning. A lot of sweat, tear, and blood went into those right set of plugins and elaborate job configurations. When users upgrade to 2.0, they need to continue to work, or else Jenkins 2.0 will be Jenkins in just the name only.

Therefore, we cannot make massive internal changes. In many ways, it has to be evolutionary instead of revolutionary, when it comes to the code. This is not a "let's redo everything from scratch" kind of 2.0. In any case, I think it's a pitfall to focus too much on internals. We all have a long list of things we want to fix and the technical debt that we want to pay down. My cautionary tale here is that of Maven 2 to Maven 3 upgrade. The developers of the project spent a lot of efforts redoing all the plumbings. Plexus gave way to Guice, and the dependency resolution engine got completely rewritten. Then to keep plugins working, more efforts were spent on building the backward compatibility layer. After something like 18 months, Maven 3 came out, which did more or less the same thing as far as users are concerned. I'm sure I'm over-simplifying this, but you get the point.

So given all that constraints, I think 2.0 should have the following 3 major pillars:

• Messaging changes, to make sure people coming into the Continuous Delivery space will get that Jenkins does what they want.
  
• Software that backs up our messages. Out of the box experience that caters to Continuous Delivery needs.
  
• Targeted internal plumbing changes that enable those goals
  

I have some concrete ideas in each of these pillars, and I'll describe them below. But I also need help from everyone to come up with, discuss, and decide what other things will advance those pillars.

Messaging:

• Domain name. It's kind of a problem that we have "ci" baked into our domain name jenkins-ci.org. We have acquired http://jenkins.cd/ How about we change the domain name? I think it sends another clear signal.
  
  
• We need more up-to-date feature list page (like http://arquillian.org/features/) that talks about things that matter to the modern users.
  
  
• We need authoritative and curated getting started guide that expands on the things listed in the features page and help people understand those features, so that we have clearly marked trails.
  
  
• This is probably out of scope for the initial 2.0 launch, but in the future we want to redo the plugin listing page as well. This is a persistent feedback that we hear from users.
  
  
• All the above things call for better infra that can handle this. Right now we have our web assets are split into Drupal and Wiki, but the former can be only touched by a few people and the latter is slow and klunky. I think this is the time to switch to some static site generator, so that everyone can contribute content through Git and pull requests, just like how we collaborate on plugins.
  

Out of the Box Experience:

• This work is already in progress, but we really need some initial setup wizard. We can use it to install plugins so that new instances come up more useful from get-go --- things like git, workflow, pipeline as code, folders, and so on. These plugins together tell the story of how we want users to use Jenkins.
  
  
• Another work that's already under way is the UX improvement, specifically the config form re-layout. This is the kind of change that helps people (literally) see that 2.0 is different. UX in general is clearly one of the places we should spend our precious disruption budget for.
  
  
• To reinforce the message that workflow is the future, CloudBees is going to open-source our workflow stage view plugin that was previously a part of CloudBees Jenkins Enterprise.
  

Internals:

• Let's define a policy to remove APIs after they are deprecated. We have talked about this in FOSDEM, and this could be as easy as "N releases after deprecation". Feedbacks from users at the San Jose JAM was that things like this is OK, but we need to help people identify plugins that will be impacted to give them earlier warnings.
  
  
• As a part of the UX rebump effort, Tom et al has been working on a brand-new way of doing frontend in Jenkins plugins. His JUC talk has some materials. Given that user experience is a major theme in 2.0, I think this internal plumbing change makes sense.
  
  
• Let's use the opportunity to update some of the libraries. I'm thinking about things like Groovy, which according to the testing done during Copenhagen Hackathon, should be compatible. This shouldn't include updates that are known to be compatibility breaking, such as Acegi Security to Spring Security (which involves package name changes.)
  
  
• Time to bump up the system requirement to Java 8 and Servlet 3.0. Let's think about what this would enable to users. Again, we talked about this a bit in FOSDEM.
  

Finally, timeline-wise, my aspirational timeline is as follows, though obviously this is largely dependent on feedback to the proposal:

• Announce the proposal publicly and have discussions to nail the details (Sep-Oct)
  
• Execution (Oct-Dec)
  
• Periodic alpha/beta releases to solicit feedbacks from users
• PR activities
• This phase concludes with the release candidate
• Plugin sweep to ensure key plugins are "2.0 ready". This is the opportunity to find issues (Jan 2016)
  
• Release (end Jan?)
  
• Drop 1.x development as soon as possible to focus on 2.x. 
  

There are a lot of things I haven't captured, but this email is aleady getting too long. Looking forward to having more conversations about this with everyone.

--

Kohsuke Kawaguchi

[cactusman]

I generally agree, but what do you think for jelly and stapler?

I hope to replace that to make plugin more easily.

But I understand difficult to seamless upgrade.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAN4CQ4zMNF-BJnZ%3DwSxYJiXBkExPvNUnaNU4S5Ru6p-PgJmUbw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--

cactusman

[Richard Bywater]

That's quite an email.... it's quite a work of art! :)  

Overall I agree with the idea and I don't want any of my reply to sound too critical. I realise that Jenkins has a great bunch of developers who do a stellar job of building up things like core, workflow, etc! Therefore please excuse me if I've worded anything in a way that doesn't come off well intended.

I think that, as alluded to, we need to make sure that 2.0 means the start of a journey to a particular goal as otherwise it seems like we are doing what all the big players out there seem to do and increase the major version number to keep customers/users happy :)

I'm not really qualified to talk to a lot of the things you mention, but I haven't seen in the email much in the form of things like visualising CD pipelines etc. Do these types of things exist today because I think to have the goal to have our place in the CD world things like that are needed. Quite a few people at my work don't think that Jenkins is particular good for CD and point in directions such as Thoughtworks Go which, of course, is now also in the open-source arena. If they don't exist they sound like features you wouldn't want to just whip up other night to keep people happy as first impressions count.

I've dragged out a few areas where I wanted to comment - apologies if I end up butchering your original message in the process:

[We collectively came quite a long way. When I started Jenkins, having a server building & testing on every commit was a cutting-edge practice. So are automatically capturing changelogs and analysing test reports. But now, those are tablestakes, and the frontier of automation has moved further. Now we are talking about building pipelines & workflows, continuously deploying to servers, leveraging containers, and/or testing pull requests before they get merged. I'm going to call this much bigger entangled automation "Continuous Delivery", to contrast this with more classical automated build & test executions (aka "Continuous Integration") that we set out to do.]

I think in general, people are "Now we are *also* talking about" CD. In my mind efficient CD pipelines start with efficient CI techniques so I think its dangerous to imply that CI is no longer a thing and its all about CD. You can easily do CI without doing CD if you work in the more classical (read locked down) IT environments.

[Domain name. It's kind of a problem that we have "ci" baked into our domain name jenkins-ci.org. We have acquired http://jenkins.cd/ How about we change the domain name? I think it sends another clear signal.]

Similar to my previous comment I think this is a dangerous move as it sounds like Jenkins is dropping CI in favour of CD. Also what happens when XYZ becomes the next big movement that Jenkins looks to add support for - would it change to http://jenkins.xyz? I think it would be a good idea to try and get a domain name that can stand the test of time - even if you then have the other domains forwarding through to it for catch-iness.

[All the above things call for better infra that can handle this. Right now we have our web assets are split into Drupal and Wiki, but the former can be only touched by a few people and the latter is slow and klunky. I think this is the time to switch to some static site generator, so that everyone can contribute content through Git and pull requests, just like how we collaborate on plugins.]

I agree that having that mix doesn't work very well but I'm not sure it is really a requirement to dump everything that we've got today and jump to another completely different mechanism to which I'm not sure how many people would feel comfortable contributing to documentation. I (as an end user) see the main issue with Confluence as being the fact that a) perhaps the structure is not setup correctly / too much dead pages within and b) the version is SOOOO old - 3.4.7 was released 28th January *2011*! If you take a look at what Atlassian are now doing with (some) of their pages is that they are fronting the public web version with Scroll Viewport (https://marketplace.atlassian.com/plugins/com.k15t.scroll.scroll-viewport) which gives you the backend Wiki editing with the nice presentation.

[Time to bump up the system requirement to Java 8 and Servlet 3.0. Let's think about what this would enable to users. Again, we talked about this a bit in FOSDEM.]

Not sure if this goal and the don't leave people sitting on 1.x goal go too well together (well the Java 8 part at least) especially given the time frames being talked about.

[Release (end Jan?)]

And on the subject of timeframes I worry that its quite tight to look at something as close as January for what seems like a big piece of work. Perhaps its not but best to not understate the time requirement as otherwise Jenkins becomes like some other projects that keep saying that "the next big version" is coming out soon.

---

Cheers

Richard.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAKBnHw-xjXtJHWVUFsZ%2B5vrJQJYfCmw2YNRE6Fb8h2EOotga2w%40mail.gmail.com.

[Kohsuke Kawaguchi]

As you say, for compatibility reasons I don't think we can remove them.

But that "brand-new way of doing frontend development" that I mentioned touches a related space. It borrows a lot of tooling from NodeJS, such as writing JavaScript in CommonJS modules, using JavaScript template libraries like handlebar, etc. My hope is that that provides a different way of building pages that require high degree of interactivity.

The workflow stage view is built using this technology. I'll let Tom chime in to talk more about this.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAKBnHw-xjXtJHWVUFsZ%2B5vrJQJYfCmw2YNRE6Fb8h2EOotga2w%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Kohsuke Kawaguchi

[domi]

As Richard, I also think that we would need some sort of a concept for a CD pipeline. I consider my self as a quite advanced jenkins user, but still I end up redesigning some sort of a CD pipeline with every new installation or even project from scratch with different plugins then I used before. CD is a great concept and sure it looks different for everyone - but people wanna have guidance and then they will be able to change it the way it works for them.

In my point of view Jenkins 2.0 would need some sort of a definition for it and give answers/proposals how to do these steps together (and visualise):

- build some sort of artifact (this stage might include unit tests)

- promote artifacts for integration tests 

- do a release build

- store the released artifacts (depending on the type also make it public available, e.g. for maven, npm, bower,…)

- promote released artifacts to test env 

- trigger tests on test env

- promote released artifacts to prod env 

(for sure the list is not complete and every step has to be plugable for different technologies, but you get the idea)

I know we have all the components/plugins to solve all these issues, but its hard for a newbe (and also experienced users) to really get this right. Workflow for sure is great to go for something like this, but I think we need something on top of it (maybe its just a good documentation) to really get people into this.

/Domi

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAMui945HN1dgibt69rDEsDHOXUoOH1gyH97p3MN3bAZEoQ4m_A%40mail.gmail.com.

[nicolas de loof]

[Time to bump up the system requirement to Java 8 and Servlet 3.0. Let's think about what this would enable to users. Again, we talked about this a bit in FOSDEM.]

Spring changed major version number as they changed the minimal JDK requirement, even the API didn't changed. I think it make sense to use 2.0 at least for this single technical shift. The fact some didn't yet upgraded to 1.8 has been discussed already, we are talking here about Jenkins runtime, not the projects it builds and deploy, that can target Java 1.1 ...

For people who can't get 1.8 ... I'd say "run jenkins docker image" (docker is always the solution, isn't it ?)

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/F34FB690-3946-47E6-8AC9-CB7BA5859D11%40fortysix.ch.

[Michael Neale]

Richard - I would love to talk more about pipeline visualisations (an area I have been looking at, and started working on). Can talk on this list or off list. I think it is really important as well, and totally within reach. Workflow is a powerful base to build on for a lot of things, more so than most realise. 

[Kanstantsin Shautsou]

Will be glad to see:

- normal Queue handling with normal Cloud/provisioning API. We are leaving in clouds 3-4 years but jenkins still can't satisfy this requirement.

- reworked freestyle job, at least merge builders and publishers in repeatable way.

About CI->CD. 

As technical person i see no benefits as it's all just a part of "automation". No matter what automate: builds, deploys, flows, etc. Automation - is what saves your time and money in any field (even non-computer).

[Daniel Beck]

On 25.09.2015, at 09:04, nicolas de loof <nicolas...@gmail.com> wrote:

> The fact some didn't yet upgraded to 1.8 has been discussed already, we are talking here about Jenkins runtime, not the projects it builds and deploy, that can target Java 1.1 ...

1 in 8 Jenkins jobs are based on the Maven job type, whose minimum JDK version is tied to what can run Jenkins.

[Stephen Connolly]

w00t so if I read that correctly then we're dropping the evil job type
for Jenkins 2.0 ;-)

*unless the Maven project does a Maven 3.4.x after JDK9 is released
and before Jenkins 2.0 as then Maven 3.4.x will only be supported on
Java 9 and Java 9-1

> --
> You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/F28E47EB-0F8C-4D9C-98E8-FB39BC13E853%40beckweb.net.

[Tom Fennelly]

On Friday, September 25, 2015 at 7:21:34 AM UTC+1, Kohsuke Kawaguchi wrote:

As you say, for compatibility reasons I don't think we can remove them.

But that "brand-new way of doing frontend development" that I mentioned touches a related space. It borrows a lot of tooling from NodeJS, such as writing JavaScript in CommonJS modules, using JavaScript template libraries like handlebar, etc. My hope is that that provides a different way of building pages that require high degree of interactivity.

The workflow stage view is built using this technology. I'll let Tom chime in to talk more about this.

As far as stapler and jelly are concerned, I'd say the following:

1. We're probably stuck with it.
   
2. Even though there's much about them that I think sucks, I also think they have some very important qualities that have contributed greatly towards making Jenkins as popular as it is. The provide an easy way for people (often with little or no UI skills) to get going with Jenkins and create their first plugin that can contribute elements to the Jenkins UI. So, we shouldn't throw out the baby with the bathwater.

I suppose the best place to start wrt the newer methodologies we've been adopting is to take a look at the following docs:

• jenkins-js-modules: A very simple JavaScript bundle loader tailored to the Jenkins "environment". The main reason I point to this is because the docs there try to explain the problems/challenges (+ there's a slide dec which people might find useful).
• jenkins-js-builder: An NPM module to tries to simplify the process of working with this stuff. Some docs there too.
• jenkins-js-test: No prices for guessing what this is about. Builds on Jasmine and jsdom, making them a bit easier to use together. But, people can always use whatever they want.

PLEASE note that we are not proposing to make any of this a mandatory part of learning or working with Jenkins. If you are happy with Stapler+Jelly and don't want to be bothered with any of this, then keep rock'n !! See this FAQ.

[Tom Moore]

This is a great thread.   I do want to raise one point though.   This should be a "clean" repacking and try to avoid inclusion of plugin functionality in the install that may not be needed or desired.   Your mentioning of Git for example.  Jenkins is used by people who utilize CVS, SVN, Git, Bazaar, Visual Source Safe, Team Foundation Server, Perforce, etc, etc, etc.   So installation of Git as a default is not going to necessarily be something desired by all.    And none of these are even needed to run the base application for configuration.  

I'd prefer identifying the plugin "groups" we know people will nearly always use, such as a SCM, Authentication, or a Builder (Maven, MSBuild, Unity, etc) and the first time a Jenkins server is started and the UI logged into then the user is sent into a wizard to select the ones to download for the first time.   Obviously we wouldn't want to walk them through all 1100 plugins.  But I would think the three groups I mentioned would be pretty much a common decision for nearly all installations.

[Kanstantsin Shautsou]

>
> I'd prefer identifying the plugin "groups" we know people will nearly always use, such as a SCM, Authentication, or a Builder (Maven, MSBuild, Unity, etc) and the first time a Jenkins server is started and the UI logged into then the user is sent into a wizard to select the ones to download for the first time. Obviously we wouldn't want to walk them through all 1100 plugins. But I would think the three groups I mentioned would be pretty much a common decision for nearly all installations.

All projects that i know that started with “groups” (+ sub-groups) end with “tags”, because groups can’t be applied for all cases.
Soft dependencies with reverse dependencies can also provide better user experience. Is it possible to implement them?

[Daniel Beck]

On 25.09.2015, at 14:49, Tom Moore <tommoo...@gmail.com> wrote:

> This is a great thread. I do want to raise one point though. This should be a "clean" repacking and try to avoid inclusion of plugin functionality in the install that may not be needed or desired. Your mentioning of Git for example. Jenkins is used by people who utilize CVS, SVN, Git, Bazaar, Visual Source Safe, Team Foundation Server, Perforce, etc, etc, etc. So installation of Git as a default is not going to necessarily be something desired by all. And none of these are even needed to run the base application for configuration.
>
> I'd prefer identifying the plugin "groups" we know people will nearly always use, such as a SCM, Authentication, or a Builder (Maven, MSBuild, Unity, etc) and the first time a Jenkins server is started and the UI logged into then the user is sent into a wizard to select the ones to download for the first time. Obviously we wouldn't want to walk them through all 1100 plugins. But I would think the three groups I mentioned would be pretty much a common decision for nearly all installations.

This is actually something we're already discussing.

https://groups.google.com/d/msg/jenkinsci-dev/kRobm-cxFw8/AQyLTVDSHQAJ

The thread is a bit long, and started out completely differently, but may be worth reading.

[Jesse Glick]

On Fri, Sep 25, 2015 at 12:53 AM, Kohsuke Kawaguchi <k...@kohsuke.org> wrote:
> Let's use the opportunity to update some of the libraries. I'm thinking
> about things like Groovy, which according to the testing done during
> Copenhagen Hackathon, should be compatible.

IMO rather than compound our error, do the right thing:
https://issues.jenkins-ci.org/browse/JENKINS-29068

[Stephen Connolly]

Yes yes yes. Let's kill groovy views! I nearly have handlebar views working

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr2yZ%2Bfdy2h7UvHopdUpcUqC4zEM7jVumNoL3Ap-VwFpqg%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--
Sent from my phone

[Michael Neale]

On Friday, September 25, 2015 at 8:37:50 PM UTC+10, Kanstantsin Shautsou wrote:

About CI->CD. 

As technical person i see no benefits as it's all just a part of "automation". No matter what automate: builds, deploys, flows, etc. Automation - is what saves your time and money in any field (even non-computer).

Yes, it is all automation, so a lot of this is messaging and presentation. In a practical sense for people like us there isn't a huge technical change necessarily (perhaps the biggest thing in CD is the participation of a wider range of people in a pipeline, vs in CI when it was a fairly technical only crowd). 

 

[Kohsuke Kawaguchi]

2015-09-24 23:20 GMT-07:00 Richard Bywater <ric...@byh2o.com>:

I'm not really qualified to talk to a lot of the things you mention, but I haven't seen in the email much in the form of things like visualising CD pipelines etc. Do these types of things exist today because I think to have the goal to have our place in the CD world things like that are needed. Quite a few people at my work don't think that Jenkins is particular good for CD and point in directions such as Thoughtworks Go which, of course, is now also in the open-source arena. If they don't exist they sound like features you wouldn't want to just whip up other night to keep people happy as first impressions count.

For me, workflow stage view is the piece for visualizing pipelines. As I wrote today that's a proprietary part of CloudBees Jenkins Enterprise, but CloudBees is committing to open-sourcing it to move the dial for Jenkins 2.0.

 

[We collectively came quite a long way. When I started Jenkins, having a server building & testing on every commit was a cutting-edge practice. So are automatically capturing changelogs and analysing test reports. But now, those are tablestakes, and the frontier of automation has moved further. Now we are talking about building pipelines & workflows, continuously deploying to servers, leveraging containers, and/or testing pull requests before they get merged. I'm going to call this much bigger entangled automation "Continuous Delivery", to contrast this with more classical automated build & test executions (aka "Continuous Integration") that we set out to do.]

I think in general, people are "Now we are *also* talking about" CD. In my mind efficient CD pipelines start with efficient CI techniques so I think its dangerous to imply that CI is no longer a thing and its all about CD. You can easily do CI without doing CD if you work in the more classical (read locked down) IT environments.

I think it's probably poor wording on my part. I didn't mean to imply that CI is no longer a thing. I think of CD as a super set of CI, in the sense that everything you do under the banner CI also contributes to CD.

And I think people get this. The main concern is that if we just push CI, people will not get the message that we do more than that.

[Domain name. It's kind of a problem that we have "ci" baked into our domain name jenkins-ci.org. We have acquired http://jenkins.cd/ How about we change the domain name? I think it sends another clear signal.]

Similar to my previous comment I think this is a dangerous move as it sounds like Jenkins is dropping CI in favour of CD. Also what happens when XYZ becomes the next big movement that Jenkins looks to add support for - would it change to http://jenkins.xyz? I think it would be a good idea to try and get a domain name that can stand the test of time - even if you then have the other domains forwarding through to it for catch-iness.

I think I understand where you are coming from. That is, some of us see CD as just another buzzword, so getting that incorporated into the domain name is repeating the same mistake of having CI in jenkins-ci.org in the first place. I see the logic in that.

In this domain name change proposal, the part that I have a stronger conviction of is that we need to drop the "-ci" part because it is creating an incorrect perception in the market that Jenkins is "just CI".  In the ideal world we should get http://jenkins.org/ but unfortunately that domain is just not available.

BTW, no disrespect intended for the word CI --- it lasted good 10 years, which cannot be said for many things in this industry. And I expect CD to last just as long. If we have to change the domain name every 10 years, is that unacceptable or is it OK?

 

[All the above things call for better infra that can handle this. Right now we have our web assets are split into Drupal and Wiki, but the former can be only touched by a few people and the latter is slow and klunky. I think this is the time to switch to some static site generator, so that everyone can contribute content through Git and pull requests, just like how we collaborate on plugins.]

I agree that having that mix doesn't work very well but I'm not sure it is really a requirement to dump everything that we've got today and jump to another completely different mechanism to which I'm not sure how many people would feel comfortable contributing to documentation. I (as an end user) see the main issue with Confluence as being the fact that a) perhaps the structure is not setup correctly / too much dead pages within and b) the version is SOOOO old - 3.4.7 was released 28th January *2011*! If you take a look at what Atlassian are now doing with (some) of their pages is that they are fronting the public web version with Scroll Viewport (https://marketplace.atlassian.com/plugins/com.k15t.scroll.scroll-viewport) which gives you the backend Wiki editing with the nice presentation.

I see some people taking documentation off Confluence and into a set of markdown files in GitHub repo (workflow being one example.) I take it as a signal that people are speaking with their feet. It's been also hard to sustain. It needs constant attention, it's slow, and we cannot control the presentation to the degree Drupal lets us to.

That said, a big part of the motivation for me is to make it easier for people to contribute. So if the static site generator fails in that regard, then that's not good. I'm on the same page with you about that.

I think all that it takes is to have the "Edit" button in the page that takes you back to the edit page in GitHub.

[Time to bump up the system requirement to Java 8 and Servlet 3.0. Let's think about what this would enable to users. Again, we talked about this a bit in FOSDEM.]

Not sure if this goal and the don't leave people sitting on 1.x goal go too well together (well the Java 8 part at least) especially given the time frames being talked about.

OK, that's a good feedback. Servlet 3.0 is 6 year old technology that came with JavaEE *6*, so I'd like to think that everyone already runs it. Java8 is newer and came out 2014, so perhaps I assumed incorrectly that everyone can run Java8 if they wanted to.

I admit that my putting Java8 there is motivated more by the fact that I as a developer wants to code in Java8, as opposed to "Java8 has these additions that translate into these Jenkins features that users would like" 

[Release (end Jan?)]

And on the subject of timeframes I worry that its quite tight to look at something as close as January for what seems like a big piece of work. Perhaps its not but best to not understate the time requirement as otherwise Jenkins becomes like some other projects that keep saying that "the next big version" is coming out soon

OK. I used it to emphasize that this needs to be limited in scope, that I don't want this to turn into a multi-year project. I think you are saying that having a concrete date like this will send other messages to people.

I think we all agree that it's very difficult to put a date to software project, especially when it's not even clear exactly what Jenkins 2.0 is going to be!

 

--

Kohsuke Kawaguchi

[Kohsuke Kawaguchi]

2015-09-24 23:58 GMT-07:00 domi <do...@fortysix.ch>:

As Richard, I also think that we would need some sort of a concept for a CD pipeline. I consider my self as a quite advanced jenkins user, but still I end up redesigning some sort of a CD pipeline with every new installation or even project from scratch with different plugins then I used before. CD is a great concept and sure it looks different for everyone - but people wanna have guidance and then they will be able to change it the way it works for them.

In my point of view Jenkins 2.0 would need some sort of a definition for it and give answers/proposals how to do these steps together (and visualise):

- build some sort of artifact (this stage might include unit tests)

- promote artifacts for integration tests 

- do a release build

- store the released artifacts (depending on the type also make it public available, e.g. for maven, npm, bower,…)

- promote released artifacts to test env 

- trigger tests on test env

- promote released artifacts to prod env 

(for sure the list is not complete and every step has to be plugable for different technologies, but you get the idea)

I know we have all the components/plugins to solve all these issues, but its hard for a newbe (and also experienced users) to really get this right. Workflow for sure is great to go for something like this, but I think we need something on top of it (maybe its just a good documentation) to really get people into this.

Yes, I totally agree. The other day I saw Tom showing his experiment to add ACE editor to Jenkins Workflow config UI (which also came with a drop-down to let you insert some templates.) I've also talked with Jesse and others a number of times about how we can generate GDSL and DSLD. Something like that is clearly needed, no question.

 

--

Kohsuke Kawaguchi

[Kohsuke Kawaguchi]

2015-09-25 5:03 GMT-07:00 Stephen Connolly <stephen.al...@gmail.com>:

w00t so if I read that correctly then we're dropping the evil job type
for Jenkins 2.0 ;-)

Just so that the rest of the world knows you are joking. No we aren't. We need to keep it running.

*unless the Maven project does a Maven 3.4.x after JDK9 is released
and before Jenkins 2.0 as then Maven 3.4.x will only be supported on
Java 9 and Java 9-1

Right, didn't Maven start splitting the JVM version in which Maven itself runs vs JVM that tests run and code gets compiled? Or is it not really a reality among users?

 

On 25 September 2015 at 12:22, Daniel Beck <m...@beckweb.net> wrote:
>
> On 25.09.2015, at 09:04, nicolas de loof <nicolas...@gmail.com> wrote:
>
>> The fact some didn't yet upgraded to 1.8 has been discussed already, we are talking here about Jenkins runtime, not the projects it builds and deploy, that can target Java 1.1 ...
>
> 1 in 8 Jenkins jobs are based on the Maven job type, whose minimum JDK version is tied to what can run Jenkins.
>
> --
> You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/F28E47EB-0F8C-4D9C-98E8-FB39BC13E853%40beckweb.net.
> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CA%2BnPnMwi0pnFP9T6f%3DQLSgzefANX4i7evLjsn-cu8Dha6WtmdA%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Kohsuke Kawaguchi

[Kohsuke Kawaguchi]

OK, that makes sense.

I'm not sure how to do init.groovy. That's a life saver for a small number of people, and it really doesn't work well as plugins.

Maybe we keep Groovy but hide those classes from plugins or something like that?

We need some means to keep track of tickets we are targeting for 2.0. I put the new label "2.0" on that one.

--

Kohsuke Kawaguchi

[Tom Moore]

On Fri, Sep 25, 2015 at 9:24 PM, Kohsuke Kawaguchi <k...@kohsuke.org> wrote:

I think it's probably poor wording on my part. I didn't mean to imply that CI is no longer a thing. I think of CD as a super set of CI, in the sense that everything you do under the banner CI also contributes to CD.

I think you're starting down a good track here but let me throw in a couple thoughts.   When we look at the Continuous processes in terms of the SDLC, there are three concepts that have been developed over the years.   The first is CI, which keeps pulling in developer changes as they are committed, builds and tests them, thereby finding integration problems as early as possible.   Then came the first CD,  Continuous Delivery, which picks up the results of the CI outputs and makes sure they are always in a deploy-able state (not necessarily actually delivering them), thereby hopefully finding deployment preparation problems as early as possible, and also always having the latest good build ready to deploy when its desired.   Then finally the second CD, Continuous Deployment.  This one takes the outputs that pass the Continuous Delivery processes and gates and based on the rules/processes set up, automatically deploys them to a target system.

I think that trying to think of Continuous Deployment as a superset of CI is actually trying to make Continuous Deployment into something its not.   Continuous Integration, Continuous Delivery, and Continuous Deployment are separate but conceptually similar processes which are complimentary to each other.  Historically when trying to view similar processes through the lens of one tends to cause the others to be diminished.      Instead of doing that here, I think what we need is a new conceptual model, one that encompasses all three processes, but has each as a separate phase or workflow.  I'm just not sure what we would call it.  Continuous Integration, Delivery and Deployment (CIDD) doesn't really work for me.  

[Michael Neale]

In various blogs, books and conferences like citcon I have heard the term "continuous delivery" used to mean the overarching concepts that you mention needs a new name at the end. 

Like all terms, it can grow in meaning, and runs a risk of being meaningless, but it does seem to have settled down to something useful (a bit like "cloud" which has a pretty fuzzy definition, but overall seems to be useful). Perhaps saying "CD" as just the initials has this affect, and is "good enough" to convey the concepts you mention? 

[Stephen Connolly]

On Saturday 26 September 2015, Kohsuke Kawaguchi <k...@kohsuke.org> wrote:

2015-09-25 5:03 GMT-07:00 Stephen Connolly <stephen.al...@gmail.com>:

w00t so if I read that correctly then we're dropping the evil job type
for Jenkins 2.0 ;-)

Just so that the rest of the world knows you are joking. No we aren't. We need to keep it running.

Damn it. You pick *now* to detect my humour style of writing.... How will I be able to troll you any more!

(Yes I know we're not going to kill off the evil job type in the short term, but it's still on my list of things to do)

 

*unless the Maven project does a Maven 3.4.x after JDK9 is released
and before Jenkins 2.0 as then Maven 3.4.x will only be supported on
Java 9 and Java 9-1

Right, didn't Maven start splitting the JVM version in which Maven itself runs vs JVM that tests run and code gets compiled? Or is it not really a reality among users?

So yeah, we have toolchains, but even there we have started to limit how far back we support, eg recent versions of surefire will only fork back to Java 5.

Once we get the JDK up to 8 for Maven plugins, we'll likely only support forking back to Java 6 in surefire, 9 -> 7 (but that depends on 9 letting us compile with -target 1.7) 

 

On 25 September 2015 at 12:22, Daniel Beck <m...@beckweb.net> wrote:
>
> On 25.09.2015, at 09:04, nicolas de loof <nicolas...@gmail.com> wrote:
>
>> The fact some didn't yet upgraded to 1.8 has been discussed already, we are talking here about Jenkins runtime, not the projects it builds and deploy, that can target Java 1.1 ...
>
> 1 in 8 Jenkins jobs are based on the Maven job type, whose minimum JDK version is tied to what can run Jenkins.
>
> --
> You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/F28E47EB-0F8C-4D9C-98E8-FB39BC13E853%40beckweb.net.
> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CA%2BnPnMwi0pnFP9T6f%3DQLSgzefANX4i7evLjsn-cu8Dha6WtmdA%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Kohsuke Kawaguchi

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAN4CQ4x5FmFZivYJu0%3Dro87xuGmfZgx022Ap9%3DSCHJkDUOSq2A%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--
Sent from my phone

[nicolas de loof]

[Domain name. It's kind of a problem that we have "ci" baked into our domain name jenkins-ci.org. We have acquired http://jenkins.cd/ How about we change the domain name? I think it sends another clear signal.]

Similar to my previous comment I think this is a dangerous move as it sounds like Jenkins is dropping CI in favour of CD. Also what happens when XYZ becomes the next big movement that Jenkins looks to add support for - would it change to http://jenkins.xyz? I think it would be a good idea to try and get a domain name that can stand the test of time - even if you then have the other domains forwarding through to it for catch-iness.

I think I understand where you are coming from. That is, some of us see CD as just another buzzword, so getting that incorporated into the domain name is repeating the same mistake of having CI in jenkins-ci.org in the first place. I see the logic in that.

In this domain name change proposal, the part that I have a stronger conviction of is that we need to drop the "-ci" part because it is creating an incorrect perception in the market that Jenkins is "just CI".  In the ideal world we should get http://jenkins.org/ but unfortunately that domain is just not available.

As jenkins.org do redirect to stevejenkins.com, dit you tried to contact registrant and propose to acquire the domain name ?

 

[Tom Fennelly]

Yeah, "cd" in the domain name becomes obsolete as soon as "CX" becomes the latest buzz term (I'm sure Martin Fowler is brewing up whatever "X" is as we speak).

--
You received this message because you are subscribed to a topic in the Google Groups "Jenkins Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-dev/vbXK7JJekFw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANMVJzm33iOh3D2xU9mzbfcekVW9qHcFxvpr5%2BqKwgUd5Ab_EQ%40mail.gmail.com.

[Michael Neale]

Although as mentioned by Kohsuke before, if it lasts only 10 years perhaps that is ok?

CX. That has a ring to it now ;)

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CA%2BbPaoLjXdiaMbDSwLBk%2BW1LFya_Mb55GdcQF%2B9Y%3DnxanTy%3DuA%40mail.gmail.com.

[Tom Moore]

On Sat, Sep 26, 2015 at 2:52 AM, Michael Neale <michae...@gmail.com> wrote:

In various blogs, books and conferences like citcon I have heard the term "continuous delivery" used to mean the overarching concepts that you mention needs a new name at the end. 

Those blogs/books/conferences are doing exactly what I said we shouldn't.  Continuous Delivery comes after Continuous Integration in the lifecycle and is a separate process.  If we were to pick something that covers all of them, the only thing I've heard of is the IT term DevOps that is catching on in the IT (as opposed to sw dev) world.  But DevOps doesn't necessarily have to have continuous processes, and it typically is described as covering the whole lifecycle... hmm.... if IT DevOps is overarching and doesn't have to imply continuous processes, what do we think of Continuous DevOps   CDO?   Sure we aren't explicitly covering requirements and design *now* but I can think of primitive continuous processes to just publish the latest changes to those artifacts... and I know there are some automated validation techniques for them as well... so we *COULD* potentially have Jenkins controlling continuous cycles in every phase.
 

[Michael Neale]

My point it is already happening, hard to hold back the tide.

--

You received this message because you are subscribed to a topic in the Google Groups "Jenkins Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-dev/vbXK7JJekFw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jenkinsci-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAFzFsaoTF3BztDd_hfnAHE4HoWkueduQM_48cwvB8aOny-65Jw%40mail.gmail.com.

[Tom Moore]

its not a tide yet... just as many blogs, books and conferences use the term continuous deployment for the same meaning... and then others use devops.  The industry is searching for a common term.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAKVMTi7-bdTgA3qb%2BkG_gnAz4c%2BzhX8UW2YmbS6JV1UeEmg97Q%40mail.gmail.com.

[martinda]

I am in year 3 of using Jenkins, and my users are complaining more about slow downs than before. As an example, the "more" link at the bottom of the build history takes too long to return for my users. I have 141 projects, 30,373 builds, and 1,073,591 junit XML test results. It took me over 20 minutes of CPU time to get these values by querying the file system.

Will Jenkins 2.0 have database backend?

[Michael Neale]

That's a good question. Backing by a database would be a big step, especially as some plugins expect certain things to be in certain places. Certainly a consideration for the future. Incidentally, do you use magnetic or SSD drives?

On Sun, 27 Sep 2015 at 2:19 PM, martinda <martin....@gmail.com> wrote:

I am in year 3 of using Jenkins, and my users are complaining more about slow downs than before. As an example, the "more" link at the bottom of the build history takes too long to return for my users. I have 141 projects, 30,373 builds, and 1,073,591 junit XML test results. It took me over 20 minutes of CPU time to get these values by querying the file system.

Will Jenkins 2.0 have database backend?

--
You received this message because you are subscribed to a topic in the Google Groups "Jenkins Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-dev/vbXK7JJekFw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jenkinsci-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/c65b5bb2-cd2f-4aae-a330-cabdd9abc8a4%40googlegroups.com.

[Baptiste Mathus]

Isn't this issue just about the thing Tom started (finished?) working on some months ago? I.e. paginating the build details.

Because indeed, on a job with lots of build, you're gonna have a bad time if you click on details. (Might be already fixed, didn't check that specific point recently)

You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAKVMTi76nv3f%3DWFcPodYzS5PKTsfQkhvVp_uMSY2PS2aGHiuag%40mail.gmail.com.

[Tom Fennelly]

On Sunday, September 27, 2015 at 2:31:24 PM UTC+1, Baptiste Mathus wrote:

Isn't this issue just about the thing Tom started (finished?) working on some months ago? I.e. paginating the build details.

Because indeed, on a job with lots of build, you're gonna have a bad time if you click on details. (Might be already fixed, didn't check that specific point recently)

It's still in a PR: https://github.com/jenkinsci/jenkins/pull/1641

It's ready to be merged but I think Daniel wanted to remove some of the search features I added, leaving just the "fuzzy" search. Maybe others should declare their interest and whether they want the more advanced search or just the "fuzzy". A topic for another thread I suppose.

[Jesse Glick]

On Sun, Sep 27, 2015 at 12:18 AM, martinda <martin....@gmail.com> wrote:
> I have 141 projects,
> 30,373 builds, and 1,073,591 junit XML test results. It took me over 20
> minutes of CPU time to get these values by querying the file system.

Yes, build storage is a well-known architectural problem. PR 1641 will
not help you here.

> Will Jenkins 2.0 have database backend?

No, the changes required are too big. There is no way to do it that
quickly. I think we *should* do these changes sooner or later. For me
this seems both more important and more disruptive than the code
changes currently being discussed for 2.0.

[Nigel Magnay]

Would 2.0 be an appropriate juncture to revisit the plugin architecture?

I've often gotten bitten by classloader issues in 2nd/3rd party dependencies (e.g: plugin wants newer version of guava than jenkins-core). Shading is a reasonably complicated thing to do for a plugin, and I wonder if something cleverer could be done - without disappearing down a (say) complicated solution like OSGi.

On Fri, Sep 25, 2015 at 5:53 AM, Kohsuke Kawaguchi <k...@kohsuke.org> wrote:

Jenkins is over 10 years old, and it came quite a long way. I still remember the first few plugins that I wrote by myself, and now we have close to 1100 plugins. What's started as a hobby project that had run under my desk today boasts more than 100K installations driving half a million-ish build machines.

We collectively came quite a long way. When I started Jenkins, having a server building & testing on every commit was a cutting-edge practice. So are automatically capturing changelogs and analysing test reports. But now, those are tablestakes, and the frontier of automation has moved further. Now we are talking about building pipelines & workflows, continuously deploying to servers, leveraging containers, and/or testing pull requests before they get merged. I'm going to call this much bigger entangled automation "Continuous Delivery", to contrast this with more classical automated build & test executions (aka "Continuous Integration") that we set out to do.

The other thing I'd like to point out is that the adoption of Jenkins continues to grow at the incredible pace of 30% year over year. That is, a lot of people are starting new with Jenkins, and they are looking for us to help them get Continuous Delivery done. Therefore, this is a good time to step back and think about whether we are addressing those current user demands.

For example, despite this advance during the last 10 years and 1000+ plugins we've created, messaging in our website hasn't changed much since the first version I wrote on java.net. It spends more space talking about JNLP and zero mention of Git, pipeline, or Docker.

The fresh installation of Jenkins is not much better. The CVS support is available out of the box, but Git isn't. All the cool stuff that the community has done and its collective best practices still need be learned by each and every new user. It's bit like we are making everyone assemble LEGO blocks. That's not doing enough justice to the 30K+ users that will be joining us in this year.

So I propose we do Jenkins 2.0 to fix this.

There are three important goals that I see in Jenkins 2.0.

1. We need to claim our rightful place in Continuous Delivery. We have lots of pieces that address these modern needs, but we are not communicating this very well.
   
   
2. We need to revisit out of the box experience, so that Jenkins itself speaks that story and makes it clear what we are aiming for. Our software needs to speak for itself and tell people where we are going, so that the community can better focus efforts on the important parts.
   
   
3. And we need to do this while keeping what makes Jenkins great in the first place, which are the ecosystem, the community, and the extensibility that recognizes that one size does not fit all and let people do what they want to do.
   

Incrementing the major version sends a clear message to people that we are moving forward. That's why I think 2.0 is appropriate for this effort.

Now, 2.0 can mean a lot of different things to a lot of people, so let me outline what I think we should do and we shouldn't do.

It's very important for me to make sure that my fellow Jenkins developers understand the motivation and the goal of this proposal, because that drives much of what we should and shouldn't do. So instead of deep-diving into technical parts, please take time to try to understand why I am proposing this.

We need to contain the scope. 2.0 has to have enough in it to justify the major version number increase, but it creates a period of pause and uncertainty, so it cannot keep dragging on for too long. 2.0 cannot be everything everyone ever wanted.

We cannot do massively disruptive 2.0, because it ends up splitting the community. If users perceive that the upgrade to 2.0 is risky, enough of them will stay behind with 1.x, plugin authors would want to continue supporting them, which makes 1.x more liveable, which makes the transition slower. I do not want to end up in Python2/3 situation, and nobody wins.

That means we cannot be really breaking plugins. We cannot do s/hudson/jenkins/g in the package names because doing that will break all the plugins. 2.0 does come with the expectation that it is more disruptive than usual 1.630 to 1.631 upgrade, so we have some "disruption budget", but we have to use it really wisely.

Simiarly, for me it is an absolute requirement that we keep people's $JENKINS_HOME functioning. A lot of sweat, tear, and blood went into those right set of plugins and elaborate job configurations. When users upgrade to 2.0, they need to continue to work, or else Jenkins 2.0 will be Jenkins in just the name only.

Therefore, we cannot make massive internal changes. In many ways, it has to be evolutionary instead of revolutionary, when it comes to the code. This is not a "let's redo everything from scratch" kind of 2.0. In any case, I think it's a pitfall to focus too much on internals. We all have a long list of things we want to fix and the technical debt that we want to pay down. My cautionary tale here is that of Maven 2 to Maven 3 upgrade. The developers of the project spent a lot of efforts redoing all the plumbings. Plexus gave way to Guice, and the dependency resolution engine got completely rewritten. Then to keep plugins working, more efforts were spent on building the backward compatibility layer. After something like 18 months, Maven 3 came out, which did more or less the same thing as far as users are concerned. I'm sure I'm over-simplifying this, but you get the point.

So given all that constraints, I think 2.0 should have the following 3 major pillars:

• Messaging changes, to make sure people coming into the Continuous Delivery space will get that Jenkins does what they want.
  
• Software that backs up our messages. Out of the box experience that caters to Continuous Delivery needs.
  
• Targeted internal plumbing changes that enable those goals
  

I have some concrete ideas in each of these pillars, and I'll describe them below. But I also need help from everyone to come up with, discuss, and decide what other things will advance those pillars.

Messaging:

• Domain name. It's kind of a problem that we have "ci" baked into our domain name jenkins-ci.org. We have acquired http://jenkins.cd/ How about we change the domain name? I think it sends another clear signal.
  
  

• We need more up-to-date feature list page (like http://arquillian.org/features/) that talks about things that matter to the modern users.
  
  
• We need authoritative and curated getting started guide that expands on the things listed in the features page and help people understand those features, so that we have clearly marked trails.
  
  
• This is probably out of scope for the initial 2.0 launch, but in the future we want to redo the plugin listing page as well. This is a persistent feedback that we hear from users.
  
  

• All the above things call for better infra that can handle this. Right now we have our web assets are split into Drupal and Wiki, but the former can be only touched by a few people and the latter is slow and klunky. I think this is the time to switch to some static site generator, so that everyone can contribute content through Git and pull requests, just like how we collaborate on plugins.
  

Out of the Box Experience:

• This work is already in progress, but we really need some initial setup wizard. We can use it to install plugins so that new instances come up more useful from get-go --- things like git, workflow, pipeline as code, folders, and so on. These plugins together tell the story of how we want users to use Jenkins.
  
  
• Another work that's already under way is the UX improvement, specifically the config form re-layout. This is the kind of change that helps people (literally) see that 2.0 is different. UX in general is clearly one of the places we should spend our precious disruption budget for.
  
  
• To reinforce the message that workflow is the future, CloudBees is going to open-source our workflow stage view plugin that was previously a part of CloudBees Jenkins Enterprise.
  

Internals:

• Let's define a policy to remove APIs after they are deprecated. We have talked about this in FOSDEM, and this could be as easy as "N releases after deprecation". Feedbacks from users at the San Jose JAM was that things like this is OK, but we need to help people identify plugins that will be impacted to give them earlier warnings.
  
  
• As a part of the UX rebump effort, Tom et al has been working on a brand-new way of doing frontend in Jenkins plugins. His JUC talk has some materials. Given that user experience is a major theme in 2.0, I think this internal plumbing change makes sense.
  
  
• Let's use the opportunity to update some of the libraries. I'm thinking about things like Groovy, which according to the testing done during Copenhagen Hackathon, should be compatible. This shouldn't include updates that are known to be compatibility breaking, such as Acegi Security to Spring Security (which involves package name changes.)
  
  

• Time to bump up the system requirement to Java 8 and Servlet 3.0. Let's think about what this would enable to users. Again, we talked about this a bit in FOSDEM.
  

Finally, timeline-wise, my aspirational timeline is as follows, though obviously this is largely dependent on feedback to the proposal:

• Announce the proposal publicly and have discussions to nail the details (Sep-Oct)
  
• Execution (Oct-Dec)
  
• Periodic alpha/beta releases to solicit feedbacks from users
• PR activities
• This phase concludes with the release candidate
• Plugin sweep to ensure key plugins are "2.0 ready". This is the opportunity to find issues (Jan 2016)
  
• Release (end Jan?)
  
• Drop 1.x development as soon as possible to focus on 2.x. 
  

There are a lot of things I haven't captured, but this email is aleady getting too long. Looking forward to having more conversations about this with everyone.

--

Kohsuke Kawaguchi

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAN4CQ4zMNF-BJnZ%3DwSxYJiXBkExPvNUnaNU4S5Ru6p-PgJmUbw%40mail.gmail.com.

[Olivier Lamy]

Hi,
In the same part, make writing plugin a bit more easier :-) could help to attract more dev.


Olivier

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAPYP83Rx3Nt2ecARsqez1D%2BQ_wQbTfneSHq5cm9UQhfQ9Am3kA%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Olivier Lamy
http://twitter.com/olamy | http://linkedin.com/in/olamy

[Michael Neale]

Is the real problem that core dependencies get too old too quickly? (As opposed to some tech that allows multiple versions of things to get loaded?)

You received this message because you are subscribed to a topic in the Google Groups "Jenkins Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-dev/vbXK7JJekFw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAPYP83Rx3Nt2ecARsqez1D%2BQ_wQbTfneSHq5cm9UQhfQ9Am3kA%40mail.gmail.com.

[nicolas de loof]

if you consider 10 years of jenkins development, I don't think the issue is for dependencies to become old "too quickly", but the fact we hardly can change them without potential risk to break some plugin.

I'd like we find some technical way to isolate core classes implementation so they aren't accessible by plugins, then could exclude core dependencies from plugin classpath, and ensure those one explicitly declare dependencies they rely one.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAKVMTi6Ehmqd-Vc4JFJqi9idiutzNK_n9Xc5a6u2-zc5QhMdgw%40mail.gmail.com.

[Baptiste Mathus]

Didn't have a in-depth look into it yet, but isn't that kind of isolation what's Java9's Jigsaw supposed to bring to the game btw? 

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANMVJzmeJNsHi9XCPUfDirmcc7fKREG5Ljw_y1UbOCMf6-Y%2Beg%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Baptiste <Batmat> MATHUS - http://batmat.net
Sauvez un arbre,
Mangez un castor !

[nicolas de loof]

yes, supposed to, but not sure how and if this could apply to jenkins. Also would then require Java 9, and we already hardly can move to 8 ...

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANWgJS74qrB_DK81JthYdhXsOoc%3Dq4se7CcuJk6yB2%2Bf5STOyw%40mail.gmail.com.

[Stephen Connolly]

IIRC Jigsaw has been reduced in scope to just within the JRE... but
that may have changed again since

> https://groups.google.com/d/msgid/jenkinsci-dev/CANWgJS74qrB_DK81JthYdhXsOoc%3Dq4se7CcuJk6yB2%2Bf5STOyw%40mail.gmail.com.

[Arnaud Héritier]

On Mon, Sep 28, 2015 at 2:41 PM, nicolas de loof <nicolas...@gmail.com> wrote:

if you consider 10 years of jenkins development, I don't think the issue is for dependencies to become old "too quickly", but the fact we hardly can change them without potential risk to break some plugin.

and a part of the issue is probably our lack of tests. 

Because it is really difficult to automate integration tests with all 3rd party systems on which Jenkins is integrated to...

 

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANMVJzmeJNsHi9XCPUfDirmcc7fKREG5Ljw_y1UbOCMf6-Y%2Beg%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

-----

Arnaud Héritier

http://aheritier.net

Mail/GTalk: aheritier AT gmail DOT com

Twitter/Skype : aheritier

[Jesse Glick]

On Mon, Sep 28, 2015 at 6:36 AM, Nigel Magnay <nigel....@gmail.com> wrote:
> Would 2.0 be an appropriate juncture to revisit the plugin architecture?
>
> I've often gotten bitten by classloader issues in 2nd/3rd party dependencies
> (e.g: plugin wants newer version of guava than jenkins-core). Shading is a
> reasonably complicated thing to do for a plugin

I do not think we can make fundamental changes, but we could certainly
try to hide such libraries from the plugin class loader (or, in some
cases, split the core code relying on them into plugins), so that
plugins could use simple unshaded dependencies, for example on
`org.jenkins-ci.libs:guava12:12.0-1`.

[Stephen Connolly]

I think we could hide them from plugins modulo perhaps those compiled
against older cores... or perhaps we could treat the older core
plugins as being like the other de-bundled functionality

> --
> You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr0-3EpWTizZ2EgbVBqm4MBG6%3DZTUceGfTtb%2BZMwQNdrFQ%40mail.gmail.com.

[Kanstantsin Shautsou]

Isn't it possible to make some flag in hpi meta-inf that will switch classloading modes for plugin?

[nicolas de loof]

sure, could rely on MANIFEST's Jenkins-Version >= 2.x

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/153b5592-f7ea-4461-9a45-0a630bf072c0%40googlegroups.com.

[Jesse Glick]

On Mon, Sep 28, 2015 at 2:01 PM, Stephen Connolly
<stephen.al...@gmail.com> wrote:
> perhaps we could treat the older core plugins as being like the other de-bundled functionality

Yes, this was my thought. Just as a plugin declaring a dependency on
an older core gets an implicit dependency on any plugins subsequently
split out, so too would it get an implicit dependency on any libraries
subsequently made into plugins. Indeed there is little difference from
the perspective of the plugin with the (possible) dependency whether
that dependency happened to be defined in
`org.jenkins-ci.main:jenkins-core`, some `jenkins-module`, or some
random `WEB-INF/lib/*.jar`.

[Oleg Nenashev]

What about aggregating all the proposals in a new Jenkins JIRA project?

Seems this thread becomes barely readable due to dozens of proposals. A JIRA project would also allow to vote for issues and to categorize them (infra/code, breaking/non-breaking, etc.)

понедельник, 28 сентября 2015 г., 23:14:37 UTC+3 пользователь Jesse Glick написал:

[Daniel Beck]

My proposal is that anyone suggesting a change for Jenkins 2.0 (both new ideas and old) should label them in JIRA as "jenkins-2.0". We'll then be able to discuss them and decide which make sense. A new project isn't necessary, and this would allow nominating older issues for this.

WDY all T?

> --
> You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/061aa4a5-ef88-4b5c-bd0d-37010e2d9bb7%40googlegroups.com.

[Oleg Nenashev]

+1. We can also setup some global filters for such purpose.

INFRA project also fits well for the infrastructure stuff being discussed here

You received this message because you are subscribed to a topic in the Google Groups "Jenkins Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-dev/vbXK7JJekFw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit