Jenkins 2.0 proposal

8,768 views
Skip to first unread message

Kohsuke Kawaguchi

unread,
Sep 25, 2015, 12:54:03 AM9/25/15
to jenkin...@googlegroups.com
Jenkins is over 10 years old, and it came quite a long way. I still remember the first few plugins that I wrote by myself, and now we have close to 1100 plugins. What's started as a hobby project that had run under my desk today boasts more than 100K installations driving half a million-ish build machines.

We collectively came quite a long way. When I started Jenkins, having a server building & testing on every commit was a cutting-edge practice. So are automatically capturing changelogs and analysing test reports. But now, those are tablestakes, and the frontier of automation has moved further. Now we are talking about building pipelines & workflows, continuously deploying to servers, leveraging containers, and/or testing pull requests before they get merged. I'm going to call this much bigger entangled automation "Continuous Delivery", to contrast this with more classical automated build & test executions (aka "Continuous Integration") that we set out to do.

The other thing I'd like to point out is that the adoption of Jenkins continues to grow at the incredible pace of 30% year over year. That is, a lot of people are starting new with Jenkins, and they are looking for us to help them get Continuous Delivery done. Therefore, this is a good time to step back and think about whether we are addressing those current user demands.


For example, despite this advance during the last 10 years and 1000+ plugins we've created, messaging in our website hasn't changed much since the first version I wrote on java.net. It spends more space talking about JNLP and zero mention of Git, pipeline, or Docker.

The fresh installation of Jenkins is not much better. The CVS support is available out of the box, but Git isn't. All the cool stuff that the community has done and its collective best practices still need be learned by each and every new user. It's bit like we are making everyone assemble LEGO blocks. That's not doing enough justice to the 30K+ users that will be joining us in this year.

So I propose we do Jenkins 2.0 to fix this.

There are three important goals that I see in Jenkins 2.0.
  1. We need to claim our rightful place in Continuous Delivery. We have lots of pieces that address these modern needs, but we are not communicating this very well.

  2. We need to revisit out of the box experience, so that Jenkins itself speaks that story and makes it clear what we are aiming for. Our software needs to speak for itself and tell people where we are going, so that the community can better focus efforts on the important parts.

  3. And we need to do this while keeping what makes Jenkins great in the first place, which are the ecosystem, the community, and the extensibility that recognizes that one size does not fit all and let people do what they want to do.

Incrementing the major version sends a clear message to people that we are moving forward. That's why I think 2.0 is appropriate for this effort.



Now, 2.0 can mean a lot of different things to a lot of people, so let me outline what I think we should do and we shouldn't do.

It's very important for me to make sure that my fellow Jenkins developers understand the motivation and the goal of this proposal, because that drives much of what we should and shouldn't do. So instead of deep-diving into technical parts, please take time to try to understand why I am proposing this.

We need to contain the scope. 2.0 has to have enough in it to justify the major version number increase, but it creates a period of pause and uncertainty, so it cannot keep dragging on for too long. 2.0 cannot be everything everyone ever wanted.

We cannot do massively disruptive 2.0, because it ends up splitting the community. If users perceive that the upgrade to 2.0 is risky, enough of them will stay behind with 1.x, plugin authors would want to continue supporting them, which makes 1.x more liveable, which makes the transition slower. I do not want to end up in Python2/3 situation, and nobody wins.

That means we cannot be really breaking plugins. We cannot do s/hudson/jenkins/g in the package names because doing that will break all the plugins. 2.0 does come with the expectation that it is more disruptive than usual 1.630 to 1.631 upgrade, so we have some "disruption budget", but we have to use it really wisely.

Simiarly, for me it is an absolute requirement that we keep people's $JENKINS_HOME functioning. A lot of sweat, tear, and blood went into those right set of plugins and elaborate job configurations. When users upgrade to 2.0, they need to continue to work, or else Jenkins 2.0 will be Jenkins in just the name only.

Therefore, we cannot make massive internal changes. In many ways, it has to be evolutionary instead of revolutionary, when it comes to the code. This is not a "let's redo everything from scratch" kind of 2.0. In any case, I think it's a pitfall to focus too much on internals. We all have a long list of things we want to fix and the technical debt that we want to pay down. My cautionary tale here is that of Maven 2 to Maven 3 upgrade. The developers of the project spent a lot of efforts redoing all the plumbings. Plexus gave way to Guice, and the dependency resolution engine got completely rewritten. Then to keep plugins working, more efforts were spent on building the backward compatibility layer. After something like 18 months, Maven 3 came out, which did more or less the same thing as far as users are concerned. I'm sure I'm over-simplifying this, but you get the point.



So given all that constraints, I think 2.0 should have the following 3 major pillars:
  • Messaging changes, to make sure people coming into the Continuous Delivery space will get that Jenkins does what they want.
  • Software that backs up our messages. Out of the box experience that caters to Continuous Delivery needs.
  • Targeted internal plumbing changes that enable those goals
I have some concrete ideas in each of these pillars, and I'll describe them below. But I also need help from everyone to come up with, discuss, and decide what other things will advance those pillars.

Messaging:
  • Domain name. It's kind of a problem that we have "ci" baked into our domain name jenkins-ci.org. We have acquired http://jenkins.cd/ How about we change the domain name? I think it sends another clear signal.

  • We need more up-to-date feature list page (like http://arquillian.org/features/) that talks about things that matter to the modern users.

  • We need authoritative and curated getting started guide that expands on the things listed in the features page and help people understand those features, so that we have clearly marked trails.

  • This is probably out of scope for the initial 2.0 launch, but in the future we want to redo the plugin listing page as well. This is a persistent feedback that we hear from users.

  • All the above things call for better infra that can handle this. Right now we have our web assets are split into Drupal and Wiki, but the former can be only touched by a few people and the latter is slow and klunky. I think this is the time to switch to some static site generator, so that everyone can contribute content through Git and pull requests, just like how we collaborate on plugins.

Out of the Box Experience:
  • This work is already in progress, but we really need some initial setup wizard. We can use it to install plugins so that new instances come up more useful from get-go --- things like git, workflow, pipeline as code, folders, and so on. These plugins together tell the story of how we want users to use Jenkins.

  • Another work that's already under way is the UX improvement, specifically the config form re-layout. This is the kind of change that helps people (literally) see that 2.0 is different. UX in general is clearly one of the places we should spend our precious disruption budget for.

  • To reinforce the message that workflow is the future, CloudBees is going to open-source our workflow stage view plugin that was previously a part of CloudBees Jenkins Enterprise.

Internals:
  • Let's define a policy to remove APIs after they are deprecated. We have talked about this in FOSDEM, and this could be as easy as "N releases after deprecation". Feedbacks from users at the San Jose JAM was that things like this is OK, but we need to help people identify plugins that will be impacted to give them earlier warnings.

  • As a part of the UX rebump effort, Tom et al has been working on a brand-new way of doing frontend in Jenkins plugins. His JUC talk has some materials. Given that user experience is a major theme in 2.0, I think this internal plumbing change makes sense.

  • Let's use the opportunity to update some of the libraries. I'm thinking about things like Groovy, which according to the testing done during Copenhagen Hackathon, should be compatible. This shouldn't include updates that are known to be compatibility breaking, such as Acegi Security to Spring Security (which involves package name changes.)

  • Time to bump up the system requirement to Java 8 and Servlet 3.0. Let's think about what this would enable to users. Again, we talked about this a bit in FOSDEM.

Finally, timeline-wise, my aspirational timeline is as follows, though obviously this is largely dependent on feedback to the proposal:
  • Announce the proposal publicly and have discussions to nail the details (Sep-Oct)
  • Execution (Oct-Dec)
    • Periodic alpha/beta releases to solicit feedbacks from users
    • PR activities
    • This phase concludes with the release candidate
  • Plugin sweep to ensure key plugins are "2.0 ready". This is the opportunity to find issues (Jan 2016)
  • Release (end Jan?)
  • Drop 1.x development as soon as possible to focus on 2.x. 

There are a lot of things I haven't captured, but this email is aleady getting too long. Looking forward to having more conversations about this with everyone.

--
Kohsuke Kawaguchi

cactusman

unread,
Sep 25, 2015, 1:52:29 AM9/25/15
to jenkin...@googlegroups.com
I generally agree, but what do you think for jelly and stapler?

I hope to replace that to make plugin more easily.

But I understand difficult to seamless upgrade.


--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAN4CQ4zMNF-BJnZ%3DwSxYJiXBkExPvNUnaNU4S5Ru6p-PgJmUbw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.



--
cactusman

Richard Bywater

unread,
Sep 25, 2015, 2:20:53 AM9/25/15
to jenkin...@googlegroups.com
That's quite an email.... it's quite a work of art! :)  

Overall I agree with the idea and I don't want any of my reply to sound too critical. I realise that Jenkins has a great bunch of developers who do a stellar job of building up things like core, workflow, etc! Therefore please excuse me if I've worded anything in a way that doesn't come off well intended.

I think that, as alluded to, we need to make sure that 2.0 means the start of a journey to a particular goal as otherwise it seems like we are doing what all the big players out there seem to do and increase the major version number to keep customers/users happy :)

I'm not really qualified to talk to a lot of the things you mention, but I haven't seen in the email much in the form of things like visualising CD pipelines etc. Do these types of things exist today because I think to have the goal to have our place in the CD world things like that are needed. Quite a few people at my work don't think that Jenkins is particular good for CD and point in directions such as Thoughtworks Go which, of course, is now also in the open-source arena. If they don't exist they sound like features you wouldn't want to just whip up other night to keep people happy as first impressions count.

I've dragged out a few areas where I wanted to comment - apologies if I end up butchering your original message in the process:

[We collectively came quite a long way. When I started Jenkins, having a server building & testing on every commit was a cutting-edge practice. So are automatically capturing changelogs and analysing test reports. But now, those are tablestakes, and the frontier of automation has moved further. Now we are talking about building pipelines & workflows, continuously deploying to servers, leveraging containers, and/or testing pull requests before they get merged. I'm going to call this much bigger entangled automation "Continuous Delivery", to contrast this with more classical automated build & test executions (aka "Continuous Integration") that we set out to do.]

I think in general, people are "Now we are *also* talking about" CD. In my mind efficient CD pipelines start with efficient CI techniques so I think its dangerous to imply that CI is no longer a thing and its all about CD. You can easily do CI without doing CD if you work in the more classical (read locked down) IT environments.

[Domain name. It's kind of a problem that we have "ci" baked into our domain name jenkins-ci.org. We have acquired http://jenkins.cd/ How about we change the domain name? I think it sends another clear signal.]

Similar to my previous comment I think this is a dangerous move as it sounds like Jenkins is dropping CI in favour of CD. Also what happens when XYZ becomes the next big movement that Jenkins looks to add support for - would it change to http://jenkins.xyz? I think it would be a good idea to try and get a domain name that can stand the test of time - even if you then have the other domains forwarding through to it for catch-iness.

[All the above things call for better infra that can handle this. Right now we have our web assets are split into Drupal and Wiki, but the former can be only touched by a few people and the latter is slow and klunky. I think this is the time to switch to some static site generator, so that everyone can contribute content through Git and pull requests, just like how we collaborate on plugins.]

I agree that having that mix doesn't work very well but I'm not sure it is really a requirement to dump everything that we've got today and jump to another completely different mechanism to which I'm not sure how many people would feel comfortable contributing to documentation. I (as an end user) see the main issue with Confluence as being the fact that a) perhaps the structure is not setup correctly / too much dead pages within and b) the version is SOOOO old - 3.4.7 was released 28th January *2011*! If you take a look at what Atlassian are now doing with (some) of their pages is that they are fronting the public web version with Scroll Viewport (https://marketplace.atlassian.com/plugins/com.k15t.scroll.scroll-viewport) which gives you the backend Wiki editing with the nice presentation.

[Time to bump up the system requirement to Java 8 and Servlet 3.0. Let's think about what this would enable to users. Again, we talked about this a bit in FOSDEM.]

Not sure if this goal and the don't leave people sitting on 1.x goal go too well together (well the Java 8 part at least) especially given the time frames being talked about.

[Release (end Jan?)]

And on the subject of timeframes I worry that its quite tight to look at something as close as January for what seems like a big piece of work. Perhaps its not but best to not understate the time requirement as otherwise Jenkins becomes like some other projects that keep saying that "the next big version" is coming out soon.

---

Cheers
Richard.

Kohsuke Kawaguchi

unread,
Sep 25, 2015, 2:21:34 AM9/25/15
to jenkin...@googlegroups.com
As you say, for compatibility reasons I don't think we can remove them.

But that "brand-new way of doing frontend development" that I mentioned touches a related space. It borrows a lot of tooling from NodeJS, such as writing JavaScript in CommonJS modules, using JavaScript template libraries like handlebar, etc. My hope is that that provides a different way of building pages that require high degree of interactivity.

The workflow stage view is built using this technology. I'll let Tom chime in to talk more about this.



For more options, visit https://groups.google.com/d/optout.



--
Kohsuke Kawaguchi

domi

unread,
Sep 25, 2015, 2:58:26 AM9/25/15
to Jenkins Developers
As Richard, I also think that we would need some sort of a concept for a CD pipeline. I consider my self as a quite advanced jenkins user, but still I end up redesigning some sort of a CD pipeline with every new installation or even project from scratch with different plugins then I used before. CD is a great concept and sure it looks different for everyone - but people wanna have guidance and then they will be able to change it the way it works for them.
In my point of view Jenkins 2.0 would need some sort of a definition for it and give answers/proposals how to do these steps together (and visualise):
- build some sort of artifact (this stage might include unit tests)
- promote artifacts for integration tests 
- do a release build
- store the released artifacts (depending on the type also make it public available, e.g. for maven, npm, bower,…)
- promote released artifacts to test env 
- trigger tests on test env
- promote released artifacts to prod env 
(for sure the list is not complete and every step has to be plugable for different technologies, but you get the idea)

I know we have all the components/plugins to solve all these issues, but its hard for a newbe (and also experienced users) to really get this right. Workflow for sure is great to go for something like this, but I think we need something on top of it (maybe its just a good documentation) to really get people into this.

/Domi


nicolas de loof

unread,
Sep 25, 2015, 3:04:35 AM9/25/15
to jenkin...@googlegroups.com
[Time to bump up the system requirement to Java 8 and Servlet 3.0. Let's think about what this would enable to users. Again, we talked about this a bit in FOSDEM.]

Spring changed major version number as they changed the minimal JDK requirement, even the API didn't changed. I think it make sense to use 2.0 at least for this single technical shift. The fact some didn't yet upgraded to 1.8 has been discussed already, we are talking here about Jenkins runtime, not the projects it builds and deploy, that can target Java 1.1 ...

For people who can't get 1.8 ... I'd say "run jenkins docker image" (docker is always the solution, isn't it ?)

Michael Neale

unread,
Sep 25, 2015, 4:42:23 AM9/25/15
to Jenkins Developers
Richard - I would love to talk more about pipeline visualisations (an area I have been looking at, and started working on). Can talk on this list or off list. I think it is really important as well, and totally within reach. Workflow is a powerful base to build on for a lot of things, more so than most realise. 

Kanstantsin Shautsou

unread,
Sep 25, 2015, 6:37:50 AM9/25/15
to Jenkins Developers
Will be glad to see:
- normal Queue handling with normal Cloud/provisioning API. We are leaving in clouds 3-4 years but jenkins still can't satisfy this requirement.
- reworked freestyle job, at least merge builders and publishers in repeatable way.

About CI->CD. 
As technical person i see no benefits as it's all just a part of "automation". No matter what automate: builds, deploys, flows, etc. Automation - is what saves your time and money in any field (even non-computer).

Daniel Beck

unread,
Sep 25, 2015, 7:22:48 AM9/25/15
to jenkin...@googlegroups.com

On 25.09.2015, at 09:04, nicolas de loof <nicolas...@gmail.com> wrote:

> The fact some didn't yet upgraded to 1.8 has been discussed already, we are talking here about Jenkins runtime, not the projects it builds and deploy, that can target Java 1.1 ...

1 in 8 Jenkins jobs are based on the Maven job type, whose minimum JDK version is tied to what can run Jenkins.

Stephen Connolly

unread,
Sep 25, 2015, 8:03:47 AM9/25/15
to jenkin...@googlegroups.com
w00t so if I read that correctly then we're dropping the evil job type
for Jenkins 2.0 ;-)

*unless the Maven project does a Maven 3.4.x after JDK9 is released
and before Jenkins 2.0 as then Maven 3.4.x will only be supported on
Java 9 and Java 9-1
> --
> You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/F28E47EB-0F8C-4D9C-98E8-FB39BC13E853%40beckweb.net.

Tom Fennelly

unread,
Sep 25, 2015, 8:29:42 AM9/25/15
to Jenkins Developers
On Friday, September 25, 2015 at 7:21:34 AM UTC+1, Kohsuke Kawaguchi wrote:
As you say, for compatibility reasons I don't think we can remove them.

But that "brand-new way of doing frontend development" that I mentioned touches a related space. It borrows a lot of tooling from NodeJS, such as writing JavaScript in CommonJS modules, using JavaScript template libraries like handlebar, etc. My hope is that that provides a different way of building pages that require high degree of interactivity.

The workflow stage view is built using this technology. I'll let Tom chime in to talk more about this.

As far as stapler and jelly are concerned, I'd say the following:
  1. We're probably stuck with it.
  2. Even though there's much about them that I think sucks, I also think they have some very important qualities that have contributed greatly towards making Jenkins as popular as it is. The provide an easy way for people (often with little or no UI skills) to get going with Jenkins and create their first plugin that can contribute elements to the Jenkins UI. So, we shouldn't throw out the baby with the bathwater.
I suppose the best place to start wrt the newer methodologies we've been adopting is to take a look at the following docs:
  • jenkins-js-modules: A very simple JavaScript bundle loader tailored to the Jenkins "environment". The main reason I point to this is because the docs there try to explain the problems/challenges (+ there's a slide dec which people might find useful).
  • jenkins-js-builder: An NPM module to tries to simplify the process of working with this stuff. Some docs there too.
  • jenkins-js-test: No prices for guessing what this is about. Builds on Jasmine and jsdom, making them a bit easier to use together. But, people can always use whatever they want.
PLEASE note that we are not proposing to make any of this a mandatory part of learning or working with Jenkins. If you are happy with Stapler+Jelly and don't want to be bothered with any of this, then keep rock'n !! See this FAQ.

Tom Moore

unread,
Sep 25, 2015, 8:49:04 AM9/25/15
to Jenkins Developers
This is a great thread.   I do want to raise one point though.   This should be a "clean" repacking and try to avoid inclusion of plugin functionality in the install that may not be needed or desired.   Your mentioning of Git for example.  Jenkins is used by people who utilize CVS, SVN, Git, Bazaar, Visual Source Safe, Team Foundation Server, Perforce, etc, etc, etc.   So installation of Git as a default is not going to necessarily be something desired by all.    And none of these are even needed to run the base application for configuration.  

I'd prefer identifying the plugin "groups" we know people will nearly always use, such as a SCM, Authentication, or a Builder (Maven, MSBuild, Unity, etc) and the first time a Jenkins server is started and the UI logged into then the user is sent into a wizard to select the ones to download for the first time.   Obviously we wouldn't want to walk them through all 1100 plugins.  But I would think the three groups I mentioned would be pretty much a common decision for nearly all installations.



Kanstantsin Shautsou

unread,
Sep 25, 2015, 8:56:24 AM9/25/15
to jenkin...@googlegroups.com
>
> I'd prefer identifying the plugin "groups" we know people will nearly always use, such as a SCM, Authentication, or a Builder (Maven, MSBuild, Unity, etc) and the first time a Jenkins server is started and the UI logged into then the user is sent into a wizard to select the ones to download for the first time. Obviously we wouldn't want to walk them through all 1100 plugins. But I would think the three groups I mentioned would be pretty much a common decision for nearly all installations.
All projects that i know that started with “groups” (+ sub-groups) end with “tags”, because groups can’t be applied for all cases.
Soft dependencies with reverse dependencies can also provide better user experience. Is it possible to implement them?

Daniel Beck

unread,
Sep 25, 2015, 9:07:52 AM9/25/15
to jenkin...@googlegroups.com

On 25.09.2015, at 14:49, Tom Moore <tommoo...@gmail.com> wrote:

> This is a great thread. I do want to raise one point though. This should be a "clean" repacking and try to avoid inclusion of plugin functionality in the install that may not be needed or desired. Your mentioning of Git for example. Jenkins is used by people who utilize CVS, SVN, Git, Bazaar, Visual Source Safe, Team Foundation Server, Perforce, etc, etc, etc. So installation of Git as a default is not going to necessarily be something desired by all. And none of these are even needed to run the base application for configuration.
>
> I'd prefer identifying the plugin "groups" we know people will nearly always use, such as a SCM, Authentication, or a Builder (Maven, MSBuild, Unity, etc) and the first time a Jenkins server is started and the UI logged into then the user is sent into a wizard to select the ones to download for the first time. Obviously we wouldn't want to walk them through all 1100 plugins. But I would think the three groups I mentioned would be pretty much a common decision for nearly all installations.

This is actually something we're already discussing.

https://groups.google.com/d/msg/jenkinsci-dev/kRobm-cxFw8/AQyLTVDSHQAJ

The thread is a bit long, and started out completely differently, but may be worth reading.

Jesse Glick

unread,
Sep 25, 2015, 1:56:04 PM9/25/15
to Jenkins Dev
On Fri, Sep 25, 2015 at 12:53 AM, Kohsuke Kawaguchi <k...@kohsuke.org> wrote:
> Let's use the opportunity to update some of the libraries. I'm thinking
> about things like Groovy, which according to the testing done during
> Copenhagen Hackathon, should be compatible.

IMO rather than compound our error, do the right thing:
https://issues.jenkins-ci.org/browse/JENKINS-29068

Stephen Connolly

unread,
Sep 25, 2015, 4:26:05 PM9/25/15
to jenkin...@googlegroups.com
Yes yes yes. Let's kill groovy views! I nearly have handlebar views working
--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr2yZ%2Bfdy2h7UvHopdUpcUqC4zEM7jVumNoL3Ap-VwFpqg%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.


--
Sent from my phone

Michael Neale

unread,
Sep 25, 2015, 9:06:14 PM9/25/15
to Jenkins Developers
On Friday, September 25, 2015 at 8:37:50 PM UTC+10, Kanstantsin Shautsou wrote:

About CI->CD. 
As technical person i see no benefits as it's all just a part of "automation". No matter what automate: builds, deploys, flows, etc. Automation - is what saves your time and money in any field (even non-computer).

Yes, it is all automation, so a lot of this is messaging and presentation. In a practical sense for people like us there isn't a huge technical change necessarily (perhaps the biggest thing in CD is the participation of a wider range of people in a pipeline, vs in CI when it was a fairly technical only crowd). 

 

Kohsuke Kawaguchi

unread,
Sep 25, 2015, 9:24:26 PM9/25/15
to jenkin...@googlegroups.com
2015-09-24 23:20 GMT-07:00 Richard Bywater <ric...@byh2o.com>:
I'm not really qualified to talk to a lot of the things you mention, but I haven't seen in the email much in the form of things like visualising CD pipelines etc. Do these types of things exist today because I think to have the goal to have our place in the CD world things like that are needed. Quite a few people at my work don't think that Jenkins is particular good for CD and point in directions such as Thoughtworks Go which, of course, is now also in the open-source arena. If they don't exist they sound like features you wouldn't want to just whip up other night to keep people happy as first impressions count.

For me, workflow stage view is the piece for visualizing pipelines. As I wrote today that's a proprietary part of CloudBees Jenkins Enterprise, but CloudBees is committing to open-sourcing it to move the dial for Jenkins 2.0.
 
[We collectively came quite a long way. When I started Jenkins, having a server building & testing on every commit was a cutting-edge practice. So are automatically capturing changelogs and analysing test reports. But now, those are tablestakes, and the frontier of automation has moved further. Now we are talking about building pipelines & workflows, continuously deploying to servers, leveraging containers, and/or testing pull requests before they get merged. I'm going to call this much bigger entangled automation "Continuous Delivery", to contrast this with more classical automated build & test executions (aka "Continuous Integration") that we set out to do.]

I think in general, people are "Now we are *also* talking about" CD. In my mind efficient CD pipelines start with efficient CI techniques so I think its dangerous to imply that CI is no longer a thing and its all about CD. You can easily do CI without doing CD if you work in the more classical (read locked down) IT environments.

I think it's probably poor wording on my part. I didn't mean to imply that CI is no longer a thing. I think of CD as a super set of CI, in the sense that everything you do under the banner CI also contributes to CD.

And I think people get this. The main concern is that if we just push CI, people will not get the message that we do more than that.

[Domain name. It's kind of a problem that we have "ci" baked into our domain name jenkins-ci.org. We have acquired http://jenkins.cd/ How about we change the domain name? I think it sends another clear signal.]

Similar to my previous comment I think this is a dangerous move as it sounds like Jenkins is dropping CI in favour of CD. Also what happens when XYZ becomes the next big movement that Jenkins looks to add support for - would it change to http://jenkins.xyz? I think it would be a good idea to try and get a domain name that can stand the test of time - even if you then have the other domains forwarding through to it for catch-iness.

I think I understand where you are coming from. That is, some of us see CD as just another buzzword, so getting that incorporated into the domain name is repeating the same mistake of having CI in jenkins-ci.org in the first place. I see the logic in that.

In this domain name change proposal, the part that I have a stronger conviction of is that we need to drop the "-ci" part because it is creating an incorrect perception in the market that Jenkins is "just CI".  In the ideal world we should get http://jenkins.org/ but unfortunately that domain is just not available.

BTW, no disrespect intended for the word CI --- it lasted good 10 years, which cannot be said for many things in this industry. And I expect CD to last just as long. If we have to change the domain name every 10 years, is that unacceptable or is it OK?

 
[All the above things call for better infra that can handle this. Right now we have our web assets are split into Drupal and Wiki, but the former can be only touched by a few people and the latter is slow and klunky. I think this is the time to switch to some static site generator, so that everyone can contribute content through Git and pull requests, just like how we collaborate on plugins.]

I agree that having that mix doesn't work very well but I'm not sure it is really a requirement to dump everything that we've got today and jump to another completely different mechanism to which I'm not sure how many people would feel comfortable contributing to documentation. I (as an end user) see the main issue with Confluence as being the fact that a) perhaps the structure is not setup correctly / too much dead pages within and b) the version is SOOOO old - 3.4.7 was released 28th January *2011*! If you take a look at what Atlassian are now doing with (some) of their pages is that they are fronting the public web version with Scroll Viewport (https://marketplace.atlassian.com/plugins/com.k15t.scroll.scroll-viewport) which gives you the backend Wiki editing with the nice presentation.

I see some people taking documentation off Confluence and into a set of markdown files in GitHub repo (workflow being one example.) I take it as a signal that people are speaking with their feet. It's been also hard to sustain. It needs constant attention, it's slow, and we cannot control the presentation to the degree Drupal lets us to.

That said, a big part of the motivation for me is to make it easier for people to contribute. So if the static site generator fails in that regard, then that's not good. I'm on the same page with you about that.

I think all that it takes is to have the "Edit" button in the page that takes you back to the edit page in GitHub.

[Time to bump up the system requirement to Java 8 and Servlet 3.0. Let's think about what this would enable to users. Again, we talked about this a bit in FOSDEM.]

Not sure if this goal and the don't leave people sitting on 1.x goal go too well together (well the Java 8 part at least) especially given the time frames being talked about.

OK, that's a good feedback. Servlet 3.0 is 6 year old technology that came with JavaEE *6*, so I'd like to think that everyone already runs it. Java8 is newer and came out 2014, so perhaps I assumed incorrectly that everyone can run Java8 if they wanted to.

I admit that my putting Java8 there is motivated more by the fact that I as a developer wants to code in Java8, as opposed to "Java8 has these additions that translate into these Jenkins features that users would like" 


[Release (end Jan?)]

And on the subject of timeframes I worry that its quite tight to look at something as close as January for what seems like a big piece of work. Perhaps its not but best to not understate the time requirement as otherwise Jenkins becomes like some other projects that keep saying that "the next big version" is coming out soon

OK. I used it to emphasize that this needs to be limited in scope, that I don't want this to turn into a multi-year project. I think you are saying that having a concrete date like this will send other messages to people.

I think we all agree that it's very difficult to put a date to software project, especially when it's not even clear exactly what Jenkins 2.0 is going to be!

 
--
Kohsuke Kawaguchi

Kohsuke Kawaguchi

unread,
Sep 25, 2015, 9:29:59 PM9/25/15
to jenkin...@googlegroups.com
2015-09-24 23:58 GMT-07:00 domi <do...@fortysix.ch>:
As Richard, I also think that we would need some sort of a concept for a CD pipeline. I consider my self as a quite advanced jenkins user, but still I end up redesigning some sort of a CD pipeline with every new installation or even project from scratch with different plugins then I used before. CD is a great concept and sure it looks different for everyone - but people wanna have guidance and then they will be able to change it the way it works for them.
In my point of view Jenkins 2.0 would need some sort of a definition for it and give answers/proposals how to do these steps together (and visualise):
- build some sort of artifact (this stage might include unit tests)
- promote artifacts for integration tests 
- do a release build
- store the released artifacts (depending on the type also make it public available, e.g. for maven, npm, bower,…)
- promote released artifacts to test env 
- trigger tests on test env
- promote released artifacts to prod env 
(for sure the list is not complete and every step has to be plugable for different technologies, but you get the idea)

I know we have all the components/plugins to solve all these issues, but its hard for a newbe (and also experienced users) to really get this right. Workflow for sure is great to go for something like this, but I think we need something on top of it (maybe its just a good documentation) to really get people into this.

Yes, I totally agree. The other day I saw Tom showing his experiment to add ACE editor to Jenkins Workflow config UI (which also came with a drop-down to let you insert some templates.) I've also talked with Jesse and others a number of times about how we can generate GDSL and DSLD. Something like that is clearly needed, no question.

 
--
Kohsuke Kawaguchi

Kohsuke Kawaguchi

unread,
Sep 25, 2015, 9:35:36 PM9/25/15
to jenkin...@googlegroups.com
2015-09-25 5:03 GMT-07:00 Stephen Connolly <stephen.al...@gmail.com>:
w00t so if I read that correctly then we're dropping the evil job type
for Jenkins 2.0 ;-)

Just so that the rest of the world knows you are joking. No we aren't. We need to keep it running.

*unless the Maven project does a Maven 3.4.x after JDK9 is released
and before Jenkins 2.0 as then Maven 3.4.x will only be supported on
Java 9 and Java 9-1

Right, didn't Maven start splitting the JVM version in which Maven itself runs vs JVM that tests run and code gets compiled? Or is it not really a reality among users?
 

On 25 September 2015 at 12:22, Daniel Beck <m...@beckweb.net> wrote:
>
> On 25.09.2015, at 09:04, nicolas de loof <nicolas...@gmail.com> wrote:
>
>> The fact some didn't yet upgraded to 1.8 has been discussed already, we are talking here about Jenkins runtime, not the projects it builds and deploy, that can target Java 1.1 ...
>
> 1 in 8 Jenkins jobs are based on the Maven job type, whose minimum JDK version is tied to what can run Jenkins.
>
> --
> You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/F28E47EB-0F8C-4D9C-98E8-FB39BC13E853%40beckweb.net.
> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.



--
Kohsuke Kawaguchi

Kohsuke Kawaguchi

unread,
Sep 25, 2015, 9:41:04 PM9/25/15
to jenkin...@googlegroups.com
OK, that makes sense.

I'm not sure how to do init.groovy. That's a life saver for a small number of people, and it really doesn't work well as plugins.

Maybe we keep Groovy but hide those classes from plugins or something like that?

We need some means to keep track of tickets we are targeting for 2.0. I put the new label "2.0" on that one.

--
Kohsuke Kawaguchi

Tom Moore

unread,
Sep 26, 2015, 12:06:42 AM9/26/15
to jenkin...@googlegroups.com
On Fri, Sep 25, 2015 at 9:24 PM, Kohsuke Kawaguchi <k...@kohsuke.org> wrote:

I think it's probably poor wording on my part. I didn't mean to imply that CI is no longer a thing. I think of CD as a super set of CI, in the sense that everything you do under the banner CI also contributes to CD.

I think you're starting down a good track here but let me throw in a couple thoughts.   When we look at the Continuous processes in terms of the SDLC, there are three concepts that have been developed over the years.   The first is CI, which keeps pulling in developer changes as they are committed, builds and tests them, thereby finding integration problems as early as possible.   Then came the first CD,  Continuous Delivery, which picks up the results of the CI outputs and makes sure they are always in a deploy-able state (not necessarily actually delivering them), thereby hopefully finding deployment preparation problems as early as possible, and also always having the latest good build ready to deploy when its desired.   Then finally the second CD, Continuous Deployment.  This one takes the outputs that pass the Continuous Delivery processes and gates and based on the rules/processes set up, automatically deploys them to a target system.

I think that trying to think of Continuous Deployment as a superset of CI is actually trying to make Continuous Deployment into something its not.   Continuous Integration, Continuous Delivery, and Continuous Deployment are separate but conceptually similar processes which are complimentary to each other.  Historically when trying to view similar processes through the lens of one tends to cause the others to be diminished.      Instead of doing that here, I think what we need is a new conceptual model, one that encompasses all three processes, but has each as a separate phase or workflow.  I'm just not sure what we would call it.  Continuous Integration, Delivery and Deployment (CIDD) doesn't really work for me.  

Michael Neale

unread,
Sep 26, 2015, 2:52:16 AM9/26/15
to Jenkins Developers
In various blogs, books and conferences like citcon I have heard the term "continuous delivery" used to mean the overarching concepts that you mention needs a new name at the end. 

Like all terms, it can grow in meaning, and runs a risk of being meaningless, but it does seem to have settled down to something useful (a bit like "cloud" which has a pretty fuzzy definition, but overall seems to be useful). Perhaps saying "CD" as just the initials has this affect, and is "good enough" to convey the concepts you mention? 

Stephen Connolly

unread,
Sep 26, 2015, 3:04:58 AM9/26/15
to jenkin...@googlegroups.com


On Saturday 26 September 2015, Kohsuke Kawaguchi <k...@kohsuke.org> wrote:

2015-09-25 5:03 GMT-07:00 Stephen Connolly <stephen.al...@gmail.com>:
w00t so if I read that correctly then we're dropping the evil job type
for Jenkins 2.0 ;-)

Just so that the rest of the world knows you are joking. No we aren't. We need to keep it running.

Damn it. You pick *now* to detect my humour style of writing.... How will I be able to troll you any more!

(Yes I know we're not going to kill off the evil job type in the short term, but it's still on my list of things to do)
 

*unless the Maven project does a Maven 3.4.x after JDK9 is released
and before Jenkins 2.0 as then Maven 3.4.x will only be supported on
Java 9 and Java 9-1

Right, didn't Maven start splitting the JVM version in which Maven itself runs vs JVM that tests run and code gets compiled? Or is it not really a reality among users?

So yeah, we have toolchains, but even there we have started to limit how far back we support, eg recent versions of surefire will only fork back to Java 5.

Once we get the JDK up to 8 for Maven plugins, we'll likely only support forking back to Java 6 in surefire, 9 -> 7 (but that depends on 9 letting us compile with -target 1.7) 
 

On 25 September 2015 at 12:22, Daniel Beck <m...@beckweb.net> wrote:
>
> On 25.09.2015, at 09:04, nicolas de loof <nicolas...@gmail.com> wrote:
>
>> The fact some didn't yet upgraded to 1.8 has been discussed already, we are talking here about Jenkins runtime, not the projects it builds and deploy, that can target Java 1.1 ...
>
> 1 in 8 Jenkins jobs are based on the Maven job type, whose minimum JDK version is tied to what can run Jenkins.
>
> --
> You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/F28E47EB-0F8C-4D9C-98E8-FB39BC13E853%40beckweb.net.
> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CA%2BnPnMwi0pnFP9T6f%3DQLSgzefANX4i7evLjsn-cu8Dha6WtmdA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.



--
Kohsuke Kawaguchi

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAN4CQ4x5FmFZivYJu0%3Dro87xuGmfZgx022Ap9%3DSCHJkDUOSq2A%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.


--
Sent from my phone

nicolas de loof

unread,
Sep 26, 2015, 3:35:03 AM9/26/15
to jenkin...@googlegroups.com

[Domain name. It's kind of a problem that we have "ci" baked into our domain name jenkins-ci.org. We have acquired http://jenkins.cd/ How about we change the domain name? I think it sends another clear signal.]

Similar to my previous comment I think this is a dangerous move as it sounds like Jenkins is dropping CI in favour of CD. Also what happens when XYZ becomes the next big movement that Jenkins looks to add support for - would it change to http://jenkins.xyz? I think it would be a good idea to try and get a domain name that can stand the test of time - even if you then have the other domains forwarding through to it for catch-iness.

I think I understand where you are coming from. That is, some of us see CD as just another buzzword, so getting that incorporated into the domain name is repeating the same mistake of having CI in jenkins-ci.org in the first place. I see the logic in that.

In this domain name change proposal, the part that I have a stronger conviction of is that we need to drop the "-ci" part because it is creating an incorrect perception in the market that Jenkins is "just CI".  In the ideal world we should get http://jenkins.org/ but unfortunately that domain is just not available.


As jenkins.org do redirect to stevejenkins.com, dit you tried to contact registrant and propose to acquire the domain name ?

 

Tom Fennelly

unread,
Sep 26, 2015, 3:45:38 AM9/26/15
to Jenkins Developers
Yeah, "cd" in the domain name becomes obsolete as soon as "CX" becomes the latest buzz term (I'm sure Martin Fowler is brewing up whatever "X" is as we speak).

--
You received this message because you are subscribed to a topic in the Google Groups "Jenkins Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-dev/vbXK7JJekFw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANMVJzm33iOh3D2xU9mzbfcekVW9qHcFxvpr5%2BqKwgUd5Ab_EQ%40mail.gmail.com.

Michael Neale

unread,
Sep 26, 2015, 6:03:22 AM9/26/15
to Jenkins Developers
Although as mentioned by Kohsuke before, if it lasts only 10 years perhaps that is ok?

CX. That has a ring to it now ;)

Tom Moore

unread,
Sep 26, 2015, 8:40:49 AM9/26/15
to jenkin...@googlegroups.com
On Sat, Sep 26, 2015 at 2:52 AM, Michael Neale <michae...@gmail.com> wrote:

In various blogs, books and conferences like citcon I have heard the term "continuous delivery" used to mean the overarching concepts that you mention needs a new name at the end. 


Those blogs/books/conferences are doing exactly what I said we shouldn't.  Continuous Delivery comes after Continuous Integration in the lifecycle and is a separate process.  If we were to pick something that covers all of them, the only thing I've heard of is the IT term DevOps that is catching on in the IT (as opposed to sw dev) world.  But DevOps doesn't necessarily have to have continuous processes, and it typically is described as covering the whole lifecycle... hmm.... if IT DevOps is overarching and doesn't have to imply continuous processes, what do we think of Continuous DevOps   CDO?   Sure we aren't explicitly covering requirements and design *now* but I can think of primitive continuous processes to just publish the latest changes to those artifacts... and I know there are some automated validation techniques for them as well... so we *COULD* potentially have Jenkins controlling continuous cycles in every phase.
 

Michael Neale

unread,
Sep 26, 2015, 7:40:19 PM9/26/15
to jenkin...@googlegroups.com
My point it is already happening, hard to hold back the tide.
--
You received this message because you are subscribed to a topic in the Google Groups "Jenkins Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-dev/vbXK7JJekFw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jenkinsci-de...@googlegroups.com.

Tom Moore

unread,
Sep 26, 2015, 8:12:25 PM9/26/15
to jenkin...@googlegroups.com
its not a tide yet... just as many blogs, books and conferences use the term continuous deployment for the same meaning... and then others use devops.  The industry is searching for a common term.

martinda

unread,
Sep 27, 2015, 12:18:59 AM9/27/15
to Jenkins Developers
I am in year 3 of using Jenkins, and my users are complaining more about slow downs than before. As an example, the "more" link at the bottom of the build history takes too long to return for my users. I have 141 projects, 30,373 builds, and 1,073,591 junit XML test results. It took me over 20 minutes of CPU time to get these values by querying the file system.

Will Jenkins 2.0 have database backend?

Michael Neale

unread,
Sep 27, 2015, 5:22:26 AM9/27/15
to jenkin...@googlegroups.com
That's a good question. Backing by a database would be a big step, especially as some plugins expect certain things to be in certain places. Certainly a consideration for the future. Incidentally, do you use magnetic or SSD drives?

On Sun, 27 Sep 2015 at 2:19 PM, martinda <martin....@gmail.com> wrote:
I am in year 3 of using Jenkins, and my users are complaining more about slow downs than before. As an example, the "more" link at the bottom of the build history takes too long to return for my users. I have 141 projects, 30,373 builds, and 1,073,591 junit XML test results. It took me over 20 minutes of CPU time to get these values by querying the file system.

Will Jenkins 2.0 have database backend?

--
You received this message because you are subscribed to a topic in the Google Groups "Jenkins Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-dev/vbXK7JJekFw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jenkinsci-de...@googlegroups.com.

Baptiste Mathus

unread,
Sep 27, 2015, 9:31:24 AM9/27/15
to jenkin...@googlegroups.com

Isn't this issue just about the thing Tom started (finished?) working on some months ago? I.e. paginating the build details.

Because indeed, on a job with lots of build, you're gonna have a bad time if you click on details. (Might be already fixed, didn't check that specific point recently)

You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAKVMTi76nv3f%3DWFcPodYzS5PKTsfQkhvVp_uMSY2PS2aGHiuag%40mail.gmail.com.

Tom Fennelly

unread,
Sep 27, 2015, 10:57:35 AM9/27/15
to Jenkins Developers, m...@batmat.net
On Sunday, September 27, 2015 at 2:31:24 PM UTC+1, Baptiste Mathus wrote:

Isn't this issue just about the thing Tom started (finished?) working on some months ago? I.e. paginating the build details.

Because indeed, on a job with lots of build, you're gonna have a bad time if you click on details. (Might be already fixed, didn't check that specific point recently)



It's ready to be merged but I think Daniel wanted to remove some of the search features I added, leaving just the "fuzzy" search. Maybe others should declare their interest and whether they want the more advanced search or just the "fuzzy". A topic for another thread I suppose.

Jesse Glick

unread,
Sep 27, 2015, 12:01:04 PM9/27/15
to Jenkins Dev
On Sun, Sep 27, 2015 at 12:18 AM, martinda <martin....@gmail.com> wrote:
> I have 141 projects,
> 30,373 builds, and 1,073,591 junit XML test results. It took me over 20
> minutes of CPU time to get these values by querying the file system.

Yes, build storage is a well-known architectural problem. PR 1641 will
not help you here.

> Will Jenkins 2.0 have database backend?

No, the changes required are too big. There is no way to do it that
quickly. I think we *should* do these changes sooner or later. For me
this seems both more important and more disruptive than the code
changes currently being discussed for 2.0.

Nigel Magnay

unread,
Sep 28, 2015, 6:36:29 AM9/28/15
to jenkin...@googlegroups.com
Would 2.0 be an appropriate juncture to revisit the plugin architecture?

I've often gotten bitten by classloader issues in 2nd/3rd party dependencies (e.g: plugin wants newer version of guava than jenkins-core). Shading is a reasonably complicated thing to do for a plugin, and I wonder if something cleverer could be done - without disappearing down a (say) complicated solution like OSGi.

On Fri, Sep 25, 2015 at 5:53 AM, Kohsuke Kawaguchi <k...@kohsuke.org> wrote:
Jenkins is over 10 years old, and it came quite a long way. I still remember the first few plugins that I wrote by myself, and now we have close to 1100 plugins. What's started as a hobby project that had run under my desk today boasts more than 100K installations driving half a million-ish build machines.

We collectively came quite a long way. When I started Jenkins, having a server building & testing on every commit was a cutting-edge practice. So are automatically capturing changelogs and analysing test reports. But now, those are tablestakes, and the frontier of automation has moved further. Now we are talking about building pipelines & workflows, continuously deploying to servers, leveraging containers, and/or testing pull requests before they get merged. I'm going to call this much bigger entangled automation "Continuous Delivery", to contrast this with more classical automated build & test executions (aka "Continuous Integration") that we set out to do.

The other thing I'd like to point out is that the adoption of Jenkins continues to grow at the incredible pace of 30% year over year. That is, a lot of people are starting new with Jenkins, and they are looking for us to help them get Continuous Delivery done. Therefore, this is a good time to step back and think about whether we are addressing those current user demands.


For example, despite this advance during the last 10 years and 1000+ plugins we've created, messaging in our website hasn't changed much since the first version I wrote on java.net. It spends more space talking about JNLP and zero mention of Git, pipeline, or Docker.

The fresh installation of Jenkins is not much better. The CVS support is available out of the box, but Git isn't. All the cool stuff that the community has done and its collective best practices still need be learned by each and every new user. It's bit like we are making everyone assemble LEGO blocks. That's not doing enough justice to the 30K+ users that will be joining us in this year.

So I propose we do Jenkins 2.0 to fix this.

There are three important goals that I see in Jenkins 2.0.
  1. We need to claim our rightful place in Continuous Delivery. We have lots of pieces that address these modern needs, but we are not communicating this very well.

  2. We need to revisit out of the box experience, so that Jenkins itself speaks that story and makes it clear what we are aiming for. Our software needs to speak for itself and tell people where we are going, so that the community can better focus efforts on the important parts.

  3. And we need to do this while keeping what makes Jenkins great in the first place, which are the ecosystem, the community, and the extensibility that recognizes that one size does not fit all and let people do what they want to do.

Incrementing the major version sends a clear message to people that we are moving forward. That's why I think 2.0 is appropriate for this effort.



Now, 2.0 can mean a lot of different things to a lot of people, so let me outline what I think we should do and we shouldn't do.

It's very important for me to make sure that my fellow Jenkins developers understand the motivation and the goal of this proposal, because that drives much of what we should and shouldn't do. So instead of deep-diving into technical parts, please take time to try to understand why I am proposing this.

We need to contain the scope. 2.0 has to have enough in it to justify the major version number increase, but it creates a period of pause and uncertainty, so it cannot keep dragging on for too long. 2.0 cannot be everything everyone ever wanted.

We cannot do massively disruptive 2.0, because it ends up splitting the community. If users perceive that the upgrade to 2.0 is risky, enough of them will stay behind with 1.x, plugin authors would want to continue supporting them, which makes 1.x more liveable, which makes the transition slower. I do not want to end up in Python2/3 situation, and nobody wins.

That means we cannot be really breaking plugins. We cannot do s/hudson/jenkins/g in the package names because doing that will break all the plugins. 2.0 does come with the expectation that it is more disruptive than usual 1.630 to 1.631 upgrade, so we have some "disruption budget", but we have to use it really wisely.

Simiarly, for me it is an absolute requirement that we keep people's $JENKINS_HOME functioning. A lot of sweat, tear, and blood went into those right set of plugins and elaborate job configurations. When users upgrade to 2.0, they need to continue to work, or else Jenkins 2.0 will be Jenkins in just the name only.

Therefore, we cannot make massive internal changes. In many ways, it has to be evolutionary instead of revolutionary, when it comes to the code. This is not a "let's redo everything from scratch" kind of 2.0. In any case, I think it's a pitfall to focus too much on internals. We all have a long list of things we want to fix and the technical debt that we want to pay down. My cautionary tale here is that of Maven 2 to Maven 3 upgrade. The developers of the project spent a lot of efforts redoing all the plumbings. Plexus gave way to Guice, and the dependency resolution engine got completely rewritten. Then to keep plugins working, more efforts were spent on building the backward compatibility layer. After something like 18 months, Maven 3 came out, which did more or less the same thing as far as users are concerned. I'm sure I'm over-simplifying this, but you get the point.



So given all that constraints, I think 2.0 should have the following 3 major pillars:
  • Messaging changes, to make sure people coming into the Continuous Delivery space will get that Jenkins does what they want.
  • Software that backs up our messages. Out of the box experience that caters to Continuous Delivery needs.
  • Targeted internal plumbing changes that enable those goals
I have some concrete ideas in each of these pillars, and I'll describe them below. But I also need help from everyone to come up with, discuss, and decide what other things will advance those pillars.

Messaging:
    • Domain name. It's kind of a problem that we have "ci" baked into our domain name jenkins-ci.org. We have acquired http://jenkins.cd/ How about we change the domain name? I think it sends another clear signal.

    • We need more up-to-date feature list page (like http://arquillian.org/features/) that talks about things that matter to the modern users.

    • We need authoritative and curated getting started guide that expands on the things listed in the features page and help people understand those features, so that we have clearly marked trails.

    • This is probably out of scope for the initial 2.0 launch, but in the future we want to redo the plugin listing page as well. This is a persistent feedback that we hear from users.

    • All the above things call for better infra that can handle this. Right now we have our web assets are split into Drupal and Wiki, but the former can be only touched by a few people and the latter is slow and klunky. I think this is the time to switch to some static site generator, so that everyone can contribute content through Git and pull requests, just like how we collaborate on plugins.

      Out of the Box Experience:
      • This work is already in progress, but we really need some initial setup wizard. We can use it to install plugins so that new instances come up more useful from get-go --- things like git, workflow, pipeline as code, folders, and so on. These plugins together tell the story of how we want users to use Jenkins.

      • Another work that's already under way is the UX improvement, specifically the config form re-layout. This is the kind of change that helps people (literally) see that 2.0 is different. UX in general is clearly one of the places we should spend our precious disruption budget for.

      • To reinforce the message that workflow is the future, CloudBees is going to open-source our workflow stage view plugin that was previously a part of CloudBees Jenkins Enterprise.

      Internals:
      • Let's define a policy to remove APIs after they are deprecated. We have talked about this in FOSDEM, and this could be as easy as "N releases after deprecation". Feedbacks from users at the San Jose JAM was that things like this is OK, but we need to help people identify plugins that will be impacted to give them earlier warnings.

      • As a part of the UX rebump effort, Tom et al has been working on a brand-new way of doing frontend in Jenkins plugins. His JUC talk has some materials. Given that user experience is a major theme in 2.0, I think this internal plumbing change makes sense.

      • Let's use the opportunity to update some of the libraries. I'm thinking about things like Groovy, which according to the testing done during Copenhagen Hackathon, should be compatible. This shouldn't include updates that are known to be compatibility breaking, such as Acegi Security to Spring Security (which involves package name changes.)

      • Time to bump up the system requirement to Java 8 and Servlet 3.0. Let's think about what this would enable to users. Again, we talked about this a bit in FOSDEM.

        Finally, timeline-wise, my aspirational timeline is as follows, though obviously this is largely dependent on feedback to the proposal:
        • Announce the proposal publicly and have discussions to nail the details (Sep-Oct)
        • Execution (Oct-Dec)
          • Periodic alpha/beta releases to solicit feedbacks from users
          • PR activities
          • This phase concludes with the release candidate
        • Plugin sweep to ensure key plugins are "2.0 ready". This is the opportunity to find issues (Jan 2016)
        • Release (end Jan?)
        • Drop 1.x development as soon as possible to focus on 2.x. 

        There are a lot of things I haven't captured, but this email is aleady getting too long. Looking forward to having more conversations about this with everyone.

        --
        Kohsuke Kawaguchi

        --
        You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
        To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

        Olivier Lamy

        unread,
        Sep 28, 2015, 7:57:33 AM9/28/15
        to jenkin...@googlegroups.com
        Hi,
        In the same part, make writing plugin a bit more easier :-) could help to attract more dev.


        Olivier


        For more options, visit https://groups.google.com/d/optout.

        Michael Neale

        unread,
        Sep 28, 2015, 7:59:43 AM9/28/15
        to jenkin...@googlegroups.com
        Is the real problem that core dependencies get too old too quickly? (As opposed to some tech that allows multiple versions of things to get loaded?)

        You received this message because you are subscribed to a topic in the Google Groups "Jenkins Developers" group.
        To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-dev/vbXK7JJekFw/unsubscribe.
        To unsubscribe from this group and all its topics, send an email to jenkinsci-de...@googlegroups.com.
        To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAPYP83Rx3Nt2ecARsqez1D%2BQ_wQbTfneSHq5cm9UQhfQ9Am3kA%40mail.gmail.com.

        nicolas de loof

        unread,
        Sep 28, 2015, 8:42:19 AM9/28/15
        to jenkin...@googlegroups.com
        if you consider 10 years of jenkins development, I don't think the issue is for dependencies to become old "too quickly", but the fact we hardly can change them without potential risk to break some plugin.

        I'd like we find some technical way to isolate core classes implementation so they aren't accessible by plugins, then could exclude core dependencies from plugin classpath, and ensure those one explicitly declare dependencies they rely one.




        Baptiste Mathus

        unread,
        Sep 28, 2015, 8:47:07 AM9/28/15
        to jenkin...@googlegroups.com
        Didn't have a in-depth look into it yet, but isn't that kind of isolation what's Java9's Jigsaw supposed to bring to the game btw? 


        For more options, visit https://groups.google.com/d/optout.



        --
        Baptiste <Batmat> MATHUS - http://batmat.net
        Sauvez un arbre,
        Mangez un castor !

        nicolas de loof

        unread,
        Sep 28, 2015, 8:52:28 AM9/28/15
        to jenkin...@googlegroups.com
        yes, supposed to, but not sure how and if this could apply to jenkins. Also would then require Java 9, and we already hardly can move to 8 ...

        Stephen Connolly

        unread,
        Sep 28, 2015, 8:56:02 AM9/28/15
        to jenkin...@googlegroups.com
        IIRC Jigsaw has been reduced in scope to just within the JRE... but
        that may have changed again since
        > https://groups.google.com/d/msgid/jenkinsci-dev/CANWgJS74qrB_DK81JthYdhXsOoc%3Dq4se7CcuJk6yB2%2Bf5STOyw%40mail.gmail.com.

        Arnaud Héritier

        unread,
        Sep 28, 2015, 10:02:42 AM9/28/15
        to jenkin...@googlegroups.com
        On Mon, Sep 28, 2015 at 2:41 PM, nicolas de loof <nicolas...@gmail.com> wrote:
        if you consider 10 years of jenkins development, I don't think the issue is for dependencies to become old "too quickly", but the fact we hardly can change them without potential risk to break some plugin.


        and a part of the issue is probably our lack of tests. 
        Because it is really difficult to automate integration tests with all 3rd party systems on which Jenkins is integrated to...


         

        For more options, visit https://groups.google.com/d/optout.



        --
        -----
        Arnaud Héritier
        Mail/GTalk: aheritier AT gmail DOT com
        Twitter/Skype : aheritier

        Jesse Glick

        unread,
        Sep 28, 2015, 12:51:19 PM9/28/15
        to Jenkins Dev
        On Mon, Sep 28, 2015 at 6:36 AM, Nigel Magnay <nigel....@gmail.com> wrote:
        > Would 2.0 be an appropriate juncture to revisit the plugin architecture?
        >
        > I've often gotten bitten by classloader issues in 2nd/3rd party dependencies
        > (e.g: plugin wants newer version of guava than jenkins-core). Shading is a
        > reasonably complicated thing to do for a plugin

        I do not think we can make fundamental changes, but we could certainly
        try to hide such libraries from the plugin class loader (or, in some
        cases, split the core code relying on them into plugins), so that
        plugins could use simple unshaded dependencies, for example on
        `org.jenkins-ci.libs:guava12:12.0-1`.

        Stephen Connolly

        unread,
        Sep 28, 2015, 2:01:39 PM9/28/15
        to jenkin...@googlegroups.com
        I think we could hide them from plugins modulo perhaps those compiled
        against older cores... or perhaps we could treat the older core
        plugins as being like the other de-bundled functionality
        > --
        > You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
        > To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
        > To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr0-3EpWTizZ2EgbVBqm4MBG6%3DZTUceGfTtb%2BZMwQNdrFQ%40mail.gmail.com.

        Kanstantsin Shautsou

        unread,
        Sep 28, 2015, 2:37:42 PM9/28/15
        to Jenkins Developers
        Isn't it possible to make some flag in hpi meta-inf that will switch classloading modes for plugin?

        nicolas de loof

        unread,
        Sep 28, 2015, 3:03:04 PM9/28/15
        to jenkin...@googlegroups.com
        sure, could rely on MANIFEST's Jenkins-Version >= 2.x

        Jesse Glick

        unread,
        Sep 28, 2015, 4:14:37 PM9/28/15
        to Jenkins Dev
        On Mon, Sep 28, 2015 at 2:01 PM, Stephen Connolly
        <stephen.al...@gmail.com> wrote:
        > perhaps we could treat the older core plugins as being like the other de-bundled functionality

        Yes, this was my thought. Just as a plugin declaring a dependency on
        an older core gets an implicit dependency on any plugins subsequently
        split out, so too would it get an implicit dependency on any libraries
        subsequently made into plugins. Indeed there is little difference from
        the perspective of the plugin with the (possible) dependency whether
        that dependency happened to be defined in
        `org.jenkins-ci.main:jenkins-core`, some `jenkins-module`, or some
        random `WEB-INF/lib/*.jar`.

        Oleg Nenashev

        unread,
        Sep 28, 2015, 4:56:05 PM9/28/15
        to Jenkins Developers
        What about aggregating all the proposals in a new Jenkins JIRA project?

        Seems this thread becomes barely readable due to dozens of proposals. A JIRA project would also allow to vote for issues and to categorize them (infra/code, breaking/non-breaking, etc.)

        понедельник, 28 сентября 2015 г., 23:14:37 UTC+3 пользователь Jesse Glick написал:

        Daniel Beck

        unread,
        Sep 28, 2015, 5:31:13 PM9/28/15
        to jenkin...@googlegroups.com
        My proposal is that anyone suggesting a change for Jenkins 2.0 (both new ideas and old) should label them in JIRA as "jenkins-2.0". We'll then be able to discuss them and decide which make sense. A new project isn't necessary, and this would allow nominating older issues for this.

        WDY all T?
        > --
        > You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
        > To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
        > To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/061aa4a5-ef88-4b5c-bd0d-37010e2d9bb7%40googlegroups.com.

        Oleg Nenashev

        unread,
        Sep 28, 2015, 5:51:00 PM9/28/15
        to JenkinsCI Developers
        +1. We can also setup some global filters for such purpose.
        INFRA project also fits well for the infrastructure stuff being discussed here

        You received this message because you are subscribed to a topic in the Google Groups "Jenkins Developers" group.
        To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-dev/vbXK7JJekFw/unsubscribe.
        To unsubscribe from this group and all its topics, send an email to jenkinsci-de...@googlegroups.com.
        To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/B4338EB6-0D6F-4701-AD59-FC5C92B4DE54%40beckweb.net.

        Jesse Glick

        unread,
        Sep 28, 2015, 6:03:29 PM9/28/15
        to Jenkins Dev
        On Mon, Sep 28, 2015 at 5:30 PM, Daniel Beck <m...@beckweb.net> wrote:
        > anyone suggesting a change for Jenkins 2.0 (both new ideas and old) should label them in JIRA as "jenkins-2.0".

        KK already started a `2.0` label.

        Daniel Beck

        unread,
        Sep 28, 2015, 6:25:41 PM9/28/15
        to jenkin...@googlegroups.com
        Right, thanks for the reminder.

        Michael Neale

        unread,
        Sep 28, 2015, 7:37:05 PM9/28/15
        to jenkin...@googlegroups.com
        That would be this label? 

        (currently only with one issue under it?)

        --
        You received this message because you are subscribed to a topic in the Google Groups "Jenkins Developers" group.
        To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-dev/vbXK7JJekFw/unsubscribe.
        To unsubscribe from this group and all its topics, send an email to jenkinsci-de...@googlegroups.com.
        To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/D7546C53-275C-4445-B967-94C44B421915%40beckweb.net.

        Daniel Beck

        unread,
        Sep 28, 2015, 7:47:42 PM9/28/15
        to jenkin...@googlegroups.com
        The URL is https://issues.jenkins-ci.org/issues/?jql=labels%20%3D%202.0

        Not surprising there are no issues yet, as it's just a weekend old. Go forth and nominate things for 2.0 inclusion :-)
        > You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
        > To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
        > To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAKVMTi4qvJ1Goicnd5hSHRoTeCddpS7n0R9HQ0C1spitZH3u8g%40mail.gmail.com.

        Kohsuke Kawaguchi

        unread,
        Sep 28, 2015, 10:40:43 PM9/28/15
        to jenkin...@googlegroups.com
        OK, let's find out just to cover all the bases, but I'm not holding my breath. That guy has 58K followers on Twitter, which to me suggests he's probably got a good traffic.

        And it's not like we can spend a lot of money on this...

        2015-09-26 0:34 GMT-07:00 nicolas de loof <nicolas...@gmail.com>:

        [Domain name. It's kind of a problem that we have "ci" baked into our domain name jenkins-ci.org. We have acquired http://jenkins.cd/ How about we change the domain name? I think it sends another clear signal.]

        Similar to my previous comment I think this is a dangerous move as it sounds like Jenkins is dropping CI in favour of CD. Also what happens when XYZ becomes the next big movement that Jenkins looks to add support for - would it change to http://jenkins.xyz? I think it would be a good idea to try and get a domain name that can stand the test of time - even if you then have the other domains forwarding through to it for catch-iness.

        I think I understand where you are coming from. That is, some of us see CD as just another buzzword, so getting that incorporated into the domain name is repeating the same mistake of having CI in jenkins-ci.org in the first place. I see the logic in that.

        In this domain name change proposal, the part that I have a stronger conviction of is that we need to drop the "-ci" part because it is creating an incorrect perception in the market that Jenkins is "just CI".  In the ideal world we should get http://jenkins.org/ but unfortunately that domain is just not available.


        As jenkins.org do redirect to stevejenkins.com, dit you tried to contact registrant and propose to acquire the domain name ?

         

        --
        You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
        To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

        For more options, visit https://groups.google.com/d/optout.



        --
        Kohsuke Kawaguchi

        Kohsuke Kawaguchi

        unread,
        Sep 28, 2015, 10:43:51 PM9/28/15
        to jenkin...@googlegroups.com
        Yeah, I tend to agree. It is an important issue, but as Jesse says, I'm afraid it's too disruptive than all the boundaries that we operate in.

        We should still take it on, but it shouldn't be tied to Jenkins 2.0.

        --
        Kohsuke Kawaguchi

        Kohsuke Kawaguchi

        unread,
        Sep 28, 2015, 10:46:56 PM9/28/15
        to jenkin...@googlegroups.com
        Do you have some specifics in mind?


        For more options, visit https://groups.google.com/d/optout.



        --
        Kohsuke Kawaguchi

        Kohsuke Kawaguchi

        unread,
        Sep 28, 2015, 10:48:06 PM9/28/15
        to jenkin...@googlegroups.com
        Can you file this as a ticket (or maybe you already know the existing ticket) and mark it for 2.0?

        --
        You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
        To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

        For more options, visit https://groups.google.com/d/optout.



        --
        Kohsuke Kawaguchi

        Tom Fennelly

        unread,
        Sep 29, 2015, 10:37:24 AM9/29/15
        to Jenkins Developers
        The new Plugin Install Wizard is an example of the sort of UX improvement we could target for a 2.0 release. See this short video of the current progress there: https://youtu.be/9pq5tHm4nWs

        It is using the modularised JavaScript approaches we talked about earlier in this thread

        Antonio Muñiz

        unread,
        Sep 29, 2015, 2:34:04 PM9/29/15
        to jenkin...@googlegroups.com
        There are a lot of backend related technical stuff to be improved in
        2.0, and we should go for them even breaking some backward
        compatibility, but not too much, I mean, having to completely rewrite
        plugins is a clear no-go from my point of view, but if some plugins
        require some adjustments to work in 2.0 then it would be ok.

        But what I really would like to see in 2.0 is a UI change (just in the
        way shown by Tom in his screencast). Jenkins (with the huge plugins
        base) already has all the mechanics to do CI/CD really well, but the
        UI is sometimes hard to use. Probably the current Jelly approach must
        be kept just to ensure a good level of backward compatibility, but
        there is a lot of things that could be done directly in core that
        would really improve the UX, and (as in backend code) some breakages
        should be accepted (and fixed in plugins).

        IMO the most important part of 2.0 should be the UX improvement.
        > --
        > You received this message because you are subscribed to the Google Groups
        > "Jenkins Developers" group.
        > To unsubscribe from this group and stop receiving emails from it, send an
        > email to jenkinsci-de...@googlegroups.com.
        > To view this discussion on the web visit
        > https://groups.google.com/d/msgid/jenkinsci-dev/e23aac54-ae6e-48b1-9cfd-434b6d30fa1e%40googlegroups.com.
        >
        > For more options, visit https://groups.google.com/d/optout.



        --
        Antonio Muñiz
        Software Engineer
        CloudBees, Inc.

        Artur Szostak

        unread,
        Sep 29, 2015, 3:15:06 PM9/29/15
        to jenkin...@googlegroups.com
        My 2 cents...

        >>> I have 141 projects,
        >>> 30,373 builds, and 1,073,591 junit XML test results. It took me over 20
        >>> minutes of CPU time to get these values by querying the file system.
        >>
        >> Yes, build storage is a well-known architectural problem. PR 1641 will
        >> not help you here.

        Just to add another data point that might be interesting. I am running a large Jenkins instance (version 1.565.1) with 123 different plugins installed. It contains, 1,021 projects (711 are matrix jobs) with 26,040 builds (454,575 matrix sub-job builds) and 14,988,971 test results. The build results take up about 1.8TB disk space.
        The Jenkins master runs on a 24 core machine, with 64GB RAM, 2.5TB RAID 10 storage (~ 200 MB/sec peak I/O). The build slaves are run on secondary machines.

        The startup time for this Jenkins master is about 40-60min. Taking the raw I/O performance on this system into account, the theoretical absolute lower limit should be just under 6min.
        Reading all my test XML reports directly from the filesystem took about 31min.
        For comparison, accessing these same results through the REST API takes about 45min.


        >>> Will Jenkins 2.0 have database backend?
        >>
        >> No, the changes required are too big. There is no way to do it that
        >> quickly. I think we *should* do these changes sooner or later. For me
        >> this seems both more important and more disruptive than the code
        >> changes currently being discussed for 2.0.
        >
        > Yeah, I tend to agree. It is an important issue, but as Jesse says,
        > I'm afraid it's too disruptive than all the boundaries that we operate in.
        >
        > We should still take it on, but it shouldn't be tied to Jenkins 2.0.

        Not knowing much about the internals of Jenkins, my suspicion is that most plugins deal with I/O by themselves. If this is indeed the case, then in my experience it is usually bad for performance to let everyone do their own thing for I/O. I would then suggest as a first step to enforce plugins to use a unified I/O interface.
        Later it would be much easier to redirect that to a DB backend.
        Perhaps this idea has already been raised by the core developers. :-)

        Kind regards.

        Artur

        Jesse Glick

        unread,
        Sep 29, 2015, 6:06:03 PM9/29/15
        to Jenkins Dev
        On Tue, Sep 29, 2015 at 3:14 PM, Artur Szostak <aszo...@partner.eso.org> wrote:
        > I would then suggest as a first step to enforce plugins to use a unified I/O interface.

        Yes well this is the hard part. :-)

        Michael Neale

        unread,
        Sep 29, 2015, 6:09:47 PM9/29/15
        to Jenkins Dev
        It would be great to have such huge workspaces as part of a startup benchmark test suite (startup time is mentioned increasingly, and probably highlights a lot of issues for improvement).

        That is one big workspace to profile on!

        (Even 6 minutes is a long time, clearly eagerly loading things is not desired)
        --
        You received this message because you are subscribed to a topic in the Google Groups "Jenkins Developers" group.
        To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-dev/vbXK7JJekFw/unsubscribe.
        To unsubscribe from this group and all its topics, send an email to jenkinsci-de...@googlegroups.com.
        To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr3d9TFBDUeBQ%3DmewV5y02Ap0cxywnCvkUjcoq3z5b%3D1eg%40mail.gmail.com.

        Surya Gaddipati

        unread,
        Sep 29, 2015, 7:59:12 PM9/29/15
        to Jenkins Developers
        Great post Kohsuke. Thank you for creating Jenkins. 

        >Time to bump up the system requirement to Java 8 and Servlet 3.0. Let's think about what this would enable to users. 

        I've been using Java 8 for serverside rendering of react fontend for dotci using Nashorn. It has been working beautifully. 

        Servlet 3.0 would be great for doing push to browser updates instead of constant polling that I have to do now. This has been a terrible for scaling our jenkins setup. 



        It would also be nice to move wiki pages to github readme's. Not really important but just a suggestion.

        Very excited to see this post. Will respond more once i get to my computer.

        Surya

        Olivier Lamy

        unread,
        Sep 29, 2015, 9:41:18 PM9/29/15
        to jenkin...@googlegroups.com
        Oh That's really impressive!! 
        I think the javascript approaches is a good answer to attract more people to develop plugins (especially ui part)

        Olivier

        On 30 September 2015 at 00:37, Tom Fennelly <tom.fe...@gmail.com> wrote:
        The new Plugin Install Wizard is an example of the sort of UX improvement we could target for a 2.0 release. See this short video of the current progress there: https://youtu.be/9pq5tHm4nWs

        It is using the modularised JavaScript approaches we talked about earlier in this thread

        --
        You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
        To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
        To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/e23aac54-ae6e-48b1-9cfd-434b6d30fa1e%40googlegroups.com.

        For more options, visit https://groups.google.com/d/optout.

        R. Tyler Croy

        unread,
        Sep 29, 2015, 10:45:01 PM9/29/15
        to jenkin...@googlegroups.com
        (replies inline)

        On Tue, 29 Sep 2015, Surya Gaddipati wrote:

        > It would also be nice to move wiki pages to github readme's. Not really
        > important but just a suggestion.

        I've got a very love/hate relationship with Confluence (I love to hate it :))
        but the one useful aspect of having centralized documentation is ease of
        linking and search indexing!

        Perhaps what would be useful for plugin developers would be to write their
        README's and then incorporate some tooling to populate a wiki page with that
        data upon release?

        I.e. a wiki page might be:

        {{pluginheadermacrothing}}

        {{github-readme-macro}}

        other-custom-content.



        Just thinking out loud, please don't consider this as me volunteering to write
        Confluence plugins! :)
        > > 1. We need to claim our rightful place in Continuous Delivery. We have
        > > lots of pieces that address these modern needs, but we are not
        > > communicating this very well.
        > >
        > > 2. We need to revisit out of the box experience, so that Jenkins
        > > itself speaks that story and makes it clear what we are aiming for. Our
        > > software needs to speak for itself and tell people where we are going, so
        > > that the community can better focus efforts on the important parts.
        > >
        > > 3. And we need to do this while keeping what makes Jenkins great in
        > > - Messaging changes, to make sure people coming into the Continuous
        > > Delivery space will get that Jenkins does what they want.
        > > - Software that backs up our messages. Out of the box experience that
        > > caters to Continuous Delivery needs.
        > > - Targeted internal plumbing changes that enable those goals
        > >
        > > I have some concrete ideas in each of these pillars, and I'll describe
        > > them below. But I also need help from everyone to come up with, discuss,
        > > and decide what other things will advance those pillars.
        > >
        > > Messaging:
        > >
        > > - Domain name. It's kind of a problem that we have "ci" baked into our
        > > domain name jenkins-ci.org. We have acquired http://jenkins.cd/ How
        > > about we change the domain name? I think it sends another clear signal.
        > >
        > > - We need more up-to-date feature list page (like
        > > http://arquillian.org/features/) that talks about things that matter
        > > to the modern users.
        > >
        > > - We need authoritative and curated getting started guide that expands
        > > on the things listed in the features page and help people understand those
        > > features, so that we have clearly marked trails.
        > >
        > > - This is probably out of scope for the initial 2.0 launch, but in the
        > > future we want to redo the plugin listing page as well. This is a
        > > persistent feedback that we hear from users.
        > >
        > > - All the above things call for better infra that can handle this.
        > > Right now we have our web assets are split into Drupal and Wiki, but the
        > > former can be only touched by a few people and the latter is slow and
        > > klunky. I think this is the time to switch to some static site generator,
        > > so that everyone can contribute content through Git and pull requests, just
        > > like how we collaborate on plugins.
        > >
        > >
        > > Out of the Box Experience:
        > >
        > > - This work is already in progress, but we really need some initial
        > > setup wizard. We can use it to install plugins so that new instances come
        > > up more useful from get-go --- things like git, workflow, pipeline as code,
        > > folders, and so on. These plugins together tell the story of how we want
        > > users to use Jenkins.
        > >
        > > - Another work that's already under way is the UX improvement,
        > > specifically the config form re-layout. This is the kind of change that
        > > helps people (literally) see that 2.0 is different. UX in general is
        > > clearly one of the places we should spend our precious disruption budget
        > > for.
        > >
        > > - To reinforce the message that workflow is the future, CloudBees is
        > > going to open-source our workflow stage view plugin that was previously a
        > > part of CloudBees Jenkins Enterprise.
        > >
        > >
        > > Internals:
        > >
        > > - Let's define a policy to remove APIs after they are deprecated. We
        > > have talked about this in FOSDEM, and this could be as easy as "N releases
        > > after deprecation". Feedbacks from users at the San Jose JAM was that
        > > things like this is OK, but we need to help people identify plugins that
        > > will be impacted to give them earlier warnings.
        > >
        > > - As a part of the UX rebump effort, Tom et al has been working on a
        > > brand-new way of doing frontend in Jenkins plugins. His JUC talk has some
        > > materials. Given that user experience is a major theme in 2.0, I think this
        > > internal plumbing change makes sense.
        > >
        > > - Let's use the opportunity to update some of the libraries. I'm
        > > thinking about things like Groovy, which according to the testing done
        > > during Copenhagen Hackathon, should be compatible. This shouldn't include
        > > updates that are known to be compatibility breaking, such as Acegi Security
        > > to Spring Security (which involves package name changes.)
        > >
        > > - Time to bump up the system requirement to Java 8 and Servlet 3.0.
        > > Let's think about what this would enable to users. Again, we talked about
        > > this a bit in FOSDEM.
        > >
        > >
        > > Finally, timeline-wise, my aspirational timeline is as follows, though
        > > obviously this is largely dependent on feedback to the proposal:
        > >
        > > - Announce the proposal publicly and have discussions to nail the
        > > details (Sep-Oct)
        > > - Execution (Oct-Dec)
        > > - Periodic alpha/beta releases to solicit feedbacks from users
        > > - PR activities
        > > - This phase concludes with the release candidate
        > > - Plugin sweep to ensure key plugins are "2.0 ready". This is the
        > > opportunity to find issues (Jan 2016)
        > > - Release (end Jan?)
        > > - Drop 1.x development as soon as possible to focus on 2.x.
        > >
        > >
        > > There are a lot of things I haven't captured, but this email is aleady
        > > getting too long. Looking forward to having more conversations about this
        > > with everyone.
        > >
        > > --
        > > Kohsuke Kawaguchi
        > >
        >
        > --
        > You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
        > To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
        > To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/53d66e8a-888f-4434-9c6d-04dc97d550e1%40googlegroups.com.
        > For more options, visit https://groups.google.com/d/optout.


        - R. Tyler Croy

        ------------------------------------------------------
        Code: <https://github.com/rtyler>
        Chatter: <https://twitter.com/agentdero>

        % gpg --keyserver keys.gnupg.net --recv-key 3F51E16F
        ------------------------------------------------------
        signature.asc

        Tom Fennelly

        unread,
        Sep 30, 2015, 4:04:07 AM9/30/15
        to Jenkins Developers
        From a UI perspective, another thing I would like to see baked into a 2.0 is push notifications to the browser so we can get rid of the polling that happens in many places (e.g. the build history). It would need to be bi-directional (websockets). Something based on SSE/Long-poll would work.


        --
        You received this message because you are subscribed to a topic in the Google Groups "Jenkins Developers" group.
        To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-dev/vbXK7JJekFw/unsubscribe.
        To unsubscribe from this group and all its topics, send an email to jenkinsci-de...@googlegroups.com.
        To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/20150930024339.GH2353%40blackberry.coupleofllamas.com.

        Tom Fennelly

        unread,
        Sep 30, 2015, 4:25:31 AM9/30/15
        to Jenkins Developers
        On Wednesday, September 30, 2015 at 9:04:07 AM UTC+1, Tom Fennelly wrote:
        From a UI perspective, another thing I would like to see baked into a 2.0 is push notifications to the browser so we can get rid of the polling that happens in many places (e.g. the build history). It would need to be bi-directional (websockets). Something based on SSE/Long-poll would work.

        Sorry .... typo there .... I meant to say "... it would NOT need to be bidirectional ..."

        Michael Neale

        unread,
        Sep 30, 2015, 4:30:01 AM9/30/15
        to jenkin...@googlegroups.com
        On Wed, Sep 30, 2015 at 9:59 AM Surya Gaddipati <suryap...@gmail.com> wrote:
        Great post Kohsuke. Thank you for creating Jenkins. 

        >Time to bump up the system requirement to Java 8 and Servlet 3.0. Let's think about what this would enable to users. 

        I've been using Java 8 for serverside rendering of react fontend for dotci using Nashorn. It has been working beautifully. 

        Oh that is quite interesting. By react you mean react from facebook? 

        Tom Fennelly

        unread,
        Sep 30, 2015, 5:42:38 AM9/30/15
        to Jenkins Developers
        On 30 September 2015 at 09:29, Michael Neale <michae...@gmail.com> wrote:


        On Wed, Sep 30, 2015 at 9:59 AM Surya Gaddipati <suryap...@gmail.com> wrote:
        Great post Kohsuke. Thank you for creating Jenkins. 

        >Time to bump up the system requirement to Java 8 and Servlet 3.0. Let's think about what this would enable to users. 

        I've been using Java 8 for serverside rendering of react fontend for dotci using Nashorn. It has been working beautifully. 

        Oh that is quite interesting. By react you mean react from facebook? 


        We purposely stayed away from using the likes of React/Polymer etc in the install wizard. We feel it would be too "controversial" to use in Jenkins Core. However, I think they could be very interesting options in a plugin, so making it easy to use these would be a great thing I think.

        Tom Fennelly

        unread,
        Sep 30, 2015, 5:46:53 AM9/30/15
        to Jenkins Developers
        On 30 September 2015 at 00:59, Surya Gaddipati <suryap...@gmail.com> wrote:
        Servlet 3.0 would be great for doing push to browser updates instead of constant polling that I have to do now. This has been a terrible for scaling our jenkins setup. 

        Sorry I missed this Surya .... +1000 I think adding nice clean APIs for doing push notifications (as opposed to every plugin etc opening a long poll /event-stream request to Jenkins etc) would be a huge step forward. It would
        1. greatly simplify the programming model
        2. better ux
        3. take load off the backend

         

        James Nord

        unread,
        Sep 30, 2015, 6:31:49 PM9/30/15
        to Jenkins Developers
        OT: if you can buy some ssd disks and raid them so you can get space and redundancy you will probalg see an order of magnitude difference. I had start up times of 3 hours which fell to 3 minutes... As for tests have you tuned the jvm memory settings? If not then the results get garbage collected whilst loading other results that are needed which keads to the results being loaded again which leads to.... And eventually enough loads and gcs pass that it works again.
        Apart from artifact/log archival/retreival most of the io in jenkins will be loading small files and random Iops is king not sequential bandwidth. YMMV standard disclaimers apply

        Andrew Bayer

        unread,
        Oct 1, 2015, 5:24:08 AM10/1/15
        to jenkin...@googlegroups.com
        Some thoughts...

        jenkins.cd is a bad idea. It has a weird feel to it - I've never seen another .cd domain. It's more narrow and marketing-specific than jenkins-ci.org, as has been said. Honestly, it feels more like a startup domain than an OSS project domain. If we can get jenkins.org or another of the more common TLDs, sure, that's an improvement, but jenkins.cd just doesn't feel right.

        I'm not comfortable with the timeframe - this is our best opportunity to make major architectural changes to core for a long time, and we should really take advantage of that. Let's have real discussions about breaking compatibility in some places, re-doing the backend storage (I strongly feel we should move core's storage (and quasi-core plugins like junit-plugin, Maven/matrix job types, etc) to an abstraction layer - so that it becomes possible to move builds/tests/etc to be stored in something other than XML), etc. 

        Given that I think 2.0 should be more radical a change than proposed, I also think we should keep the 1.x LTS line going for like a year - just bug fixes, of course, but we'd really need to have a path for people to take time to move off of removed functionality or plugins that won't work any more in 2.0, etc, and keeping up the LTS track is a necessary part of that.


        A.

        On Thu, Oct 1, 2015 at 12:31 AM, James Nord <james...@gmail.com> wrote:
        OT: if you can buy some ssd disks and raid them so you can get space and redundancy you will probalg see an order of magnitude difference.  I had start up times of 3 hours which fell to 3 minutes...   As for tests have you tuned the jvm memory settings? If not then the results get garbage collected whilst loading other results that are needed which keads to the results being loaded again which leads to.... And eventually enough loads and gcs pass that it works again.
        Apart from artifact/log archival/retreival most of the io in jenkins will be loading small files and random Iops is king not sequential bandwidth.  YMMV standard disclaimers apply
        --
        You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
        To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

        Michael Neale

        unread,
        Oct 1, 2015, 6:00:54 AM10/1/15
        to jenkin...@googlegroups.com
        yes picking a JS framework seems like picking a Loser these days - no matter what you do, you will pick wrong ;) Longevity isn't great. Doing as much with plain old js (possibly with jquery given it isn't going anywhere) is probably a good baseline, but it would be great to see plugins take advantage of the modularity and explore what can be done with react. 



        --
        You received this message because you are subscribed to a topic in the Google Groups "Jenkins Developers" group.
        To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-dev/vbXK7JJekFw/unsubscribe.
        To unsubscribe from this group and all its topics, send an email to jenkinsci-de...@googlegroups.com.
        To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CA%2BbPaoLXGC85XTg8yXdfQWxQ9C_2LznskYJB_om0vBKot6FNsg%40mail.gmail.com.

        Michael Neale

        unread,
        Oct 1, 2015, 6:05:56 AM10/1/15
        to jenkin...@googlegroups.com
        I sometimes feel the same way about unfamiliar TLDs, but perhaps it is us internet old timers that balk at non .org or .com things? 

        You received this message because you are subscribed to a topic in the Google Groups "Jenkins Developers" group.
        To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-dev/vbXK7JJekFw/unsubscribe.
        To unsubscribe from this group and all its topics, send an email to jenkinsci-de...@googlegroups.com.
        To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAPbPdOYtrnhrkZORCTzGxgRrMhydS%3DzEKRAS8FffPaK_24Ln2A%40mail.gmail.com.

        Kaj Kandler

        unread,
        Oct 1, 2015, 9:57:48 PM10/1/15
        to Jenkins Developers
        Go for it! Those 4 digit minor numbers are getting a bit long :-)

        I'd just go for the UX/UI theme as 2.0. Not that all other things are desirable too, but just the visual part could be a big boon and some disruption for the plugin community as well. That said, is UI going to be important going forward with DSL and Workflow?

        I also think documentation improvements could be really helpful. 
        * For example I could not find a definition of the general Queue mechanism and discipline, only references in plugins that change it. But nothing authoritative.
        * Similar, the documentation of extension points and API's is gravely lacking (what is getBuildVariables() supposed to do? Can it call getEnvironment()? Not if it wants to interoperate with a later build of the SVN plugin, which calls getBuildVariables() in the getEnvironment - poof stack overflow - who's fault is it? Oh, why does getBuildVariables() not have any access to a logger?)

        I personally love Wikis (at least those that accept Wiki syntax, which Atlassian messes up with its newer versions and standard Wysiwyg editors that break everything).

        Kohsuke Kawaguchi

        unread,
        Oct 2, 2015, 3:02:04 PM10/2/15
        to jenkin...@googlegroups.com
        Yep. I think there was a conversation a year ago or so about this, and SSE felt like a great match.


        For more options, visit https://groups.google.com/d/optout.



        --
        Kohsuke Kawaguchi

        Kohsuke Kawaguchi

        unread,
        Oct 2, 2015, 4:21:57 PM10/2/15
        to jenkin...@googlegroups.com
        2015-10-01 2:23 GMT-07:00 Andrew Bayer <andrew...@gmail.com>:
        Some thoughts...

        jenkins.cd is a bad idea. It has a weird feel to it - I've never seen another .cd domain. It's more narrow and marketing-specific than jenkins-ci.org, as has been said. Honestly, it feels more like a startup domain than an OSS project domain. If we can get jenkins.org or another of the more common TLDs, sure, that's an improvement, but jenkins.cd just doesn't feel right.

        I don't want to get into the CI vs CD definition argument, but I think CD is less narrow than CI.

        Initially I wasn't very keen on the new domain name myself, but over the time it growed on me and I started seeing the point of it. Name is important, and domain name is one of the first things that people see.

        I'm not comfortable with the timeframe - this is our best opportunity to make major architectural changes to core for a long time, and we should really take advantage of that. Let's have real discussions about breaking compatibility in some places, re-doing the backend storage (I strongly feel we should move core's storage (and quasi-core plugins like junit-plugin, Maven/matrix job types, etc) to an abstraction layer - so that it becomes possible to move builds/tests/etc to be stored in something other than XML), etc. 

        I assume you mean storing XML in something other than a file system.

        Given that I think 2.0 should be more radical a change than proposed, I also think we should keep the 1.x LTS line going for like a year - just bug fixes, of course, but we'd really need to have a path for people to take time to move off of removed functionality or plugins that won't work any more in 2.0, etc, and keeping up the LTS track is a necessary part of that.

        I just don't see how this can work out. A fundamentally incompatible 2.0 that takes a lot longer to do, which breaks massive number of plugins. And then we let those people stay on Jenkins 1.x longer. It's going to significantly delay the launch of 2.0, then delays the migration further out.


        I'm not saying storage pluggability is a bad idea, it's the opposite. It's just that with all things considered, I don't think it's the best move to spend our precious resources on it at the moment. As I wrote, this is trying to cater to people who are newly coming to Jenkins, which is big in numbers. And for those who are running huge deployments, what eBay/Paypal/CloudBees/etc are doing --- lots of small masters in an elastic environment instead of one giant monolithic master --- makes a lot of sense, and addresses # of other related problems that those guys face.

        But you aren't the first one to point out about the storage pluggability. So maybe there's some version of it that can be done reasonably that's also compatible over time. That is, we could still require one directory per build, but at least store build records and job configurations elsewhere to help startup time and build history traversal. Kind of like what DotCi does.


        A.

        On Thu, Oct 1, 2015 at 12:31 AM, James Nord <james...@gmail.com> wrote:
        OT: if you can buy some ssd disks and raid them so you can get space and redundancy you will probalg see an order of magnitude difference.  I had start up times of 3 hours which fell to 3 minutes...   As for tests have you tuned the jvm memory settings? If not then the results get garbage collected whilst loading other results that are needed which keads to the results being loaded again which leads to.... And eventually enough loads and gcs pass that it works again.
        Apart from artifact/log archival/retreival most of the io in jenkins will be loading small files and random Iops is king not sequential bandwidth.  YMMV standard disclaimers apply

        --
        You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
        To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
        To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/7f434dd9-a7c6-4477-ac46-43bf34dbb8b4%40googlegroups.com.
        For more options, visit https://groups.google.com/d/optout.

        --
        You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
        To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

        For more options, visit https://groups.google.com/d/optout.



        --
        Kohsuke Kawaguchi

        Kohsuke Kawaguchi

        unread,
        Oct 2, 2015, 4:29:28 PM10/2/15
        to jenkin...@googlegroups.com
        2015-10-01 18:57 GMT-07:00 Kaj Kandler <kajka...@conficio.com>:
        Go for it! Those 4 digit minor numbers are getting a bit long :-)

        I'd just go for the UX/UI theme as 2.0. Not that all other things are desirable too, but just the visual part could be a big boon and some disruption for the plugin community as well. That said, is UI going to be important going forward with DSL and Workflow?

        Yes, because we still want you to come to Jenkins to find out what's happening. Just that you won't have to go there to configure.

        I also think documentation improvements could be really helpful. 
        * For example I could not find a definition of the general Queue mechanism and discipline, only references in plugins that change it. But nothing authoritative.
        * Similar, the documentation of extension points and API's is gravely lacking (what is getBuildVariables() supposed to do? Can it call getEnvironment()? Not if it wants to interoperate with a later build of the SVN plugin, which calls getBuildVariables() in the getEnvironment - poof stack overflow - who's fault is it? Oh, why does getBuildVariables() not have any access to a logger?)

        OK. Build variables vs environment is a confusing one indeed.
         
        I personally love Wikis (at least those that accept Wiki syntax, which Atlassian messes up with its newer versions and standard Wysiwyg editors that break everything).

        So my take is that you like textual syntax, which is what we are proposing wrt static site.

        --
        Kohsuke Kawaguchi

        Kanstantsin Shautsou

        unread,
        Oct 2, 2015, 9:39:48 PM10/2/15
        to Jenkins Developers
        What about jenkins build process itself? Will workflow finally allow you make it in automated way and provide core logs?

        Imho .cd is bad domain for jenkins because we already have this inconvenient jenkins-ci and switching to dot cd will confuse people: aurally jenkins-cd? without dot org? Just dot cd? Better send me link...
        It seems that there are not so many domains for choice. All short generic domains already used jenkins.io, org, .com. Though it seems that jenkins.house, jenkins.is, jenkins.online, jenkins.site are available (and even jenkins.sexy, jenkins.sucks, jenkins.wtf)

        Leif Gruenwoldt

        unread,
        Oct 2, 2015, 11:15:57 PM10/2/15
        to Jenkins Developers
        Hmmm, you mentioned it's "It's kind of a problem that we have "ci" baked into our domain name". However obviously both Travis CI and Circle CI (and maybe others?) use the term "CI" in their titles and domains, but that hasn't stopped them from including great support for continuous deployment in their products. 

        I've never known anyone to not choose Jenkins just because they thought it was limited to CI. 

        With that said, I do agree that the Jenkins website could describe the product better and it'd be a good idea to include the the term continuous deployment there.

        Two cents.

        Andrew Bayer

        unread,
        Oct 5, 2015, 6:11:26 AM10/5/15
        to jenkin...@googlegroups.com
        On Fri, Oct 2, 2015 at 10:21 PM, Kohsuke Kawaguchi <k...@kohsuke.org> wrote:

        I don't want to get into the CI vs CD definition argument, but I think CD is less narrow than CI.

        Initially I wasn't very keen on the new domain name myself, but over the time it growed on me and I started seeing the point of it. Name is important, and domain name is one of the first things that people see.

        I'm still not persuaded. =)
         

        I assume you mean storing XML in something other than a file system.

        Or theoretically serializing to something other than XML - that'd be tougher, obviously, but not *much* harder.
         

        I just don't see how this can work out. A fundamentally incompatible 2.0 that takes a lot longer to do, which breaks massive number of plugins. And then we let those people stay on Jenkins 1.x longer. It's going to significantly delay the launch of 2.0, then delays the migration further out.

        I'm not advocating incompatibility for the sake of incompatibility - just saying that this is our opportunity to make significant architectural changes, so we should at least consider them.
         

        I'm not saying storage pluggability is a bad idea, it's the opposite. It's just that with all things considered, I don't think it's the best move to spend our precious resources on it at the moment. As I wrote, this is trying to cater to people who are newly coming to Jenkins, which is big in numbers. And for those who are running huge deployments, what eBay/Paypal/CloudBees/etc are doing --- lots of small masters in an elastic environment instead of one giant monolithic master --- makes a lot of sense, and addresses # of other related problems that those guys face.

        Yeah. A bunch of things are just not really viable with the current storage - HA most notably, etc. 


        But you aren't the first one to point out about the storage pluggability. So maybe there's some version of it that can be done reasonably that's also compatible over time. That is, we could still require one directory per build, but at least store build records and job configurations elsewhere to help startup time and build history traversal. Kind of like what DotCi does.

        Yeah, that's pretty much exactly what I'm hoping for.

        A.

        Baptiste Mathus

        unread,
        Oct 5, 2015, 7:20:15 AM10/5/15
        to jenkin...@googlegroups.com
        2015-10-05 12:10 GMT+02:00 Andrew Bayer <andrew...@gmail.com>:
        On Fri, Oct 2, 2015 at 10:21 PM, Kohsuke Kawaguchi <k...@kohsuke.org> wrote:

        I don't want to get into the CI vs CD definition argument, but I think CD is less narrow than CI.

        Initially I wasn't very keen on the new domain name myself, but over the time it growed on me and I started seeing the point of it. Name is important, and domain name is one of the first things that people see.

        I'm still not persuaded. =)

        Personally still trying to decide if it's visionnary or weird ;-). 
        But indeed, jenkins.cd is also growing on me. Maybe others system will just follow The Jenkins Way like they all struggle to do currently ;-). 

         

        I assume you mean storing XML in something other than a file system.

        Or theoretically serializing to something other than XML - that'd be tougher, obviously, but not *much* harder.
         

        I just don't see how this can work out. A fundamentally incompatible 2.0 that takes a lot longer to do, which breaks massive number of plugins. And then we let those people stay on Jenkins 1.x longer. It's going to significantly delay the launch of 2.0, then delays the migration further out.

        I'm not advocating incompatibility for the sake of incompatibility - just saying that this is our opportunity to make significant architectural changes, so we should at least consider them.
         

        I'm not saying storage pluggability is a bad idea, it's the opposite. It's just that with all things considered, I don't think it's the best move to spend our precious resources on it at the moment. As I wrote, this is trying to cater to people who are newly coming to Jenkins, which is big in numbers. And for those who are running huge deployments, what eBay/Paypal/CloudBees/etc are doing --- lots of small masters in an elastic environment instead of one giant monolithic master --- makes a lot of sense, and addresses # of other related problems that those guys face.

        Yeah. A bunch of things are just not really viable with the current storage - HA most notably, etc. 


        But you aren't the first one to point out about the storage pluggability. So maybe there's some version of it that can be done reasonably that's also compatible over time. That is, we could still require one directory per build, but at least store build records and job configurations elsewhere to help startup time and build history traversal. Kind of like what DotCi does.

        Yeah, that's pretty much exactly what I'm hoping for.


        2.0 may also be the right time to try and incorporate some of that Google's feedback. 

        -- Baptiste

        martinda

        unread,
        Oct 6, 2015, 8:19:21 AM10/6/15
        to Jenkins Developers, m...@batmat.net
        Definitely not fixed in LTS 1.609.3. It is so bad my browser becomes inoperative as soon as I click "More..." in the build history, and I have to restart it. My users are complaining loud about this. Build history pagination would be an awesome necessity. If there is a Jira I have not found it yet, could you let me know? Thanks.

        On Sunday, September 27, 2015 at 9:31:24 AM UTC-4, Baptiste Mathus wrote:

        Isn't this issue just about the thing Tom started (finished?) working on some months ago? I.e. paginating the build details.

        Because indeed, on a job with lots of build, you're gonna have a bad time if you click on details. (Might be already fixed, didn't check that specific point recently)


        Nigel Magnay

        unread,
        Oct 6, 2015, 8:31:44 AM10/6/15
        to jenkin...@googlegroups.com

        Personally still trying to decide if it's visionnary or weird ;-). 
        But indeed, jenkins.cd is also growing on me. Maybe others system will just follow The Jenkins Way like they all struggle to do currently ;-). 


        ​Until continuous-X becomes the 'new hotness'.




         

        Daniel Beck

        unread,
        Oct 6, 2015, 8:44:29 AM10/6/15
        to jenkin...@googlegroups.com

        On 06.10.2015, at 14:19, martinda <martin....@gmail.com> wrote:

        > Definitely not fixed in LTS 1.609.3. It is so bad my browser becomes inoperative as soon as I click "More..." in the build history, and I have to restart it. My users are complaining loud about this. Build history pagination would be an awesome necessity. If there is a Jira I have not found it yet, could you let me know? Thanks.

        PR 1641 aims to resolve this. It should be in the LTS line after 1.625.x.

        That said, could we please keep this discussion on topic? The known problem in this case is the JavaScript performance since the widget was rewritten around ~1.57x/1.58x more than any reading files from disk (if you have benchmarks indicating the opposite, please let us know!). And adding pagination is neither a major new functionality nor particularly disruptive, so can be done any time.

        Andrew Bayer

        unread,
        Oct 6, 2015, 11:54:25 AM10/6/15
        to jenkin...@googlegroups.com
        So in re: backend storage - a possible pilot suggestion - unit test results. That's something that'd be really handy to be able to query/report on outside of Jenkins, and is fairly self contained...

        A.

        --
        You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
        To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

        Surya Gaddipati

        unread,
        Oct 6, 2015, 1:22:39 PM10/6/15
        to Jenkins Developers
        >Oh that is quite interesting. By react you mean react from facebook? 

        Yep. Found it really easy to do server side rendering with Nashorn.

         
         >We purposely stayed away from using the likes of React/Polymer etc in the install wizard. We feel it would be too "controversial" to use in Jenkins Core.  

        don't want to start a flamewar here and hijack this thread. Just curious what the controversy is ?  

        I think 2.0 UI should be totally Javascript with server side and client side rendering.


        Surya Gaddipati

        unread,
        Oct 6, 2015, 2:18:55 PM10/6/15
        to Jenkins Developers
        Regarding backend solution. We use DotCi and store all builds/logs in mongodb.( Still experimenting how to properly store logs in the db, but we have it working on staging). 

        The only things on disks are plugins and folder config.xml ( because of this issue https://github.com/jenkinsci/jenkins/pull/1762, jenkins deletes anything not on disk from memory ). I need to spend time to fix the issue in jenkins properly. 


        Once that is done. We can have things like, deploying on heroku , true load balancing with multiple masters. 

        One more thing that is preventing from from jenkins from being used in any serious installations is heavily thread locked Queue implementation. We are having to do strange workarounds with our jenkins because it threadlocks under even medium loads ( I saw this mentioned in google's slides for JUC too, curious what their solution was ) . 

        An extension point for queue would be great so we can store queue in redis or something.


        Surya

        Richard Bywater

        unread,
        Oct 6, 2015, 4:02:42 PM10/6/15
        to jenkin...@googlegroups.com
        That would be a good compromise on the whole discussion. Unless someone has an issue with aligning us as a build system? :)

        Richard.

        --
        You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
        To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

        Michael Neale

        unread,
        Oct 6, 2015, 6:25:38 PM10/6/15
        to jenkin...@googlegroups.com
        On "controversy", I think it may be due to the longevity of stuff in Jenkins being far beyond what most popular JavaScript frameworks lifetime appears to be (slight exaggeration). Jquery is a notable exception. Perhaps react will be another (I know lots of people do get really good results with it). Whatever is used, will need to stick around for a long time.

        --
        You received this message because you are subscribed to a topic in the Google Groups "Jenkins Developers" group.
        To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-dev/vbXK7JJekFw/unsubscribe.
        To unsubscribe from this group and all its topics, send an email to jenkinsci-de...@googlegroups.com.
        To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/4855a7f7-fd5f-4e81-9703-72b9818a72cd%40googlegroups.com.

        Michael Neale

        unread,
        Oct 6, 2015, 6:26:48 PM10/6/15
        to jenkin...@googlegroups.com
        Surya, in your case has being DB backed helped load times? (If you can say)

        Surya Gaddipati

        unread,
        Oct 6, 2015, 10:49:58 PM10/6/15
        to Jenkins Developers
        >Surya, in your case has being DB backed helped load times? (If you can say)

         Yes, definitely.  That was the original impetus behind moving stuff into the database. We went from > 45 mins load time to < 1 min. 

        Michael Neale

        unread,
        Oct 6, 2015, 11:03:16 PM10/6/15
        to jenkin...@googlegroups.com
        Remarkable.

        On Wed, 7 Oct 2015 at 1:50 PM, Surya Gaddipati <suryap...@gmail.com> wrote:
        >Surya, in your case has being DB backed helped load times? (If you can say)

         Yes, definitely.  That was the original impetus behind moving stuff into the database. We went from > 45 mins load time to < 1 min. 

        --
        You received this message because you are subscribed to a topic in the Google Groups "Jenkins Developers" group.
        To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-dev/vbXK7JJekFw/unsubscribe.
        To unsubscribe from this group and all its topics, send an email to jenkinsci-de...@googlegroups.com.

        Andrew Bayer

        unread,
        Oct 7, 2015, 8:35:30 AM10/7/15
        to jenkin...@googlegroups.com
        Yeah, I'm not shocked - directory traversal + disk i/o is nowhere near as efficient as pretty much any database ever written. =)

        I'm really excited about the kind of analytics that could be done with build data in a database backend - even if it's just serialized XML and not document-native to the DB.

        A.

        --
        You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
        To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
        To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAKVMTi63wrax1ewP1v9DC3N7sGifg0RG%2BuetE5x3dSTcw5t7kg%40mail.gmail.com.

        Kohsuke Kawaguchi

        unread,
        Oct 7, 2015, 1:39:43 PM10/7/15
        to jenkin...@googlegroups.com
        Right, I remember looking at DotCi and thinking that the way it moved the storage to mongodb points toward an abstraction we can build.

        Similar hook already exists for artifacts, and then we can provide auxiliary BLOB store for plugins that write random bits of data under builds, jobs, etc.

        Over time we can move things one by one to the BLOB store like that, then at that point we have filesystem free Jenkins.


        --
        You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
        To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

        For more options, visit https://groups.google.com/d/optout.



        --
        Kohsuke Kawaguchi

        Andrew Bayer

        unread,
        Oct 7, 2015, 3:57:22 PM10/7/15
        to jenkin...@googlegroups.com
        So looking back over the thread - my concerns have pretty much all been about missed opportunities to address stability and performance issues that may require more work in core than we normally do in a release (not necessarily compatibility-breaking ones). In the 7 or so years I've been involved with the project, there've been many times when we've said "Well, maybe we'll do that in Jenkins 2.0" so seeing 2.0 coming without the opportunity to make those sorts of changes takes a bit to get used to. =)

        But! The branding/messaging stuff has value - well, most of it - I'll fight to the death against jenkins.cd. =) I know I've been a really big fan of curated plugin packs or something along those lines to help users get up and running, and the UI framework definitely needs a complete refresh. Yeah, my dreams of Jenkins 2.0 going out the door with a drastically revamped storage backend and better support for analytics aren't going to happen - but that doesn't mean they *won't* happen in 2.20, 2.30, etc. The plugin compatibility testing is going to be something hugely useful for making significant changes to the core in the future - if we can get a good regression testing framework that covers a broad array of plugin use cases, we can be more liberal in changing core in the future. 

        2.0 isn't going to be what I've always thought 2.0 would be. It's not going to fit the semantic versioning definition of a major version change. But 2.0 is still a good way to label the kind of UX/on boarding changes that are so central to what Kohsuke's proposing. Let's take this as an opportunity to figure out how to do planning for significant changes and new features in core that take time to deliver - this time, it's user experience, look and feel, and out of the box that are the main focus, but perhaps when 2.0 is out the door, we will focus another couple months of work on storage and memory utilization, or plugin cleanup, or...

        A.

        Baptiste Mathus

        unread,
        Oct 7, 2015, 4:05:41 PM10/7/15
        to jenkin...@googlegroups.com
        Another idea I'll dump here. it's still a bit fluffy, but anyway.

        I think that with the config-management/devops trend, it would be a good thing that Jenkins can really be configured from scratch through (most simple possible) CLI/API calls. It's currently possible but requires a quite high-level knowledge of Jenkins internals to achieve (groovy scripts, and so on).

        For the jobs part, IMO things like the Job DSL Plugin to handle job versioning and durable management are great. And that would be great to have an equivalent for the instance. 
        Something like a high-level language (say DSL) to describe the server configuration.config itself.

        Maybe this is not something necessarily breaking things, hence not necessarily related to 2.0, but perhaps some contract/interface could be introduced in plugins to kind of standardize that discovery and be able to offer a standard API to configure Jenkins?

        The area to handle off the top of my head:
        * Core configuration (slaves, tools...)
        * Plugins to install (see also https://github.com/jenkinsci/docker/blob/master/README.md#installing-more-tools for reference/example)
        * Plugins config
        * Jobs

        My 2 cents


        For more options, visit https://groups.google.com/d/optout.



        --
        Baptiste <Batmat> MATHUS - http://batmat.net
        Sauvez un arbre,
        Mangez un castor !

        Kanstantsin Shautsou

        unread,
        Oct 7, 2015, 4:16:30 PM10/7/15
        to jenkin...@googlegroups.com
        Fully agree. 

        Atm Jenkins 2.0 sound like 2.0 for users and not 2.0 for developers.

        --
        You received this message because you are subscribed to a topic in the Google Groups "Jenkins Developers" group.
        To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-dev/vbXK7JJekFw/unsubscribe.
        To unsubscribe from this group and all its topics, send an email to jenkinsci-de...@googlegroups.com.
        To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAPbPdOZaTbv6LQqPvDV56tWSxbX1Au%3DBC2uBcAwc0cxqL17OpA%40mail.gmail.com.
        It is loading more messages.
        0 new messages