[2.0] slipping date

236 views
Skip to first unread message

Kohsuke Kawaguchi

unread,
Mar 28, 2016, 8:45:07 PM3/28/16
to jenkin...@googlegroups.com, jenkins...@googlegroups.com
In today's 2.0 status check call, Daniel Beck as 2.0 release manager had raised that we still have too many issues in flight and it's unrealistic to expect that we can fix them all in RC. So we decided to add one more week till RC.

Then we realized that this is exactly the same thing that happened with beta. So it looks like it'd be a reasonable estimate that RC to release is going to need 2 weeks, too. That way we don't have to make a fool of ourselves by slipping it once again two weeks from now.

That brings the updated time line to ...
  • April 6: Create first release candidate (RC)
  • April 20: Create 2.0 proper release with last RC
  • April 26: More coordinated launch marketing activities commence
For the up-to-date information, please check the 2.0 Wiki page. My apologies for any inconvenience this may cause to anyone.

--
Kohsuke Kawaguchi

Kohsuke Kawaguchi

unread,
Mar 28, 2016, 8:54:36 PM3/28/16
to jenkin...@googlegroups.com
(For dev list...)

This impacts the branching & 1.x release freeze plan announced by Daniel earlier, too.
  • April 3: 1.656 release, which will be the last 1.x release
  • April 6: RC.
    master will be merged into 2.0. Ideally we're not going to merge crazy stuff into 'master' between now and next Wednesday to keep the risk of the 2.0 RC being terrible low. Any unmerged pull requests against '2.0' will be closed, and '2.0' will be merged into 'master'. This means that from that point on, anything happening on the 'master' branch will be released as 2.1, as it makes no sense to continue releasing 1.x weeklies while 2.0 code is frozen and we're preparing the release.
  • April 10: weekly release will be skipped
  • Apirl 17: weekly release will be skipped
  • April 20: 2.0 GA
  • April 26: More coordinated launch marketing activities commence
  • May 1:  2.1 and weekly release resumes

Also, we agreed that "Major" and above fixes are only allowed until March 30, "Critical" and above are allowed only until RC, and only "Blocker" past that point, so that we do march toward the conversion and prevent the extra time from getting used on random fixes that end up destabilzing.

Thanks for focusing the effort around 2.0, especially those of you who are working on fixes that are targeted for 2.0. Others, please continue to kick the tire and report problems you find!

--
Kohsuke Kawaguchi

Kanstantsin Shautsou

unread,
Apr 4, 2016, 8:06:32 PM4/4/16
to Jenkins Developers
So, everybody would have 1.x without fixes for 2 months and 2.x that would be unstable? 

Daniel Beck

unread,
Apr 4, 2016, 8:30:19 PM4/4/16
to jenkin...@googlegroups.com

On 05.04.2016, at 02:06, Kanstantsin Shautsou <kanstan...@gmail.com> wrote:

> 1.x without fixes for 2 months and 2.x that would be unstable?

Not sure what you're referring to. As you can see in the plan posted by Kohsuke, we'll skip just two weekly releases, something we did around New Year's without notable issues, and before then in November. I can't remember a single substantial complaint. "We don't get to upgrade Jenkins often enough" is a complaint I don't expect to hear any time soon.

And the reason we're skipping a few weeklies is to ensure as much as we can that 2.0 will not be a terrible release because it pulls in a broken change from master at the last minute. AFAICT it gets much more testing than any weekly release of Jenkins in recent history, probably even LTS releases.

And if there's actually a critical bug in 1.656 (too recent to tell from feedback), we'll find a way to post a 1.657 with a bug fix for it.

Daniel Beck

unread,
Apr 6, 2016, 8:36:35 PM4/6/16
to jenkin...@googlegroups.com

On 29.03.2016, at 02:54, Kohsuke Kawaguchi <k...@kohsuke.org> wrote:

> • April 6: RC.
> master will be merged into 2.0. Ideally we're not going to merge crazy stuff into 'master' between now and next Wednesday to keep the risk of the 2.0 RC being terrible low. Any unmerged pull requests against '2.0' will be closed, and '2.0' will be merged into 'master'. This means that from that point on, anything happening on the 'master' branch will be released as 2.1, as it makes no sense to continue releasing 1.x weeklies while 2.0 code is frozen and we're preparing the release.

Hi everyone,

KK is starting to release the 2.0 RC now.

Given the late hour, I will merge 2.0 into master tomorrow.

Daniel

Daniel Beck

unread,
Apr 7, 2016, 6:06:56 AM4/7/16
to jenkin...@googlegroups.com

On 07.04.2016, at 02:37, Daniel Beck <m...@beckweb.net> wrote:

> Given the late hour, I will merge 2.0 into master tomorrow.

The merge commit is here: https://github.com/daniel-beck/jenkins/commit/1fe9cf7b7ada45230f2bc5b8e2f1bdb93175ff9f

If you'd like to give it a final review, the time to do so is right now. I'll hold off pushing this for a few hours (and I won't do additional rounds if I have to resolve a conflict later). Please note that any findings will need to be addressed in followup changes, likely towards 2.1.

A reminder:

The version 2.0 will be released from the '2.0' branch. Any changes to 'master' will be targeting 2.1 (I'll adjust the POM version later).

Daniel Beck

unread,
Apr 7, 2016, 10:24:54 AM4/7/16
to jenkin...@googlegroups.com
I pushed the merge to 'master'. So anything targeting 2.1+ can be now proposed in pull requests to that branch.

Anything happening on '2.0' branch will be limited to critical fixes for the 2.0 release specifically.
> --
> You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/41A4284B-F8C7-4B43-B413-3E1CD1CC942A%40beckweb.net.
> For more options, visit https://groups.google.com/d/optout.
>

Robert Sandell

unread,
Apr 7, 2016, 11:16:53 AM4/7/16
to jenkin...@googlegroups.com
Where is the RC?


For more options, visit https://groups.google.com/d/optout.



--
Robert Sandell
Software Engineer
CloudBees Inc.

Daniel Beck

unread,
Apr 7, 2016, 11:21:24 AM4/7/16
to jenkin...@googlegroups.com

On 07.04.2016, at 17:16, Robert Sandell <rsan...@cloudbees.com> wrote:

> Where is the RC?

Same URL as the betas, except for Docker and that's listed on https://hub.docker.com/r/jenkinsci/jenkins/tags/ -- I just need to get the 2.0 page updated and submit the blog post. Haven't had the time to do so yet, unfortunately.

Antonio Muñiz

unread,
Apr 8, 2016, 4:18:53 AM4/8/16
to jenkin...@googlegroups.com
I'm trying to diagnose the OOME in tests on
https://ci.jenkins-ci.org/job/jenkins_main_trunk after merging 2.0
into master.
Running tests locally.
> --
> You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/15137931-8AD6-4132-9176-DF6F439977F5%40beckweb.net.
> For more options, visit https://groups.google.com/d/optout.



--
Antonio Muñiz
Software Engineer
CloudBees, Inc.

Antonio Muñiz

unread,
Apr 8, 2016, 4:24:56 AM4/8/16
to jenkin...@googlegroups.com
The build is using JDK 7, could it be configured to use JDK 8?
(https://ci.jenkins-ci.org/job/jenkins_pipeline/branch/2.0 is stable
with JDK 8)

Arnaud Héritier

unread,
Apr 8, 2016, 4:33:09 AM4/8/16
to jenkin...@googlegroups.com
Why do we use Java 8 for CI ?
Java 7 is the oldest version we are supporting for now it must run with it :(


For more options, visit https://groups.google.com/d/optout.



--
-----
Arnaud Héritier
Mail/GTalk: aheritier AT gmail DOT com
Twitter/Skype : aheritier

Antonio Muñiz

unread,
Apr 8, 2016, 4:59:53 AM4/8/16
to jenkin...@googlegroups.com
On Fri, Apr 8, 2016 at 10:32 AM, Arnaud Héritier <aher...@gmail.com> wrote:
> Java 7 is the oldest version we are supporting for now it must run with it
> :(


`<java.level>7</java.level>` is set anyways. So building with JDK 8
should be fine.
BTW the test suite ran successfully in my local box - max memory
consumption during tests: 500MB - (JDK 8, Maven 3.3.3)

But before changing anything, could someone check that `celery` node
is ok? Last time Daniel checked there were a lot of zombie processes
eating the memory (which leaded to OOM errors in new builds).

Kanstantsin Shautsou

unread,
Apr 8, 2016, 5:55:20 AM4/8/16
to jenkin...@googlegroups.com
Limit threading in build and check what you did with pom and threading in pom/test plugins.
> --
> You received this message because you are subscribed to a topic in the Google Groups "Jenkins Developers" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-dev/f4iUVX7w_PE/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to jenkinsci-de...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAJc7kzQmaq1AA_CF7EL89K009gTXVQ52p8D5W1XXwOj6kYxgVg%40mail.gmail.com.
signature.asc

Antonio Muñiz

unread,
Apr 8, 2016, 6:22:08 AM4/8/16
to jenkin...@googlegroups.com
On Fri, Apr 8, 2016 at 11:55 AM, Kanstantsin Shautsou
<kanstan...@gmail.com> wrote:
> check what you did with pom and threading in pom/test plugins.

Nothing was done, just -Xmx1g (which is not required IMO).

I think celery is not being able to manage the load and that's the
root cause of OOM errors. Arnaud just checked that the physical memory
was completely full some minutes ago (so 2.0 builds are getting OOM
even before hitting their 1GB limit).

I guess the number of concurrent builds increased in the last months?

If the problem persists, this would need to be reopen/merged to give
us some light: https://github.com/jenkinsci/jenkins/pull/2220

Antonio Muñiz

unread,
Apr 8, 2016, 6:57:38 AM4/8/16
to jenkin...@googlegroups.com
https://ci.jenkins-ci.org/job/jenkins_main_trunk/4555 is running. No
memory issues so far, but a test seems to be blocked for some other
reason.

Kanstantsin Shautsou

unread,
Apr 8, 2016, 7:17:24 AM4/8/16
to jenkin...@googlegroups.com
Your local runs wouldn’t reproduce anything because you have different hardware.
Check threading/forking in maven test plugins (surefire/failsafe). 

--
Antonio Muñiz
Software Engineer
CloudBees, Inc.

--
You received this message because you are subscribed to a topic in the Google Groups "Jenkins Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-dev/f4iUVX7w_PE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jenkinsci-de...@googlegroups.com.
signature.asc

Antonio Muñiz

unread,
Apr 8, 2016, 7:20:38 AM4/8/16
to jenkin...@googlegroups.com
One of the forked booters finished, the other one is blocked in a test. Any idea?

"Executing retainMasterLabelWhenNoSlaveDefined(jenkins.model.MasterBuildConfigurationTest)" prio=10 tid=0x00007f47f4007800 nid=0x3384 in Object.wait() [0x00007f47fbe88000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00000000c664a970> (a jenkins.model.Jenkins$8)
	at java.lang.Object.wait(Object.java:503)
	at org.jvnet.hudson.reactor.Reactor.execute(Reactor.java:267)
	- locked <0x00000000c664a970> (a jenkins.model.Jenkins$8)
	at jenkins.InitReactorRunner.run(InitReactorRunner.java:44)
	at jenkins.model.Jenkins.executeReactor(Jenkins.java:1020)
	at jenkins.model.Jenkins.<init>(Jenkins.java:864)
	at hudson.model.Hudson.<init>(Hudson.java:85)
	at org.jvnet.hudson.test.JenkinsRule.newHudson(JenkinsRule.java:571)
	at org.jvnet.hudson.test.JenkinsRule.before(JenkinsRule.java:358)
	at org.jvnet.hudson.test.JenkinsRule$2.evaluate(JenkinsRule.java:516)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/B1984CD6-146D-4D84-B794-B47E1093C5CB%40gmail.com.

For more options, visit https://groups.google.com/d/optout.

Kanstantsin Shautsou

unread,
Apr 8, 2016, 7:25:22 AM4/8/16
to jenkin...@googlegroups.com
Check dmesg, for OOM killer
signature.asc

Antonio Muñiz

unread,
Apr 8, 2016, 7:33:06 AM4/8/16
to jenkin...@googlegroups.com
On Fri, Apr 8, 2016 at 1:25 PM, Kanstantsin Shautsou
<kanstan...@gmail.com> wrote:
> Check dmesg, for OOM killer


It's not a memory issue I believe, the test is just blocked for some reason.

Robert Sandell

unread,
Apr 8, 2016, 9:03:20 AM4/8/16
to jenkin...@googlegroups.com
Well Antonio might be up to something with the OOMs anyways. I am also experiencing OOMs on my test instance where I just updated the war from 1.650 to 2.0-rc and installed the "recommended plugins" ontop of what I already had, seems like I need to tweak both max heap and perm size from what was working in 1.x.
But I also had a very minimal set of plugins to begin with.

/B

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.



--

Jesse Glick

unread,
Apr 8, 2016, 9:21:04 AM4/8/16
to Jenkins Dev
On Fri, Apr 8, 2016 at 4:32 AM, Arnaud Héritier <aher...@gmail.com> wrote:
> Why do we use Java 8 for CI ?
> Java 7 is the oldest version we are supporting

The build already uses `-target 7` and runs Animal Sniffer to prevent
(accidental) use of Java 8+ APIs. 8 drops the PermGen, which in my
experience makes it much easier to keep functional tests running on
CI.

So the only real question is about test coverage of behavioral
differences between 7 and 8. If we run CI tests on one, then we lose
coverage for the other. We *allow* Jenkins to be run on 7 but
*recommend* 8, so it is better to test on 8, the platform we expect
most users to run.

Now having a daily or weekly CI job to rerun core (and, for that
matter, plugin and ATH) tests on 7 would be great, to catch
regressions. While we are it, we should run the tests on a UNC share
in Windows Server 2008, and try to make them break.

Daniel Beck

unread,
Apr 20, 2016, 2:03:30 AM4/20/16
to jenkin...@googlegroups.com

> On 29.03.2016, at 02:54, Kohsuke Kawaguchi <k...@kohsuke.org> wrote:
>
> • April 3: 1.656 release, which will be the last 1.x release
>

FYI As a test run of the new infra, KK released 1.657 today, which will be followed by 1.658.

2.0 will be released later today or tomorrow, so these shouldn't matter too much.

Christopher Orr

unread,
Apr 20, 2016, 8:10:34 AM4/20/16
to jenkin...@googlegroups.com
Will the lack of Update Centre files being generated affect the 2.0 release, if it happens today?

Regards,
Chris

PS. Yay! :)
Reply all
Reply to author
Forward
0 new messages