Jenkins 2.188 release is stuck

64 views
Skip to first unread message

Kohsuke Kawaguchi

unread,
Aug 6, 2019, 12:53:57 AM8/6/19
to JenkinsCI Developers
Currently the release of 2.188 is stuck due to a failing jenkins.triggers.TriggerTest. The log is https://gist.github.com/kohsuke/7ba26b2d9f4a2a49d832610d069d0ab4

I'll look into this further later, but if anyone has any thoughts on what might be wrong, that'd be greatly appreciated.

--
Kohsuke Kawaguchi

Oleg Nenashev

unread,
Aug 6, 2019, 2:54:30 AM8/6/19
to Jenkins Developers
I was unable to reproduce it locally on Maven 3.6.0 and Windows.
Would it be possible to provide a crashdump?

Thanks in advance,
Oleg

Lloyd Chang

unread,
Aug 6, 2019, 3:36:21 AM8/6/19
to Jenkins Developers

Lloyd Chang

unread,
Aug 6, 2019, 3:57:15 AM8/6/19
to Jenkins Developers
clarification: the quoted version number "2.20.1" was Surefire, and not Jenkins.

Kohsuke Kawaguchi

unread,
Aug 6, 2019, 11:42:18 AM8/6/19
to JenkinsCI Developers
Thanks Lloyd. With Surefire 2.20.1, the error message changes more explicitly to timeout: https://gist.github.com/kohsuke/4d31b8dbcbfe1926e71ccba08f7f9555

The test case in question includes an invocation of 'interactiveBreak', which is surely wrong? When I remove that line, the test passes.

Oleg, how does this test pass for you (and presumably the CI system that gated this change) with the interactiveBreak invocation intact?



--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/1bd559a1-7c52-4a60-91ad-9c2f97ebe8d5%40googlegroups.com.


--
Kohsuke Kawaguchi

Kohsuke Kawaguchi

unread,
Aug 6, 2019, 11:49:24 AM8/6/19
to JenkinsCI Developers
I created https://github.com/jenkinsci/jenkins/pull/4142 to get us past this blockage.
--
Kohsuke Kawaguchi

Oleg Nenashev

unread,
Aug 6, 2019, 12:04:51 PM8/6/19
to Jenkins Developers
Oleg, how does this test pass for you (and presumably the CI system that gated this change) with the interactiveBreak invocation intact?

I do not run tests locally when integrating changes, I rely on CI.
And CI was green, not sure why.

The issue itself came from https://github.com/jenkinsci/jenkins/pull/3802 , and it definitely did not get noticed during code reviews.



 

On Tuesday, August 6, 2019 at 5:49:24 PM UTC+2, Kohsuke Kawaguchi wrote:
I created https://github.com/jenkinsci/jenkins/pull/4142 to get us past this blockage.

On Tue, Aug 6, 2019 at 8:42 AM Kohsuke Kawaguchi <k...@kohsuke.org> wrote:
Thanks Lloyd. With Surefire 2.20.1, the error message changes more explicitly to timeout: https://gist.github.com/kohsuke/4d31b8dbcbfe1926e71ccba08f7f9555

The test case in question includes an invocation of 'interactiveBreak', which is surely wrong? When I remove that line, the test passes.

Oleg, how does this test pass for you (and presumably the CI system that gated this change) with the interactiveBreak invocation intact?



On Tue, Aug 6, 2019 at 12:57 AM Lloyd Chang <lloyd...@gmail.com> wrote:
clarification: the quoted version number "2.20.1" was Surefire, and not Jenkins.

On Tuesday, August 6, 2019 at 12:36:21 AM UTC-7, Lloyd Chang wrote:
> We have it fixed in 2.20.1.

On Monday, August 5, 2019 at 11:54:30 PM UTC-7, Oleg Nenashev wrote:
I was unable to reproduce it locally on Maven 3.6.0 and Windows.
Would it be possible to provide a crashdump?

Thanks in advance,
Oleg


On Tuesday, August 6, 2019 at 6:53:57 AM UTC+2, Kohsuke Kawaguchi wrote:
Currently the release of 2.188 is stuck due to a failing jenkins.triggers.TriggerTest. The log is https://gist.github.com/kohsuke/7ba26b2d9f4a2a49d832610d069d0ab4

I'll look into this further later, but if anyone has any thoughts on what might be wrong, that'd be greatly appreciated.

--
Kohsuke Kawaguchi

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkin...@googlegroups.com.


--
Kohsuke Kawaguchi


--
Kohsuke Kawaguchi

Oleg Nenashev

unread,
Aug 6, 2019, 12:12:34 PM8/6/19
to Jenkins Developers
Should we just mark the method as `@Restricted(NoExternalUse.class)` in JTH so that all production build fail by default?
This method is IMO dangerous by default

You received this message because you are subscribed to a topic in the Google Groups "Jenkins Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-dev/l2vGNx289vE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/5bae9334-db08-4753-b893-c8fb8187ccc2%40googlegroups.com.

Kohsuke Kawaguchi

unread,
Aug 6, 2019, 12:13:52 PM8/6/19
to JenkinsCI Developers
I was referring to your attempt to reproduce this earlier this thread.

My guess on why this passes the CI is that it runs in an environment where stdin is /dev/null, which I suspect will make interactiveBreak() a no-op. When I run a release or "mvn test" from a terminal, it actually does block.

If my guess is right, it's probably possible to harden interactiveBreak() to detect that it's reading from /dev/null and fail the test.

To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/5bae9334-db08-4753-b893-c8fb8187ccc2%40googlegroups.com.


--
Kohsuke Kawaguchi

Kohsuke Kawaguchi

unread,
Aug 6, 2019, 12:14:26 PM8/6/19
to JenkinsCI Developers
Up to you --- I just came up with another suggestion, but whatever works for you is fine by me.

To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAPfivLB5R0y3MYOUYHpSzZMsJCOMmUY1XiwtnAWY_b5pHaib0w%40mail.gmail.com.


--
Kohsuke Kawaguchi

Jesse Glick

unread,
Aug 6, 2019, 2:24:32 PM8/6/19
to Jenkins Dev
On Tue, Aug 6, 2019 at 12:12 PM Oleg Nenashev <o.v.ne...@gmail.com> wrote:
> Should we just mark the method as `@Restricted(NoExternalUse.class)` in JTH so that all production build fail by default?

Would have no effect—access modifiers are ignored in test sources.

Oleg Nenashev

unread,
Aug 7, 2019, 8:41:13 AM8/7/19
to Jenkins Developers
It looks like the new release attempt did not go well
There was https://repo.jenkins-ci.org/snapshots/org/jenkins-ci/main/jenkins-war/2.188-SNAPSHOT/ deployed this night, and the release is not there so far.


On Tuesday, August 6, 2019 at 8:24:32 PM UTC+2, Jesse Glick wrote:

Kohsuke Kawaguchi

unread,
Aug 7, 2019, 10:57:58 AM8/7/19
to JenkinsCI Developers
Yep, still investigating. I'm burning 2.188 and moving to 2.189.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/1053517a-513b-4ba6-92ab-ca5c3abed392%40googlegroups.com.


--
Kohsuke Kawaguchi

Kohsuke Kawaguchi

unread,
Aug 7, 2019, 2:46:28 PM8/7/19
to JenkinsCI Developers
I think 2.189 finally went through.
--
Kohsuke Kawaguchi

Oleg Nenashev

unread,
Aug 11, 2019, 2:09:45 AM8/11/19
to Jenkins Developers
Thanks for the release Kohsuke!
2.188 looks OK according to JIRA and community stats.

Regarding the today's weekly, I think we can skip it. ci.jenkins.io has been unstable since Monday, and I do not feel confident about integrating changes while the maintainers bandwidth is saturated by DW-JW. So I have not merged anything and do not plan to do so until the CI instance stabilizes, sorry to all contributors.

Best regards,
Oleg


On Wednesday, August 7, 2019 at 8:46:28 PM UTC+2, Kohsuke Kawaguchi wrote:
I think 2.189 finally went through.

On Wed, Aug 7, 2019 at 7:57 AM Kohsuke Kawaguchi <k...@kohsuke.org> wrote:
Yep, still investigating. I'm burning 2.188 and moving to 2.189.

On Wed, Aug 7, 2019 at 5:41 AM Oleg Nenashev <o.v.n...@gmail.com> wrote:
It looks like the new release attempt did not go well
There was https://repo.jenkins-ci.org/snapshots/org/jenkins-ci/main/jenkins-war/2.188-SNAPSHOT/ deployed this night, and the release is not there so far.


On Tuesday, August 6, 2019 at 8:24:32 PM UTC+2, Jesse Glick wrote:
On Tue, Aug 6, 2019 at 12:12 PM Oleg Nenashev <o.v.n...@gmail.com> wrote:
> Should we just mark the method as `@Restricted(NoExternalUse.class)` in JTH so that all production build fail by default?

Would have no effect—access modifiers are ignored in test sources.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkin...@googlegroups.com.


--
Kohsuke Kawaguchi


--
Kohsuke Kawaguchi

Kohsuke Kawaguchi

unread,
Aug 11, 2019, 2:18:24 AM8/11/19
to jenkin...@googlegroups.com
OK. Thanks for the heads-up 

To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/98e5b0ea-9462-4cd7-bc65-bf91f7cdf87e%40googlegroups.com.
--
Kohsuke Kawaguchi

Oleg Nenashev

unread,
Aug 16, 2019, 9:01:13 AM8/16/19
to Jenkins Developers
I think we can continue weekly releases on Sunday/Monday as usual. We have restored the CI (see https://groups.google.com/forum/#!topic/jenkinsci-dev/v9d-XosOp2s) and integrated a bunch of changes, so we have something for the next weekly releases.


On Sunday, August 11, 2019 at 8:18:24 AM UTC+2, Kohsuke Kawaguchi wrote:
OK. Thanks for the heads-up 
Reply all
Reply to author
Forward
0 new messages