[JIRA] [ec2-plugin] (JENKINS-15931) Build is hanging on newly created EC2 slaves

[francisu@gmail.com (JIRA)]

Francis Upton closed an issue as Cannot Reproduce

Old issue, please refile if broken. Jenkins / JENKINS-15931 Build is hanging on newly created EC2 slaves Change By: Francis Upton Status: Open Closed Resolution: Cannot Reproduce Add Comment

This message was sent by Atlassian JIRA (v6.4.2#64017-sha1:e244265)

[eckhard.voellm@nokia.com (JIRA)]

Eckhard Völlm reopened an issue

We have exactly this isse here frequently with jobs having a large number of testcases. What else information is needed to fix this problem ? Hereafter the job that got stuck at 16.01 10:47:23. Today morning i pressed the stop button at 8:24, and you see the 16.01 10:47:22 </ul> 16.01 10:47:23 [xUnit] [INFO] - Starting to record. 16.01 10:47:23 [xUnit] [INFO] - Processing BoostTest-1.x (default) 16.01 10:47:23 [xUnit] [INFO] - [BoostTest-1.x (default)] - 2 test report file(s) were found with the pattern '*.xml' relative to '/home/alcatel/workspace/R8Light' for the testing framework 'BoostTest-1.x (default)'. 18.01 08:24:25 [xUnit] [WARNING] - Caught exception of unexpected type class java.lang.InterruptedException, rethrowing 18.01 08:24:25 ERROR: Step ‘Publish xUnit test result report’ aborted due to exception: 18.01 08:24:25 java.lang.InterruptedException 18.01 08:24:25 at java.lang.Object.wait(Native Method) 18.01 08:24:25 at hudson.remoting.Request.call(Request.java:147) 18.01 08:24:25 at hudson.remoting.Channel.call(Channel.java:780) 18.01 08:24:25 at hudson.FilePath.act(FilePath.java:979) 18.01 08:24:25 at hudson.FilePath.act(FilePath.java:968) 18.01 08:24:25 at org.jenkinsci.plugins.xunit.XUnitProcessor.performTests(XUnitProcessor.java:138) 18.01 08:24:25 at org.jenkinsci.plugins.xunit.XUnitProcessor.performXUnit(XUnitProcessor.java:81) 18.01 08:24:25 at org.jenkinsci.plugins.xunit.XUnitPublisher.perform(XUnitPublisher.java:112) 18.01 08:24:25 at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20) 18.01 08:24:25 at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:782) 18.01 08:24:25 at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:723) 18.01 08:24:25 at hudson.model.Build$BuildExecution.post2(Build.java:185) 18.01 08:24:25 at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:668) 18.01 08:24:25 at hudson.model.Run.execute(Run.java:1763) 18.01 08:24:25 at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) 18.01 08:24:25 at hudson.model.ResourceController.execute(ResourceController.java:98) 18.01 08:24:25 at hudson.model.Executor.run(Executor.java:410) 18.01 08:24:25 [PostBuildScript] - Execution post build scripts. 18.01 08:24:25 [R8Light] $ /bin/bash -xe /tmp/hudson4109735917483500943.sh

Jenkins / JENKINS-15931 Build is hanging on newly created EC2 slaves

Change By: Eckhard Völlm Resolution: Cannot Reproduce Status: Closed Reopened

Add Comment

[eckhard.voellm@nokia.com (JIRA)]

Eckhard Völlm assigned an issue to Gregory Boissinot

Jenkins / JENKINS-15931 Build is hanging on newly created EC2 slaves

Change By: Eckhard Völlm Assignee: Gregory Boissinot

Add Comment

[eckhard.voellm@nokia.com (JIRA)]

Eckhard Völlm commented on JENKINS-15931

Re: Build is hanging on newly created EC2 slaves Hi Gregory, can you help me here, or do you know an expert for this xUnit problem ? Thanks in advance, Eckhard !

Add Comment

[eckhard.voellm@nokia.com (JIRA)]

Eckhard Völlm updated an issue

Jenkins / JENKINS-15931

Build is hanging on newly created EC2 slaves

Here i added the full Java trace from core dump of the hanging Jenkins Jobs. Change By: Eckhard Völlm Attachment: hs_err_pid15419.log

Add Comment

[eckhard.voellm@nokia.com (JIRA)]

Eckhard Völlm commented on JENKINS-15931

Re: Build is hanging on newly created EC2 slaves Jenkins version 1.6.43 xUnit version 1.99

Add Comment

[eckhard.voellm@nokia.com (JIRA)]

Eckhard Völlm edited a comment on JENKINS-15931

Re: Build is hanging on newly created EC2 slaves

Hi Gregory, can you help me here , or do you know an expert for this xUnit problem  ?

Thanks in advance, Eckhard !

Add Comment

[eckhard.voellm@nokia.com (JIRA)]

Eckhard Völlm edited a comment on JENKINS-15931

Re: Build is hanging on newly created EC2 slaves

Jenkins version 1.6.43 xUnit version  1.99

Here a thread where Java threads also were hanging at  "java.util.zip.ZipFile.getEntry": https://github.com/aws/aws-sdk-java/issues/238 same as we observe here. They also found a solution, IMHO: "And yes, using this system property gets rid of the thread blocks :+1: Thank you so much for the suggestion!"

Add Comment

[eckhard.voellm@nokia.com (JIRA)]

Eckhard Völlm edited a comment on JENKINS-15931

Re: Build is hanging on newly created EC2 slaves

Jenkins version 1.6.43 xUnit version  1.99

java version "1.7.0_91" OpenJDK Runtime Environment (IcedTea 2.6.3) (7u91-2.6.3-0ubuntu0.14.04.1) OpenJDK 64-Bit Server VM (build 24.91-b01, mixed mode) Linux slave 4.2.0-19-lowlatency #23~14.04.1-Ubuntu SMP PREEMPT Thu Nov 12 13:19:01 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

Here a thread where Java threads also were hanging at  "java.util.zip.ZipFile.getEntry": https://github.com/aws/aws-sdk-java/issues/238 same as we observe here. They also found a solution, IMHO: "And yes, using this system property gets rid of the thread blocks :+1: Thank you so much for the suggestion!"

Add Comment

[francisu@gmail.com (JIRA)]

Francis Upton commented on JENKINS-15931

Re: Build is hanging on newly created EC2 slaves

Does not appear to be an ec2 plugin issue.

Add Comment

[francisu@gmail.com (JIRA)]

Francis Upton updated an issue

Jenkins / JENKINS-15931

Build is hanging on newly created EC2 slaves

Change By: Francis Upton Component/s: ec2-plugin

Add Comment

[eckhard.voellm@nokia.com (JIRA)]

Eckhard Völlm commented on JENKINS-15931

Re: Build is hanging on newly created EC2 slaves Yes, we do not have ec2 plugin in use, i can confirm that. And we decided to try an upgrade of Java version now, using oracle 1.8.0_66-b17 since last week, Since we did that, there was no hanging, but time is too short to say something, not many test were running since snowzilla hit eastcost.

Add Comment

[eckhard.voellm@nokia.com (JIRA)]

Eckhard Völlm edited a comment on JENKINS-15931

Re: Build is hanging on newly created EC2 slaves

Yes, we do not have ec2 plugin in use, i can confirm that. And we decided to try an upgrade of Java version now, using oracle  1.8.0_66-b17 since last week, Since we did that, there was no hanging, but time is too short to say something, not many test were running since snowzilla hit eastcost.

Propose to switch Subject, to "Build is hanging in xUnit plugin processing"

Add Comment