build hangs on slave

[Kenneth Westelinck]

Dear list,

I have a Jenkins master - slave setup. The master is a centos box, slave is a win 2008r2 box. I have a long running build (+/- 90 minutes) that sometimes builds, but starts to hang most of the times. The build runs find on developer's Linux / win 7 boxes with same JDK / maven version. So it has to be a jenkins thing.

This is what I get on the slave:

Thread Dump

Channel reader thread: channel

"Channel reader thread: channel" Id=9 Group=main RUNNABLE (in native)
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(Unknown Source)
	at java.net.SocketInputStream.read(Unknown Source)
	at java.io.FilterInputStream.read(Unknown Source)
	at java.io.BufferedInputStream.fill(Unknown Source)
	at java.io.BufferedInputStream.read(Unknown Source)
	-  locked java.io.BufferedInputStream@41ed34a8
	at java.io.ObjectInputStream$PeekInputStream.peek(Unknown Source)
	at java.io.ObjectInputStream$BlockDataInputStream.peek(Unknown Source)
	at java.io.ObjectInputStream$BlockDataInputStream.peekByte(Unknown Source)
	at java.io.ObjectInputStream.readObject0(Unknown Source)
	at java.io.ObjectInputStream.readObject(Unknown Source)
	at hudson.remoting.Channel$ReaderThread.run(Channel.java:1127)

D:\data\hudson\tools\Java_1.6.29/bin/java -Xms512m -Xmx2048m -XX:PermSize=512m -XX:MaxPermSize=1024m -cp D:\data\hudson\maven3-agent.jar;D:\data\hudson\tools\maven-3.0.3\boot\plexus-classworlds-2.4.jar org.jvnet.hudson.maven3.agent.Maven3Main D:\data\hudson\tools\maven-3.0.3 D:\data\hudson\slave.jar D:\data\hudson\maven3-interceptor.jar 53670: stdout copier

"D:\data\hudson\tools\Java_1.6.29/bin/java -Xms512m -Xmx2048m -XX:PermSize=512m -XX:MaxPermSize=1024m -cp D:\data\hudson\maven3-agent.jar;D:\data\hudson\tools\maven-3.0.3\boot\plexus-classworlds-2.4.jar org.jvnet.hudson.maven3.agent.Maven3Main D:\data\hudson\tools\maven-3.0.3 D:\data\hudson\slave.jar D:\data\hudson\maven3-interceptor.jar 53670: stdout copier" Id=9205 Group=main RUNNABLE (in native)
	at java.io.FileInputStream.readBytes(Native Method)
	at java.io.FileInputStream.read(Unknown Source)
	at java.io.BufferedInputStream.read1(Unknown Source)
	at java.io.BufferedInputStream.read(Unknown Source)
	-  locked java.io.BufferedInputStream@77215ab7
	at java.io.FilterInputStream.read(Unknown Source)
	at hudson.util.StreamCopyThread.run(StreamCopyThread.java:60)

main

"main" Id=1 Group=main WAITING on hudson.remoting.Channel@455d0ff7
	at java.lang.Object.wait(Native Method)
	-  waiting on hudson.remoting.Channel@455d0ff7
	at java.lang.Object.wait(Object.java:503)
	at hudson.remoting.Channel.join(Channel.java:798)
	at hudson.remoting.Launcher.main(Launcher.java:420)
	at hudson.remoting.Launcher.runOnSocket(Launcher.java:314)
	at hudson.remoting.Launcher.runAsTcpServer(Launcher.java:310)
	at hudson.remoting.Launcher.run(Launcher.java:203)
	at hudson.remoting.Launcher.main(Launcher.java:168)

Ping thread for channel hudson.remoting.Channel@455d0ff7:channel

"Ping thread for channel hudson.remoting.Channel@455d0ff7:channel" Id=10 Group=main TIMED_WAITING
	at java.lang.Thread.sleep(Native Method)
	at hudson.remoting.PingThread.run(PingThread.java:86)

Pipe writer thread: channel

"Pipe writer thread: channel" Id=12 Group=main WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@52f8a456
	at sun.misc.Unsafe.park(Native Method)
	-  waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@52f8a456
	at java.util.concurrent.locks.LockSupport.park(Unknown Source)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)
	at java.util.concurrent.LinkedBlockingQueue.take(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.lang.Thread.run(Unknown Source)

pool-1-thread-85

"pool-1-thread-85" Id=9227 Group=main RUNNABLE (in native)
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(Unknown Source)
	at java.net.SocketInputStream.read(Unknown Source)
	at java.io.FilterInputStream.read(Unknown Source)
	at hudson.remoting.ProxyInputStream$Buffer.read(ProxyInputStream.java:110)
	at hudson.remoting.ProxyInputStream$Chunk.perform(ProxyInputStream.java:132)
	at hudson.remoting.ProxyInputStream$Chunk.perform(ProxyInputStream.java:119)
	at hudson.remoting.Request$2.run(Request.java:287)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
	at java.util.concurrent.FutureTask.run(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.lang.Thread.run(Unknown Source)

	Number of locked synchronizers = 1
	- java.util.concurrent.ThreadPoolExecutor$Worker@7141e2ae

pool-1-thread-86

"pool-1-thread-86" Id=9280 Group=main RUNNABLE
	at sun.management.ThreadImpl.dumpThreads0(Native Method)
	at sun.management.ThreadImpl.dumpAllThreads(Unknown Source)
	at hudson.Functions.getThreadInfos(Functions.java:861)
	at hudson.util.RemotingDiagnostics$GetThreadDump.call(RemotingDiagnostics.java:93)
	at hudson.util.RemotingDiagnostics$GetThreadDump.call(RemotingDiagnostics.java:89)
	at hudson.remoting.UserRequest.perform(UserRequest.java:118)
	at hudson.remoting.UserRequest.perform(UserRequest.java:48)
	at hudson.remoting.Request$2.run(Request.java:287)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
	at java.util.concurrent.FutureTask.run(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.lang.Thread.run(Unknown Source)

	Number of locked synchronizers = 1
	- java.util.concurrent.ThreadPoolExecutor$Worker@16d19a86

Attach Listener

"Attach Listener" Id=4 Group=system RUNNABLE

Finalizer

"Finalizer" Id=3 Group=system WAITING on java.lang.ref.ReferenceQueue$Lock@3c1c2368
	at java.lang.Object.wait(Native Method)
	-  waiting on java.lang.ref.ReferenceQueue$Lock@3c1c2368
	at java.lang.ref.ReferenceQueue.remove(Unknown Source)
	at java.lang.ref.ReferenceQueue.remove(Unknown Source)
	at java.lang.ref.Finalizer$FinalizerThread.run(Unknown Source)

Reference Handler

"Reference Handler" Id=2 Group=system WAITING on java.lang.ref.Reference$Lock@5b589fac
	at java.lang.Object.wait(Native Method)
	-  waiting on java.lang.ref.Reference$Lock@5b589fac
	at java.lang.Object.wait(Object.java:503)
	at java.lang.ref.Reference$ReferenceHandler.run(Unknown Source)


Can someone more knowledgeable shed a light on this. (if you need more info, don't hesitate to ask)
Many thanks in advance.

regards,

Kenneth 

[Kenneth Westelinck]

Sorry, forgot to mention, this is running the latest and greatest 1.450.

[Santosh Hadawale]

Hi, I am also having similar Jenkins Master-Slave configuration and facing the same issue while building the job.
The build which suppose to take 40 mins for successful execution, is running more than 2 hrs and still running.

[Kenneth Westelinck]

We decided to replace Jenkins with Bamboo. We had far too many issues with Jenkins.

--
You received this message because you are subscribed to a topic in the Google Groups "Jenkins Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-users/HIqesMjZoYg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jenkinsci-use...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.