Help troubleshooting Jenkins slow page load times.

[Maxfield Stewart]

Hello!

We recently upgraded our Jenkins deploy from 1.480 to 1.484. Technically we were trying to upgrade to 1.492 which was the latest at the time.  That upgrade failed for various plugin compatibility problems and we downgraded, Eventually settling on 1.484.  In the process we inherited terribly slow page load times and are having trouble figuring out how to troubleshoot/identify the cause.  Here's a list of what we know right now:

* CPU: The java process for jenkins sits around %4-%8 CPU utlization

* Memory: We have a 5GB heap allocated, which we use about 1.5GB.  We do have a memory leak, but page load times are slow even on initial restart, the leak takes about a week to become a problem.

* Garbage Collection: We're executing Full GC's rarely, maybe once every 10-15 minutes, they last about a second until the memory leak hits capacity.

* Disk Utilization: Nominal, no obvious signs that the drives are I/O locking.

Page loads are intermittantly bad, and worse once we crest 20-30 http sessions on the box.  "Bad" is defined as we're often waiting 4-8 seconds for a page load, on any page in Jenkins.  Experimenting with the load times, it appears to be some kind of locking/resource contention as you can make many requests at once, they'll all "wait" and then load at the same time after 4-8 seconds.  When we ran 1.480 our page load times were under a second, extremely snappy.

I've looked at running java threads and nothing looks clearly locked.  There were some Jenkins patch notes indicating that the number of HTTP workers was reduced from 1000 to 10 between version 1.480 and 1.484 but I have no way of proving if that impacted us or not.

I'm looking for guidance on ways I can troubleshoot this problem and identify the issue.  This is an extremely active Jenkins environment and I cannot just do certain things, like unilaterally disable all plugins, or continuously restart Jenkins, we're running CI's here nearly 24/7.  I was hoping there was something specific I can look for either with the JavaMelody information or other monitoring tools.  Are there any best practices here?

[Maxfield Stewart]

Here for example is a thread dump. We're currently experience 10-18 second page load times.  I note that JavaMelody reports "HTTP Mean Time" is 60ms right now.

1/23/13 1:35 PM

===== Threads on SOME-MA...@xxx.xxx.xxx.xxx =====

"Attach Listener" daemon prio=5 RUNNABLE

"AWT-Windows" daemon prio=6 RUNNABLE

sun.awt.windows.WToolkit.eventLoop(Native Method)

sun.awt.windows.WToolkit.run(Unknown Source)

java.lang.Thread.run(Unknown Source)

"Channel reader thread: SOME-MACHINE-02" prio=5 RUNNABLE

java.net.SocketInputStream.socketRead0(Native Method)

java.net.SocketInputStream.read(Unknown Source)

java.net.SocketInputStream.read(Unknown Source)

java.io.BufferedInputStream.fill(Unknown Source)

java.io.BufferedInputStream.read(Unknown Source)

java.io.ObjectInputStream$PeekInputStream.peek(Unknown Source)

java.io.ObjectInputStream$BlockDataInputStream.peek(Unknown Source)

java.io.ObjectInputStream$BlockDataInputStream.peekByte(Unknown Source)

java.io.ObjectInputStream.readObject0(Unknown Source)

java.io.ObjectInputStream.readObject(Unknown Source)

hudson.remoting.Command.readFrom(Command.java:90)

hudson.remoting.ClassicCommandTransport.read(ClassicCommandTransport.java:59)

hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:48)

"Channel reader thread: SOME-MACHINE-03" prio=5 RUNNABLE

java.net.SocketInputStream.socketRead0(Native Method)

java.net.SocketInputStream.read(Unknown Source)

java.net.SocketInputStream.read(Unknown Source)

java.io.BufferedInputStream.fill(Unknown Source)

java.io.BufferedInputStream.read(Unknown Source)

java.io.ObjectInputStream$PeekInputStream.peek(Unknown Source)

java.io.ObjectInputStream$BlockDataInputStream.peek(Unknown Source)

java.io.ObjectInputStream$BlockDataInputStream.peekByte(Unknown Source)

java.io.ObjectInputStream.readObject0(Unknown Source)

java.io.ObjectInputStream.readObject(Unknown Source)

hudson.remoting.Command.readFrom(Command.java:90)

hudson.remoting.ClassicCommandTransport.read(ClassicCommandTransport.java:59)

hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:48)

"Channel reader thread: SOME-MACHINE-04" prio=5 RUNNABLE

java.net.SocketInputStream.socketRead0(Native Method)

java.net.SocketInputStream.read(Unknown Source)

java.net.SocketInputStream.read(Unknown Source)

java.io.BufferedInputStream.fill(Unknown Source)

java.io.BufferedInputStream.read(Unknown Source)

java.io.ObjectInputStream$PeekInputStream.peek(Unknown Source)

java.io.ObjectInputStream$BlockDataInputStream.peek(Unknown Source)

java.io.ObjectInputStream$BlockDataInputStream.peekByte(Unknown Source)

java.io.ObjectInputStream.readObject0(Unknown Source)

java.io.ObjectInputStream.readObject(Unknown Source)

hudson.remoting.Command.readFrom(Command.java:90)

hudson.remoting.ClassicCommandTransport.read(ClassicCommandTransport.java:59)

hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:48)

"Channel reader thread: SOME-MACHINE-05" prio=5 RUNNABLE

java.net.SocketInputStream.socketRead0(Native Method)

java.net.SocketInputStream.read(Unknown Source)

java.net.SocketInputStream.read(Unknown Source)

java.io.BufferedInputStream.fill(Unknown Source)

java.io.BufferedInputStream.read(Unknown Source)

java.io.ObjectInputStream$PeekInputStream.peek(Unknown Source)

java.io.ObjectInputStream$BlockDataInputStream.peek(Unknown Source)

java.io.ObjectInputStream$BlockDataInputStream.peekByte(Unknown Source)

java.io.ObjectInputStream.readObject0(Unknown Source)

java.io.ObjectInputStream.readObject(Unknown Source)

hudson.remoting.Command.readFrom(Command.java:90)

hudson.remoting.ClassicCommandTransport.read(ClassicCommandTransport.java:59)

hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:48)

"Channel reader thread: SOME-MACHINE-06" prio=5 RUNNABLE

java.net.SocketInputStream.socketRead0(Native Method)

java.net.SocketInputStream.read(Unknown Source)

java.net.SocketInputStream.read(Unknown Source)

java.io.BufferedInputStream.fill(Unknown Source)

java.io.BufferedInputStream.read(Unknown Source)

java.io.ObjectInputStream$PeekInputStream.peek(Unknown Source)

java.io.ObjectInputStream$BlockDataInputStream.peek(Unknown Source)

java.io.ObjectInputStream$BlockDataInputStream.peekByte(Unknown Source)

java.io.ObjectInputStream.readObject0(Unknown Source)

java.io.ObjectInputStream.readObject(Unknown Source)

hudson.remoting.Command.readFrom(Command.java:90)

hudson.remoting.ClassicCommandTransport.read(ClassicCommandTransport.java:59)

hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:48)

"Channel reader thread: SOME-MACHINE-07" prio=5 RUNNABLE

java.net.SocketInputStream.socketRead0(Native Method)

java.net.SocketInputStream.read(Unknown Source)

java.net.SocketInputStream.read(Unknown Source)

java.io.BufferedInputStream.fill(Unknown Source)

java.io.BufferedInputStream.read(Unknown Source)

java.io.ObjectInputStream$PeekInputStream.peek(Unknown Source)

java.io.ObjectInputStream$BlockDataInputStream.peek(Unknown Source)

java.io.ObjectInputStream$BlockDataInputStream.peekByte(Unknown Source)

java.io.ObjectInputStream.readObject0(Unknown Source)

java.io.ObjectInputStream.readObject(Unknown Source)

hudson.remoting.Command.readFrom(Command.java:90)

hudson.remoting.ClassicCommandTransport.read(ClassicCommandTransport.java:59)

hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:48)

"Channel reader thread: SOME-MACHINE-08" prio=5 RUNNABLE

java.net.SocketInputStream.socketRead0(Native Method)

java.net.SocketInputStream.read(Unknown Source)

java.net.SocketInputStream.read(Unknown Source)

java.io.BufferedInputStream.fill(Unknown Source)

java.io.BufferedInputStream.read(Unknown Source)

java.io.ObjectInputStream$PeekInputStream.peek(Unknown Source)

java.io.ObjectInputStream$BlockDataInputStream.peek(Unknown Source)

java.io.ObjectInputStream$BlockDataInputStream.peekByte(Unknown Source)

java.io.ObjectInputStream.readObject0(Unknown Source)

java.io.ObjectInputStream.readObject(Unknown Source)

hudson.remoting.Command.readFrom(Command.java:90)

hudson.remoting.ClassicCommandTransport.read(ClassicCommandTransport.java:59)

hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:48)

"Channel reader thread: some-machine-01" prio=5 RUNNABLE

java.net.SocketInputStream.socketRead0(Native Method)

java.net.SocketInputStream.read(Unknown Source)

java.net.SocketInputStream.read(Unknown Source)

java.io.BufferedInputStream.fill(Unknown Source)

java.io.BufferedInputStream.read(Unknown Source)

java.io.ObjectInputStream$PeekInputStream.peek(Unknown Source)

java.io.ObjectInputStream$BlockDataInputStream.peek(Unknown Source)

java.io.ObjectInputStream$BlockDataInputStream.peekByte(Unknown Source)

java.io.ObjectInputStream.readObject0(Unknown Source)

java.io.ObjectInputStream.readObject(Unknown Source)

hudson.remoting.Command.readFrom(Command.java:90)

hudson.remoting.ClassicCommandTransport.read(ClassicCommandTransport.java:59)

hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:48)

"Channel reader thread: some-machine-02" prio=5 RUNNABLE

java.net.SocketInputStream.socketRead0(Native Method)

java.net.SocketInputStream.read(Unknown Source)

java.net.SocketInputStream.read(Unknown Source)

java.io.BufferedInputStream.fill(Unknown Source)

java.io.BufferedInputStream.read(Unknown Source)

java.io.ObjectInputStream$PeekInputStream.peek(Unknown Source)

java.io.ObjectInputStream$BlockDataInputStream.peek(Unknown Source)

java.io.ObjectInputStream$BlockDataInputStream.peekByte(Unknown Source)

java.io.ObjectInputStream.readObject0(Unknown Source)

java.io.ObjectInputStream.readObject(Unknown Source)

hudson.remoting.Command.readFrom(Command.java:90)

hudson.remoting.ClassicCommandTransport.read(ClassicCommandTransport.java:59)

hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:48)

"Channel reader thread: some-machine-03" prio=5 RUNNABLE

java.net.SocketInputStream.socketRead0(Native Method)

java.net.SocketInputStream.read(Unknown Source)

java.net.SocketInputStream.read(Unknown Source)

java.io.BufferedInputStream.fill(Unknown Source)

java.io.BufferedInputStream.read(Unknown Source)

java.io.ObjectInputStream$PeekInputStream.peek(Unknown Source)

java.io.ObjectInputStream$BlockDataInputStream.peek(Unknown Source)

java.io.ObjectInputStream$BlockDataInputStream.peekByte(Unknown Source)

java.io.ObjectInputStream.readObject0(Unknown Source)

java.io.ObjectInputStream.readObject(Unknown Source)

hudson.remoting.Command.readFrom(Command.java:90)

hudson.remoting.ClassicCommandTransport.read(ClassicCommandTransport.java:59)

hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:48)

"Channel reader thread: BVT-1" prio=5 RUNNABLE

java.net.SocketInputStream.socketRead0(Native Method)

java.net.SocketInputStream.read(Unknown Source)

java.net.SocketInputStream.read(Unknown Source)

java.io.BufferedInputStream.fill(Unknown Source)

java.io.BufferedInputStream.read(Unknown Source)

java.io.ObjectInputStream$PeekInputStream.peek(Unknown Source)

java.io.ObjectInputStream$BlockDataInputStream.peek(Unknown Source)

java.io.ObjectInputStream$BlockDataInputStream.peekByte(Unknown Source)

java.io.ObjectInputStream.readObject0(Unknown Source)

java.io.ObjectInputStream.readObject(Unknown Source)

hudson.remoting.Command.readFrom(Command.java:90)

hudson.remoting.ClassicCommandTransport.read(ClassicCommandTransport.java:59)

hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:48)

"Channel reader thread: BVT-MacOSX" daemon prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

com.trilead.ssh2.StreamGobbler.read(StreamGobbler.java:144)

java.io.ObjectInputStream$PeekInputStream.peek(Unknown Source)

java.io.ObjectInputStream$BlockDataInputStream.peek(Unknown Source)

java.io.ObjectInputStream$BlockDataInputStream.peekByte(Unknown Source)

java.io.ObjectInputStream.readObject0(Unknown Source)

java.io.ObjectInputStream.readObject(Unknown Source)

hudson.remoting.Command.readFrom(Command.java:90)

hudson.remoting.ClassicCommandTransport.read(ClassicCommandTransport.java:59)

hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:48)

"Channel reader thread: Channel to Maven [C:\Program Files\Java\jdk1.6.0_29/bin/java, -Xmx4096M, -Xms4096M, -XX:MaxPermSize=512M, -Dtangosol.coherence.ttl=0, -Dtangosol.coherence.clusterport=27851, -Dtangosol.coherence.cluster=jmc_laxbuild, -XX:ReservedCodeCacheSize=128m, -cp, C:\jenkins\maven3-agent.jar;C:\apache-maven-3.0.4\boot\plexus-classworlds-2.4.jar, org.jvnet.hudson.maven3.agent.Maven3Main, C:\apache-maven-3.0.4, C:\jenkins\slave.jar, C:\jenkins\maven3-interceptor.jar, 59104] / waiting for hudson.remoting.Channel@6a554075:SOME-MACHINE-03" prio=5 TIMED_WAITING

java.lang.Object.wait(Native Method)

hudson.remoting.Request.call(Request.java:146)

hudson.remoting.ProxyInputStream._read(ProxyInputStream.java:74)

hudson.remoting.ProxyInputStream.read(ProxyInputStream.java:80)

hudson.remoting.RemoteInputStream.read(RemoteInputStream.java:91)

java.io.BufferedInputStream.read1(Unknown Source)

java.io.BufferedInputStream.read(Unknown Source)

java.io.BufferedInputStream.fill(Unknown Source)

java.io.BufferedInputStream.read(Unknown Source)

java.io.ObjectInputStream$PeekInputStream.peek(Unknown Source)

java.io.ObjectInputStream$BlockDataInputStream.peek(Unknown Source)

java.io.ObjectInputStream$BlockDataInputStream.peekByte(Unknown Source)

java.io.ObjectInputStream.readObject0(Unknown Source)

java.io.ObjectInputStream.readObject(Unknown Source)

hudson.remoting.Command.readFrom(Command.java:90)

hudson.remoting.ClassicCommandTransport.read(ClassicCommandTransport.java:59)

hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:48)

"Channel reader thread: XXXmobile" daemon prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

com.trilead.ssh2.StreamGobbler.read(StreamGobbler.java:144)

java.io.ObjectInputStream$PeekInputStream.peek(Unknown Source)

java.io.ObjectInputStream$BlockDataInputStream.peek(Unknown Source)

java.io.ObjectInputStream$BlockDataInputStream.peekByte(Unknown Source)

java.io.ObjectInputStream.readObject0(Unknown Source)

java.io.ObjectInputStream.readObject(Unknown Source)

hudson.remoting.Command.readFrom(Command.java:90)

hudson.remoting.ClassicCommandTransport.read(ClassicCommandTransport.java:59)

hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:48)

"Channel reader thread: XXXbuild-1" daemon prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

com.trilead.ssh2.StreamGobbler.read(StreamGobbler.java:144)

java.io.ObjectInputStream$PeekInputStream.peek(Unknown Source)

java.io.ObjectInputStream$BlockDataInputStream.peek(Unknown Source)

java.io.ObjectInputStream$BlockDataInputStream.peekByte(Unknown Source)

java.io.ObjectInputStream.readObject0(Unknown Source)

java.io.ObjectInputStream.readObject(Unknown Source)

hudson.remoting.Command.readFrom(Command.java:90)

hudson.remoting.ClassicCommandTransport.read(ClassicCommandTransport.java:59)

hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:48)

"Channel reader thread: XXXbuild-2" daemon prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

com.trilead.ssh2.StreamGobbler.read(StreamGobbler.java:144)

java.io.ObjectInputStream$PeekInputStream.peek(Unknown Source)

java.io.ObjectInputStream$BlockDataInputStream.peek(Unknown Source)

java.io.ObjectInputStream$BlockDataInputStream.peekByte(Unknown Source)

java.io.ObjectInputStream.readObject0(Unknown Source)

java.io.ObjectInputStream.readObject(Unknown Source)

hudson.remoting.Command.readFrom(Command.java:90)

hudson.remoting.ClassicCommandTransport.read(ClassicCommandTransport.java:59)

hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:48)

"ComThread for Finalizing set up" daemon prio=5 RUNNABLE

com4j.Win32Lock.suspend0(Native Method)

com4j.Win32Lock.suspend(Win32Lock.java:37)

com4j.ComThread.run0(ComThread.java:146)

com4j.ComThread.run(ComThread.java:134)

"ComThread for Handling GET /configure : RequestHandlerThread[#1036]" daemon prio=5 RUNNABLE

com4j.Win32Lock.suspend0(Native Method)

com4j.Win32Lock.suspend(Win32Lock.java:37)

com4j.ComThread.run0(ComThread.java:146)

com4j.ComThread.run(ComThread.java:134)

"ComThread for Handling GET /configure : RequestHandlerThread[#1038]" daemon prio=5 RUNNABLE

com4j.Win32Lock.suspend0(Native Method)

com4j.Win32Lock.suspend(Win32Lock.java:37)

com4j.ComThread.run0(ComThread.java:146)

com4j.ComThread.run(ComThread.java:134)

"ComThread for Handling GET /configure : RequestHandlerThread[#450]" daemon prio=5 RUNNABLE

com4j.Win32Lock.suspend0(Native Method)

com4j.Win32Lock.suspend(Win32Lock.java:37)

com4j.ComThread.run0(ComThread.java:146)

com4j.ComThread.run(ComThread.java:134)

"ComThread for Handling GET /configure : RequestHandlerThread[#78]" daemon prio=5 RUNNABLE

com4j.Win32Lock.suspend0(Native Method)

com4j.Win32Lock.suspend(Win32Lock.java:37)

com4j.ComThread.run0(ComThread.java:146)

com4j.ComThread.run(ComThread.java:134)

"ComThread for Handling GET /configure : RequestHandlerThread[#974]" daemon prio=5 RUNNABLE

com4j.Win32Lock.suspend0(Native Method)

com4j.Win32Lock.suspend(Win32Lock.java:37)

com4j.ComThread.run0(ComThread.java:146)

com4j.ComThread.run(ComThread.java:134)

"ComThread for Handling POST /configSubmit : RequestHandlerThread[#982]" daemon prio=5 RUNNABLE

com4j.Win32Lock.suspend0(Native Method)

com4j.Win32Lock.suspend(Win32Lock.java:37)

com4j.ComThread.run0(ComThread.java:146)

com4j.ComThread.run(ComThread.java:134)

"ConnectorThread:[ajp13-8009]" daemon prio=5 RUNNABLE

java.net.DualStackPlainSocketImpl.waitForNewConnection(Native Method)

java.net.DualStackPlainSocketImpl.socketAccept(Unknown Source)

java.net.AbstractPlainSocketImpl.accept(Unknown Source)

java.net.PlainSocketImpl.accept(Unknown Source)

java.net.ServerSocket.implAccept(Unknown Source)

java.net.ServerSocket.accept(Unknown Source)

winstone.ajp13.Ajp13Listener.run(Ajp13Listener.java:116)

java.lang.Thread.run(Unknown Source)

"ConnectorThread:[http-8080]" daemon prio=5 RUNNABLE

java.net.DualStackPlainSocketImpl.waitForNewConnection(Native Method)

java.net.DualStackPlainSocketImpl.socketAccept(Unknown Source)

java.net.AbstractPlainSocketImpl.accept(Unknown Source)

java.net.PlainSocketImpl.accept(Unknown Source)

java.net.ServerSocket.implAccept(Unknown Source)

java.net.ServerSocket.accept(Unknown Source)

winstone.HttpListener.run(HttpListener.java:136)

java.lang.Thread.run(Unknown Source)

"DestroyJavaVM" prio=5 RUNNABLE

"Executor #0 for SOME-MACHINE-02" prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

hudson.util.OneShotEvent.block(OneShotEvent.java:72)

hudson.model.Queue.pop(Queue.java:875)

hudson.model.Executor.grabJob(Executor.java:284)

hudson.model.Executor.run(Executor.java:205)

"Executor #0 for SOME-MACHINE-03" prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

hudson.util.OneShotEvent.block(OneShotEvent.java:72)

hudson.model.Queue.pop(Queue.java:875)

hudson.model.Executor.grabJob(Executor.java:284)

hudson.model.Executor.run(Executor.java:205)

"Executor #0 for SOME-MACHINE-04" prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

hudson.util.OneShotEvent.block(OneShotEvent.java:72)

hudson.model.Queue.pop(Queue.java:875)

hudson.model.Executor.grabJob(Executor.java:284)

hudson.model.Executor.run(Executor.java:205)

"Executor #0 for SOME-MACHINE-05 : executing gsm-impl #458 / waiting for hudson.remoting.Channel@4bff6bf4:SOME-MACHINE-05" prio=5 TIMED_WAITING

java.lang.Object.wait(Native Method)

hudson.remoting.Request.call(Request.java:146)

hudson.remoting.Channel.call(Channel.java:664)

hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:158)

$Proxy41.join(Unknown Source)

hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:868)

hudson.Launcher$ProcStarter.join(Launcher.java:352)

hudson.tasks.Maven.perform(Maven.java:265)

hudson.plugins.sonar.utils.SonarMaven.executeMaven(SonarMaven.java:150)

hudson.plugins.sonar.SonarPublisher.executeSonar(SonarPublisher.java:332)

hudson.plugins.sonar.SonarPublisher.perform(SonarPublisher.java:293)

hudson.tasks.BuildStepMonitor$3.perform(BuildStepMonitor.java:36)

hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:717)

hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:692)

hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.post2(MavenModuleSetBuild.java:994)

hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:639)

hudson.model.Run.execute(Run.java:1527)

hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:477)

hudson.model.ResourceController.execute(ResourceController.java:88)

hudson.model.Executor.run(Executor.java:236)

"Executor #0 for SOME-MACHINE-06" prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

hudson.util.OneShotEvent.block(OneShotEvent.java:72)

hudson.model.Queue.pop(Queue.java:875)

hudson.model.Executor.grabJob(Executor.java:284)

hudson.model.Executor.run(Executor.java:205)

"Executor #0 for SOME-MACHINE-07" prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

hudson.util.OneShotEvent.block(OneShotEvent.java:72)

hudson.model.Queue.pop(Queue.java:875)

hudson.model.Executor.grabJob(Executor.java:284)

hudson.model.Executor.run(Executor.java:205)

"Executor #0 for SOME-MACHINE-08" prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

hudson.util.OneShotEvent.block(OneShotEvent.java:72)

hudson.model.Queue.pop(Queue.java:875)

hudson.model.Executor.grabJob(Executor.java:284)

hudson.model.Executor.run(Executor.java:205)

"Executor #0 for some-machine-01" prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

hudson.util.OneShotEvent.block(OneShotEvent.java:72)

hudson.model.Queue.pop(Queue.java:875)

hudson.model.Executor.grabJob(Executor.java:284)

hudson.model.Executor.run(Executor.java:205)

"Executor #0 for some-machine-02" prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

hudson.util.OneShotEvent.block(OneShotEvent.java:72)

hudson.model.Queue.pop(Queue.java:875)

hudson.model.Executor.grabJob(Executor.java:284)

hudson.model.Executor.run(Executor.java:205)

"Executor #0 for some-machine-03" prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

hudson.util.OneShotEvent.block(OneShotEvent.java:72)

hudson.model.Queue.pop(Queue.java:875)

hudson.model.Executor.grabJob(Executor.java:284)

hudson.model.Executor.run(Executor.java:205)

"Executor #0 for BVT-1 : executing BVT-CI-MAIN-Win32 #1416 / waiting for hudson.remoting.Channel@288b2277:BVT-1" daemon prio=5 TIMED_WAITING

java.lang.Object.wait(Native Method)

hudson.remoting.Request.call(Request.java:146)

hudson.remoting.Channel.call(Channel.java:664)

hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:158)

$Proxy41.join(Unknown Source)

hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:868)

hudson.Launcher$ProcStarter.join(Launcher.java:352)

hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:82)

hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58)

hudson.plugins.templateproject.ProxyBuilder.perform(ProxyBuilder.java:83)

hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)

hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:717)

hudson.model.Build$BuildExecution.build(Build.java:199)

hudson.model.Build$BuildExecution.doRun(Build.java:160)

hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)

hudson.model.Run.execute(Run.java:1502)

hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)

hudson.model.ResourceController.execute(ResourceController.java:88)

hudson.model.Executor.run(Executor.java:236)

"Executor #0 for BVT-MacOSX" prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

hudson.util.OneShotEvent.block(OneShotEvent.java:72)

hudson.model.Queue.pop(Queue.java:875)

hudson.model.Executor.grabJob(Executor.java:284)

hudson.model.Executor.run(Executor.java:205)

"Executor #0 for BVT-MacOSX-Test" prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

hudson.util.OneShotEvent.block(OneShotEvent.java:72)

hudson.model.Queue.pop(Queue.java:875)

hudson.model.Executor.grabJob(Executor.java:284)

hudson.model.Executor.run(Executor.java:205)

"Executor #0 for BVT-Win32-Test" prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

hudson.util.OneShotEvent.block(OneShotEvent.java:72)

hudson.model.Queue.pop(Queue.java:875)

hudson.model.Executor.grabJob(Executor.java:284)

hudson.model.Executor.run(Executor.java:205)

"Executor #0 for XXX-BUILD-3" prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

hudson.util.OneShotEvent.block(OneShotEvent.java:72)

hudson.model.Queue.pop(Queue.java:875)

hudson.model.Executor.grabJob(Executor.java:284)

hudson.model.Executor.run(Executor.java:205)

"Executor #0 for XXXmobile" prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

hudson.util.OneShotEvent.block(OneShotEvent.java:72)

hudson.model.Queue.pop(Queue.java:875)

hudson.model.Executor.grabJob(Executor.java:284)

hudson.model.Executor.run(Executor.java:205)

"Executor #0 for XXXbuild-1 : executing New-Code-CI-__MAIN__-Mac #870 / waiting for hudson.remoting.Channel@7b253755:XXXbuild-1" prio=5 TIMED_WAITING

java.lang.Object.wait(Native Method)

hudson.remoting.Request.call(Request.java:146)

hudson.remoting.Channel.call(Channel.java:664)

hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:158)

$Proxy41.join(Unknown Source)

hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:868)

hudson.Launcher$ProcStarter.join(Launcher.java:352)

hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:82)

hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58)

hudson.plugins.templateproject.ProxyBuilder.perform(ProxyBuilder.java:83)

hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)

hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:717)

hudson.model.Build$BuildExecution.build(Build.java:199)

hudson.model.Build$BuildExecution.doRun(Build.java:160)

hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)

hudson.model.Run.execute(Run.java:1502)

hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)

hudson.model.ResourceController.execute(ResourceController.java:88)

hudson.model.Executor.run(Executor.java:236)

"Executor #0 for XXXbuild-2" prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

hudson.util.OneShotEvent.block(OneShotEvent.java:72)

hudson.model.Queue.pop(Queue.java:875)

hudson.model.Executor.grabJob(Executor.java:284)

hudson.model.Executor.run(Executor.java:205)

"Executor #0 for master" prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

hudson.util.OneShotEvent.block(OneShotEvent.java:72)

hudson.model.Queue.pop(Queue.java:875)

hudson.model.Executor.grabJob(Executor.java:284)

hudson.model.Executor.run(Executor.java:205)

"Executor #1 for SOME-MACHINE-02" prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

hudson.util.OneShotEvent.block(OneShotEvent.java:72)

hudson.model.Queue.pop(Queue.java:875)

hudson.model.Executor.grabJob(Executor.java:284)

hudson.model.Executor.run(Executor.java:205)

"Executor #1 for SOME-MACHINE-03" prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

hudson.util.OneShotEvent.block(OneShotEvent.java:72)

hudson.model.Queue.pop(Queue.java:875)

hudson.model.Executor.grabJob(Executor.java:284)

hudson.model.Executor.run(Executor.java:205)

"Executor #1 for SOME-MACHINE-04" prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

hudson.util.OneShotEvent.block(OneShotEvent.java:72)

hudson.model.Queue.pop(Queue.java:875)

hudson.model.Executor.grabJob(Executor.java:284)

hudson.model.Executor.run(Executor.java:205)

"Executor #1 for SOME-MACHINE-05" prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

hudson.util.OneShotEvent.block(OneShotEvent.java:72)

hudson.model.Queue.pop(Queue.java:875)

hudson.model.Executor.grabJob(Executor.java:284)

hudson.model.Executor.run(Executor.java:205)

"Executor #1 for SOME-MACHINE-06" prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

hudson.util.OneShotEvent.block(OneShotEvent.java:72)

hudson.model.Queue.pop(Queue.java:875)

hudson.model.Executor.grabJob(Executor.java:284)

hudson.model.Executor.run(Executor.java:205)

"Executor #1 for SOME-MACHINE-07 : executing New-ContentBundle-__MAIN__-DO_NOT_SHIP #2714 / waiting for hudson.remoting.Channel@42131a6a:SOME-MACHINE-07" prio=5 TIMED_WAITING

java.lang.Object.wait(Native Method)

hudson.remoting.Request.call(Request.java:146)

hudson.remoting.Channel.call(Channel.java:664)

hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:158)

$Proxy41.join(Unknown Source)

hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:868)

hudson.Launcher$ProcStarter.join(Launcher.java:352)

hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:82)

hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58)

hudson.plugins.templateproject.ProxyBuilder.perform(ProxyBuilder.java:83)

hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)

hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:717)

hudson.model.Build$BuildExecution.build(Build.java:199)

hudson.model.Build$BuildExecution.doRun(Build.java:160)

hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)

hudson.model.Run.execute(Run.java:1502)

hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)

hudson.model.ResourceController.execute(ResourceController.java:88)

hudson.model.Executor.run(Executor.java:236)

"Executor #1 for SOME-MACHINE-08" prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

hudson.util.OneShotEvent.block(OneShotEvent.java:72)

hudson.model.Queue.pop(Queue.java:875)

hudson.model.Executor.grabJob(Executor.java:284)

hudson.model.Executor.run(Executor.java:205)

"Executor #1 for some-machine-01" prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

hudson.util.OneShotEvent.block(OneShotEvent.java:72)

hudson.model.Queue.pop(Queue.java:875)

hudson.model.Executor.grabJob(Executor.java:284)

hudson.model.Executor.run(Executor.java:205)

"Executor #1 for some-machine-02" prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

hudson.util.OneShotEvent.block(OneShotEvent.java:72)

hudson.model.Queue.pop(Queue.java:875)

hudson.model.Executor.grabJob(Executor.java:284)

hudson.model.Executor.run(Executor.java:205)

"Executor #1 for some-machine-03" prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

hudson.util.OneShotEvent.block(OneShotEvent.java:72)

hudson.model.Queue.pop(Queue.java:875)

hudson.model.Executor.grabJob(Executor.java:284)

hudson.model.Executor.run(Executor.java:205)

"Executor #1 for XXX-BUILD-3" prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

hudson.util.OneShotEvent.block(OneShotEvent.java:72)

hudson.model.Queue.pop(Queue.java:875)

hudson.model.Executor.grabJob(Executor.java:284)

hudson.model.Executor.run(Executor.java:205)

"Executor #1 for XXXmobile" prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

hudson.util.OneShotEvent.block(OneShotEvent.java:72)

hudson.model.Queue.pop(Queue.java:875)

hudson.model.Executor.grabJob(Executor.java:284)

hudson.model.Executor.run(Executor.java:205)

"Executor #1 for XXXbuild-1" prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

hudson.util.OneShotEvent.block(OneShotEvent.java:72)

hudson.model.Queue.pop(Queue.java:875)

hudson.model.Executor.grabJob(Executor.java:284)

hudson.model.Executor.run(Executor.java:205)

"Executor #1 for XXXbuild-2" prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

hudson.util.OneShotEvent.block(OneShotEvent.java:72)

hudson.model.Queue.pop(Queue.java:875)

hudson.model.Executor.grabJob(Executor.java:284)

hudson.model.Executor.run(Executor.java:205)

"Executor #2 for SOME-MACHINE-02" prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

hudson.util.OneShotEvent.block(OneShotEvent.java:72)

hudson.model.Queue.pop(Queue.java:875)

hudson.model.Executor.grabJob(Executor.java:284)

hudson.model.Executor.run(Executor.java:205)

"Executor #2 for SOME-MACHINE-03 : executing Core - Platform Module - MAIN #1902 / waiting for hudson.slaves.Channels$1@16e03dcd:Channel to Maven [C:\Program Files\Java\jdk1.6.0_29/bin/java, -Xmx4096M, -Xms4096M, -XX:MaxPermSize=512M, -Dtangosol.coherence.ttl=0, -Dtangosol.coherence.clusterport=27851, -Dtangosol.coherence.cluster=jmc_laxbuild, -XX:ReservedCodeCacheSize=128m, -cp, C:\jenkins\maven3-agent.jar;C:\apache-maven-3.0.4\boot\plexus-classworlds-2.4.jar, org.jvnet.hudson.maven3.agent.Maven3Main, C:\apache-maven-3.0.4, C:\jenkins\slave.jar, C:\jenkins\maven3-interceptor.jar, 59104]" prio=5 TIMED_WAITING

java.lang.Object.wait(Native Method)

hudson.remoting.Request.call(Request.java:146)

hudson.remoting.Channel.call(Channel.java:664)

hudson.maven.ProcessCache$MavenProcess.call(ProcessCache.java:156)

hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:791)

hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)

hudson.model.Run.execute(Run.java:1502)

hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:477)

hudson.model.ResourceController.execute(ResourceController.java:88)

hudson.model.Executor.run(Executor.java:236)

"Executor #2 for SOME-MACHINE-04 : executing Platform-Server-CI-MAIN #2942 / waiting for hudson.remoting.Channel@364c8db2:SOME-MACHINE-04" prio=5 TIMED_WAITING

java.lang.Object.wait(Native Method)

hudson.remoting.Request.call(Request.java:146)

hudson.remoting.Channel.call(Channel.java:664)

hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:158)

$Proxy41.join(Unknown Source)

hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:868)

hudson.Launcher$ProcStarter.join(Launcher.java:352)

hudson.tasks.Maven.perform(Maven.java:265)

hudson.plugins.templateproject.ProxyBuilder.perform(ProxyBuilder.java:83)

hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)

hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:717)

hudson.model.Build$BuildExecution.build(Build.java:199)

hudson.model.Build$BuildExecution.doRun(Build.java:160)

hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)

hudson.model.Run.execute(Run.java:1502)

hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)

hudson.model.ResourceController.execute(ResourceController.java:88)

hudson.model.Executor.run(Executor.java:236)

"Executor #2 for SOME-MACHINE-05" prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

hudson.util.OneShotEvent.block(OneShotEvent.java:72)

hudson.model.Queue.pop(Queue.java:875)

hudson.model.Executor.grabJob(Executor.java:284)

hudson.model.Executor.run(Executor.java:205)

"Executor #2 for SOME-MACHINE-06" prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

hudson.util.OneShotEvent.block(OneShotEvent.java:72)

hudson.model.Queue.pop(Queue.java:875)

hudson.model.Executor.grabJob(Executor.java:284)

hudson.model.Executor.run(Executor.java:205)

"Executor #2 for SOME-MACHINE-07 : executing New-ContentBundle-__MAIN__ #4083 / waiting for hudson.remoting.Channel@42131a6a:SOME-MACHINE-07" prio=5 TIMED_WAITING

java.lang.Object.wait(Native Method)

hudson.remoting.Request.call(Request.java:146)

hudson.remoting.Channel.call(Channel.java:664)

hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:158)

$Proxy41.join(Unknown Source)

hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:868)

hudson.Launcher$ProcStarter.join(Launcher.java:352)

hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:82)

hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58)

hudson.plugins.templateproject.ProxyBuilder.perform(ProxyBuilder.java:83)

hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)

hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:717)

hudson.model.Build$BuildExecution.build(Build.java:199)

hudson.model.Build$BuildExecution.doRun(Build.java:160)

hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)

hudson.model.Run.execute(Run.java:1502)

hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)

hudson.model.ResourceController.execute(ResourceController.java:88)

hudson.model.Executor.run(Executor.java:236)

"Executor #2 for SOME-MACHINE-08 : executing New-Content-CI-__MAIN__ #5931 / waiting for hudson.remoting.Channel@41741938:SOME-MACHINE-08" prio=5 TIMED_WAITING

java.lang.Object.wait(Native Method)

hudson.remoting.Request.call(Request.java:146)

hudson.remoting.Channel.call(Channel.java:664)

hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:158)

$Proxy41.join(Unknown Source)

hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:868)

hudson.Launcher$ProcStarter.join(Launcher.java:352)

hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:82)

hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58)

hudson.plugins.templateproject.ProxyBuilder.perform(ProxyBuilder.java:83)

hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)

hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:717)

hudson.model.Build$BuildExecution.build(Build.java:199)

hudson.model.Build$BuildExecution.doRun(Build.java:160)

hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)

hudson.model.Run.execute(Run.java:1502)

hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)

hudson.model.ResourceController.execute(ResourceController.java:88)

hudson.model.Executor.run(Executor.java:236)

"Executor #2 for some-machine-01" prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

hudson.util.OneShotEvent.block(OneShotEvent.java:72)

hudson.model.Queue.pop(Queue.java:875)

hudson.model.Executor.grabJob(Executor.java:284)

hudson.model.Executor.run(Executor.java:205)

"Executor #2 for some-machine-02" prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

hudson.util.OneShotEvent.block(OneShotEvent.java:72)

hudson.model.Queue.pop(Queue.java:875)

hudson.model.Executor.grabJob(Executor.java:284)

hudson.model.Executor.run(Executor.java:205)

"Executor #2 for some-machine-03" prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

hudson.util.OneShotEvent.block(OneShotEvent.java:72)

hudson.model.Queue.pop(Queue.java:875)

hudson.model.Executor.grabJob(Executor.java:284)

hudson.model.Executor.run(Executor.java:205)

"Executor #2 for XXXmobile" prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

hudson.util.OneShotEvent.block(OneShotEvent.java:72)

hudson.model.Queue.pop(Queue.java:875)

hudson.model.Executor.grabJob(Executor.java:284)

hudson.model.Executor.run(Executor.java:205)

"Executor #2 for XXXbuild-1" prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

hudson.util.OneShotEvent.block(OneShotEvent.java:72)

hudson.model.Queue.pop(Queue.java:875)

hudson.model.Executor.grabJob(Executor.java:284)

hudson.model.Executor.run(Executor.java:205)

"Executor #2 for XXXbuild-2" prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

hudson.util.OneShotEvent.block(OneShotEvent.java:72)

hudson.model.Queue.pop(Queue.java:875)

hudson.model.Executor.grabJob(Executor.java:284)

hudson.model.Executor.run(Executor.java:205)

"Executor #3 for SOME-MACHINE-02" prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

hudson.util.OneShotEvent.block(OneShotEvent.java:72)

hudson.model.Queue.pop(Queue.java:875)

hudson.model.Executor.grabJob(Executor.java:284)

hudson.model.Executor.run(Executor.java:205)

"Executor #3 for SOME-MACHINE-03" prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

hudson.util.OneShotEvent.block(OneShotEvent.java:72)

hudson.model.Queue.pop(Queue.java:875)

hudson.model.Executor.grabJob(Executor.java:284)

hudson.model.Executor.run(Executor.java:205)

"Executor #3 for SOME-MACHINE-04" prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

hudson.util.OneShotEvent.block(OneShotEvent.java:72)

hudson.model.Queue.pop(Queue.java:875)

hudson.model.Executor.grabJob(Executor.java:284)

hudson.model.Executor.run(Executor.java:205)

"Executor #3 for SOME-MACHINE-05" prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

hudson.util.OneShotEvent.block(OneShotEvent.java:72)

hudson.model.Queue.pop(Queue.java:875)

hudson.model.Executor.grabJob(Executor.java:284)

hudson.model.Executor.run(Executor.java:205)

"Executor #3 for SOME-MACHINE-06" prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

hudson.util.OneShotEvent.block(OneShotEvent.java:72)

hudson.model.Queue.pop(Queue.java:875)

hudson.model.Executor.grabJob(Executor.java:284)

hudson.model.Executor.run(Executor.java:205)

"Executor #3 for SOME-MACHINE-07" prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

hudson.util.OneShotEvent.block(OneShotEvent.java:72)

hudson.model.Queue.pop(Queue.java:875)

hudson.model.Executor.grabJob(Executor.java:284)

hudson.model.Executor.run(Executor.java:205)

"Executor #3 for SOME-MACHINE-08" prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

hudson.util.OneShotEvent.block(OneShotEvent.java:72)

hudson.model.Queue.pop(Queue.java:875)

hudson.model.Executor.grabJob(Executor.java:284)

hudson.model.Executor.run(Executor.java:205)

"Executor #3 for some-machine-01" prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

hudson.util.OneShotEvent.block(OneShotEvent.java:72)

hudson.model.Queue.pop(Queue.java:875)

hudson.model.Executor.grabJob(Executor.java:284)

hudson.model.Executor.run(Executor.java:205)

"Executor #3 for some-machine-02 : executing LoadTest-Monitor #288 / waiting for hudson.remoting.Channel@6686064c:some-machine-02" prio=5 TIMED_WAITING

java.lang.Object.wait(Native Method)

hudson.remoting.Request.call(Request.java:146)

hudson.remoting.Channel.call(Channel.java:664)

hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:158)

$Proxy41.join(Unknown Source)

hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:868)

hudson.Launcher$ProcStarter.join(Launcher.java:352)

hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:82)

hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58)

hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)

hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:717)

hudson.model.Build$BuildExecution.build(Build.java:199)

hudson.model.Build$BuildExecution.doRun(Build.java:160)

hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)

hudson.model.Run.execute(Run.java:1502)

hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)

hudson.model.ResourceController.execute(ResourceController.java:88)

hudson.model.Executor.run(Executor.java:236)

"Executor #3 for some-machine-03 : executing DWDWargDeploy #211 / waiting for hudson.remoting.Channel@367c02a:some-machine-03" prio=5 TIMED_WAITING

java.lang.Object.wait(Native Method)

hudson.remoting.Request.call(Request.java:146)

hudson.remoting.Channel.call(Channel.java:664)

hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:158)

$Proxy41.join(Unknown Source)

hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:868)

hudson.Launcher$ProcStarter.join(Launcher.java:352)

hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:82)

hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58)

hudson.plugins.templateproject.ProxyBuilder.perform(ProxyBuilder.java:83)

hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)

hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:717)

hudson.model.Build$BuildExecution.build(Build.java:199)

hudson.model.Build$BuildExecution.doRun(Build.java:160)

hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)

hudson.model.Run.execute(Run.java:1502)

hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)

hudson.model.ResourceController.execute(ResourceController.java:88)

hudson.model.Executor.run(Executor.java:236)

"Executor #3 for XXXmobile" prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

hudson.util.OneShotEvent.block(OneShotEvent.java:72)

hudson.model.Queue.pop(Queue.java:875)

hudson.model.Executor.grabJob(Executor.java:284)

hudson.model.Executor.run(Executor.java:205)

"Executor #3 for XXXbuild-1" prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

hudson.util.OneShotEvent.block(OneShotEvent.java:72)

hudson.model.Queue.pop(Queue.java:875)

hudson.model.Executor.grabJob(Executor.java:284)

hudson.model.Executor.run(Executor.java:205)

"Executor #3 for XXXbuild-2" prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

hudson.util.OneShotEvent.block(OneShotEvent.java:72)

hudson.model.Queue.pop(Queue.java:875)

hudson.model.Executor.grabJob(Executor.java:284)

hudson.model.Executor.run(Executor.java:205)

"Finalizer" daemon prio=8 WAITING

java.lang.Object.wait(Native Method)

java.lang.ref.ReferenceQueue.remove(Unknown Source)

java.lang.ref.ReferenceQueue.remove(Unknown Source)

java.lang.ref.Finalizer$FinalizerThread.run(Unknown Source)

"IM-Reconnector-Thread" daemon prio=5 TIMED_WAITING

sun.misc.Unsafe.park(Native Method)

java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)

java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(Unknown Source)

java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(Unknown Source)

java.util.concurrent.Semaphore.tryAcquire(Unknown Source)

hudson.plugins.im.IMConnectionProvider$ConnectorRunnable.run(IMConnectionProvider.java:195)

java.lang.Thread.run(Unknown Source)

"Java2D Disposer" daemon prio=10 WAITING

java.lang.Object.wait(Native Method)

java.lang.ref.ReferenceQueue.remove(Unknown Source)

java.lang.ref.ReferenceQueue.remove(Unknown Source)

sun.java2d.Disposer.run(Unknown Source)

java.lang.Thread.run(Unknown Source)

"javamelody" daemon prio=5 TIMED_WAITING

java.lang.Object.wait(Native Method)

java.util.TimerThread.mainLoop(Unknown Source)

java.util.TimerThread.run(Unknown Source)

"Jenkins cron thread" prio=5 TIMED_WAITING

java.lang.Object.wait(Native Method)

java.util.TimerThread.mainLoop(Unknown Source)

java.util.TimerThread.run(Unknown Source)

"Jenkins UDP 33848 monitoring thread" prio=5 RUNNABLE

java.net.TwoStacksPlainDatagramSocketImpl.receive0(Native Method)

java.net.TwoStacksPlainDatagramSocketImpl.receive(Unknown Source)

java.net.DatagramSocket.receive(Unknown Source)

hudson.UDPBroadcastThread.run(UDPBroadcastThread.java:79)

"JmDNS(SOME-MACHINE-01.local.).State.Timer" prio=5 TIMED_WAITING

java.lang.Object.wait(Native Method)

java.util.TimerThread.mainLoop(Unknown Source)

java.util.TimerThread.run(Unknown Source)

"JmDNS(SOME-MACHINE-01.local.).Timer" daemon prio=5 TIMED_WAITING

java.lang.Object.wait(Native Method)

java.util.TimerThread.mainLoop(Unknown Source)

java.util.TimerThread.run(Unknown Source)

"jrobin " daemon prio=5 TIMED_WAITING

java.lang.Object.wait(Native Method)

java.util.TimerThread.mainLoop(Unknown Source)

java.util.TimerThread.run(Unknown Source)

"LauncherControlThread[ControlPort=-1]" prio=5 TIMED_WAITING

java.lang.Thread.sleep(Native Method)

winstone.Launcher.run(Launcher.java:265)

java.lang.Thread.run(Unknown Source)

"NioSocketAcceptor-1" prio=5 RUNNABLE

sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)

sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(Unknown Source)

sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(Unknown Source)

sun.nio.ch.WindowsSelectorImpl.doSelect(Unknown Source)

sun.nio.ch.SelectorImpl.lockAndDoSelect(Unknown Source)

sun.nio.ch.SelectorImpl.select(Unknown Source)

sun.nio.ch.SelectorImpl.select(Unknown Source)

org.apache.mina.transport.socket.nio.NioSocketAcceptor.select(NioSocketAcceptor.java:285)

org.apache.mina.core.polling.AbstractPollingIoAcceptor$Acceptor.run(AbstractPollingIoAcceptor.java:401)

org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)

java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

java.lang.Thread.run(Unknown Source)

"Ping thread for channel hudson.remoting.Channel@244c7c34:XXXmobile" daemon prio=5 TIMED_WAITING

java.lang.Thread.sleep(Native Method)

hudson.remoting.PingThread.run(PingThread.java:86)

"Ping thread for channel hudson.remoting.Channel@288b2277:BVT-1" daemon prio=5 TIMED_WAITING

java.lang.Thread.sleep(Native Method)

hudson.remoting.PingThread.run(PingThread.java:86)

"Ping thread for channel hudson.remoting.Channel@2b89081e:XXXbuild-2" daemon prio=5 TIMED_WAITING

java.lang.Thread.sleep(Native Method)

hudson.remoting.PingThread.run(PingThread.java:86)

"Ping thread for channel hudson.remoting.Channel@364c8db2:SOME-MACHINE-04" daemon prio=5 TIMED_WAITING

java.lang.Thread.sleep(Native Method)

hudson.remoting.PingThread.run(PingThread.java:86)

"Ping thread for channel hudson.remoting.Channel@367c02a:some-machine-03" daemon prio=5 TIMED_WAITING

java.lang.Thread.sleep(Native Method)

hudson.remoting.PingThread.run(PingThread.java:86)

"Ping thread for channel hudson.remoting.Channel@3b82bcb0:some-machine-01" daemon prio=5 TIMED_WAITING

java.lang.Thread.sleep(Native Method)

hudson.remoting.PingThread.run(PingThread.java:86)

"Ping thread for channel hudson.remoting.Channel@41741938:SOME-MACHINE-08" daemon prio=5 TIMED_WAITING

java.lang.Thread.sleep(Native Method)

hudson.remoting.PingThread.run(PingThread.java:86)

"Ping thread for channel hudson.remoting.Channel@42131a6a:SOME-MACHINE-07" daemon prio=5 TIMED_WAITING

java.lang.Thread.sleep(Native Method)

hudson.remoting.PingThread.run(PingThread.java:86)

"Ping thread for channel hudson.remoting.Channel@4bff6bf4:SOME-MACHINE-05" daemon prio=5 TIMED_WAITING

java.lang.Thread.sleep(Native Method)

hudson.remoting.PingThread.run(PingThread.java:86)

"Ping thread for channel hudson.remoting.Channel@6686064c:some-machine-02" daemon prio=5 TIMED_WAITING

java.lang.Thread.sleep(Native Method)

hudson.remoting.PingThread.run(PingThread.java:86)

"Ping thread for channel hudson.remoting.Channel@6a554075:SOME-MACHINE-03" daemon prio=5 TIMED_WAITING

java.lang.Thread.sleep(Native Method)

hudson.remoting.PingThread.run(PingThread.java:86)

"Ping thread for channel hudson.remoting.Channel@6cd0d116:SOME-MACHINE-06" daemon prio=5 TIMED_WAITING

java.lang.Thread.sleep(Native Method)

hudson.remoting.PingThread.run(PingThread.java:86)

"Ping thread for channel hudson.remoting.Channel@7b253755:XXXbuild-1" daemon prio=5 TIMED_WAITING

java.lang.Thread.sleep(Native Method)

hudson.remoting.PingThread.run(PingThread.java:86)

"Ping thread for channel hudson.remoting.Channel@a3d234c:BVT-MacOSX" daemon prio=5 TIMED_WAITING

java.lang.Thread.sleep(Native Method)

hudson.remoting.PingThread.run(PingThread.java:86)

"Ping thread for channel hudson.remoting.Channel@eb8cfa7:SOME-MACHINE-02" daemon prio=5 TIMED_WAITING

java.lang.Thread.sleep(Native Method)

hudson.remoting.PingThread.run(PingThread.java:86)

"Pipe writer thread: SOME-MACHINE-02" prio=5 WAITING

sun.misc.Unsafe.park(Native Method)

java.util.concurrent.locks.LockSupport.park(Unknown Source)

java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)

java.util.concurrent.LinkedBlockingQueue.take(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

java.lang.Thread.run(Unknown Source)

"Pipe writer thread: SOME-MACHINE-03" prio=5 WAITING

sun.misc.Unsafe.park(Native Method)

java.util.concurrent.locks.LockSupport.park(Unknown Source)

java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)

java.util.concurrent.LinkedBlockingQueue.take(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

java.lang.Thread.run(Unknown Source)

"Pipe writer thread: SOME-MACHINE-04" prio=5 WAITING

sun.misc.Unsafe.park(Native Method)

java.util.concurrent.locks.LockSupport.park(Unknown Source)

java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)

java.util.concurrent.LinkedBlockingQueue.take(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

java.lang.Thread.run(Unknown Source)

"Pipe writer thread: SOME-MACHINE-05" prio=5 WAITING

sun.misc.Unsafe.park(Native Method)

java.util.concurrent.locks.LockSupport.park(Unknown Source)

java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)

java.util.concurrent.LinkedBlockingQueue.take(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

java.lang.Thread.run(Unknown Source)

"Pipe writer thread: SOME-MACHINE-06" prio=5 WAITING

sun.misc.Unsafe.park(Native Method)

java.util.concurrent.locks.LockSupport.park(Unknown Source)

java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)

java.util.concurrent.LinkedBlockingQueue.take(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

java.lang.Thread.run(Unknown Source)

"Pipe writer thread: SOME-MACHINE-07" prio=5 WAITING

sun.misc.Unsafe.park(Native Method)

java.util.concurrent.locks.LockSupport.park(Unknown Source)

java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)

java.util.concurrent.LinkedBlockingQueue.take(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

java.lang.Thread.run(Unknown Source)

"Pipe writer thread: SOME-MACHINE-08" prio=5 WAITING

sun.misc.Unsafe.park(Native Method)

java.util.concurrent.locks.LockSupport.park(Unknown Source)

java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)

java.util.concurrent.LinkedBlockingQueue.take(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

java.lang.Thread.run(Unknown Source)

"Pipe writer thread: some-machine-01" prio=5 WAITING

sun.misc.Unsafe.park(Native Method)

java.util.concurrent.locks.LockSupport.park(Unknown Source)

java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)

java.util.concurrent.LinkedBlockingQueue.take(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

java.lang.Thread.run(Unknown Source)

"Pipe writer thread: some-machine-02" prio=5 WAITING

sun.misc.Unsafe.park(Native Method)

java.util.concurrent.locks.LockSupport.park(Unknown Source)

java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)

java.util.concurrent.LinkedBlockingQueue.take(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

java.lang.Thread.run(Unknown Source)

"Pipe writer thread: some-machine-03" prio=5 WAITING

sun.misc.Unsafe.park(Native Method)

java.util.concurrent.locks.LockSupport.park(Unknown Source)

java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)

java.util.concurrent.LinkedBlockingQueue.take(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

java.lang.Thread.run(Unknown Source)

"Pipe writer thread: BVT-1" prio=5 WAITING

sun.misc.Unsafe.park(Native Method)

java.util.concurrent.locks.LockSupport.park(Unknown Source)

java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)

java.util.concurrent.LinkedBlockingQueue.take(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

java.lang.Thread.run(Unknown Source)

"Pipe writer thread: Channel to Maven [C:\Program Files\Java\jdk1.6.0_29/bin/java, -Xmx4096M, -Xms4096M, -XX:MaxPermSize=512M, -Dtangosol.coherence.ttl=0, -Dtangosol.coherence.clusterport=27851, -Dtangosol.coherence.cluster=jmc_laxbuild, -XX:ReservedCodeCacheSize=128m, -cp, C:\jenkins\maven3-agent.jar;C:\apache-maven-3.0.4\boot\plexus-classworlds-2.4.jar, org.jvnet.hudson.maven3.agent.Maven3Main, C:\apache-maven-3.0.4, C:\jenkins\slave.jar, C:\jenkins\maven3-interceptor.jar, 59104]" prio=5 WAITING

sun.misc.Unsafe.park(Native Method)

java.util.concurrent.locks.LockSupport.park(Unknown Source)

java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)

java.util.concurrent.LinkedBlockingQueue.take(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

java.lang.Thread.run(Unknown Source)

"Pipe writer thread: XXXmobile" daemon prio=5 WAITING

sun.misc.Unsafe.park(Native Method)

java.util.concurrent.locks.LockSupport.park(Unknown Source)

java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)

java.util.concurrent.LinkedBlockingQueue.take(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

java.lang.Thread.run(Unknown Source)

"Pipe writer thread: XXXbuild-1" daemon prio=5 WAITING

sun.misc.Unsafe.park(Native Method)

java.util.concurrent.locks.LockSupport.park(Unknown Source)

java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)

java.util.concurrent.LinkedBlockingQueue.take(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

java.lang.Thread.run(Unknown Source)

"Pipe writer thread: XXXbuild-2" daemon prio=5 WAITING

sun.misc.Unsafe.park(Native Method)

java.util.concurrent.locks.LockSupport.park(Unknown Source)

java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)

java.util.concurrent.LinkedBlockingQueue.take(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

java.lang.Thread.run(Unknown Source)

"pool-14-thread-1" daemon prio=5 RUNNABLE

java.io.FileInputStream.readBytes(Native Method)

java.io.FileInputStream.read(Unknown Source)

java.io.BufferedInputStream.fill(Unknown Source)

java.io.BufferedInputStream.read(Unknown Source)

hudson.remoting.ClassicCommandTransport.create(ClassicCommandTransport.java:98)

hudson.remoting.Channel.<init>(Channel.java:391)

hudson.remoting.Channel.<init>(Channel.java:387)

hudson.remoting.Channel.<init>(Channel.java:348)

hudson.remoting.Channel.<init>(Channel.java:344)

hudson.slaves.SlaveComputer.setChannel(SlaveComputer.java:316)

hudson.slaves.CommandLauncher.launch(CommandLauncher.java:122)

hudson.slaves.SlaveComputer$1.call(SlaveComputer.java:200)

java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)

java.util.concurrent.FutureTask.run(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

java.lang.Thread.run(Unknown Source)

"pool-14-thread-1157" daemon prio=5 TIMED_WAITING

sun.misc.Unsafe.park(Native Method)

java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)

java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(Unknown Source)

java.util.concurrent.SynchronousQueue$TransferStack.transfer(Unknown Source)

java.util.concurrent.SynchronousQueue.poll(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

java.lang.Thread.run(Unknown Source)

"pool-14-thread-2" daemon prio=5 RUNNABLE

java.io.FileInputStream.readBytes(Native Method)

java.io.FileInputStream.read(Unknown Source)

java.io.BufferedInputStream.fill(Unknown Source)

java.io.BufferedInputStream.read(Unknown Source)

hudson.remoting.ClassicCommandTransport.create(ClassicCommandTransport.java:98)

hudson.remoting.Channel.<init>(Channel.java:391)

hudson.remoting.Channel.<init>(Channel.java:387)

hudson.remoting.Channel.<init>(Channel.java:348)

hudson.remoting.Channel.<init>(Channel.java:344)

hudson.slaves.SlaveComputer.setChannel(SlaveComputer.java:316)

hudson.slaves.CommandLauncher.launch(CommandLauncher.java:122)

hudson.slaves.SlaveComputer$1.call(SlaveComputer.java:200)

java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)

java.util.concurrent.FutureTask.run(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

java.lang.Thread.run(Unknown Source)

"pool-14-thread-8" daemon prio=5 RUNNABLE

java.net.DualStackPlainSocketImpl.accept0(Native Method)

java.net.DualStackPlainSocketImpl.socketAccept(Unknown Source)

java.net.AbstractPlainSocketImpl.accept(Unknown Source)

java.net.PlainSocketImpl.accept(Unknown Source)

java.net.ServerSocket.implAccept(Unknown Source)

java.net.ServerSocket.accept(Unknown Source)

com.youdevise.hudson.slavestatus.SocketHTTPListener.waitForConnection(SlaveListener.java:132)

com.youdevise.hudson.slavestatus.SlaveListener$1.run(SlaveListener.java:63)

com.youdevise.hudson.slavestatus.Daemon.go(Daemon.java:16)

com.youdevise.hudson.slavestatus.SlaveListener.call(SlaveListener.java:83)

hudson.remoting.LocalChannel$1.call(LocalChannel.java:52)

java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)

java.util.concurrent.FutureTask.run(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

java.lang.Thread.run(Unknown Source)

"pool-15-thread-1" daemon prio=5 WAITING

sun.misc.Unsafe.park(Native Method)

java.util.concurrent.locks.LockSupport.park(Unknown Source)

java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)

java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)

java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

java.lang.Thread.run(Unknown Source)

"pool-3-thread-1" daemon prio=5 WAITING

sun.misc.Unsafe.park(Native Method)

java.util.concurrent.locks.LockSupport.park(Unknown Source)

java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)

java.util.concurrent.LinkedBlockingQueue.take(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

java.lang.Thread.run(Unknown Source)

"pool-4-thread-1" prio=5 TIMED_WAITING

sun.misc.Unsafe.park(Native Method)

java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)

java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(Unknown Source)

java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)

java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

java.lang.Thread.run(Unknown Source)

"pool-797-thread-1" prio=5 WAITING

sun.misc.Unsafe.park(Native Method)

java.util.concurrent.locks.LockSupport.park(Unknown Source)

java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)

java.util.concurrent.LinkedBlockingQueue.take(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

java.lang.Thread.run(Unknown Source)

"pool-797-thread-10" prio=5 WAITING

sun.misc.Unsafe.park(Native Method)

java.util.concurrent.locks.LockSupport.park(Unknown Source)

java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)

java.util.concurrent.LinkedBlockingQueue.take(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

java.lang.Thread.run(Unknown Source)

"pool-797-thread-2" prio=5 WAITING

sun.misc.Unsafe.park(Native Method)

java.util.concurrent.locks.LockSupport.park(Unknown Source)

java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)

java.util.concurrent.LinkedBlockingQueue.take(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

java.lang.Thread.run(Unknown Source)

"pool-797-thread-3" prio=5 WAITING

sun.misc.Unsafe.park(Native Method)

java.util.concurrent.locks.LockSupport.park(Unknown Source)

java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)

java.util.concurrent.LinkedBlockingQueue.take(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

java.lang.Thread.run(Unknown Source)

"pool-797-thread-4" prio=5 WAITING

sun.misc.Unsafe.park(Native Method)

java.util.concurrent.locks.LockSupport.park(Unknown Source)

java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)

java.util.concurrent.LinkedBlockingQueue.take(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

java.lang.Thread.run(Unknown Source)

"pool-797-thread-5" prio=5 WAITING

sun.misc.Unsafe.park(Native Method)

java.util.concurrent.locks.LockSupport.park(Unknown Source)

java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)

java.util.concurrent.LinkedBlockingQueue.take(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

java.lang.Thread.run(Unknown Source)

"pool-797-thread-7" prio=5 WAITING

sun.misc.Unsafe.park(Native Method)

java.util.concurrent.locks.LockSupport.park(Unknown Source)

java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)

java.util.concurrent.LinkedBlockingQueue.take(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

java.lang.Thread.run(Unknown Source)

"pool-797-thread-8" prio=5 WAITING

sun.misc.Unsafe.park(Native Method)

java.util.concurrent.locks.LockSupport.park(Unknown Source)

java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)

java.util.concurrent.LinkedBlockingQueue.take(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

java.lang.Thread.run(Unknown Source)

"pool-797-thread-9" prio=5 WAITING

sun.misc.Unsafe.park(Native Method)

java.util.concurrent.locks.LockSupport.park(Unknown Source)

java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)

java.util.concurrent.LinkedBlockingQueue.take(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

java.lang.Thread.run(Unknown Source)

"Reference Handler" daemon prio=10 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

java.lang.ref.Reference$ReferenceHandler.run(Unknown Source)

"RequestHandlerThread[#1026]" daemon prio=5 RUNNABLE

java.net.SocketInputStream.socketRead0(Native Method)

java.net.SocketInputStream.read(Unknown Source)

java.net.SocketInputStream.read(Unknown Source)

java.net.SocketInputStream.read(Unknown Source)

winstone.WinstoneInputStream.read(WinstoneInputStream.java:49)

javax.servlet.ServletInputStream.readLine(ServletInputStream.java:27)

winstone.WinstoneInputStream.readLine(WinstoneInputStream.java:108)

winstone.HttpListener.parseURI(HttpListener.java:238)

winstone.RequestHandlerThread.run(RequestHandlerThread.java:75)

java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)

java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)

java.util.concurrent.FutureTask.run(Unknown Source)

winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77)

java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

java.lang.Thread.run(Unknown Source)

"RequestHandlerThread[#1033]" daemon prio=5 RUNNABLE

java.lang.Thread.dumpThreads(Native Method)

java.lang.Thread.getAllStackTraces(Unknown Source)

net.bull.javamelody.JavaInformations.buildThreadInformationsList(JavaInformations.java:286)

net.bull.javamelody.JavaInformations.<init>(JavaInformations.java:187)

net.bull.javamelody.MonitoringController.doActionIfNeededAndReport(MonitoringController.java:134)

net.bull.javamelody.MonitoringFilter.doMonitoring(MonitoringFilter.java:340)

net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:170)

net.bull.javamelody.PluginMonitoringFilter.doFilter(PluginMonitoringFilter.java:86)

org.jvnet.hudson.plugins.monitoring.HudsonMonitoringFilter.doFilter(HudsonMonitoringFilter.java:84)

hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:98)

hudson.plugins.audit_trail.AuditTrailFilter.doFilter(AuditTrailFilter.java:66)

hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:98)

hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:87)

winstone.FilterConfiguration.execute(FilterConfiguration.java:194)

winstone.RequestDispatcher.doFilter(RequestDispatcher.java:366)

hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:47)

winstone.FilterConfiguration.execute(FilterConfiguration.java:194)

winstone.RequestDispatcher.doFilter(RequestDispatcher.java:366)

hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)

hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51)

hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)

org.acegisecurity.ui.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:166)

hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)

org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)

hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)

org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142)

hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)

org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271)

hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)

org.acegisecurity.ui.basicauth.BasicProcessingFilter.doFilter(BasicProcessingFilter.java:173)

hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)

jenkins.security.ApiTokenFilter.doFilter(ApiTokenFilter.java:63)

hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)

org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)

hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:66)

hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)

hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76)

hudson.security.HudsonFilter.doFilter(HudsonFilter.java:164)

winstone.FilterConfiguration.execute(FilterConfiguration.java:194)

winstone.RequestDispatcher.doFilter(RequestDispatcher.java:366)

org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:50)

winstone.FilterConfiguration.execute(FilterConfiguration.java:194)

winstone.RequestDispatcher.doFilter(RequestDispatcher.java:366)

hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:81)

winstone.FilterConfiguration.execute(FilterConfiguration.java:194)

winstone.RequestDispatcher.doFilter(RequestDispatcher.java:366)

winstone.RequestDispatcher.forward(RequestDispatcher.java:331)

winstone.RequestHandlerThread.processRequest(RequestHandlerThread.java:215)

winstone.RequestHandlerThread.run(RequestHandlerThread.java:138)

java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)

java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)

java.util.concurrent.FutureTask.run(Unknown Source)

winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77)

java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

java.lang.Thread.run(Unknown Source)

"RequestHandlerThread[#1034]" daemon prio=5 RUNNABLE

java.net.SocketInputStream.socketRead0(Native Method)

java.net.SocketInputStream.read(Unknown Source)

java.net.SocketInputStream.read(Unknown Source)

java.net.SocketInputStream.read(Unknown Source)

winstone.WinstoneInputStream.read(WinstoneInputStream.java:49)

javax.servlet.ServletInputStream.readLine(ServletInputStream.java:27)

winstone.WinstoneInputStream.readLine(WinstoneInputStream.java:108)

winstone.HttpListener.parseURI(HttpListener.java:238)

winstone.RequestHandlerThread.run(RequestHandlerThread.java:75)

java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)

java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)

java.util.concurrent.FutureTask.run(Unknown Source)

winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77)

java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

java.lang.Thread.run(Unknown Source)

"RequestHandlerThread[#1038]" daemon prio=5 TIMED_WAITING

sun.misc.Unsafe.park(Native Method)

java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)

java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(Unknown Source)

java.util.concurrent.SynchronousQueue$TransferStack.transfer(Unknown Source)

java.util.concurrent.SynchronousQueue.poll(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

java.lang.Thread.run(Unknown Source)

"RequestHandlerThread[#1050]" daemon prio=5 RUNNABLE

java.net.SocketInputStream.socketRead0(Native Method)

java.net.SocketInputStream.read(Unknown Source)

java.net.SocketInputStream.read(Unknown Source)

java.net.SocketInputStream.read(Unknown Source)

winstone.WinstoneInputStream.read(WinstoneInputStream.java:49)

javax.servlet.ServletInputStream.readLine(ServletInputStream.java:27)

winstone.WinstoneInputStream.readLine(WinstoneInputStream.java:108)

winstone.HttpListener.parseURI(HttpListener.java:238)

winstone.RequestHandlerThread.run(RequestHandlerThread.java:75)

java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)

java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)

java.util.concurrent.FutureTask.run(Unknown Source)

winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77)

java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

java.lang.Thread.run(Unknown Source)

"RequestHandlerThread[#1051]" daemon prio=5 RUNNABLE

java.net.SocketInputStream.socketRead0(Native Method)

java.net.SocketInputStream.read(Unknown Source)

java.net.SocketInputStream.read(Unknown Source)

java.net.SocketInputStream.read(Unknown Source)

winstone.WinstoneInputStream.read(WinstoneInputStream.java:49)

javax.servlet.ServletInputStream.readLine(ServletInputStream.java:27)

winstone.WinstoneInputStream.readLine(WinstoneInputStream.java:108)

winstone.HttpListener.parseURI(HttpListener.java:238)

winstone.RequestHandlerThread.run(RequestHandlerThread.java:75)

java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)

java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)

java.util.concurrent.FutureTask.run(Unknown Source)

winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77)

java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

java.lang.Thread.run(Unknown Source)

"RequestHandlerThread[#1054]" daemon prio=5 RUNNABLE

java.net.SocketInputStream.socketRead0(Native Method)

java.net.SocketInputStream.read(Unknown Source)

java.net.SocketInputStream.read(Unknown Source)

java.net.SocketInputStream.read(Unknown Source)

winstone.WinstoneInputStream.read(WinstoneInputStream.java:49)

javax.servlet.ServletInputStream.readLine(ServletInputStream.java:27)

winstone.WinstoneInputStream.readLine(WinstoneInputStream.java:108)

winstone.HttpListener.parseURI(HttpListener.java:238)

winstone.RequestHandlerThread.run(RequestHandlerThread.java:75)

java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)

java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)

java.util.concurrent.FutureTask.run(Unknown Source)

winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77)

java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

java.lang.Thread.run(Unknown Source)

"RequestHandlerThread[#1056]" daemon prio=5 RUNNABLE

java.net.SocketInputStream.socketRead0(Native Method)

java.net.SocketInputStream.read(Unknown Source)

java.net.SocketInputStream.read(Unknown Source)

java.net.SocketInputStream.read(Unknown Source)

winstone.WinstoneInputStream.read(WinstoneInputStream.java:49)

javax.servlet.ServletInputStream.readLine(ServletInputStream.java:27)

winstone.WinstoneInputStream.readLine(WinstoneInputStream.java:108)

winstone.HttpListener.parseURI(HttpListener.java:238)

winstone.RequestHandlerThread.run(RequestHandlerThread.java:75)

java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)

java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)

java.util.concurrent.FutureTask.run(Unknown Source)

winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77)

java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

java.lang.Thread.run(Unknown Source)

"RequestHandlerThread[#1058]" daemon prio=5 RUNNABLE

java.net.SocketInputStream.socketRead0(Native Method)

java.net.SocketInputStream.read(Unknown Source)

java.net.SocketInputStream.read(Unknown Source)

java.net.SocketInputStream.read(Unknown Source)

winstone.WinstoneInputStream.read(WinstoneInputStream.java:49)

javax.servlet.ServletInputStream.readLine(ServletInputStream.java:27)

winstone.WinstoneInputStream.readLine(WinstoneInputStream.java:108)

winstone.HttpListener.parseURI(HttpListener.java:238)

winstone.RequestHandlerThread.run(RequestHandlerThread.java:75)

java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)

java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)

java.util.concurrent.FutureTask.run(Unknown Source)

winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77)

java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

java.lang.Thread.run(Unknown Source)

"RequestHandlerThread[#1059]" daemon prio=5 RUNNABLE

java.net.SocketInputStream.socketRead0(Native Method)

java.net.SocketInputStream.read(Unknown Source)

java.net.SocketInputStream.read(Unknown Source)

java.net.SocketInputStream.read(Unknown Source)

winstone.WinstoneInputStream.read(WinstoneInputStream.java:49)

javax.servlet.ServletInputStream.readLine(ServletInputStream.java:27)

winstone.WinstoneInputStream.readLine(WinstoneInputStream.java:108)

winstone.HttpListener.parseURI(HttpListener.java:238)

winstone.RequestHandlerThread.run(RequestHandlerThread.java:75)

java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)

java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)

java.util.concurrent.FutureTask.run(Unknown Source)

winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77)

java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

java.lang.Thread.run(Unknown Source)

"RequestHandlerThread[#1060]" daemon prio=5 RUNNABLE

java.net.SocketInputStream.socketRead0(Native Method)

java.net.SocketInputStream.read(Unknown Source)

java.net.SocketInputStream.read(Unknown Source)

java.net.SocketInputStream.read(Unknown Source)

winstone.WinstoneInputStream.read(WinstoneInputStream.java:49)

javax.servlet.ServletInputStream.readLine(ServletInputStream.java:27)

winstone.WinstoneInputStream.readLine(WinstoneInputStream.java:108)

winstone.HttpListener.parseURI(HttpListener.java:238)

winstone.RequestHandlerThread.run(RequestHandlerThread.java:75)

java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)

java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)

java.util.concurrent.FutureTask.run(Unknown Source)

winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77)

java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

java.lang.Thread.run(Unknown Source)

"RequestHandlerThread[#1061]" daemon prio=5 TIMED_WAITING

sun.misc.Unsafe.park(Native Method)

java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)

java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(Unknown Source)

java.util.concurrent.SynchronousQueue$TransferStack.transfer(Unknown Source)

java.util.concurrent.SynchronousQueue.poll(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

java.lang.Thread.run(Unknown Source)

"RequestHandlerThread[#1062]" daemon prio=5 RUNNABLE

java.net.SocketInputStream.socketRead0(Native Method)

java.net.SocketInputStream.read(Unknown Source)

java.net.SocketInputStream.read(Unknown Source)

java.net.SocketInputStream.read(Unknown Source)

winstone.WinstoneInputStream.read(WinstoneInputStream.java:49)

javax.servlet.ServletInputStream.readLine(ServletInputStream.java:27)

winstone.WinstoneInputStream.readLine(WinstoneInputStream.java:108)

winstone.HttpListener.parseURI(HttpListener.java:238)

winstone.RequestHandlerThread.run(RequestHandlerThread.java:75)

java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)

java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)

java.util.concurrent.FutureTask.run(Unknown Source)

winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77)

java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

java.lang.Thread.run(Unknown Source)

"RequestHandlerThread[#1063]" daemon prio=5 RUNNABLE

java.net.SocketInputStream.socketRead0(Native Method)

java.net.SocketInputStream.read(Unknown Source)

java.net.SocketInputStream.read(Unknown Source)

java.net.SocketInputStream.read(Unknown Source)

winstone.WinstoneInputStream.read(WinstoneInputStream.java:49)

javax.servlet.ServletInputStream.readLine(ServletInputStream.java:27)

winstone.WinstoneInputStream.readLine(WinstoneInputStream.java:108)

winstone.HttpListener.parseURI(HttpListener.java:238)

winstone.RequestHandlerThread.run(RequestHandlerThread.java:75)

java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)

java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)

java.util.concurrent.FutureTask.run(Unknown Source)

winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77)

java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

java.lang.Thread.run(Unknown Source)

"RequestHandlerThread[#176]" daemon prio=5 RUNNABLE

java.net.SocketInputStream.socketRead0(Native Method)

java.net.SocketInputStream.read(Unknown Source)

java.net.SocketInputStream.read(Unknown Source)

java.net.SocketInputStream.read(Unknown Source)

winstone.WinstoneInputStream.read(WinstoneInputStream.java:49)

javax.servlet.ServletInputStream.readLine(ServletInputStream.java:27)

winstone.WinstoneInputStream.readLine(WinstoneInputStream.java:108)

winstone.HttpListener.parseURI(HttpListener.java:238)

winstone.RequestHandlerThread.run(RequestHandlerThread.java:75)

java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)

java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)

java.util.concurrent.FutureTask.run(Unknown Source)

winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77)

java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

java.lang.Thread.run(Unknown Source)

"RequestHandlerThread[#906]" daemon prio=5 RUNNABLE

java.net.SocketInputStream.socketRead0(Native Method)

java.net.SocketInputStream.read(Unknown Source)

java.net.SocketInputStream.read(Unknown Source)

java.net.SocketInputStream.read(Unknown Source)

winstone.WinstoneInputStream.read(WinstoneInputStream.java:49)

javax.servlet.ServletInputStream.readLine(ServletInputStream.java:27)

winstone.WinstoneInputStream.readLine(WinstoneInputStream.java:108)

winstone.HttpListener.parseURI(HttpListener.java:238)

winstone.RequestHandlerThread.run(RequestHandlerThread.java:75)

java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)

java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)

java.util.concurrent.FutureTask.run(Unknown Source)

winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77)

java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

java.lang.Thread.run(Unknown Source)

"RequestHandlerThread[#940]" daemon prio=5 RUNNABLE

java.net.SocketInputStream.socketRead0(Native Method)

java.net.SocketInputStream.read(Unknown Source)

java.net.SocketInputStream.read(Unknown Source)

java.net.SocketInputStream.read(Unknown Source)

winstone.WinstoneInputStream.read(WinstoneInputStream.java:49)

javax.servlet.ServletInputStream.readLine(ServletInputStream.java:27)

winstone.WinstoneInputStream.readLine(WinstoneInputStream.java:108)

winstone.HttpListener.parseURI(HttpListener.java:238)

winstone.RequestHandlerThread.run(RequestHandlerThread.java:75)

java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)

java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)

java.util.concurrent.FutureTask.run(Unknown Source)

winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77)

java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

java.lang.Thread.run(Unknown Source)

"RequestHandlerThread[#957]" daemon prio=5 RUNNABLE

java.net.SocketInputStream.socketRead0(Native Method)

java.net.SocketInputStream.read(Unknown Source)

java.net.SocketInputStream.read(Unknown Source)

java.net.SocketInputStream.read(Unknown Source)

winstone.WinstoneInputStream.read(WinstoneInputStream.java:49)

javax.servlet.ServletInputStream.readLine(ServletInputStream.java:27)

winstone.WinstoneInputStream.readLine(WinstoneInputStream.java:108)

winstone.HttpListener.parseURI(HttpListener.java:238)

winstone.RequestHandlerThread.run(RequestHandlerThread.java:75)

java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)

java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)

java.util.concurrent.FutureTask.run(Unknown Source)

winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77)

java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

java.lang.Thread.run(Unknown Source)

"RequestHandlerThread[#965]" daemon prio=5 RUNNABLE

java.net.SocketInputStream.socketRead0(Native Method)

java.net.SocketInputStream.read(Unknown Source)

java.net.SocketInputStream.read(Unknown Source)

java.net.SocketInputStream.read(Unknown Source)

winstone.WinstoneInputStream.read(WinstoneInputStream.java:49)

javax.servlet.ServletInputStream.readLine(ServletInputStream.java:27)

winstone.WinstoneInputStream.readLine(WinstoneInputStream.java:108)

winstone.HttpListener.parseURI(HttpListener.java:238)

winstone.RequestHandlerThread.run(RequestHandlerThread.java:75)

java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)

java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)

java.util.concurrent.FutureTask.run(Unknown Source)

winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77)

java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

java.lang.Thread.run(Unknown Source)

"RequestHandlerThread[#971]" daemon prio=5 RUNNABLE

java.net.SocketInputStream.socketRead0(Native Method)

java.net.SocketInputStream.read(Unknown Source)

java.net.SocketInputStream.read(Unknown Source)

java.net.SocketInputStream.read(Unknown Source)

winstone.WinstoneInputStream.read(WinstoneInputStream.java:49)

javax.servlet.ServletInputStream.readLine(ServletInputStream.java:27)

winstone.WinstoneInputStream.readLine(WinstoneInputStream.java:108)

winstone.HttpListener.parseURI(HttpListener.java:238)

winstone.RequestHandlerThread.run(RequestHandlerThread.java:75)

java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)

java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)

java.util.concurrent.FutureTask.run(Unknown Source)

winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77)

java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

java.lang.Thread.run(Unknown Source)

"RequestHandlerThread[#974]" daemon prio=5 RUNNABLE

java.net.SocketInputStream.socketRead0(Native Method)

java.net.SocketInputStream.read(Unknown Source)

java.net.SocketInputStream.read(Unknown Source)

java.net.SocketInputStream.read(Unknown Source)

winstone.WinstoneInputStream.read(WinstoneInputStream.java:49)

javax.servlet.ServletInputStream.readLine(ServletInputStream.java:27)

winstone.WinstoneInputStream.readLine(WinstoneInputStream.java:108)

winstone.HttpListener.parseURI(HttpListener.java:238)

winstone.RequestHandlerThread.run(RequestHandlerThread.java:75)

java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)

java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)

java.util.concurrent.FutureTask.run(Unknown Source)

winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77)

java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

java.lang.Thread.run(Unknown Source)

"RequestHandlerThread[#995]" daemon prio=5 RUNNABLE

java.net.SocketInputStream.socketRead0(Native Method)

java.net.SocketInputStream.read(Unknown Source)

java.net.SocketInputStream.read(Unknown Source)

java.net.SocketInputStream.read(Unknown Source)

winstone.WinstoneInputStream.read(WinstoneInputStream.java:49)

javax.servlet.ServletInputStream.readLine(ServletInputStream.java:27)

winstone.WinstoneInputStream.readLine(WinstoneInputStream.java:108)

winstone.HttpListener.parseURI(HttpListener.java:238)

winstone.RequestHandlerThread.run(RequestHandlerThread.java:75)

java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)

java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)

java.util.concurrent.FutureTask.run(Unknown Source)

winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77)

java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

java.lang.Thread.run(Unknown Source)

"SocketListener(SOME-MACHINE-01.local.)" daemon prio=5 RUNNABLE

java.net.TwoStacksPlainDatagramSocketImpl.receive0(Native Method)

java.net.TwoStacksPlainDatagramSocketImpl.receive(Unknown Source)

java.net.DatagramSocket.receive(Unknown Source)

javax.jmdns.impl.SocketListener.run(SocketListener.java:41)

"stderr copier for remote agent on some-machine-01" daemon prio=5 RUNNABLE

java.io.FileInputStream.readBytes(Native Method)

java.io.FileInputStream.read(Unknown Source)

hudson.util.StreamCopyThread.run(StreamCopyThread.java:60)

"stderr copier for remote agent on some-machine-02" daemon prio=5 RUNNABLE

java.io.FileInputStream.readBytes(Native Method)

java.io.FileInputStream.read(Unknown Source)

hudson.util.StreamCopyThread.run(StreamCopyThread.java:60)

"stderr copier for remote agent on BVT-MacOSX" daemon prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

com.trilead.ssh2.StreamGobbler.read(StreamGobbler.java:213)

com.trilead.ssh2.StreamGobbler.read(StreamGobbler.java:171)

hudson.util.StreamCopyThread.run(StreamCopyThread.java:60)

"stderr copier for remote agent on XXXmobile" daemon prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

com.trilead.ssh2.StreamGobbler.read(StreamGobbler.java:213)

com.trilead.ssh2.StreamGobbler.read(StreamGobbler.java:171)

hudson.util.StreamCopyThread.run(StreamCopyThread.java:60)

"stderr copier for remote agent on XXXbuild-1" daemon prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

com.trilead.ssh2.StreamGobbler.read(StreamGobbler.java:213)

com.trilead.ssh2.StreamGobbler.read(StreamGobbler.java:171)

hudson.util.StreamCopyThread.run(StreamCopyThread.java:60)

"stderr copier for remote agent on XXXbuild-2" daemon prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

com.trilead.ssh2.StreamGobbler.read(StreamGobbler.java:213)

com.trilead.ssh2.StreamGobbler.read(StreamGobbler.java:171)

hudson.util.StreamCopyThread.run(StreamCopyThread.java:60)

"TCP slave agent listener port=0" prio=5 RUNNABLE

java.net.DualStackPlainSocketImpl.accept0(Native Method)

java.net.DualStackPlainSocketImpl.socketAccept(Unknown Source)

java.net.AbstractPlainSocketImpl.accept(Unknown Source)

java.net.PlainSocketImpl.accept(Unknown Source)

java.net.ServerSocket.implAccept(Unknown Source)

java.net.ServerSocket.accept(Unknown Source)

hudson.TcpSlaveAgentListener.run(TcpSlaveAgentListener.java:91)

"Thread-14" daemon prio=5 RUNNABLE

java.net.SocketInputStream.socketRead0(Native Method)

java.net.SocketInputStream.read(Unknown Source)

java.net.SocketInputStream.read(Unknown Source)

com.trilead.ssh2.crypto.cipher.CipherInputStream.fill_buffer(CipherInputStream.java:41)

com.trilead.ssh2.crypto.cipher.CipherInputStream.internal_read(CipherInputStream.java:52)

com.trilead.ssh2.crypto.cipher.CipherInputStream.getBlock(CipherInputStream.java:79)

com.trilead.ssh2.crypto.cipher.CipherInputStream.read(CipherInputStream.java:108)

com.trilead.ssh2.transport.TransportConnection.receiveMessage(TransportConnection.java:232)

com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:677)

com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:475)

java.lang.Thread.run(Unknown Source)

"Thread-15" daemon prio=5 RUNNABLE

java.net.SocketInputStream.socketRead0(Native Method)

java.net.SocketInputStream.read(Unknown Source)

java.net.SocketInputStream.read(Unknown Source)

com.trilead.ssh2.crypto.cipher.CipherInputStream.fill_buffer(CipherInputStream.java:41)

com.trilead.ssh2.crypto.cipher.CipherInputStream.internal_read(CipherInputStream.java:52)

com.trilead.ssh2.crypto.cipher.CipherInputStream.getBlock(CipherInputStream.java:79)

com.trilead.ssh2.crypto.cipher.CipherInputStream.read(CipherInputStream.java:108)

com.trilead.ssh2.transport.TransportConnection.receiveMessage(TransportConnection.java:232)

com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:677)

com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:475)

java.lang.Thread.run(Unknown Source)

"Thread-16" daemon prio=5 RUNNABLE

java.net.SocketInputStream.socketRead0(Native Method)

java.net.SocketInputStream.read(Unknown Source)

java.net.SocketInputStream.read(Unknown Source)

com.trilead.ssh2.crypto.cipher.CipherInputStream.fill_buffer(CipherInputStream.java:41)

com.trilead.ssh2.crypto.cipher.CipherInputStream.internal_read(CipherInputStream.java:52)

com.trilead.ssh2.crypto.cipher.CipherInputStream.getBlock(CipherInputStream.java:79)

com.trilead.ssh2.crypto.cipher.CipherInputStream.read(CipherInputStream.java:108)

com.trilead.ssh2.transport.TransportConnection.receiveMessage(TransportConnection.java:232)

com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:677)

com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:475)

java.lang.Thread.run(Unknown Source)

"Thread-17" daemon prio=5 RUNNABLE

java.net.SocketInputStream.socketRead0(Native Method)

java.net.SocketInputStream.read(Unknown Source)

java.net.SocketInputStream.read(Unknown Source)

com.trilead.ssh2.crypto.cipher.CipherInputStream.fill_buffer(CipherInputStream.java:41)

com.trilead.ssh2.crypto.cipher.CipherInputStream.internal_read(CipherInputStream.java:52)

com.trilead.ssh2.crypto.cipher.CipherInputStream.getBlock(CipherInputStream.java:79)

com.trilead.ssh2.crypto.cipher.CipherInputStream.read(CipherInputStream.java:108)

com.trilead.ssh2.transport.TransportConnection.receiveMessage(TransportConnection.java:232)

com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:677)

com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:475)

java.lang.Thread.run(Unknown Source)

"Thread-18" daemon prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

com.trilead.ssh2.channel.ChannelManager.getChannelData(ChannelManager.java:985)

com.trilead.ssh2.channel.ChannelInputStream.read(ChannelInputStream.java:58)

com.trilead.ssh2.channel.ChannelInputStream.read(ChannelInputStream.java:70)

com.trilead.ssh2.StreamGobbler$GobblerThread.run(StreamGobbler.java:48)

"Thread-19" daemon prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

com.trilead.ssh2.channel.ChannelManager.getChannelData(ChannelManager.java:985)

com.trilead.ssh2.channel.ChannelInputStream.read(ChannelInputStream.java:58)

com.trilead.ssh2.channel.ChannelInputStream.read(ChannelInputStream.java:70)

com.trilead.ssh2.StreamGobbler$GobblerThread.run(StreamGobbler.java:48)

"Thread-20" daemon prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

com.trilead.ssh2.channel.ChannelManager.getChannelData(ChannelManager.java:985)

com.trilead.ssh2.channel.ChannelInputStream.read(ChannelInputStream.java:58)

com.trilead.ssh2.channel.ChannelInputStream.read(ChannelInputStream.java:70)

com.trilead.ssh2.StreamGobbler$GobblerThread.run(StreamGobbler.java:48)

"Thread-21" daemon prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

com.trilead.ssh2.channel.ChannelManager.getChannelData(ChannelManager.java:985)

com.trilead.ssh2.channel.ChannelInputStream.read(ChannelInputStream.java:58)

com.trilead.ssh2.channel.ChannelInputStream.read(ChannelInputStream.java:70)

com.trilead.ssh2.StreamGobbler$GobblerThread.run(StreamGobbler.java:48)

"Thread-22" daemon prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

com.trilead.ssh2.channel.ChannelManager.getChannelData(ChannelManager.java:985)

com.trilead.ssh2.channel.ChannelInputStream.read(ChannelInputStream.java:58)

com.trilead.ssh2.channel.ChannelInputStream.read(ChannelInputStream.java:70)

com.trilead.ssh2.StreamGobbler$GobblerThread.run(StreamGobbler.java:48)

"Thread-23" daemon prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

com.trilead.ssh2.channel.ChannelManager.getChannelData(ChannelManager.java:985)

com.trilead.ssh2.channel.ChannelInputStream.read(ChannelInputStream.java:58)

com.trilead.ssh2.channel.ChannelInputStream.read(ChannelInputStream.java:70)

com.trilead.ssh2.StreamGobbler$GobblerThread.run(StreamGobbler.java:48)

"Thread-24" daemon prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

com.trilead.ssh2.channel.ChannelManager.getChannelData(ChannelManager.java:985)

com.trilead.ssh2.channel.ChannelInputStream.read(ChannelInputStream.java:58)

com.trilead.ssh2.channel.ChannelInputStream.read(ChannelInputStream.java:70)

com.trilead.ssh2.StreamGobbler$GobblerThread.run(StreamGobbler.java:48)

"Thread-25" daemon prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

com.trilead.ssh2.channel.ChannelManager.getChannelData(ChannelManager.java:985)

com.trilead.ssh2.channel.ChannelInputStream.read(ChannelInputStream.java:58)

com.trilead.ssh2.channel.ChannelInputStream.read(ChannelInputStream.java:70)

com.trilead.ssh2.StreamGobbler$GobblerThread.run(StreamGobbler.java:48)

"Thread-9" daemon prio=5 RUNNABLE

sun.net.dns.ResolverConfigurationImpl.notifyAddrChange0(Native Method)

sun.net.dns.ResolverConfigurationImpl$AddressChangeListener.run(Unknown Source)

"ViewJob reload thread" daemon prio=5 TIMED_WAITING

java.lang.Object.wait(Native Method)

hudson.model.ViewJob$ReloadThread.getNext(ViewJob.java:171)

hudson.model.ViewJob$ReloadThread.run(ViewJob.java:188)

"Waiting to acquire /jenkins/workspace/DWDWargDeploy : SCM polling for hudson.model.FreeStyleProject@51f4c07[DWDWargDeploy]" prio=5 WAITING

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:503)

hudson.slaves.WorkspaceList.acquire(WorkspaceList.java:232)

hudson.slaves.WorkspaceList.acquire(WorkspaceList.java:211)

hudson.model.AbstractProject._poll(AbstractProject.java:1396)

hudson.model.AbstractProject.poll(AbstractProject.java:1335)

hudson.triggers.SCMTrigger$Runner.runPolling(SCMTrigger.java:420)

hudson.triggers.SCMTrigger$Runner.run(SCMTrigger.java:449)

hudson.util.SequentialExecutionQueue$QueueEntry.run(SequentialExecutionQueue.java:118)

java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)

java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)

java.util.concurrent.FutureTask.run(Unknown Source)

java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

java.lang.Thread.run(Unknown Source)

"WinstoneHostConfigurationMgmt:default" daemon prio=5 TIMED_WAITING

java.lang.Thread.sleep(Native Method)

winstone.HostConfiguration.run(HostConfiguration.java:174)

java.lang.Thread.run(Unknown Source)

[Maxfield Stewart]

So we resolved this issue for anyone tracking a similar problem.  The root cause appears to be a change in 1.482 where the number of winstone worker threads was reduced in code from 1000 to 10.  By reconfiguring our winstone startup to use more threads we restored performance to our Jenkins environment. We set the startup options to:

--handlerCountMax=500 --handlerCountStartup=50

On Wednesday, January 23, 2013 1:19:27 PM UTC-8, Maxfield Stewart wrote:

[Slide]

Can you file a bug on this? That change sounds pretty major...

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
Website: http://earl-of-code.com

[Peter Jönsson]

For reference, here is a issue regarding this:

https://issues.jenkins-ci.org/browse/JENKINS-16474

> <mailto:jenkinsci-dev%2Bunsu...@googlegroups.com>.