[JIRA] [durable-task-plugin] (JENKINS-34150) Pipeline Batch hangs

[d@danield.us (JIRA)]

Daniel Daughertydp created an issue

Jenkins / JENKINS-34150 Pipeline Batch hangs Issue Type: Bug Assignee: Jesse Glick Components: durable-task-plugin, workflow-plugin Created: 2016/Apr/11 8:26 PM Environment: Jenkins 2.0RC1 Windows 2008r2, Windows 2012r2, Windows 7 Labels: workflow Priority: Critical Reporter: Daniel Daughertydp When batch task is included in a Pipeline job it will hang on completion of the batch task. I can see in task manager that the job startup it logs data to jenkins-log.txt. The batch completes and I see in task manager that the batch is nolonger running. But Jenkins is still waiting for the task to complete. I do not see jenkins-result.txt writen to the workspace tmp durabletask directory. If I create the file manually or run the workflow-wrap.bat manually the task completes. This is an itermitent bug. Task might work 3 time then fail 5 times then work 8 times. No change to the system during this time. I am setting the job to run every min to see what the stats look like for longer run. job: node { bat 'ping 127.0.0.1 -n 10' echo 'batch completed' } Could be any command you want ping is just an easy one to have it take a little bit of time. And require nothing else installed on machine. I see many other task like this I have tested on serveral differnt machines using a base install of Jenkins. Add Comment

This message was sent by Atlassian JIRA (v6.4.2#64017-sha1:e244265)

[tyler@monkeypox.org (JIRA)]

R. Tyler Croy updated an issue

Jenkins / JENKINS-34150 Pipeline Batch hangs

Change By: R. Tyler Croy

When batch task is included in a Pipeline job it will hang on completion of the batch task.  I can see in task manager that the job startup it logs data to jenkins-log.txt. The batch completes and I see in task manager that the batch is nolonger running.  But Jenkins is still waiting for the task to complete.  I do not see jenkins-result.txt writen to the workspace tmp durabletask directory.  If I create the file manually or run the workflow-wrap.bat manually the task completes.  This is an itermitent bug.  Task might work 3 time then fail 5 times then work 8 times.  No change to the system during this time.  I am setting the job to run every min to see what the stats look like for longer run.   job:

{code}

node {   bat 'ping 127.0.0.1 -n 10'  echo 'batch completed' }

{code}

Could be any command you want ping is just an easy one to have it take a little bit of time.  And require nothing else installed on machine.   I see many other task like this I have tested on serveral differnt machines using a base install of Jenkins.

Add Comment

[tyler@monkeypox.org (JIRA)]

R. Tyler Croy updated an issue

Jenkins / JENKINS-34150 Pipeline Batch hangs

Change By: R. Tyler Croy

Labels: 2.0 workflow

Add Comment

[tyler@monkeypox.org (JIRA)]

R. Tyler Croy updated an issue

Jenkins / JENKINS-34150 Pipeline Batch hangs

I have just tried this on a Windows Server 2012R2 instance on EC2 with the Jenkins 2.0RC1 installer. Using the Pipeline script: node { bat 'ping 127.0.0.1 -n 10' echo 'pinging done!' } I was able to successfully run the Pipeline a few times. I added a sleep 30 and was also able to abort the Pipeline successfully. This is with: Durable Task 1.9 Pipeline 2.0 Pipeline Job, Pipeline Groovy 2.1

Change By: R. Tyler Croy

Attachment: jenkins2-pipeline-batch-nohang.png

Add Comment

[tyler@monkeypox.org (JIRA)]

R. Tyler Croy updated an issue

Jenkins / JENKINS-34150 Pipeline Batch hangs

Change By: R. Tyler Croy

Labels: 2.0  testfest  workflow

Add Comment

[tyler@monkeypox.org (JIRA)]

R. Tyler Croy resolved as Cannot Reproduce

Daniel Daughertydp I'm going to mark this as "Cannot Reproduce" but please re-open if you have additional reproduction steps that I could use to help verify.

Jenkins / JENKINS-34150 Pipeline Batch hangs

Change By: R. Tyler Croy

Status: Open Resolved Assignee: Jesse Glick R. Tyler Croy Resolution: Cannot Reproduce

Add Comment

[tyler@monkeypox.org (JIRA)]

R. Tyler Croy updated an issue

Jenkins / JENKINS-34150 Pipeline Batch hangs

Change By: R. Tyler Croy

Attachment: jenkins2-pipeline-batch-hang.png

Add Comment

[tyler@monkeypox.org (JIRA)]

R. Tyler Croy reopened an issue

I think I actually found the reproduction case for this! If you click the "Build Now" button a lot to where pipelines are running in parallel the pipelines will all hang.

Jenkins / JENKINS-34150 Pipeline Batch hangs

Change By: R. Tyler Croy

Resolution: Cannot Reproduce Status: Resolved Reopened

Add Comment

[tyler@monkeypox.org (JIRA)]

R. Tyler Croy commented on JENKINS-34150

Re: Pipeline Batch hangs Thread dump from a Windows 2012R2 server with two hanging Pipeline executions Thread Dump Computer.threadPoolForRemoting [#6] "Computer.threadPoolForRemoting [#6]" Id=197 Group=main TIMED_WAITING on java.util.concurrent.SynchronousQueue$TransferStack@d13463 at sun.misc.Unsafe.park(Native Method) - waiting on java.util.concurrent.SynchronousQueue$TransferStack@d13463 at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(Unknown Source) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(Unknown Source) at java.util.concurrent.SynchronousQueue.poll(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Computer.threadPoolForRemoting [#7] "Computer.threadPoolForRemoting [#7]" Id=198 Group=main TIMED_WAITING on java.util.concurrent.SynchronousQueue$TransferStack@d13463 at sun.misc.Unsafe.park(Native Method) - waiting on java.util.concurrent.SynchronousQueue$TransferStack@d13463 at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(Unknown Source) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(Unknown Source) at java.util.concurrent.SynchronousQueue.poll(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) DestroyJavaVM "DestroyJavaVM" Id=16 Group=main RUNNABLE FilePath.localPool [#5] "FilePath.localPool [#5]" Id=208 Group=main RUNNABLE at sun.management.ThreadImpl.dumpThreads0(Native Method) at sun.management.ThreadImpl.dumpAllThreads(Unknown Source) at hudson.Functions.getThreadInfos(Functions.java:1195) at hudson.util.RemotingDiagnostics$GetThreadDump.call(RemotingDiagnostics.java:98) at hudson.util.RemotingDiagnostics$GetThreadDump.call(RemotingDiagnostics.java:95) at hudson.remoting.LocalChannel$1.call(LocalChannel.java:52) at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Number of locked synchronizers = 1 - java.util.concurrent.ThreadPoolExecutor$Worker@1f9039d Handling GET /threadDump from 127.0.0.1 : RequestHandlerThread[#10] Jenkins/threadDump.jelly "Handling GET /threadDump from 127.0.0.1 : RequestHandlerThread[#10] Jenkins/threadDump.jelly" Id=150 Group=main TIMED_WAITING on java.util.concurrent.FutureTask@295f13 at sun.misc.Unsafe.park(Native Method) - waiting on java.util.concurrent.FutureTask@295f13 at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source) at java.util.concurrent.FutureTask.awaitDone(Unknown Source) at java.util.concurrent.FutureTask.get(Unknown Source) at hudson.remoting.LocalChannel$2.get(LocalChannel.java:81) at jenkins.model.Jenkins.getAllThreadDumps(Jenkins.java:3522) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at org.apache.commons.jexl.util.introspection.UberspectImpl$VelMethodImpl.invoke(UberspectImpl.java:258) at org.apache.commons.jexl.parser.ASTMethod.execute(ASTMethod.java:104) at org.apache.commons.jexl.parser.ASTReference.execute(ASTReference.java:83) at org.apache.commons.jexl.parser.ASTReference.value(ASTReference.java:57) at org.apache.commons.jexl.parser.ASTReferenceExpression.value(ASTReferenceExpression.java:51) at org.apache.commons.jexl.ExpressionImpl.evaluate(ExpressionImpl.java:80) at hudson.ExpressionFactory2$JexlExpression.evaluate(ExpressionFactory2.java:74) at org.apache.commons.jelly.tags.core.CoreTagLibrary$3.run(CoreTagLibrary.java:134) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.kohsuke.stapler.jelly.CallTagLibScript$1.run(CallTagLibScript.java:99) at org.apache.commons.jelly.tags.define.InvokeBodyTag.doTag(InvokeBodyTag.java:91) at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.apache.commons.jelly.tags.core.CoreTagLibrary$1.run(CoreTagLibrary.java:98) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105) at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:120) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.kohsuke.stapler.jelly.CallTagLibScript$1.run(CallTagLibScript.java:99) at org.apache.commons.jelly.tags.define.InvokeBodyTag.doTag(InvokeBodyTag.java:91) at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105) at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:120) at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105) at org.kohsuke.stapler.jelly.JellyViewScript.run(JellyViewScript.java:95) at org.kohsuke.stapler.jelly.DefaultScriptInvoker.invokeScript(DefaultScriptInvoker.java:63) at org.kohsuke.stapler.jelly.DefaultScriptInvoker.invokeScript(DefaultScriptInvoker.java:53) at org.kohsuke.stapler.jelly.JellyFacet$1.dispatch(JellyFacet.java:95) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:649) at org.kohsuke.stapler.Stapler.service(Stapler.java:238) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:812) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1669) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:135) at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:126) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:49) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84) at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76) at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:171) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:49) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:81) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:553) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) at org.eclipse.jetty.server.Server.handle(Server.java:499) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257) at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544) at winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Number of locked synchronizers = 1 - java.util.concurrent.ThreadPoolExecutor$Worker@fa28b Jenkins cron thread "Jenkins cron thread" Id=17 Group=main WAITING on java.util.TaskQueue@1679501 at java.lang.Object.wait(Native Method) - waiting on java.util.TaskQueue@1679501 at java.lang.Object.wait(Unknown Source) at java.util.TimerThread.mainLoop(Unknown Source) at java.util.TimerThread.run(Unknown Source) Jenkins UDP 33848 monitoring thread "Jenkins UDP 33848 monitoring thread" Id=45 Group=main RUNNABLE (in native) at java.net.TwoStacksPlainDatagramSocketImpl.receive0(Native Method) - locked java.net.TwoStacksPlainDatagramSocketImpl@16ef542 at java.net.TwoStacksPlainDatagramSocketImpl.receive(Unknown Source) - locked java.net.TwoStacksPlainDatagramSocketImpl@16ef542 at java.net.DatagramSocket.receive(Unknown Source) - locked java.net.DatagramPacket@f55b4d - locked java.net.MulticastSocket@1bb58c1 at hudson.UDPBroadcastThread.run(UDPBroadcastThread.java:83) jenkins.util.Timer [#10] "jenkins.util.Timer [#10]" Id=41 Group=main WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@182ace7 at sun.misc.Unsafe.park(Native Method) - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@182ace7 at java.util.concurrent.locks.LockSupport.park(Unknown Source) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) jenkins.util.Timer [#1] "jenkins.util.Timer [#1]" Id=18 Group=main WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@182ace7 at sun.misc.Unsafe.park(Native Method) - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@182ace7 at java.util.concurrent.locks.LockSupport.park(Unknown Source) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) jenkins.util.Timer [#2] "jenkins.util.Timer [#2]" Id=30 Group=main TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@182ace7 at sun.misc.Unsafe.park(Native Method) - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@182ace7 at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(Unknown Source) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) jenkins.util.Timer [#3] "jenkins.util.Timer [#3]" Id=31 Group=main WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@182ace7 at sun.misc.Unsafe.park(Native Method) - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@182ace7 at java.util.concurrent.locks.LockSupport.park(Unknown Source) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) jenkins.util.Timer [#4] "jenkins.util.Timer [#4]" Id=32 Group=main WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@182ace7 at sun.misc.Unsafe.park(Native Method) - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@182ace7 at java.util.concurrent.locks.LockSupport.park(Unknown Source) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) jenkins.util.Timer [#5] "jenkins.util.Timer [#5]" Id=33 Group=main WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@182ace7 at sun.misc.Unsafe.park(Native Method) - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@182ace7 at java.util.concurrent.locks.LockSupport.park(Unknown Source) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) jenkins.util.Timer [#6] "jenkins.util.Timer [#6]" Id=34 Group=main WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@182ace7 at sun.misc.Unsafe.park(Native Method) - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@182ace7 at java.util.concurrent.locks.LockSupport.park(Unknown Source) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) jenkins.util.Timer [#7] "jenkins.util.Timer [#7]" Id=35 Group=main WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@182ace7 at sun.misc.Unsafe.park(Native Method) - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@182ace7 at java.util.concurrent.locks.LockSupport.park(Unknown Source) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) jenkins.util.Timer [#8] "jenkins.util.Timer [#8]" Id=36 Group=main WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@182ace7 at sun.misc.Unsafe.park(Native Method) - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@182ace7 at java.util.concurrent.locks.LockSupport.park(Unknown Source) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) jenkins.util.Timer [#9] "jenkins.util.Timer [#9]" Id=37 Group=main WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@182ace7 at sun.misc.Unsafe.park(Native Method) - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@182ace7 at java.util.concurrent.locks.LockSupport.park(Unknown Source) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) JmDNS(WIN-N2IVJVRIP82.local.).State.Timer "JmDNS(WIN-N2IVJVRIP82.local.).State.Timer" Id=51 Group=main TIMED_WAITING on java.util.TaskQueue@12bccb2 at java.lang.Object.wait(Native Method) - waiting on java.util.TaskQueue@12bccb2 at java.util.TimerThread.mainLoop(Unknown Source) at java.util.TimerThread.run(Unknown Source) JmDNS(WIN-N2IVJVRIP82.local.).Timer "JmDNS(WIN-N2IVJVRIP82.local.).Timer" Id=50 Group=main TIMED_WAITING on java.util.TaskQueue@b48798 at java.lang.Object.wait(Native Method) - waiting on java.util.TaskQueue@b48798 at java.util.TimerThread.mainLoop(Unknown Source) at java.util.TimerThread.run(Unknown Source) LauncherControlThread[ControlPort=-1] "LauncherControlThread[ControlPort=-1]" Id=14 Group=main TIMED_WAITING at java.lang.Thread.sleep(Native Method) at winstone.Launcher.run(Launcher.java:243) at java.lang.Thread.run(Unknown Source) NioSocketAcceptor-1 "NioSocketAcceptor-1" Id=43 Group=main RUNNABLE (in native) at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method) at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(Unknown Source) at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(Unknown Source) at sun.nio.ch.WindowsSelectorImpl.doSelect(Unknown Source) at sun.nio.ch.SelectorImpl.lockAndDoSelect(Unknown Source) - locked sun.nio.ch.Util$2@150341e - locked java.util.Collections$UnmodifiableSet@1372a4a - locked sun.nio.ch.WindowsSelectorImpl@350cfd at sun.nio.ch.SelectorImpl.select(Unknown Source) at sun.nio.ch.SelectorImpl.select(Unknown Source) at org.apache.mina.transport.socket.nio.NioSocketAcceptor.select(NioSocketAcceptor.java:238) at org.apache.mina.core.polling.AbstractPollingIoAcceptor$Acceptor.run(AbstractPollingIoAcceptor.java:432) at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Number of locked synchronizers = 1 - java.util.concurrent.ThreadPoolExecutor$Worker@15a9d55 org.eclipse.jetty.server.session.HashSessionManager@1ea3742Timer "org.eclipse.jetty.server.session.HashSessionManager@1ea3742Timer" Id=9 Group=main TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@da471 at sun.misc.Unsafe.park(Native Method) - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@da471 at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(Unknown Source) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) RequestHandlerThread[#13] "RequestHandlerThread[#13]" Id=194 Group=main TIMED_WAITING on java.util.concurrent.SynchronousQueue$TransferStack@16c689 at sun.misc.Unsafe.park(Native Method) - waiting on java.util.concurrent.SynchronousQueue$TransferStack@16c689 at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(Unknown Source) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(Unknown Source) at java.util.concurrent.SynchronousQueue.poll(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) RequestHandlerThread[#1]-selector-ServerConnectorManager@15e4a26/0 "RequestHandlerThread[#1]-selector-ServerConnectorManager@15e4a26/0" Id=12 Group=main RUNNABLE (in native) at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method) at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(Unknown Source) at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(Unknown Source) at sun.nio.ch.WindowsSelectorImpl.doSelect(Unknown Source) at sun.nio.ch.SelectorImpl.lockAndDoSelect(Unknown Source) - locked sun.nio.ch.Util$2@1f243b0 - locked java.util.Collections$UnmodifiableSet@552324 - locked sun.nio.ch.WindowsSelectorImpl@16f6b72 at sun.nio.ch.SelectorImpl.select(Unknown Source) at sun.nio.ch.SelectorImpl.select(Unknown Source) at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:600) at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:549) at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52) at winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Number of locked synchronizers = 1 - java.util.concurrent.ThreadPoolExecutor$Worker@12646b RequestHandlerThread[#2]-acceptor-0@14888b2-ServerConnector@24a83d{HTTP/1.1}{0.0.0.0:8080} "RequestHandlerThread[#2]-acceptor-0@14888b2-ServerConnector@24a83d{HTTP/1.1}{0.0.0.0:8080}" Id=13 Group=main RUNNABLE (in native) at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at sun.nio.ch.ServerSocketChannelImpl.accept(Unknown Source) at sun.nio.ch.ServerSocketChannelImpl.accept(Unknown Source) - locked java.lang.Object@1bb3a34 at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:377) at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:500) at winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Number of locked synchronizers = 1 - java.util.concurrent.ThreadPoolExecutor$Worker@136aa0c RequestHandlerThread[#3] "RequestHandlerThread[#3]" Id=24 Group=main TIMED_WAITING on java.util.concurrent.SynchronousQueue$TransferStack@16c689 at sun.misc.Unsafe.park(Native Method) - waiting on java.util.concurrent.SynchronousQueue$TransferStack@16c689 at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(Unknown Source) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(Unknown Source) at java.util.concurrent.SynchronousQueue.poll(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) RequestHandlerThread[#9] "RequestHandlerThread[#9]" Id=149 Group=main TIMED_WAITING on java.util.concurrent.SynchronousQueue$TransferStack@16c689 at sun.misc.Unsafe.park(Native Method) - waiting on java.util.concurrent.SynchronousQueue$TransferStack@16c689 at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(Unknown Source) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(Unknown Source) at java.util.concurrent.SynchronousQueue.poll(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Scheduler-5482987 "Scheduler-5482987" Id=23 Group=main TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1e28f0d at sun.misc.Unsafe.park(Native Method) - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1e28f0d at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(Unknown Source) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) SocketListener(WIN-N2IVJVRIP82.local.) "SocketListener(WIN-N2IVJVRIP82.local.)" Id=49 Group=main RUNNABLE (in native) at java.net.TwoStacksPlainDatagramSocketImpl.receive0(Native Method) - locked java.net.TwoStacksPlainDatagramSocketImpl@ca2349 at java.net.TwoStacksPlainDatagramSocketImpl.receive(Unknown Source) - locked java.net.TwoStacksPlainDatagramSocketImpl@ca2349 at java.net.DatagramSocket.receive(Unknown Source) - locked java.net.DatagramPacket@bf36c4 - locked java.net.MulticastSocket@10433fd at javax.jmdns.impl.SocketListener.run(SocketListener.java:41) TCP agent listener port=0 "TCP agent listener port=0" Id=44 Group=main RUNNABLE (in native) at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at sun.nio.ch.ServerSocketChannelImpl.accept(Unknown Source) at sun.nio.ch.ServerSocketChannelImpl.accept(Unknown Source) - locked java.lang.Object@14a0c50 at hudson.TcpSlaveAgentListener.run(TcpSlaveAgentListener.java:110) Attach Listener "Attach Listener" Id=4 Group=system RUNNABLE Finalizer "Finalizer" Id=3 Group=system WAITING on java.lang.ref.ReferenceQueue$Lock@13ef3d7 at java.lang.Object.wait(Native Method) - waiting on java.lang.ref.ReferenceQueue$Lock@13ef3d7 at java.lang.ref.ReferenceQueue.remove(Unknown Source) at java.lang.ref.ReferenceQueue.remove(Unknown Source) at java.lang.ref.Finalizer$FinalizerThread.run(Unknown Source) Java2D Disposer "Java2D Disposer" Id=10 Group=system WAITING on java.lang.ref.ReferenceQueue$Lock@19b9fb1 at java.lang.Object.wait(Native Method) - waiting on java.lang.ref.ReferenceQueue$Lock@19b9fb1 at java.lang.ref.ReferenceQueue.remove(Unknown Source) at java.lang.ref.ReferenceQueue.remove(Unknown Source) at sun.java2d.Disposer.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Reference Handler "Reference Handler" Id=2 Group=system WAITING on java.lang.ref.Reference$Lock@1522b39 at java.lang.Object.wait(Native Method) - waiting on java.lang.ref.Reference$Lock@1522b39 at java.lang.Object.wait(Unknown Source) at java.lang.ref.Reference$ReferenceHandler.run(Unknown Source)

Add Comment

[tyler@monkeypox.org (JIRA)]

R. Tyler Croy commented on JENKINS-34150

Re: Pipeline Batch hangs A thread dump from the Pipeline execution itself: Thread #2 at DSL.bat(Native Method) at WorkflowScript.run(WorkflowScript:2) at DSL.node(Native Method) at WorkflowScript.run(WorkflowScript:1)

Add Comment

[tyler@monkeypox.org (JIRA)]

R. Tyler Croy assigned an issue to Antonio Muñiz

Jenkins / JENKINS-34150 Pipeline Batch hangs

Change By: R. Tyler Croy

Assignee: R. Tyler Croy Antonio Muñiz

Add Comment

[amuniz@cloudbees.com (JIRA)]

Antonio Muñiz commented on JENKINS-34150

Re: Pipeline Batch hangs Something to fix in durable-task, nothing to do with Pipeline I believe.

Add Comment

[rufermarc@gmail.com (JIRA)]

Marc Rufer commented on JENKINS-34150

Re: Pipeline Batch hangs I had the same issue with the following Jenkinsfile node { checkout scm bat 'Powershell.exe -executionpolicy remotesigned -File Build.ps1' } Batch only hangs if there are several executions running in parallel

Add Comment

[rufermarc@gmail.com (JIRA)]

Marc Rufer edited a comment on JENKINS-34150

Re: Pipeline Batch hangs

I had the same issue with the following Jenkinsfile

{code:java}

node {   checkout scm   bat 'Powershell.exe -executionpolicy remotesigned -File Build.ps1' }

{code} Batch  only  hangs ,  if there are  several  multiple  executions running in parallel . Sometimes the problem occurs as well if there is only one job running

Add Comment

[d@danield.us (JIRA)]

Daniel Daughertydp commented on JENKINS-34150

Re: Pipeline Batch hangs I agree with Antonio that this is a durable task issue. While trouble shooting this I reverted a logging change in durable task and the issue could not longer be replicated. I suspected a locking issue with the results file where the durable task was locking the results file causing the batch to not be able to create it. But the current 1.9 version of the durable task plugin does not capture the output of the wraper batch file. When I enable the logging of that information the issue no longer shows up. True heisenbug. The act of looking for the cause of the problem causes it to nolonger exist. I submitted pull request to the durable task plugin with the 3 line change that I made. Reverting to older logging code in that project. https://github.com/jenkinsci/durable-task-plugin/pull/20 While this resolves the issue I do not feel that it is a fix. Logging or not logging should not cause batch to work or not work. It is masking the underlying issue where the result file does not get created. But with out the logging you can't see why the result file is not created. Multiple executions is one of the ways that I have triggered it also.

Add Comment

[karing.martin@gmail.com (JIRA)]

Martin Karing commented on JENKINS-34150

Re: Pipeline Batch hangs It is a bug in the durable task plugin. I created a test to reproduce it and submitted a fix to resolve the issue. Both neatly bundled as a pull request: https://github.com/jenkinsci/durable-task-plugin/pull/21

Add Comment

[amuniz@cloudbees.com (JIRA)]

Antonio Muñiz commented on JENKINS-34150

Re: Pipeline Batch hangs After some hours trying combinations (and installing Windows VMs), I've not been able to reproduce it. Tried on: Jenkins 2.0 RC1 (running on Ubuntu, Durable Task 1.9) + Agent on Windows Server 2008R2 (JDK8) Jenkins 2.0 RC1 (running on OSX, Durable Task 1.9) + Agent on Windows Server 2012R2 (JDK8) My test script is: node ('windows') { bat 'ping 192.168.1.108 -n 10' // this is the Jenkins master IP in my local network echo 'batch completed' } I executed up to 10 concurrent builds. All of them finished successfully.

Add Comment

[karing.martin@gmail.com (JIRA)]

Martin Karing commented on JENKINS-34150

Re: Pipeline Batch hangs I developed the test case and the fix on a Windows 10 development box. I encountered the issue on a windows 2012 r2 server. On both systems the hanging happened consistently. How ever on Windows 2012 I never encountered the problem in case only one script was executing. But starting with two scripts there was a fair chance of one of the scripts hanging. My test case on Windows 10 caused the hanging problem in 100% of the tests even with a single running task. My best guess is that it is a timing issue, the test case I provided queries the exit status as fast as possible and causes the hanging issue this way. The problem is that the batch script does not create the file while the plugin is checking the file system for the presence of the file. I do not know why this happens, in my mind the file creation should be unrelated to the file presence check, but it seems on Windows systems it is not.

Add Comment

[amuniz@cloudbees.com (JIRA)]

Antonio Muñiz started work on JENKINS-34150

Change By: Antonio Muñiz Status: Reopened In Progress

Add Comment

[amuniz@cloudbees.com (JIRA)]

Antonio Muñiz commented on JENKINS-34150

Re: Pipeline Batch hangs Martin Karing Thanks for the info. I'm going to move the master to a Windows box too as it's the only thing I see my environment differing from yours.

Add Comment

[amuniz@cloudbees.com (JIRA)]

Antonio Muñiz commented on JENKINS-34150

Re: Pipeline Batch hangs The issue is reproducible only if two batch durable tasks run concurrently on the master node.

Add Comment

[amuniz@cloudbees.com (JIRA)]

Antonio Muñiz commented on JENKINS-34150

Re: Pipeline Batch hangs For some reason the result file is not being created, this line must be failing to execute (but I can not find any log): echo %ERRORLEVEL% > "[JENKINS_HOME]\my-job@tmp\durable-acc8d5a4\jenkins-result.txt" While the build was hanging, I manually executed jenkins-wrap.bat and it made the execution to finish (as the result file was created). Trying to see why the result file is not being written.

Add Comment

[dbeck@cloudbees.com (JIRA)]

Daniel Beck commented on JENKINS-34150

Re: Pipeline Batch hangs Wild guess: Spaces in JENKINS_HOME path when installing using the installer?

Add Comment

[amuniz@cloudbees.com (JIRA)]

Antonio Muñiz commented on JENKINS-34150

Re: Pipeline Batch hangs No. It's reproducible for me without spaces in the path (and didn't use the installer but direct java -jar mode).

Add Comment

[dbeck@cloudbees.com (JIRA)]

Daniel Beck updated an issue

Jenkins / JENKINS-34150 Pipeline Batch hangs

Change By: Daniel Beck Labels: 2.0  2.0-planned  testfest workflow

Add Comment

[amuniz@cloudbees.com (JIRA)]

Antonio Muñiz commented on JENKINS-34150

Re: Pipeline Batch hangs Not a regression in core at least, reproduced in 1.651.1 (Pipeline Durable Task Step 2.0 + Durable Task 1.9)

Add Comment

[d@danield.us (JIRA)]

Daniel Daughertydp commented on JENKINS-34150

Re: Pipeline Batch hangs Antonio, If you apply my pull request (20) you will see the logs for the jenkins-wraper.bat to include the echo command run. But you will no longer experince the error. Just as you will not enconter the error if you apply Martins pull (21). The main action that both change is that a reference to Launcher.ProcStarter ps is maintained after doLaunch() is called. This to me speaks of a possible GC issue where the Proc is destroyed before task completes. And that causes the wraper to not finish. But when I watch performance monitor in windows as the job is runing I don't see the command prompts being killed early. So may locking issue as Martin mentioned earlier. Where checking for the existince of the result file prevents the result from being created. But that does not explain why keeping a reference to the PS instance causes the bug to no longer happen. That speaks more to something happening due to GC. If this was C I would call it a use after free errror. Where PS is nolonger reference but expected to continue to do things.

Add Comment

[karing.martin@gmail.com (JIRA)]

Martin Karing commented on JENKINS-34150

Re: Pipeline Batch hangs Daniel Daughertydp: Actually it does explain why it works when maintaining the reference to the process causes the bug to disappear. My pull requests works because it does not monitor the presence of the file, but it waits until the process is terminated and only after the process is no longer present is looks for the file. This way there is no file checking done as long as the process is active and the batch file is able to create the result file without any problems.

Add Comment

[amuniz@cloudbees.com (JIRA)]

Antonio Muñiz commented on JENKINS-34150

Re: Pipeline Batch hangs This way there is no file checking done as long as the process is active Right. As I noted in the PR, keeping that reference is just what this plugin is trying to avoid. I think the GC theory is probably the culprit. Perhaps if we keep a transient instance field private transient Launcher.ProcStarter ps in WindowsBatchScript it is prevented to be collected. I'm currently testing this option.

Add Comment

[amuniz@cloudbees.com (JIRA)]

Antonio Muñiz commented on JENKINS-34150

Re: Pipeline Batch hangs No, it does not work.

Add Comment

[karing.martin@gmail.com (JIRA)]

Martin Karing commented on JENKINS-34150

Re: Pipeline Batch hangs This issue is really annoying. I tried to track it with the SysInternals Process Monitor. As soon as the monitor runs the issue does not happen any more. Also I tried to alter the wrapping batch file to check if the file was created after writing the error code and if not try again. This does not resolve the issue. It seems like the batch file "sees" the jenkins-result.txt during it's execution.

Add Comment

[karing.martin@gmail.com (JIRA)]

Martin Karing commented on JENKINS-34150

Re: Pipeline Batch hangs Okay, the reason why the loop in the batch file did not work is that something is killing the entire batch file structure before it finishes. I have no idea why this happens and I can't track it because process monitor seems to do something that stops this from happening. I was able to get it running by changing the script so the execution of the wrapper is done by a additional "start" command. This cause command line windows to popup all over my desktop, but it allowed the entire thing to execute properly. This approach has the massive disadvantage that there is no way to terminate the script process in case the script itself hangs or something like this because it runs fully detached and there is no reference to the process. On the other hand the purpose of this plugin is to allow scripts to run across restarts of Jenkins. So it has to run as a detached process from Jenkins, so the JVM doesn't tear it down along with it, but we need a serializable reference to the process so it's possible to locate it again. Just in case it is required to terminate it after a Jenkins reboot.

Add Comment

[kkegao@gmail.com (JIRA)]

KK G commented on JENKINS-34150

Re: Pipeline Batch hangs It's related to https://issues.jenkins-ci.org/browse/JENKINS-33164. Just attach a simple repro from that bug: Pipeline code is: node('master') { for(int i=0; i < 100; ++i) { bat('echo "Hello from batch file."' + i.toString()) }} Click "build now" 5 times. All 5 jobs got stuck on windows OS. Please help. Thanks.

Add Comment

[amuniz@cloudbees.com (JIRA)]

Antonio Muñiz commented on JENKINS-34150

Re: Pipeline Batch hangs I was not able to reproduce the issue in a debug session and did not manage to diagnose why jenkins-wrapper.bat is not fully executed (so jenkins-result.txt is not created) and the bat step never finishes. If someone with more Windows background can throw some light here, it would be great. Perhaps the additional start command proposed by Martin Karing is the less ugly fix, what others think? Jesse Glick ? In the meantime, the workaround is to use a build agent (other than master), even being in the same physical machine.

Add Comment

[karing.martin@gmail.com (JIRA)]

Martin Karing commented on JENKINS-34150

Re: Pipeline Batch hangs I was able to track down that the batch process is forcefully terminated. If you run the wrapper batch by hand and close the command line window before the process finishes you get exactly the same behaviour. The main and the child processes are terminated and no files are created. The only thing that is attached to the command line actually is java. So the termination has to come from there. The thing I wonder is: Can all this even work across a jenkins restart? If Java terminates its child processes this would kill no command line execution no matter what. I think there are solutions to work around this using powershell or the scripting host. But those may be blocked on the host system.

Add Comment

[kkegao@gmail.com (JIRA)]

KK G commented on JENKINS-34150

Re: Pipeline Batch hangs Just gave a try. "workaround is to use a build agent (other than master), even being in the same physical machine." really works! Thanks. At least, I can proceed. BTW, I notice that for the same machine, master node has info, "Windows Server 2012 R2 (x86)", while client has info, "Windows Server 2012 R2 (amd64)". I doubt if the bug trigger corner case related to machine architecture.

Add Comment

[kkegao@gmail.com (JIRA)]

KK G updated an issue

Jenkins / JENKINS-34150 Pipeline Batch hangs

Change By: KK G Environment: Jenkins 2.0RC1   , Jenkins LTS Windows 2008r2, Windows 2012r2, Windows 7

Add Comment

[kkegao@gmail.com (JIRA)]

KK G updated an issue

Jenkins / JENKINS-34150 Pipeline Batch hangs

Change By: KK G Environment: Jenkins 2.0RC1, Jenkins LTS Windows 2008r2, Windows 2012r2, Windows 7 , Window 10

Add Comment

[kkegao@gmail.com (JIRA)]

KK G updated an issue

Jenkins / JENKINS-34150 Pipeline Batch hangs

Change By: KK G Environment: Jenkins 2.0RC1, Jenkins LTS Windows 2008r2, Windows 2012r2, Windows 7,  Window  Windows  10

Add Comment

[kkegao@gmail.com (JIRA)]

KK G updated an issue

Jenkins / JENKINS-34150 Pipeline Batch hangs

Change By: KK G Environment: Jenkins 2.0RC1, Jenkins  1.6*  LTS Windows 2008r2, Windows 2012r2, Windows 7, Windows 10

Add Comment

[jglick@cloudbees.com (JIRA)]

Jesse Glick updated an issue

Jenkins / JENKINS-34150 Pipeline Batch hangs

Change By: Jesse Glick Component/s: workflow-plugin

Add Comment

[jglick@cloudbees.com (JIRA)]

Jesse Glick commented on JENKINS-34150

Re: Pipeline Batch hangs Can all this even work across a jenkins restart? If you are using a master executor, not generally. (On Unix, it works under some conditions but not others.) You are strongly recommended to use an agent rather than master executors in general. In particular, if you have any kind of layered security on your Jenkins installation—whereby people configuring jobs (or permitted to edit build scripts in SCM) are not Jenkins administrators—you must not have a master executor, or any pretense at security is gone. Even if only one physical computer is available, you must configure a separate service account for builds. All that said, if the problem can be fixed—or at least clearly diagnosed and reported—without breaking anything for the more general use case of an agent on another machine, obviously we want to apply a fix.

Add Comment

[jglick@cloudbees.com (JIRA)]

Jesse Glick updated an issue

Jenkins / JENKINS-34150 Pipeline Batch hangs

Change By: Jesse Glick Labels: 2.0 2.0-planned testfest  windows  workflow

Add Comment

[luebbe.opensource@gmail.com (JIRA)]

Lübbe Onken commented on JENKINS-34150

Re: Pipeline Batch hangs This hanging batch bug has bitten me heavily too. Failing batch jobs always terminated, successful jobs never did. A working solution for me is to explicitely return an exit code from any batch call. I'm on Windows 7 Professional. So: {{ echo 'Successful step' bat '''dir exit /B %ERRORLEVEL%''' echo 'Failing step' bat '''find /c "_no.file" "_no.file" exit /B %ERRORLEVEL%''' echo 'Never execute step' bat '''dir" exit /B %ERRORLEVEL%''' }} successfully terminates step one and returns from the batch execution, the second step fails the build and the third step never gets executed. Can somebody please confirm that this solution works for them too?

Add Comment

[luebbe.opensource@gmail.com (JIRA)]

Lübbe Onken commented on JENKINS-34150

Re: Pipeline Batch hangs Looks like I was too optimistic. The solution always worked with short running batch jobs, like dir, but it didn't with long running jobs, like a NAnt build. Is it possible that there is a race condition? Some state is checked very quickly after a task is started and a simple "dir" is quick enough to deliver the result in time and a slower task isn't?

Add Comment

[luebbe.opensource@gmail.com (JIRA)]

Lübbe Onken edited a comment on JENKINS-34150

Re: Pipeline Batch hangs

Looks like I was too optimistic. The solution always worked with short running batch jobs, like dir, but it didn't with long running jobs, like a NAnt build.

Is it possible that there is a race condition? Some state is checked very quickly after a task is started  and a . A  simple "dir" is quick enough to deliver the result in time and a slower task isn't?

Add Comment

[carpentier.ch@gmail.com (JIRA)]

Christophe Carpentier commented on JENKINS-34150

Re: Pipeline Batch hangs That would explain the iconsistent results during my tests. Weird. Anyway, I've encountered this both on failed and successful jobs.

Add Comment

[amuniz@cloudbees.com (JIRA)]

Antonio Muñiz stopped work on JENKINS-34150

Change By: Antonio Muñiz Status: In Progress Open

Add Comment

[reasno@hotmail.com (JIRA)]

rens hoskens commented on JENKINS-34150

Re: Pipeline Batch hangs Have the same issue on Windows Server 2008 R2. Hope it will get fixed soon (or a plain maven command would be usefull as well)

Add Comment

[reasno@hotmail.com (JIRA)]

rens hoskens edited a comment on JENKINS-34150

Re: Pipeline Batch hangs

Have the same issue on Windows Server 2008 R2. Hope it will get fixed soon (or a plain maven command would be usefull as well)

{code:groovy} node {     mvn 'clean package -DskipTests=true' } def mvn(args) {     bat "${tool 'Maven 3.3.9'}/bin/mvn ${args}" } {code}

Add Comment

[reasno@hotmail.com (JIRA)]

rens hoskens edited a comment on JENKINS-34150

Re: Pipeline Batch hangs

Have the same issue on Windows Server 2008 R2. Hope it will get fixed soon (or a plain maven command would be usefull as well)

{code: groovy none }

node {     mvn 'clean package -DskipTests=true' } def mvn(args) {     bat "${tool 'Maven 3.3.9'}/bin/mvn ${args}" } {code}

Add Comment

[nick.sonneveld@smartward.com.au (JIRA)]

Nick Sonneveld commented on JENKINS-34150

Re: Pipeline Batch hangs I have mentioned in other related tickets but I just want to point out that I have seen this behaviour with a linux master and multiple windows agents (with 5-10 executors on each). You could try this example code. I haven't tested it but it's similar to our Jenkinsfile where we have branches doing chunks of a test. Cancelling the job in the middle of execution sometimes puts the agents in a weird state too. def branches = [:] for (int i = 0; i < 64; i++) { def id = "branch-${i}" branches[id] node ('windows') { for (int j = 0; j < 8; j++) { bat 'ping 127.0.0.1 -n 10' } } } parallel branches

Add Comment

[nick.sonneveld@smartward.com.au (JIRA)]

Nick Sonneveld edited a comment on JENKINS-34150

Re: Pipeline Batch hangs

I have mentioned in other related tickets but I just want to point out that I have seen this behaviour with a linux master and multiple windows agents (with 5-10 executors on each).  You could try this example code. I haven't tested it but it's similar to our Jenkinsfile where we have branches doing chunks of a test.  Cancelling the job in the middle of execution sometimes puts the agents in a weird state too.

{code:java}

def branches = [:] for (int i = 0; i < 64; i++) { def id = "branch-${i}"

branches[id]  = {

node ('windows') { for (int j = 0; j < 8; j++) {     bat 'ping 127.0.0.1 -n 10'  } } } }

parallel branches {code}

Add Comment

[g.kuijer@auxil.nl (JIRA)]

Gijs Kuijer commented on JENKINS-34150

Re: Pipeline Batch hangs I have the same exact issue on a windows server 2012 R2 with a Jenkins 2.1 installations and all plugins fully updated. I have installed the Github organization folder plugin to scan my organization. My jenkins file has a simple batch job to use MSBuild to build our project and a batch job for analysis of sonar. The job randomly hangs after one of these two jobs. Is there any progress on this issue?

Add Comment

[maaltan@gmail.com (JIRA)]

maaltan natlaam commented on JENKINS-34150

Re: Pipeline Batch hangs I have this issue also. my batch invocation is: bat ''' call %BUILD_CONFIG_PATH% setenv.cmd perl <custom build manager script that typically runs for 90 minutes> ''' This has broken the job completely. I cannot terminate the job, nor can i start another (this is an incremental build with a fixed workspace location so i dont want to run concurrently). If i restart jenkins the job restarts and immediately hangs again. Are there any workaround short of full uninstall/reinstall of jenkins to recover this job setup?

Add Comment

[ttt43ttt@126.com (JIRA)]

Wilson Tian commented on JENKINS-34150

Re: Pipeline Batch hangs I encounter this issue too. I'm running a maven job using Unable to find source-code formatter for language: groovy. Available languages are: actionscript, html, java, javascript, none, sql, xhtml, xml bat "${mavenHome}\\bin\\mvn clean package" . But the job always hangs at last and never exits. Is there any workaround?

Add Comment

[ttt43ttt@126.com (JIRA)]

Wilson Tian edited a comment on JENKINS-34150

Re: Pipeline Batch hangs I encounter this issue too. I'm running a maven job using {code: groovy java }bat "${mavenHome}\\bin\\mvn clean package"{code}. But the job always hangs at last and never exits. Is there any workaround?

Add Comment

[maaltan@gmail.com (JIRA)]

maaltan natlaam commented on JENKINS-34150

Re: Pipeline Batch hangs I found a workaround (better than reinstalling at least). 1. Shutdown jenkins service 2. go to <install>/jobs/<jobname>/ and delete the <jobnumber> folder. 3. Restart jenkins. There is probably a flag somewhere in that folder you can set to prevent the job from "restarting" after restart. Since ive hit this bug about 80% of the runs I've tried so far. this workaround is unusable in any kind of production environment. At least you can run the job again though. I guess another workaround would be allow multiple instances of the job to run and clean up the zombies once a day or something? Also, it seems this happens more when i view the console output via jenkins ui while the job is running.

Add Comment

[nick.sonneveld@smartward.com.au (JIRA)]

Nick Sonneveld commented on JENKINS-34150

Re: Pipeline Batch hangs Another workaround that doesn't involved deleting jobs (but isn't a long term solution) is to realise that batch steps create two batch files in the @tmp directory (which is relative to where batch is run, so it might be in the workspace if you've changed the directory, or just outside it): a jenkins-main.bat and a jenkins-wrap.bat. The main bat file contains your commands. The wrap bat file will run the main bat, pipe output to a log file and finally writes a result file. The bug concerns the wrap bat file not completing so the result file is never written. You can search for the file and run the final line manually (looks like echo %errorlevel% > ...\jenkins-result.txt), or run the wrap batch file again if you don't mind it performing the same operation again.

Add Comment

[nick.sonneveld@smartward.com.au (JIRA)]

Nick Sonneveld edited a comment on JENKINS-34150

Re: Pipeline Batch hangs Another workaround that doesn't  involved  involve  deleting jobs (but  also  isn't a long term solution) is to realise that batch steps create two batch files in the @tmp directory (which is relative to where batch is run, so it might be in the workspace if you've changed the directory, or just outside it): a jenkins-main.bat and a jenkins-wrap.bat. The main bat file contains your commands. The wrap bat file will run the main bat, pipe output to a log file and finally writes a result file.

The bug concerns the wrap bat file not completing so the result file is never written.  You can search for the file and run the final line manually (looks like echo %errorlevel% > ...\jenkins-result.txt), or run the wrap batch file again if you don't mind it performing the same operation again.

Add Comment

[maaltan@gmail.com (JIRA)]

maaltan natlaam commented on JENKINS-34150

Re: Pipeline Batch hangs cmd /c ""<script> > ".../jenkins-log.txt"" 2>&1 echo %ERRORLEVEL% > "...\..@tmp\durable-cf7a3b23\jenkins-result.txt" Perhaps using "call" will work better. that will leverage the current cmd shell to execute the batch. Ive found it more stable than launching a second cmd from a batch file. "start" is another option. That gives you a subshell that is detached from the main shell. There are parameters that prevent that though. other bonuses of start is ability to set process priorities,etc. If you know of the jar/class i need to hack off hand to make this change, i'll try to give it a shot today. I am probably going to end up grabbing the jenkins source at some point but that will probably be later next week if then.

Add Comment

[nick.sonneveld@smartward.com.au (JIRA)]

Nick Sonneveld commented on JENKINS-34150

Re: Pipeline Batch hangs There is a pull request being worked on by Martin Karing you might want to look at and comment on. Link is attached to this issue https://github.com/jenkinsci/durable-task-plugin/pull/21

Add Comment

[maaltan@gmail.com (JIRA)]

maaltan natlaam commented on JENKINS-34150

Re: Pipeline Batch hangs First off, i found a better workaround. First go to the console screen for the job. click the abort button in upper right area. scroll to bottom and wait about 10 seconds. You will see a link allowing you to force kill the job. Started At: 05-06-2016 20:00:08 Ended At: 05-06-2016 20:02:19 Build Lasted: 2 minutes 10 seconds Highest Error Code: 0 <hang here> Aborted by admin Sending interrupt signal to process Click here to forcibly terminate running steps Terminating bat [Pipeline] } [Pipeline] // node [Pipeline] End of Pipeline Finished: ABORTED Unfortunately I'm seeing an almost 100% chance of hangs on my machine so still pretty useless. ------------------------- I acquired Karing's code and tested it. it terminates my bat build steps after about 3-5 seconds no matter what the state is. I didnt dig too much into that. I reverted to baseline and tried my suggestions. None of them work. In fact, i can't prove that the script call from jenkins-wrap.bat ever returns... here is my current attempt at jenkins-wrap.bat: cmd /c ""...\jenkins-main.bat"" > "...\jenkins-log.txt" 2>&1 :retry echo writing jenkins.results >> "...\jenkins-log.txt" echo %ERRORLEVEL% > "...\jenkins-result.txt" if not exist "...\jenkins-result.txt" goto retry It is supposed to jackhammer that results file until it is created. I see no "writing jenkins.results" in the logs, therefore the wrapper script is terminating early. The same thing happens if i replace cmd /c with call or start. (note: "..." is a placeholder for my real paths not some kind of relative path thing. sorry for confusion.)

Add Comment

[camille@techsoft3d.com (JIRA)]

Camille Baud commented on JENKINS-34150

Re: Pipeline Batch hangs Sucessfulyl reproduced with: node("master") { for( int i = 0; i < 100; ++i ) { echo "i=${i}" bat "ping JENKINSSRVMGR" } } ran multiple times;

Add Comment

[camille@techsoft3d.com (JIRA)]

Camille Baud updated an issue

Jenkins / JENKINS-34150 Pipeline Batch hangs

Change By: Camille Baud Comment: Sucessfulyl reproduced with: {code:java}

node("master") {   for( int i = 0; i < 100; ++i ) {     echo "i=${i}"     bat "ping JENKINSSRVMGR"   } }

{code} ran multiple times;

Add Comment

[jnord@cloudbees.com (JIRA)]

James Nord commented on JENKINS-34150

Re: Pipeline Batch hangs I believe I have a 100% reproducible test-case for this issue It seems that in the case that the parent process has been killed (e.g. the slave dies) then all though the script terminates successfully and the wrapper terminates successfully (checked with process monitor) there is no attempt to create the result file. where the parent process has not been killed I never see this issue. All I needed to do to fix the issue I was observing is add @echo off as the first line of the wrapper script. Basically I believe it is trying to echo the commands to be run before running the commands and as there is no longer anything consuming the wrappers input/output when echoing the command it is doomed to fail (but strangely not with an exit code that implies something died!??!

Add Comment

[vyazelenko@yahoo.com (JIRA)]

Dmitry Vyazelenko commented on JENKINS-34150

Re: Pipeline Batch hangs I'm also having a build hang with simple pipeline that uses bat script to run Gradle tasks: node { timeout(time: 10, unit: 'MINUTES') { timestamps { stage 'Checkout' git ... stage 'Tests' bat 'gradlew test' step([$class: 'JUnitResultArchiver', testResults: 'build/test-results/*.xml']) } } }

Add Comment

[vyazelenko@yahoo.com (JIRA)]

Dmitry Vyazelenko updated an issue

Jenkins / JENKINS-34150 Pipeline Batch hangs

Change By: Dmitry Vyazelenko Priority: Critical Blocker

Add Comment

[jnord@cloudbees.com (JIRA)]

James Nord commented on JENKINS-34150

Re: Pipeline Batch hangs Fo anyone observing the issue you can try installing the build from PR24 or PR21 and see if this resolves your issue. (I would start with PR24 first as it is a much smaller change, but then I am biased!)

Add Comment

[jnord@cloudbees.com (JIRA)]

James Nord edited a comment on JENKINS-34150

Re: Pipeline Batch hangs Fo For  anyone observing the issue you can try installing [the build from PR24 | https://jenkins.ci.cloudbees.com/job/plugins/job/durable-task-plugin/99/org.jenkins-ci.plugins$durable-task/artifact/org.jenkins-ci.plugins/durable-task/1.10-SNAPSHOT/durable-task-1.10-SNAPSHOT.hpi] or [PR21 | https://jenkins.ci.cloudbees.com/job/plugins/job/durable-task-plugin/97/org.jenkins-ci.plugins$durable-task/artifact/org.jenkins-ci.plugins/durable-task/1.10-SNAPSHOT/durable-task-1.10-SNAPSHOT.hpi] and see if this resolves your issue.  (I would start with PR24 first as it is a much smaller change, but then I am biased!)

Add Comment

[carpentier.ch@gmail.com (JIRA)]

Christophe Carpentier commented on JENKINS-34150

Re: Pipeline Batch hangs PR24 fixes my particular issue.

Add Comment

[carpentier.ch@gmail.com (JIRA)]

Christophe Carpentier updated an issue

Jenkins / JENKINS-34150 Pipeline Batch hangs

Change By: Christophe Carpentier Attachment: How.png

Add Comment

[scm_issue_link@java.net (JIRA)]

SCM/JIRA link daemon commented on JENKINS-34150

Re: Pipeline Batch hangs Code changed in jenkins User: James Nord Path: src/main/java/org/jenkinsci/plugins/durabletask/WindowsBatchScript.java http://jenkins-ci.org/commit/durable-task-plugin/d156ebfbcdb70666757ff48127d0597bd5891a61 Log: JENKINS-34150 Fixes my observed issue. I have a reproducable tests case in a propratary implementation using this code that is 100% reproducable. The simple "@echo off" fixes the failing test for me.

It seems that in the case that the parent process has been killed (e.g. the slave dies) then all though the script terminates successfully and the wrapper terminates successfully (checked with process monitor) there is no attempt to create the result file. where the parent process has not been killed I never see this issue. All I needed to do to fix the issue I was observing is add @echo off as the first line of the wrapper script. Basically I believe it is trying to echo the commands to be run before running the commands and as there is no longer anything consuming the wrappers input/output when echoing the command it is doomed to fail (but strangely not with an exit code that implies something died!??!

Add Comment

[scm_issue_link@java.net (JIRA)]

SCM/JIRA link daemon commented on JENKINS-34150

Re: Pipeline Batch hangs Code changed in jenkins User: Jesse Glick Path: src/main/java/org/jenkinsci/plugins/durabletask/WindowsBatchScript.java http://jenkins-ci.org/commit/durable-task-plugin/8a2537cf28c826ad91d4ce14cd657712364c8953 Log: Merge pull request #24 from jtnord/jenkins-34150 [FIXED JENKINS-34150] Fixes my observed issue. Compare: https://github.com/jenkinsci/durable-task-plugin/compare/0f09bb54a1b7...8a2537cf28c8

Add Comment

[scm_issue_link@java.net (JIRA)]

SCM/JIRA link daemon resolved as Fixed

Jenkins / JENKINS-34150 Pipeline Batch hangs

Change By: SCM/JIRA link daemon Status: Open Resolved Resolution: Fixed

Add Comment

[svanoort@cloudbees.com (JIRA)]

Sam Van Oort commented on JENKINS-34150

Re: Pipeline Batch hangs Most important single-line-of-code change I've seen recently.

Add Comment

[junkimu@java.net (JIRA)]

Junichi Kimura commented on JENKINS-34150

Re: Pipeline Batch hangs The test job below used to consistently get stuck at the 8mins stage. After upgrading the Durable Task Plugin to 1.10, the job passed successfully (once so far). node Unknown macro: { stage '1min' bat ''' @ECHO OFF FOR /L %%A IN (0,1,60) DO ( ECHO %%A PING 192.0.2.1 -n 1 -w 1000 >NUL ) EXIT /B 0 ''' stage '5mins' bat ''' @ECHO OFF FOR /L %%A IN (0,1,300) DO ( ECHO %%A PING 192.0.2.1 -n 1 -w 1000 >NUL ) EXIT /B 0 ''' stage '8mins' bat ''' @ECHO OFF FOR /L %%A IN (0,1,480) DO ( ECHO %%A PING 192.0.2.1 -n 1 -w 1000 >NUL ) EXIT /B 0 ''' stage '10mins' bat ''' @ECHO OFF FOR /L %%A IN (0,1,600) DO ( ECHO %%A PING 192.0.2.1 -n 1 -w 1000 >NUL ) EXIT /B 0 ''' stage '30mins' bat ''' @ECHO OFF FOR /L %%A IN (0,1,1800) DO ( ECHO %%A PING 192.0.2.1 -n 1 -w 1000 >NUL ) EXIT /B 0 ''' stage '1hr' bat ''' @ECHO OFF FOR /L %%A IN (0,1,3600) DO ( ECHO %%A PING 192.0.2.1 -n 1 -w 1000 >NUL ) EXIT /B 0 ''' }

Add Comment

[junkimu@java.net (JIRA)]

Junichi Kimura edited a comment on JENKINS-34150

Re: Pipeline Batch hangs

The test job below used to consistently get stuck at the 8mins stage.  After upgrading the Durable Task Plugin to 1.10, the job passed successfully (once so far).

{ quote} { node {

} {quote } }

Add Comment

[junkimu@java.net (JIRA)]

Junichi Kimura edited a comment on JENKINS-34150

Re: Pipeline Batch hangs

The test job below used to consistently get stuck at the 8mins stage.  After upgrading the Durable Task Plugin to 1.10, the job passed successfully (once so far).

{{

Add Comment

[junkimu@java.net (JIRA)]

Junichi Kimura edited a comment on JENKINS-34150

Re: Pipeline Batch hangs

The test job below used to consistently get stuck at the 8mins stage.  After upgrading the Durable Task Plugin to 1.10, the job passed successfully (once so far).

{ { code:java}

{code } }

Add Comment

[g.kuijer@auxil.nl (JIRA)]

Gijs Kuijer commented on JENKINS-34150

Re: Pipeline Batch hangs Great job! Totally solves my issues!

Add Comment

[d@danield.us (JIRA)]

Daniel Daugherty commented on JENKINS-34150

Re: Pipeline Batch hangs Initial testing here show issue is resolved Just getting back from vacation so nice to see this resolved. Thanks all for the good work.

Add Comment

[rufermarc@gmail.com (JIRA)]

Marc Rufer commented on JENKINS-34150

Re: Pipeline Batch hangs Thanks guys. Great job. Updating to the newest version of the durable-task-plugin solved the issue for me as well!

Add Comment

[sven.brosi@lidl.com (JIRA)]

Sven Brosi reopened an issue

Hello, since the version 1.20 of the Durable Task plugin we encounter the same behavior described above.  Our worker nodes have Win2012 and Win2018 environment. A quick solution was to downgrade the version of this plugin to 1.18. Then everything works again in the DSL Jenkinsfiles with bat(ch) step.

Jenkins / JENKINS-34150 Pipeline Batch hangs

Change By: Sven Brosi Resolution: Fixed Status: Resolved Reopened

Add Comment

This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e)

[stevengfoster@gmail.com (JIRA)]

Steven Foster commented on JENKINS-34150

Re: Pipeline Batch hangs I'm encountering the same issue after updating to 1.20 from 1.18

Add Comment

[svanoort@cloudbees.com (JIRA)]

Sam Van Oort updated an issue

Jenkins / JENKINS-34150 Pipeline Batch hangs

Change By: Sam Van Oort Attachment: durable-task.hpi

Add Comment

[svanoort@cloudbees.com (JIRA)]

Sam Van Oort commented on JENKINS-34150

Re: Pipeline Batch hangs Steven Foster Does it work if you set 'returnStdOut: true'? If so, I have an attached hotfix for you to try – please let us know if this resolves it. durable-task.hpi

Add Comment

[svanoort@cloudbees.com (JIRA)]

Sam Van Oort commented on JENKINS-34150

Re: Pipeline Batch hangs Sven Brosi Does it work if you set 'returnStdOut: true', and if so, please try the attached hotfix and let us know if this resolves it.

Add Comment

[stevengfoster@gmail.com (JIRA)]

Steven Foster commented on JENKINS-34150

Re: Pipeline Batch hangs returnStdout: true has the same result. confirmed the process is finished on the machine.

Add Comment

[svanoort@cloudbees.com (JIRA)]

Sam Van Oort commented on JENKINS-34150

Re: Pipeline Batch hangs Steven Foster In the control directory in the build agent's workspace for this job (where there's a jenkins-main.bat and jenkins-wrap.bat) do you see a jenkins-result.txt file, a jenkins-result.txt.tmp, or both? I'm trying to figure out what actually triggered this because it did not fail any of our unit tests that should have explicitly covered this functionality.

Add Comment

[stevengfoster@gmail.com (JIRA)]

Steven Foster commented on JENKINS-34150

Re: Pipeline Batch hangs just the .tmp

Add Comment

[svanoort@cloudbees.com (JIRA)]

Sam Van Oort commented on JENKINS-34150

Re: Pipeline Batch hangs Steven Foster Okay, that means the "move" operation failed to rename the file, which should basically never happen. Would you be able to hop on #jenkins IRC briefly to discuss (I'm svanoort there)? It should be a quick and trivial fix but since we can't reproduce the issue in our own environment, it would be super-helpful to be able to see a case where it happened.

Add Comment

[sven.brosi@lidl.com (JIRA)]

Sven Brosi commented on JENKINS-34150

Re: Pipeline Batch hangs Sam Van Oort: Can not test it again, as it is a corporate jenkins and i get up and down grade the plugins on the fly. How can i still help you?

Add Comment

[sven.brosi@lidl.com (JIRA)]

Sven Brosi edited a comment on JENKINS-34150

Re: Pipeline Batch hangs [~svanoort]: Can not test it again, as it is a corporate jenkins and i get can not  up - and down grade downgrade the plugins on the fly.

How can i still help you?

Add Comment

[svanoort@cloudbees.com (JIRA)]

Sam Van Oort updated an issue

Jenkins / JENKINS-34150 Pipeline Batch hangs Change By: Sam Van Oort Attachment: durable-task.hpi

Add Comment

[svanoort@cloudbees.com (JIRA)]

Sam Van Oort commented on JENKINS-34150

Re: Pipeline Batch hangs Sven Brosi Don't worry about it, Steven Foster was thankfully available to help debug in an environment where this is reproducible (thanks!). I'm attaching one more hotfix version for him to try out which should fully resolve the issue. durable-task.hpi

Add Comment

[svanoort@cloudbees.com (JIRA)]

Sam Van Oort commented on JENKINS-34150

Re: Pipeline Batch hangs This issue duplicates symptoms of JENKINS-50025 but the root causes are significantly different, so that is being tracked separately.

Add Comment

[svanoort@cloudbees.com (JIRA)]

Sam Van Oort closed an issue as Fixed

Sven Brosi Steven Foster I'm closing THIS issue because while the result is the same this has a different cause and is resolved in JENKINS-50025.

Jenkins / JENKINS-34150 Pipeline Batch hangs

Change By: Sam Van Oort

Status: Reopened Closed Resolution: Fixed

Add Comment

[svanoort@cloudbees.com (JIRA)]

Sam Van Oort commented on JENKINS-34150

Re: Pipeline Batch hangs Sven Brosi Steven Foster Released fix after review and testing as durable-task-plugin 1.21

Add Comment