[JIRA] (JENKINS-58627) jenkins agent getting terminated with version 2.176.2

[slide.o.mix@gmail.com (JIRA)]

Alex Earl updated an issue

Jenkins / JENKINS-58627 jenkins agent getting terminated with version 2.176.2 Change By: Alex Earl Summary: jenkins slave agent getting terminated with version 2.176.2 Hi, Jenkins slave agent is geting terminated with new version 2.176.2 as explained below   OS - windows 2012 R2 (both Master and Slave agent ) Java - 1.8.0_211 (both Mater and Salve slave ) Jenkins Version - 2.176.2 Launch Method - Java Web start, JNLP(Launch by connecting it to the Master)   Issue - I have a project in Jenkins which fails mainly because the slave agent terminates after 'Starting ServiceWrapper in the CLI mode' during execution.Finds a child process conhost.exe and failed due to Killing as fallback. This is happening only for a project/ pipeline, where I have python script which pings a internal URL (https://URL/ping), invokes the network emulator tool installed, adjust the network latency and also write to a file/ push to ELK. I have a internet proxy set up using .pac file.  It completes all the steps in project, at the end the slave agent terminates, marking the build as failure  Note - We also have another old version of jenkins 2.76 installed where the same project/ pipeline works with out any issue. Now we want to upgrade to latest jenkins we are seeing issue with slave agent getting terminated. I now have to close the network emulator tool every time and restart the jenkins service to bring it back online.  Error as obsreved in Jenkins-slave.wrapper.log:- 2019-07-24 03:51:59,751 INFO  - Stopping jenkinsslave-e__jenkins 2019-07-24 03:51:59,751 DEBUG - ProcessKill 6872 2019-07-24 03:51:59,860 INFO  - Found child process: 6408 Name: conhost.exe 2019-07-24 03:51:59,892 INFO  - Stopping process 6408 2019-07-24 03:51:59,907 INFO  - Send SIGINT 6408 2019-07-24 03:51:59,907 WARN  - SIGINT to 6408 failed - Killing as fallback 2019-07-24 03:51:59,907 INFO  - Stopping process 6872 2019-07-24 03:51:59,907 INFO  - Send SIGINT 6872 2019-07-24 03:51:59,907 WARN  - SIGINT to 6872 failed - Killing as fallback 2019-07-24 03:51:59,907 INFO  - Finished jenkinsslave-e__jenkins 2019-07-24 03:51:59,907 DEBUG - Completed. Exit code is 0   Error as obsreved in jenkins-slave.error.log:- Jul 24, 2019 3:57:23 AM hudson.remoting.jnlp.Main$CuiListener statusJul 24, 2019 3:57:23 AM hudson.remoting.jnlp.Main$CuiListener statusINFO: Terminated Jul 24, 2019 3:57:23 AM hudson.util.ProcessTree getWARNING: Error while determining if vetoers existhudson.remoting.RequestAbortedException: java.nio.channels.ClosedChannelException at hudson.remoting.Request.abort(Request.java:340) at hudson.remoting.Channel.terminate(Channel.java:1040) at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer.onReadClosed(ChannelApplicationLayer.java:209) at org.jenkinsci.remoting.protocol.ApplicationLayer.onRecvClosed(ApplicationLayer.java:222) at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecvClosed(ProtocolStack.java:816) at org.jenkinsci.remoting.protocol.FilterLayer.onRecvClosed(FilterLayer.java:287) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.onRecvClosed(SSLEngineFilterLayer.java:172) at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecvClosed(ProtocolStack.java:816) at org.jenkinsci.remoting.protocol.NetworkLayer.onRecvClosed(NetworkLayer.java:154) at org.jenkinsci.remoting.protocol.impl.BIONetworkLayer.access$1800(BIONetworkLayer.java:48) at org.jenkinsci.remoting.protocol.impl.BIONetworkLayer$Reader.run(BIONetworkLayer.java:264) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:93) at java.lang.Thread.run(Unknown Source) Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to JNLP4-connect connection to jenkinsmater/IP address:33131 at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1743) at hudson.remoting.Request.call(Request.java:202) at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:286) at com.sun.proxy.$Proxy6.fetch3(Unknown Source) at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:209) at java.lang.ClassLoader.loadClass(Unknown Source) at java.lang.ClassLoader.loadClass(Unknown Source) at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(Unknown Source) at java.lang.ClassLoader.defineClass(Unknown Source) at hudson.remoting.RemoteClassLoader.loadClassFile(RemoteClassLoader.java:383) at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:307) at java.lang.ClassLoader.loadClass(Unknown Source) at java.lang.ClassLoader.loadClass(Unknown Source) at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(Unknown Source) at java.lang.ClassLoader.defineClass(Unknown Source) at hudson.remoting.RemoteClassLoader.loadClassFile(RemoteClassLoader.java:383) at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:307) at java.lang.ClassLoader.loadClass(Unknown Source) at java.lang.ClassLoader.loadClass(Unknown Source) at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(Unknown Source) at java.lang.ClassLoader.defineClass(Unknown Source) at hudson.remoting.RemoteClassLoader.loadClassFile(RemoteClassLoader.java:383) at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:307) at java.lang.ClassLoader.loadClass(Unknown Source) at java.lang.ClassLoader.loadClass(Unknown Source) at hudson.util.ProcessTree.get(ProcessTree.java:415) at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1103) at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1094) at hudson.remoting.UserRequest.perform(UserRequest.java:212) at hudson.remoting.UserRequest.perform(UserRequest.java:54) at hudson.remoting.Request$2.run(Request.java:369) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at java.util.concurrent.FutureTask.run(Unknown Source) ... 4 moreCaused by: java.nio.channels.ClosedChannelException ... 7 more Jul 24, 2019 3:57:23 AM hudson.util.ProcessTree getWARNING: Failed to load winp. Reverting to the defaultjava.lang.NoClassDefFoundError: hudson/util/ProcessTree$WindowsOSProcessException at hudson.util.ProcessTree.get(ProcessTree.java:430) at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1103) at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1094) at hudson.remoting.UserRequest.perform(UserRequest.java:212) at hudson.remoting.UserRequest.perform(UserRequest.java:54) at hudson.remoting.Request$2.run(Request.java:369) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:93) at java.lang.Thread.run(Unknown Source)Caused by: java.lang.ClassNotFoundException: hudson.util.ProcessTree$WindowsOSProcessException at java.net.URLClassLoader.findClass(Unknown Source) at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:171) at java.lang.ClassLoader.loadClass(Unknown Source) at java.lang.ClassLoader.loadClass(Unknown Source) ... 12 more Jul 24, 2019 3:57:23 AM hudson.remoting.Request$2 runINFO: Failed to send back a reply to the request hudson.remoting.Request$2@626d990c: hudson.remoting.ChannelClosedException: Channel "hudson.remoting.Channel@4d834d1e:JNLP4-connect connection to jenkinsmaster/IP Address:33131": channel is already closed Jul 24, 2019 3:57:33 AM hudson.remoting.jnlp.Main$CuiListener statusINFO: Performing onReconnect operation.   Can someone please help with this issue ?   Thanks, Chethan Add Comment

This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d)

[chetuiyer@gmail.com (JIRA)]

Chethan Mahabaleswara updated an issue

Jenkins / JENKINS-58627 jenkins agent getting terminated with version 2.176.2

More information about the issue -    I'm having an issue while using Networl Connection Emulator that it just terminates the jenkins agent process as explained below:  Scenario 1 - I have jenkins agent set up in Windows 2012 machine where connection emulator is installed. I run a python script through jenkins pipeline to set the latency and open the connection emulator using python  function "subprocess.Popen(["schtasks","/Run","/TN","StartNetworkEmulator"]) ", which opens emulator from windows task scheduler as a last step.  Once the connection emulator is opened, it interferes with the jenkins agent process and terminates the jenkins agent and master communication.  I have to close the emulator and restart the jenkins agent windows service to make agent communicate with jenkins master  Scenario 2 - There are no jenkins jobs or pipeline running. Jenkins agent is active and connected to Master.  When I open and run Connection emulator manually by going C:\Program Files, the jenkins agent and master communication is broken.    Sometimes I see Linkage error and few times "Error while determining if vetoers exist" The error log observed in build and slave are attached.  Note - Same job or pipeline works with no issues with old jenkins version 2.76 having remoting version 3.11.  But we want to upgrade to new windows server where I installed latest jenkins version.   Is this a bug with latest version of jenkins ?  Appreciate any help in fixing this issue   Thanks, Chethan Change By: Chethan Mahabaleswara Attachment: Error log_in Build.txt Attachment: Agent Log_V1.txt

Add Comment

[chetuiyer@gmail.com (JIRA)]

Chethan Mahabaleswara commented on JENKINS-58627

Re: jenkins agent getting terminated with version 2.176.2 Hello Oleg, I see the fix is released as part of jenskins  V 2.181, for Component  remoting [15489]. But when i tried with jenkins version 2.187, observed the same issue/ error I had also tried jenkins version 2.176.2, but I see fix is not released in this version.

Add Comment