I have set up the connection between Jenkins and AWS via Amazon EC2 plugin. Jenkins master cloud config: !Capture.PNG!!Capture2.PNG!!Capture3.PNG|width=1016,height=736!
The node connects via the Amazon plugin and then creates a new connection via Swarm plugin and the job ends up running on the connection made through swarm. (This is because my jobs include TestComplete & FlaUI and winRM is not quite suited for their requirements).
Jobs that take under 25 min run successfully, anything that goes over 25-26 min fails with the following:
{code:java}
12:49:46 java.io.IOException: Backing channel 'JNLP4-connect connection from 10.230.0.101/10.230.0.101:49724' is disconnected.
12:49:46 at hudson.remoting.RemoteInvocationHandler.channelOrFail(RemoteInvocationHandler.java:214)
12:49:46 at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:283)
12:49:46 at com.sun.proxy.$Proxy89.isAlive(Unknown Source)
12:49:46 at hudson.Launcher$RemoteLauncher$ProcImpl.isAlive(Launcher.java:1172)
12:49:46 at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:1164)
12:49:46 at hudson.Launcher$ProcStarter.join(Launcher.java:492)
12:49:46 at hudson.plugins.gradle.Gradle.performTask(Gradle.java:333)
12:49:46 at hudson.plugins.gradle.Gradle.perform(Gradle.java:225)
12:49:46 at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
12:49:46 at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:741)
12:49:46 at hudson.model.Build$BuildExecution.build(Build.java:206)
12:49:46 at hudson.model.Build$BuildExecution.doRun(Build.java:163)
12:49:46 at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
12:49:46 at hudson.model.Run.execute(Run.java:1815)
12:49:46 at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
12:49:46 at hudson.model.ResourceController.execute(ResourceController.java:97)
12:49:46 at hudson.model.Executor.run(Executor.java:429)
12:49:46 Caused by: java.nio.channels.ClosedChannelException
12:49:46 at org.jenkinsci.remoting.protocol.NetworkLayer.onRecvClosed(NetworkLayer.java:154)
12:49:46 at org.jenkinsci.remoting.protocol.impl.NIONetworkLayer.ready(NIONetworkLayer.java:179)
12:49:46 at org.jenkinsci.remoting.protocol.IOHub$OnReady.run(IOHub.java:795)
12:49:46 at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
12:49:46 at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)
12:49:46 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
12:49:46 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
12:49:46 at java.lang.Thread.run(Thread.java:748)
{code}
On the master's log I can see:
{code:java}
ouch, stdout exception for java -jar C:\Windows\Temp\remoting.jar -workDir C:\Users\clujouch, stdout exception for java -jar C:\Windows\Temp\remoting.jar -workDir C:\Users\clujjava.lang.NumberFormatException: For input string: "4294967295" at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) at java.lang.Integer.parseInt(Integer.java:583) at java.lang.Integer.parseInt(Integer.java:615) at hudson.plugins.ec2.win.winrm.WinRMClient.slurpOutput(WinRMClient.java:151) at hudson.plugins.ec2.win.winrm.WindowsProcess$1.run(WindowsProcess.java:99)
Sep 27, 2019 10:45:27 AM INFO hudson.remoting.SynchronousCommandTransport$ReaderThread runI/O error in channel Idle timeout of EC2 (Itiviti AWS) - Windows Jenkins node autoconnecting to deb-jenkins-prd using Swarm plugin (i- 03c88c3229acf1e7a 000908b57bb5d82a7 ) java.io.EOFException at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java after 30 idle minutes, instance statusRUNNING
Sep 30, 2019 8 : 2681) at java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java 40 : 3156) at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:862) at java.io.ObjectInputStream.<init>(ObjectInputStream.java:358) at 45 AM INFO hudson. remoting plugins . ObjectInputStreamEx ec2 . <init>(ObjectInputStreamEx.java EC2AbstractSlave idleTimeout
EC2 instance idle time expired : 49) at hudson.remoting.Command.readFrom(Command.java:140) at hudson.remoting.Command.readFrom(Command.java:126) at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:36) at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:63)Caused: java.io.IOException: Unexpected termination of the channel at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:77) i-000908b57bb5d82a7
Sep 27 30 , 2019 10 8 : 45 40 : 27 46 AM SEVERE INFO hudson.plugins.ec2. win.winrm.WinRMClient sendRequestI/O Exception in HTTP POSTjava.io.IOException: Attempted read from closed stream. at org.apache.http.impl.io.ContentLengthInputStream.read EC2OndemandSlave terminate
Terminated EC2 instance ( ContentLengthInputStream.java:165 terminated ) at org.apache.http.conn.EofSensorInputStream.read(EofSensorInputStream.java : 135) at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java i-000908b57bb5d82a7
Sep 30, 2019 8 : 284) at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java 40 : 326) at sun 46 AM INFO jenkins . nio slaves . cs.StreamDecoder.read(StreamDecoder.java DefaultJnlpSlaveReceiver channelClosed
IOHub#1 : 178) at java.io.InputStreamReader.read(InputStreamReader.java Worker[channel : 184) at java. io nio . Reader channels . read(Reader SocketChannel[connected local=/172 . java:140) at org 17 . apache 0 . http.util.EntityUtils.toString(EntityUtils.java 2 : 227) at org 40440 remote=10 . apache 230 . http 0 . util 71/10 . EntityUtils 230 . toString(EntityUtils 0 . java 71 : 308) at hudson 49735]] / Computer . plugins.ec2.win.winrm.WinRMClient.sendRequest(WinRMClient.java threadPoolForRemoting [#85772] for ec2amaz-glc1084 terminated : 261) at hudson.plugins.ec2.win.winrm.WinRMClient.sendRequest(WinRMClient. java :188) at hudson . plugins nio . ec2 channels . win.winrm.WinRMClient.sendInput(WinRMClient.java:120) at hudson.plugins.ec2.win.winrm.WindowsProcess$2.run(WindowsProcess.java:134) ClosedChannelException
Sep 27 30 , 2019 10 8 : 45 40 : 27 46 AM WARNING INFO hudson. plugins model . ec2.win.winrm.WindowsProcess$2 runouch, STDIN exception for java Run execute
aws - jar C:\Windows\Temp\remoting.jar ul - workDir C trader-extension-master-desk-uitests-listorders #22 main build action completed : \Users\clujjava.io.IOException FAILURE
Sep 30, 2019 8 : Attempted read from closed stream. at org.apache.http.impl.io.ContentLengthInputStream.read(ContentLengthInputStream.java 40 : 165) at org.apache.http.conn.EofSensorInputStream.read(EofSensorInputStream.java:135) at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284) at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326) at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178) at java.io.InputStreamReader.read(InputStreamReader.java:184) at java.io.Reader.read(Reader.java:140) at org.apache.http.util.EntityUtils.toString(EntityUtils.java:227) at org.apache.http.util.EntityUtils.toString(EntityUtils.java:308) at 46 AM INFO hudson.plugins.ec2. win.winrm.WinRMClient.sendRequest(WinRMClient.java:261)Caused: hudson.plugins.ec2.win.winrm.RuntimeIOException: I/O Exception Attempted read EC2OndemandSlave terminate
Removed EC2 instance from closed stream. at hudson.plugins.ec2.win.winrm.WinRMClient.sendRequest(WinRMClient.java jenkins master : 276) at hudson.plugins.ec2.win.winrm.WinRMClient.sendRequest(WinRMClient.java:188) at hudson.plugins.ec2.win.winrm.WinRMClient.sendInput(WinRMClient.java:120) at hudson.plugins.ec2.win.winrm.WindowsProcess$2.run(WindowsProcess.java:134) i-000908b57bb5d82a7
{code}
Any help in tracking down the problem is much appreciated. |
|
